Device and method
By selecting layers based on compression efficiency and popularity, the device optimizes data transfer and storage of divided machine learning models across cloud and edge servers, addressing data output challenges and maintaining security in communication-heavy environments.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- NTT DOCOMO INC
- Filing Date
- 2025-01-16
- Publication Date
- 2026-07-23
AI Technical Summary
The increase in data output by some layers of a divided machine learning model complicates efficient arrangement and usage, particularly in communication-heavy environments.
A device and method that efficiently arrange and use a divided machine learning model by selecting layers based on compression efficiency and popularity, distributing them across cloud and edge servers for split inference.
This approach reduces data communication while maintaining security levels, allowing efficient use of AI models in congested areas by optimizing data transfer and storage.
Smart Images
Figure JP2025001198_23072026_PF_FP_ABST
Abstract
Description
Device and Method
[0008]
[0001] The present invention relates to a device and a method.
[0002] Patent Document 1 describes dividing a so-called machine learning model into several parts and selecting, for example, a first part of a layer of the machine learning model for execution at a first node and a second part of a layer for remote execution for execution at a second node.
[0003] Japanese Patent Application Laid-Open No. 2023-4860
[0004] When the machine learning model is divided into several parts and communication is performed between the layers of each machine learning model as described in Patent Document 1, the security level becomes high. However, there is a problem that the amount of data output by some of those layers increases.
[0005] Therefore, an object of the present invention is to provide a device and a method for efficiently arranging and using a divided machine learning model.
[0006] The device of the present disclosure includes a storage unit that stores an AI model composed of a plurality of layers, and a processing unit that selects, as a target to be downloaded from an external device, one or more layers among the plurality of layers constituting the AI model, the output data from each layer satisfying a condition.
[0007] According to the present invention, a divided machine learning model can be efficiently arranged and used.
[0008] Figure 1 is a diagram showing the system configuration including the cloud server 100 of this disclosure, and is a diagram showing the process when a user terminal 400a downloads an AI model from the cloud server 100. Figure 2 is a diagram showing the operation when the area computer 200 downloads an AI model from the base station computer 300a. Figure 3 is a diagram showing the operation when a user terminal 400b downloads a desired AI model (layer 1) from the area computer 200. Figure 4 is a diagram showing that one AI model has been divided into multiple layers. Figure 5 is a block diagram showing the functional configuration of the cloud server 100. Figure 6(a) is a diagram showing a specific example of the download count DB, and Figure 6(b) is a diagram showing a specific example of the hit list. Figure 7 is a diagram showing a specific example of the division location DB. Figure 8 is a block diagram showing the functional configuration of the area computer 200. Figure 9 is a flowchart showing the process of checking the division locations of the AI model on the cloud server 100. Figure 10 is a flowchart showing the detailed operation of process S103. Figure 11 is a flowchart showing the process by which the cloud server 100 places an AI model on the area computer 200. Figure 12 is a flowchart showing the detailed processing of process S202. Figure 13 is a flowchart showing the operation of the user terminal 400. Figure 14 is a diagram showing the compression efficiency of the data output from the AI model relative to the data input to the AI model. Figure 15 is a diagram showing the layers downloaded for each AI model. This figure shows an example of the hardware configuration of a cloud server 100, an area computer 200, and a base station computer according to one embodiment of the present disclosure.
[0009] Embodiments of this disclosure will be described with reference to the attached drawings. Where possible, the same parts will be denoted by the same reference numerals, and redundant descriptions will be omitted.
[0010] Figures 1 to 3 show the system configuration including the cloud server 100 of this disclosure. As shown in the figures, the system consists of the cloud server 100, area computer 200, base station computer 300a, base station computer 300b, user terminal 400a, and user terminal 400b. The base station computer 300 is a so-called MEC (Multi-access Edge Computing) computer. The base station computer 300 is a computer located near or inside a base station and can store data transmitted and received via the base station.
[0011] The area computer 200 is positioned in a location where communication with base stations such as user terminals 400a may be congested. If a base station is in a congested area, the area computer 200 will be positioned near base station computer 300a or base station computer 300b. The area computer 200 is connected via a network to download AI models from base station computer 300a, etc. In addition, the area computer 200 has wireless communication capabilities such as Wi-Fi to download AI models from user terminals 400 upon request.
[0012] The cloud server 100 stores at least one AI model. As shown in Figure 1, initially or for the first time, the user terminal 400a accesses the cloud server 100 via the base station computer 300a to download the desired AI model. At this time, the base station computer 300a caches (stores) the downloaded AI model. The cache in the base station computer 300a is changed at regular intervals. Furthermore, RAN (Radio Access Network) communication processing may be handled by a GPU installed in the base station computer 300.
[0013] Furthermore, the cloud server 100 is responsible for receiving requests to download the AI model, processing the download, and calculating the AI model that has been divided for split inference. This cloud server 100 stores the AI model that has been divided into multiple layers. Figure 4 shows that one AI model has been divided into multiple layers. As shown in the figure, the AI model is divided into layers such as the input layer, hidden layer 1, hidden layer 2, and output layer. One or more of these layers are stored in the area computer 200, and the other layers are stored separately in the cloud server 100. In Figures 1 to 3, only layers 1 and 2 are shown for convenience, but as shown in Figure 4, the AI model consists of layers 1 (input layer) to 4 (output layer).
[0014] Figure 2 shows the operation of the area computer 200 when downloading an AI model from a base station computer 300a. The area computer 200 may be placed, for example, inside a moving train, or in areas such as stations and airports. Even if the area computer 200 is placed inside a train, it can determine which base station computer 300a is nearby because it knows where the train will pass. Therefore, the area computer 200 can download the AI model from a nearby base station computer 300a. As shown in the figure, based on instructions from the cloud server 100, the area computer 200 downloads the layer 1 portion of the AI model from the nearby base station computer 300a. In the figure, the AI model is assumed to be divided into layer 1 and layer 2. The cloud server 100 may also instruct the area computer 200 on which base station computer 300a to download from. Furthermore, as mentioned above, the area computer 200 stores information (identification information, etc.) of nearby base station computers 300a, and may download from that base station computer when a download instruction is received.
[0015] Then, as shown in Figure 3, user terminal 400b, which is different from user terminal 400a, accesses the area computer 200 and downloads the desired AI model (layer 1). User terminal 400 also performs partial calculations of the AI model that has been divided for split inference.
[0016] In other words, the area computer 200 receives an inquiry from the user terminal 400b regarding a desired AI model and provides that AI model to the user terminal 400b. Here, a segmented AI model (layer 1) is provided. The area computer 200 may also refresh (update) the AI model to be downloaded at regular intervals.
[0017] As shown in Figure 2, the base station computer 300a caches (stores) the AI model. The cloud server 100 then sends a download instruction for the AI model to the area computer 200. Upon receiving the instruction, the area computer 200 searches for and accesses a nearby base station computer 300a. The area computer 200 then downloads layer 1 of the AI model specified in the download instruction from the base station computer 300a.
[0018] As shown in Figure 3, the user terminal 400b accesses the nearby area computer 200 using a wireless LAN and downloads the AI model (layer 1) stored in the area computer 200. This allows the user terminal 400 to perform split inference using layer 1 of the AI model. That is, the user terminal 400 performs inference using layer 1, and the output from layer 1 is sent to the cloud server 100, where inference processing is performed by layer 2, which is responsible for subsequent processing. The processed inference results from the cloud server 100 are then sent to the user terminal 400b.
[0019] In this disclosure, the cloud server 100 may generate a list of popular AI models based on the number of downloads of each AI model. The cloud server 100 may also calculate branching guidelines (layers to divide) based on the compression efficiency of each AI model. Furthermore, the cloud server 100 may decide which AI models to install on each area computer 200 and to what extent.
[0020] The base station computer 300b may take over the calculation of the AI model that has been divided for split inference, which is one of the functions of the base station computer 300a. This base station computer 300b is presumed to store the AI model for split inference, be equipped with a GPU for this purpose, and have surplus resources.
[0021] Next, the functional configuration of the cloud server 100 will be described. Figure 5 is a block diagram showing the functional configuration of the cloud server 100. As shown in the figure, the cloud server 100 is composed of an AI model storage unit 101, a list generation unit 102, a check unit 103, a division location storage unit 104, a model placement processing unit 105, and a download count acquisition unit 106.
[0022] The AI model storage unit 101 stores multiple types of AI models, such as healthcare AI applications and asset management AI applications. These AI models are divided into layers for split inference. The AI model storage unit 101 also stores a download count database that stores the number of downloads for each AI model. Figure 6(a) shows a specific example of this download count database. As shown in the figure, the number of downloads is stored in association with each AI model identifier. The number of downloads is counted each time an AI model is downloaded by the model placement processing unit 105. The download count acquisition unit 106 may acquire the number of downloads of AI models downloaded from the base station computer 300a and store it in the download count database.
[0023] The list generation unit 102 refers to the AI model storage unit 101 and generates a hit list of AI models with a high number of downloads from among the multiple types of AI models. The hit list is a list showing the popularity ranking of the AI models, arranged in descending order of download count. This list generation unit 102 generates a list based on the number of downloads for each area computer 200. Figure 6(b) shows a specific example of a hit list. As shown in the figure, the identifiers of the AI models are arranged according to the ranking based on the number of downloads. Note that the hit list does not need to be arranged in order; any list that shows the popularity ranking is sufficient. In this disclosure, a popularity ranking for the area is created by checking the AI models downloaded from the base station computer 300a.
[0024] The check unit 103 is responsible for checking the compression efficiency between each layer of each AI model. Specifically, the check unit 103 performs a predetermined representative query on each AI model, calculates the output compression efficiency (or data volume) for each layer, and identifies layers that are equal to or greater than the threshold T. In other words, the check unit 103 identifies layers where the amount of output data is small compared to the amount of input data. As will be described later, the output data from each layer is compressed using a data distribution method. This is because the data content is biased depending on the output from each layer, making data distribution compression efficient. This compression and decompression process is performed by the check unit 103, and the compression and decompression of the output data from each layer are repeated.
[0025] The division location storage unit 104 is the part that stores the division location DB which describes the layers confirmed by the check unit 103. Figure 7 shows a specific example of the division location DB. As shown in the figure, the division location DB describes the division layer number (identifier) associated with each identifier representing the AI model.
[0026] The model placement processing unit 105 is responsible for placing (storing) AI models for each area computer 200. Specifically, the model placement processing unit 105 instructs the area computer 200 to download AI models from the base station computer 300a and to specify the layers to be downloaded (including their ranking). Based on these instructions, the area computer 200 downloads the specified AI model layers from the base station computer 300a in order of popularity.
[0027] The model placement processing unit 105 determines which AI model to download, which layers of that AI model to download, by referring to the division location DB and the download count DB.
[0028] The download count acquisition unit 106 is responsible for acquiring the download count for each AI model downloaded from the user terminal 400 in response to a request, from each base station computer 300a. Since the AI models downloaded to the user terminal 400 are downloaded via each base station computer 300a, each base station computer 300a recognizes the downloaded AI models and stores their download counts. The download count acquisition unit 106 acquires these download counts and stores them in the download count DB of the AI model storage unit 101.
[0029] Figure 8 is a block diagram showing the functional configuration of the area computer 200. As shown in the figure, the area computer 200 is configured to include an AI model storage unit 201 and a communication unit 202.
[0030] The AI model storage unit 201 is a part that stores one or more AI models. The AI models stored here are divided parts of the AI model (parts separated into layers).
[0031] The communication unit 202 is responsible for downloading AI models (layer by layer) stored (cached) in the base station computer 300a in response to instructions from the cloud server 100 (the AI model to be downloaded, its layers, and their popularity ranking). The communication unit 202 retrieves information on the access destinations of nearby base station computers 300a stored in a storage unit (not shown) and performs the download based on that information. The communication unit 202 also transmits (allows download) AI models (layer by layer) stored in the AI model storage unit 201 to the user terminal 400 in response to a request from the user terminal 400.
[0032] The following describes the AI model deployment process using the cloud server 100 and area computer 200 configured in this manner.
[0033] Figure 9 is a flowchart showing the process for verifying the division points of the AI model on the cloud server 100. The download count acquisition unit 106 acquires information from multiple nearby base station computers 300 about which AI models have been downloaded and how many times (i.e., the number of downloads for each AI model) (S101). The download count acquisition unit 106 stores a download count DB in the AI model storage unit 101, which contains the number of downloads.
[0034] The list generation unit 102 generates a hit list (popular list) of AI models that are expected to have a certain hit rate (S102). That is, the list generation unit 102 generates a list of AI models sorted in order of the number of downloads.
[0035] The checking unit 103 checks for each AI model the division point (layer) where the compression efficiency between the output of the previous layer and the current layer is equal to or greater than the threshold T (S103). The checking unit 103 checks the layer that will be divided and stores the identifier of that layer in the division point DB.
[0036] Figure 10 is a flowchart showing the detailed operation of the above process S103. The check unit 103 makes a typical query for the AI model to be checked for division (S103a). For example, if the AI model is a healthcare application, a typical query might be, "I have been experiencing persistent headaches and dizziness. Could this be a sign of something?" If the AI model is an asset management application, a typical query might be, "What investment areas are likely to be affected by recent interest rate increases?" These typical queries are based on information predetermined by the creator of the AI model and stored in the memory (not shown) of the cloud server 100.
[0037] Then, the check unit 103 sets i = 1 (S103b) and determines whether the compressed data size of the output of layer i (which constitutes the AI model) minus the compressed data size of the output of layer i+1 (which constitutes the AI model) > threshold T (S103c). If the compressed data size of the output of layer i (which constitutes the AI model) minus the compressed data size of the output of layer i+1 (which constitutes the AI model) > threshold T (S103c: YES), the i = i+1 process is performed without any further processing (S103d). This indicates that the compression efficiency does not exceed the threshold.
[0038] Furthermore, in processing S103c, if the compressed data size of the output of layer i minus the compressed data size of the output of layer i+1 is not greater than the threshold T (S103c: NO), the branching guideline based on compression efficiency is set to i (S103e). In other words, layer i is set as the division point. The check unit 103 stores layer i in the AI model storage unit 101 (division point DB) as the division point of its AI model (S103f).
[0039] In this way, in the cloud server 100, the splitting points are checked and stored for each AI model. When the cloud server 100 provides the split AI model (each layer) for split inference, it gives an instruction to store the layer based on this splitting point in the area computer 200 or the like.
[0040] Note that in processes S103c and S103e, the determination of the compression efficiency is only made once, but it is not limited to this. In process S103c, even if the answer is NO, it may be repeated multiple times by changing the layer, and the layer with the best compression efficiency may be selected.
[0041] Process S103c will be described in more detail. FIG. 14 is a diagram showing the compression efficiency of the data output from the AI model with respect to the data input to the AI model. In the present disclosure, for the sake of convenience, it is shown as a two-dimensional vector, but in reality, it is shown in hundreds of dimensions.
[0042] As shown in FIG. 14(a), “Where is the capital of Japan?” is input into the AI model.
[0043] Then, in the AI model, it is decomposed into each token of “Where”, “is”, “the”, “capital”, “of”, “Japan”, “?” (FIG. 14(a)). That is, the input sentence is tokenized at the initial stage of natural language processing (NLP). Tokenization is a process of decomposing a sentence into the smallest units such as words and symbols. In the case of the present disclosure, it is decomposed into seven tokens of “Where”, “is”, “the”, “capital”, “of”, “Japan”, “?”.
[0044] Then, “word embedding” is performed on each tokenized word. Word embedding means mapping each word into a high-dimensional vector space. Thereby, the semantic similarity and relationship of words can be numerically expressed.
[0045] In this disclosure, for example, the word "Japan" is represented by a vector such as "0.47, -0.12, 0.26, ...", and is represented by a 768-dimensional vector. This vector contains several implicit representations. For example, although there is no explicit label, "Japan" implicitly represents geographical information such as "island nation," information about its political system such as "democratic country," and economic information such as "developed country." Therefore, the positions (vectors) of each token are generally far apart.
[0046] Then, in the Transformer, the vectors of each token are recalculated, taking into account the relationships between each token. The position of "Japan" is transformed to a position where its relationship with "capital" is emphasized. This indicates a shift from representing "Japan" in all directions semantically to one related to capital (Figure 14(b)). As a result, the distance between Japan and capital decreases, and the dispersion of the vector can be suppressed. In this disclosure, this vector is compressed in a data distribution type, and a layer is sought that minimizes its value.
[0047] In the above, the vector data are the data output from each Transformer Block (layer). As shown in Figure 14(c), the data output from Transformer Block (Layer 1) is input to the next layer, Transformer Block (Layer 2). In this disclosure, the layer at which to split can be determined by comparing the compression efficiency of the data output from each Transformer Block. In an AI model, although the output dimensions of the intermediate layers are the same, the internal representation of the data should be different, so the compression effect can be verified by compressing the data distribution and comparing the results.
[0048] Next, the process in which the cloud server 100 arranges an AI model (a divided AI model divided into layers) in the area computer 200 will be described. FIG. 11 is a flowchart showing the process. The model arrangement processing unit 105 sets j = 1 (S201) and arranges layer j (initially layer 1) of each AI model in the area computer 200 (S202). That is, the model arrangement processing unit 105 issues a download instruction to the area computer 200 by specifying layer j of the AI model, and the area computer 200 performs a download process of the AI model of layer j from the cloud server 100.
[0049] The model arrangement processing unit 105 accesses the area computer 200 and performs a process of checking the storage capacity of the area computer 200, and determines that the storage capacity of the area computer 200 is equal to or greater than the threshold S and that there is a layer in which the division location is layer j or higher in any of the AI models (S203). Then, when the division location is layer j or higher, the model arrangement processing unit 105 sets j = j + 1 to determine the next layer (S204).
[0050] For example, when the layer of the division location is 2 (layer 2 is the division location), the model arrangement processing unit 105 determines that the division location (layer 2) is layer j or higher (initially layer 1), sets j = i + 1 (S204), and performs an arrangement process for the next layer (S202).
[0051] If the model arrangement processing unit 105 does not determine in process S203 that the storage capacity of the area computer 200 is equal to or greater than the threshold S and that there is a layer in which the division location is layer j or higher in any of the AI models (S203: NO), the arrangement process is terminated (S205).
[0052] Next, the details of process S202 will be explained. Figure 12 is a flowchart showing the detailed processing of process S202. The model placement processing unit 105 sets k = 1 (S202a). The model placement processing unit 105 determines whether the k-ranked layer j is at or above the layer i of the division point, and whether layer j is within the storage capacity of the area computer 200 (below the threshold) (S202b).
[0053] Then, when the model placement processing unit 105 determines that the k-rank most popular layer j is at or above layer i of the division point and that layer j is within the storage capacity of the area computer (S202b: YES), it causes the area computer 200 to download the k-rank most popular layer j from the cache of the nearby base station computer 300a (S202c). In other words, the model placement processing unit 105 sends an instruction to the area computer 200 to download layer j of the AI model from the base station computer 300a.
[0054] Then, the model placement processing unit 105 sets k = k + 1 (S202d), and then proceeds to (Figure 11: S203). Here, the area computer 200 downloads the AI model from a nearby base station computer 300a. Whether or not a base station is nearby is determined by the positional relationship between the area computer 200 and the base station computer 300a, and when the area computer 200 is to download the model, the corresponding base station computer 300a may be predetermined. That is, when the area computer 200a receives an instruction to download the AI model from the cloud server 100, it may be configured to access a predetermined base station computer 300 and perform the download process.
[0055] In this way, the AI model stored on the cloud server 100 is stored on the area computer 200. At that time, the area computer 200 stores the AI model divided into layers (for convenience, referred to as the divided AI model). This divided AI model is divided in order to perform split inference, and the data size output from this divided AI model is adjusted to be small.
[0056] Figure 15 shows the layers downloaded for each AI model. As shown in the figure, the AI models are downloaded layer by layer in the order of the hit list (Figure 15(a)). As shown in Figure 15(b), layer 2 is downloaded for AI model A, but for AI model B, only up to layer 1 is downloaded, and layer 2 is not downloaded. This is because, in the above process, layer 2 of AI model B was determined to have poor compression efficiency.
[0057] Similarly, in Figure 15(c), layer 2 of AI model A is not downloaded, but layer 3 of AI model C is downloaded. In this way, the AI models are downloaded layer by layer in order of popularity on the area computer 200.
[0058] Next, we will explain the process by which the user terminal 400 downloads the AI model from the area computer 200. Figure 13 is a flowchart showing the operation of the user terminal 400.
[0059] The user terminal 400 accepts a request from the user to download an AI model (S301). The user terminal 400 searches to see if the area computer 200 is located near the user terminal 400 (S302). The search method is a known technology and includes, for example, mDNS, ARP scan, and MEC service discovery API.
[0060] When the user terminal 400 searches for the area computer 200 (S303: YES), it connects to the area computer 200 (S304). The user terminal 400 then requests the area computer 200 to download an AI model (S305). This download request includes an identifier to specify the AI model. The area computer 200 determines whether or not the AI model exists by determining whether or not it has the AI model corresponding to that identifier stored in its memory.
[0061] When the user terminal 400 receives notification from the area computer 200 that a specified AI model exists (S306: YES), it selects that AI model and downloads it from the area computer 200 (S307).
[0062] In process S303, if there is no area computer in the vicinity of the user terminal 400 (S303: NO), or if the user terminal 400 receives notification that the specified AI model is not available at the area computer (S306: NO), the user terminal 400 accesses the base station computer 300a or the cloud server 100 to download the specified AI model (S308).
[0063] As mentioned above, the area computer 200 only stores some of the layers of the AI model. The user terminal 400 downloads the AI model for those specific layers.
[0064] When the user terminal 400 operates the AI model consisting of this partial layer, it performs processing by split inference using the AI model stored in the cloud server 100, base station computer 300a, or base station computer 300b. The AI model has prior information on which cloud server 100 or base station computer 300 to use.
[0065] Furthermore, during split inference, the data output units of the user terminal 400 and the base station computer 300a or cloud server 100 each compress and transmit data, and the compressed data is received and decompressed. That is, the user terminal 400 compresses and transmits the output data of the layer, the base station computer 300a or cloud server 100 receives and decompresses the compressed data and inputs it into the layer to obtain the final output data. The base station computer 300a or cloud server 100 transmits this output data to the user terminal 400, and the user can confirm the output data by displaying it on the display unit or playing it back through the speaker, etc.
[0066] Next, the effects of the cloud server 100 configured in this way will be explained. The cloud server 100 of this disclosure includes an AI model storage unit 101 that stores an AI model composed of multiple layers. The model placement processing unit 105 selects one or more layers from among the multiple layers constituting the AI model, whose output data from each layer satisfies certain conditions, as targets to be downloaded from an external device (for example, an area computer 200).
[0067] For example, the model placement processing unit 105 identifies a division point for each AI model where the compression efficiency between the output of the previous layer and the current layer is equal to or greater than a threshold T, and stores the layer indicated by that point in the division point DB. The model placement processing unit 105 then sends an instruction to the area computer 200 to download this layer.
[0068] This configuration allows for determining how much of each AI model should be downloaded to the area computer. For example, split inference of AI models generally offers a high level of security. On the other hand, if processing is done solely in the cloud without using split inference, the amount of data transferred is small, especially at the character level, but the security level decreases. That is, at the character level, the size is the number of bytes per character multiplied by the number of characters, which is considerably less than when each token has several hundred dimensions, but the security level decreases. In this disclosure, as described above, by performing split inference of the AI model divided at the division points according to the conditions, it is possible to reduce the amount of communication between devices having the divided AI model while maintaining the security level.
[0069] For example, even if a user terminal 400 wants to download and use an AI model in a crowded place such as a train, it can download the AI model divided into layers. In other words, by downloading the AI model from the area computer 200 without going through the base station, the user terminal 400 can download the AI model with less data usage without incurring the communication charges that would normally be incurred between the terminal and the base station. Furthermore, even when performing split inference, the amount of data output from the divided AI model is small, enabling efficient split inference.
[0070] The condition for determining the above division points is that the size of the output data is smaller than the input data (output data from the previous layer). In other words, this condition is determined based on the compression effect of intermediate data by data distribution-based compression on the data output from each layer.
[0071] Furthermore, in the cloud server 100 of this disclosure, the AI model storage unit 101 stores multiple different AI models. The model placement processing unit 105 then selects the AI model to be downloaded based on the popularity ranking (ranking) of each of the different AI models. Based on this popularity ranking, the model placement processing unit 105 selects one or more layers constituting the AI model to be downloaded. This popularity ranking is based on the number of downloads of the AI models from the base station computers 300a located in each area. The number of downloads is a barometer of popularity, and it is effective to prioritize storing popular AI models in the area computer 200.
[0072] Furthermore, the model placement processing unit 105 causes the area computer 200, an external device, to download one layer of each of the multiple AI models at a predetermined hierarchical position. That is, the model placement processing unit 105 sends an instruction to the area computer 200 to download from the base station computer 300a up to the layer specified for each of the designated AI models. Once the area computer 200 has finished downloading one layer, it sequentially downloads the next layer, and repeats this process to sequentially download all the specified layers of the designated AI models.
[0073] This allows the specified AI model to be downloaded evenly to the area computer 200.
[0074] The model placement processing unit 105 may determine whether or not to download the next layer based on the status of the area computer 200 (for example, the available storage space). In this disclosure, the layers are downloaded sequentially starting from the first layer. That is, the first layer mentioned above is the first layer.
[0075] The system disclosed herein comprises a cloud server 100, an area computer 200 which is an external device and is deployed in a corresponding area, and a base station computer 300a which is installed in a base station used by the cloud server 100 for communication. In addition, multiple types of AI models are available (healthcare AI model, asset management AI model, etc.), each of which is composed of multiple layers.
[0076] The cloud server 100 then distributes the AI model to the user terminal 400 via the base station computer 300a.
[0077] Based on instructions from the cloud server 100, the area computer 200 downloads and stores the specified layer of the specified AI model from the base station computer 300a. Then, when another user terminal 400b requests the distribution of an AI model, the area computer 200 distributes the AI model stored in the area computer 200 to the other user terminal 400b.
[0078] The apparatus and method of this disclosure have the following configurations.
[0079] [1] A device comprising: a storage unit that stores an AI model composed of multiple layers; and a processing unit that selects one or more layers from among the multiple layers constituting the AI model, whose output data from each layer satisfies certain conditions, to be downloaded from an external device.
[0080] [2] The apparatus according to [1], wherein the condition is that the size of the output data from the target layer is smaller than the output data from the previous layer.
[0081] [3] The apparatus described in [2], wherein the conditions are determined based on the compression effect of intermediate data by data distribution-aware compression.
[0082] [4] The apparatus according to [1], wherein the storage unit stores a plurality of different AI models, the processing unit selects the AI model to be downloaded based on the order of the respective different AI models, and selects one or more layers constituting the AI model to be downloaded based on the order.
[0083] [5] The apparatus according to [4], wherein the processing unit causes the external device to download each of the multiple AI models at a predetermined hierarchical position, and when the download of each of the layers is completed, it sequentially causes the next layer to be downloaded to the external device.
[0084] [6] The apparatus according to [5], wherein the processing unit determines whether or not to download the next layer based on the status of the external device.
[0085] [7] The apparatus according to [5] or [6], wherein the first layer is the leading layer.
[0086] [8] The apparatus according to any one of [4] to [7], wherein the order is based on the download frequency of AI models downloaded from base station computers located in each area.
[0087] [9] A system comprising: the device described in [1]; an area computer which is an external device and is arranged in accordance with an area; and a base station computer provided at a base station used by the device for communication, wherein multiple types of AI models are provided, each consisting of multiple layers; the device distributes the AI models to user terminals via the base station computer; the base station computer stores the AI models; the area computer downloads and stores the specified layers of the specified AI models from the base station computer based on instructions from the device; and when another user terminal requests the distribution of an AI model, the area computer distributes the AI model stored in the area computer to the other user terminal.
[0088]
[10] A method for an apparatus comprising: an acquisition step of acquiring an AI model composed of multiple layers; and a processing step of selecting one or more layers from among the multiple layers constituting the AI model, the layers whose output data satisfy certain conditions, as targets to be downloaded from an external device.
[0089] The block diagram used in the description of the above embodiment shows functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may be realized by combining the one or more devices with software.
[0090] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.
[0091] For example, the cloud server 100, area computer 200, and base station computer in one embodiment of the present disclosure may function as computers that perform processing for the AI model processing method of the present disclosure. Figure 16 is a diagram showing an example of the hardware configuration of the cloud server 100, area computer 200, and base station computer according to one embodiment of the present disclosure. The cloud server 100, area computer 200, and base station computer described above may be physically configured as computer devices including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.
[0092] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the cloud server 100, area computer 200, and base station computer may include one or more of the devices shown in the diagram, or it may be configured to omit some of the devices.
[0093] Each function in the cloud server 100, area computer 200, and base station computer is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.
[0094] The processor 1001 controls the entire computer, for example, by running the operating system. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. For example, the list generation unit 102 and the check unit 103 described above may be implemented by the processor 1001.
[0095] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, the list generation unit 102 and the check unit 103 may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.
[0096] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for implementing the AI model processing method according to one embodiment of the present disclosure.
[0097] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.
[0098] The communication device 1004 is hardware (transmitting / receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 1004 may be configured to include high-frequency switches, duplexers, filters, frequency synthesizers, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the model placement processing unit 105 and the download count acquisition unit 106 described above may be implemented by the communication device 1004. The communication device 1004 may be implemented with physically or logically separated transmitting and receiving units.
[0099] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).
[0100] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.
[0101] Furthermore, the cloud server 100, area computer 200, and base station computer 300 may be configured to include hardware such as microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.
[0102] Information notification is not limited to the embodiments described herein and may be carried out by other means. For example, information notification may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.
[0103] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements using exemplary order and are not limited to the specific order presented.
[0104] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.
[0105] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).
[0106] Each aspect / embodiment described in this disclosure may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).
[0107] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure may be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Accordingly, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.
[0108] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.
[0109] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technologies (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.
[0110] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.
[0111] In addition, terms used in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of the channel and symbol may be a signal (signaling). Also, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, cell, frequency carrier, etc.
[0112] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values from a given value, or other corresponding information. For example, wireless resources may be indicated by an index.
[0113] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various channels and information elements are not restrictive in any way.
[0114] In this disclosure, terms such as "Mobile Station (MS)," "user terminal," "User Equipment (UE)," and "terminal" may be used interchangeably.
[0115] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate term.
[0116] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."
[0117] The terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables, and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain, and optical (both visible and invisible) domain.
[0118] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."
[0119] Any reference to elements using designations such as “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.
[0120] Where the terms “include,” “including,” and their variations are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to be exclusive OR.
[0121] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.
[0122] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."
[0123] 100...Cloud server, 200...Area computer, 300a...Base station computer, 300b...Base station computer, 400a...User terminal, 400b...User terminal, 101...AI model storage unit, 102...List generation unit, 103...Check unit, 104...Division location storage unit, 105...Model placement processing unit, 106...Download count acquisition unit, 201...AI model storage unit, 202...Communication unit.
Claims
1. A device comprising: a storage unit that stores an AI model composed of multiple layers; and a processing unit that selects one or more layers from among the multiple layers constituting the AI model, whose output data from each layer satisfies certain conditions, to be downloaded from an external device.
2. The apparatus according to claim 1, wherein the condition is that the size of the output data from the target layer is smaller than the output data from the previous layer.
3. The apparatus according to claim 2, wherein the conditions are determined based on the compression effect of intermediate data by data distribution-aware compression.
4. The apparatus according to claim 1, wherein the storage unit stores a plurality of different AI models, the processing unit selects the AI model to be downloaded based on the order of the respective different AI models, and selects one or more layers constituting the AI model to be downloaded based on the order.
5. The apparatus according to claim 4, wherein the processing unit causes each of the plurality of AI models to download one layer at a predetermined hierarchical position to the external device, and when the download of each of the layers is completed, it sequentially causes each of the next layers to download to the external device.
6. The apparatus according to claim 5, wherein the processing unit determines whether or not to download the next layer based on the status of the external device.
7. The apparatus according to claim 5, wherein the first layer is the leading layer.
8. The apparatus according to claim 4, wherein the order is based on the download frequency of AI models downloaded from base station computers located in each area.
9. A system comprising: the device described in claim 1; an area computer which is an external device and is arranged in accordance with an area; and a base station computer provided at a base station used by the device for communication, wherein the AI model is of a plurality of types, each consisting of a plurality of layers; the device distributes the AI model to a user terminal via the base station computer; the base station computer stores the AI model; the area computer downloads and stores a specified layer of the specified AI model from the base station computer based on instructions from the device; and when another user terminal requests the distribution of an AI model, the area computer distributes the AI model stored in the area computer to the other user terminal.
10. A method for an apparatus comprising: an acquisition step of acquiring an AI model composed of multiple layers; and a processing step of selecting one or more layers from among the multiple layers constituting the AI model, the layers whose output data satisfies certain conditions, as targets to be downloaded from an external device.