Transmitting entity, receiving entity, data transmitting method and data processing method for AI / ML application

Abstract

[Problem] To provide a transmitting entity and a receiving entity for an AI / ML application that can update inference results step by step until a desired result is obtained. [Solution] This transmission entity is provided with: a generation unit that generates, from original input data, a plurality of items of input data in which the amount of data is reduced; a data transmission unit that sequentially transmits the plurality of items of input data; and a request receiving unit that receives feedback information based on whether or not additional input data is required. The receiving entity includes: a data receiving unit that sequentially receives a plurality of items of input data; an inference unit that sequentially inputs the plurality of items of input data to a multi-level input AI / ML model and sequentially acquires a plurality of inference values; an output unit that sequentially updates the inference results on the basis of the plurality of inference values and outputs the updated inference results; a determination unit that determines whether additional input data is required; and a request transmitting unit that transmits the feedback information.

Description

Technical Field

[0001] The present disclosure relates to a sending entity, a receiving entity, a data sending method, and a data processing method for artificial intelligence / machine learning (AI / ML) applications. Background Art

[0002] AI / ML applications that are performed using cellular networks such as 5G, beyond 5G, and 6G are being discussed. For example, in such AI / ML applications, it is conceivable that input data acquired by a terminal device with low computing power is sent to a cloud server through a cellular network to utilize the high computing power of the cloud server to perform inference using an AI / ML model.

[0003] As described above, when the input data acquired at the transmitting side is transmitted to the AI / ML model at the receiving side through the cellular network, and when the amount of input data is large, it may take a long time to complete the transmission. In this case, the AI / ML model at the receiving side is not allowed to start implementing inference before the transmission of the large amount of input data is completed.

[0004] Non-patent document 1 discloses the following technology: a large amount of input data obtained on the sending side is divided into smaller input data items, each smaller input data item is sent to the AI / ML model on the receiving side via a cellular network, and the inference processing implemented using the AI / ML model is parallelized to shorten the time taken to obtain the inference result.

[0005] Reference List

[0006] Non-patent literature

[0007] Non-patent document 1: "Edge Assisted Real-time Object Detection for Mobile Augmented Reality", The 25th Annual International Conference on Mobile Computing and Networking, August 2019, Article No.: 25, pages 1-16. Summary of the invention

[0008] Technical issues

[0009] Until the transmission of all the smaller input data items obtained by division is completed, the technology disclosed in non-patent document 1 is allowed to obtain an inference result. However, in real-world applications, it may be more advantageous to obtain a rough inference result before obtaining a final inference result. For example, when classifying objects appearing in an image on an automated guided vehicle (AGV), it is more advantageous to obtain a rough inference result before obtaining a precise inference result. In addition, it is much more advantageous if the inference result is allowed to be updated in stages until the desired accuracy is obtained.

[0010] The present disclosure solves the aforementioned problems, and an object of the present disclosure is to provide a sending entity and a receiving entity for an AI / ML application that allows updating an inference result in stages until a desired result is obtained.

[0011] Solutions to the Problem

[0012] In order to solve the problems described above, a sending entity for AI / ML applications according to the present disclosure includes: an acquisition part that acquires original input data; a generator that generates multiple input data with reduced data amount from the original input data; a data transmitter that sequentially sends each input data among the multiple input data; and a request receiver that receives feedback information based on whether additional input data is needed among the multiple input data.

[0013] In addition, a receiving entity for an AI / ML application according to the present disclosure includes: a data receiver that sequentially receives individual input data among multiple input data; an inference part that sequentially inputs the individual input data among the multiple input data into a multi-input AI / ML model to sequentially obtain individual inference values ​​among multiple inference values; an output part that sequentially updates the inference results based on the multiple inference values ​​to output the updated inference results; a determination part that determines whether additional input data is required among the multiple input data; and a request transmitter that sends feedback information based on the determination of whether additional input data is required, the determination being implemented by the determination part.

[0014] In addition, according to the contents of the present disclosure, a data sending method for AI / ML applications includes: obtaining original input data; generating multiple input data with reduced data volume from the original input data; sequentially sending each input data among the multiple input data; and receiving feedback information based on whether additional input data is needed among the multiple input data.

[0015] In addition, according to the contents of the present disclosure, a data processing method for AI / ML applications includes: sequentially receiving individual input data among multiple input data; sequentially inputting the individual input data among the multiple input data into a multi-input AI / ML model to sequentially obtain individual inference values ​​among multiple inference values; sequentially updating the inference results based on the multiple inference values ​​to output updated inference results; determining whether additional input data is required among the multiple input data; and sending feedback information based on the determination of whether additional input data is required. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Two examples of using AI / ML for cellular networks are shown.

[0017] Figure 2 Three execution modes of AI / ML applications using cellular networks are shown.

[0018] Figure 3 Showing examples of separation points in a neural network model.

[0019] Figure 4 An example of implementing distributed learning for AI / ML applications executed using a cellular network is shown.

[0020] Figure 5 Various ways to implement distributed learning in AI / ML applications executed using cellular networks are shown.

[0021] Figure 6 An example of an AI / ML application including a multi-input AI / ML model is shown.

[0022] Figure 7 It is a diagram used to describe the basic concept of the present disclosure.

[0023] Figure 8 The configuration of a sending entity and a receiving entity according to the present disclosure is shown.

[0024] Fig. 9 A configuration example of a 5G system is shown.

[0025] Fig.10 A configuration example of a multi-input AI / ML model is shown.

[0026] Fig.11 An example of inputting two items of still image data of different resolutions into a multi-input AI / ML model is shown.

[0027] Fig.12 A method for generating input data according to the resolution of a still image is shown.

[0028] Fig.13An example of inputting two items of still image data of different regions into a multi-input AI / ML model is shown.

[0029] Fig.14 A method for generating input data from a region in a still image is shown.

[0030] Fig.15 A method for generating input data according to resolution and area in a still image is shown.

[0031] Fig.16 A method for generating input data according to the frame rate of moving image data is shown.

[0032] Fig.17 A first example of a method for generating input data according to compression encoding performed on an image is shown.

[0033] Fig.18 A second example of a method for generating input data according to compression encoding performed on an image is shown.

[0034] Fig.19 A method for generating input data according to an error correction code is shown.

[0035] Fig. 20 An example is shown in which two items of input data are used for hard combination.

[0036] Fig.21 It is a flowchart for describing the details of the operation performed when the hard combination is performed in the case of using the robot described above.

[0037] Fig. 22 An example is shown in which two items of input data are used for soft combination.

[0038] Fig.23 It is a flowchart for describing the details of the operation performed when the soft combination is performed in the case of using the robot described above.

[0039] Fig.24A A sequence is shown when a robot in a factory performs autonomous driving while performing image recognition.

[0040] Fig. 24B A sequence is shown when a robot in a factory performs autonomous driving while performing image recognition. DETAILED DESCRIPTION

[0041] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the accompanying drawings, similar or corresponding structural elements will be marked with the same reference numerals, and their detailed description will be omitted where appropriate.

[0042] The following is the configuration of the present disclosure.

[0043] 1. Background

[0044] 1-1. AI / ML applications executed using cellular networks

[0045] 1-2. Three execution methods of AI / ML applications

[0046] 1-2-1. Cloud learning

[0047] 1-2-2. Learning on the device

[0048] 1-2-3. Distributed Learning

[0049] 1-3. Multi-input AI / ML models

[0050] 2. Basic concepts of this disclosure

[0051] 2-1. Sending entity and receiving entity

[0052] 2-2. Configuration of the sending entity

[0053] 2-3. Configuration of the receiving entity

[0054] 3. Multi-input AI / ML models and input data according to the present disclosure

[0055] 3-1. Multi-input AI / ML models

[0056] 3-2. Method for generating input data

[0057] 3-2-1. Generation of input data based on resolution of still images

[0058] 3-2-2. Generation of input data based on regions in a still image

[0059] 3-2-3. Generation of input data based on resolution and area in a still image

[0060] 3-2-4. Generation of input data based on the frame rate of moving images

[0061] 3-2-5. Generation of input data based on compression coding of images

[0062] 3-2-6. Generation of input data based on error correction code

[0063] 3-2-7. Generation of input data for separation of AI / ML models

[0064] 4. Details of various movements according to the present disclosure

[0065] 4-1. Timing of switching method for sending input data

[0066] 4-2. Hard combination

[0067] 4-3. Soft combination

[0068] 4-4. Start sending a request for additional input data

[0069] 4-5. Stop sending requests for additional input data

[0070] 4-6. How to provide feedback

[0071] 4-7. Related information that serves as feedback information

[0072] 4-8. Changes in additional input data based on associated information

[0073] 5. Functional expansion of cellular networks according to the present disclosure

[0074] 5-1. Network slicing for 5G systems

[0075] 5-2. QoS control for 5G systems

[0076] 5-3. Multi-QoS flow control for 5G systems

[0077] 6. Examples

[0078] 7. Conclusion

[0079] <1. Background>

[0080] Regarding 3GPP Secure Artificial Intelligence (SAI), research has begun on using AI / ML for cellular networks (FS_AMMT). Figure 1 Two examples of using AI / ML for cellular networks are shown.

[0081] The first use case is an AI / ML application executed using a cellular network. In this case, the AI / ML application must run smoothly on the cellular network. In the future, it is expected that AI / ML applications will be used rapidly and widely. This leads to the need for a cellular network that can be adapted to such applications.

[0082] The second use example is the construction of cellular networks using AI / ML. Replacing part or all of the functionality of the cellular network with AI / ML makes it possible to further optimize communications. In this case, it is expected that not only the replacement of the algorithm is required, but also, for example, the change of parameters for input and output is required.

[0083] (1-1. AI / ML applications executed using cellular networks)

[0084] In this disclosure, AI / ML applications executed using cellular networks are discussed, which is the first use case described above. Future cellular networks can not only simply provide communication functions, but also provide computing power for applications executed using cellular networks. In order to implement inference processing in AI / ML models at high speed, high computing power is required, especially when providing computing power for AI / ML applications.

[0085] Specific examples of AI / ML applications executed using cellular networks may include, for example, applications involving: extended reality (XR), such as virtual reality (VR), augmented reality (AR), and mixed reality (MR); autonomous driving; robotics; unmanned transport vehicles; image recognition; and voice recognition.

[0086] Low latency and high computing power are two necessary conditions for a cellular network that is adaptable to AI / ML applications. These necessary conditions are defined as low latency and high computing power exhibited when executing AI / ML applications. Therefore, for example, as a necessary condition for low latency, low latency is not only exhibited between the terminal device and the base station, but also between the terminal device and the server device that implements the inference processing in the AI / ML model.

[0087] (1-2. Three ways to execute AI / ML)

[0088] Figure 2 Three execution modes of AI / ML applications using cellular networks are shown.

[0089] (1-2-1, Cloud Learning)

[0090] The first execution mode is cloud learning in which all processing of the AI / ML application is implemented on the cloud server side. When cloud learning is applied, all processing of the AI / ML application is implemented on the cloud server side. Therefore, the relatively low computing power of the terminal device is not a problem. However, the input data (unprocessed data) necessary for implementing the processing of the AI / ML application must be sent from the terminal device to the cloud server. This may lead to a lack of bandwidth due to the increase in bandwidth usage and may lead to an increase in latency.

[0091] (1-2-2, Learning on the device)

[0092] The second execution mode is that all processing of the AI / ML application is implemented by the terminal device in-device learning. When learning is applied in-device, it is not necessary to send input data from the terminal device to the cloud server. Therefore, the second execution mode is excellent in terms of latency, but it is not realistic for the terminal device to implement all processing of the AI / ML application alone.

[0093] (1-2-3, Distributed Learning)

[0094] The third execution mode is distributed learning (Distributed Leering) in which the processing of AI / ML applications is implemented in a distributed manner by each node, wherein the node is located between the terminal device and the cloud server. Distributed learning makes it possible to distribute the computing load to each node. In addition, each input data is processed sequentially in the node and sent to the next node as intermediate data. This makes it possible to reduce the amount of data sent compared to when the input data (unprocessed data) is sent to the cloud server. This leads to avoiding the lack of bandwidth and increased latency caused by the increase in bandwidth usage. The third execution mode is excellent in low latency. In addition, depending on the type of AI / ML application, lower latency can be achieved by using edge servers arranged in a cellular network.

[0095] When applying distributed learning, the question is which processing will be performed by which node. Figure 3 An example of a split point in a neural network model as an AI / ML model is shown. As can be seen from the figure, the computational load in the node and the delay in the transmission between nodes are different depending on the point where the split is implemented. Therefore, it is important to set the split point appropriately.

[0096] When distributed learning is applied, the AI / ML model is separated and learning is performed. Here, each AI / ML model obtained by separation performs learning according to how the separation is performed. When learning is completed, the AI / ML model obtained by separation and on which learning is performed is assigned to the corresponding node. At the time of inference, each node performs inference in a distributed manner.

[0097] Figure 4 An example of implementing distributed learning in an AI / ML application executed using a cellular network is shown. Here, Figure 4 The following example is shown: an image is captured by a robot and input to an AI / ML model, and necessary movements are implemented. In this example, after the robot captures the image, part of the inference processing of the neural network model is implemented by the robot. Subsequently, the intermediate data output from the intermediate nodes in the neural network model is sent to the edge / cloud server. Here, the intermediate data has been compressed, resulting in a short transmission delay. The rest of the inference processing of the neural network model is implemented on the edge / cloud server side, and the movement instructions given based on the inference results are sent to the robot.

[0098] Figure 5 Various ways of implementing distributed learning in AI / ML applications executed using cellular networks are shown. Not only the most basic separation between terminal devices and cloud servers can be envisioned, but also separation implemented using edge servers or separation implemented between terminal devices, for example.

[0099] (1-3, Multi-input AI / ML model)

[0100] As an AI / ML application executed using a cellular network, an AI / ML application including a multi-input AI / ML model can be envisioned. As an example of an AI / ML application including a multi-input AI / ML model, Figure 6 A case where a robot moving in a factory is used is shown.

[0101] exist Figure 6 In the example shown, the robot extracts initial information (initial input) with a limited amount of data from the image captured by the robot, and first sends the initial information to the edge / cloud server. The edge / cloud server inputs the received initial information into the multi-input AI / ML model to obtain a rough recognition result. Based on the rough recognition result, the edge / cloud server sends an initial movement instruction to the robot. For example, based on the rough recognition result indicating that an obstacle is located in front of the robot, a movement for obstacle avoidance is started. When the robot receives the initial movement instruction from the edge / cloud server, the robot starts the movement for obstacle avoidance according to the initial movement instruction.

[0102] Next, the robot extracts additional information (additional input) that is not included in the initial information from the image captured by the robot as described above, and sends the additional information to the edge / cloud server. The edge / cloud server combines the received additional information with the previously received initial information, and inputs the combined information into the multi-input AI / ML model to obtain a detailed recognition result. Based on the detailed recognition result, the edge / cloud server sends an updated movement instruction to the robot. For example, based on the detailed recognition result indicating that the obstacle in front of the robot is a person, the route of movement for obstacle avoidance is corrected to ensure sufficient safety. When the robot receives an updated movement instruction from the edge / cloud server, the robot corrects the route of movement for obstacle avoidance according to the updated movement instruction.

[0103] When applying AI / ML applications as described above, evaluation indicators such as obtaining inference results with an accuracy greater than or equal to yy within a specified time period xx are important. In other words, the end-to-end latency exhibited from the time the robot starts sending initial information to the time the robot receives updated movement instructions given based on detailed inference results is important. In addition, when the robot sequentially sends items of additional information in response to requests for additional information issued by the edge / cloud server and sequentially receives updated movement instructions given based on the items of additional information, it is also necessary to consider the process of sequentially updating the inference results in the multi-input AI / ML model until the desired accuracy is obtained. In other words, it is necessary to consider the latency of the AI / ML application while taking into account the request for additional information and the process of updating the inference results based on the additional information.

[0104] <2. Basic Concepts of the Present Disclosure>

[0105] The purpose of the present disclosure is to provide a technology that allows AI / ML applications executed using a cellular network to update inference results in stages until the desired results are obtained. To achieve the purpose, a multi-input AI / ML model is used in the present disclosure, and a feedback mechanism is provided that enables the receiving side to issue a request for additional input data to the sending side.

[0106] Specifically, if Figure 7 As shown in , the sending entity 10 generates multiple input data with reduced data volume from the original input data, and first sends the first input data among the multiple input data to the receiving entity 20. The receiving entity 20 inputs the first input data to the multi-input AI / ML model to obtain a first inference value, and sets the obtained first inference value as a first inference result. For example, when the AI / ML model is a neural network model that classifies objects appearing in an image and when the neurons in the output layer correspond to the types of objects respectively (such as humans, luggage, and another robot), the type of the object corresponding to the neuron that outputs the maximum probability value from the softmax function is set as the first inference result.

[0107] When the accuracy of the first inference result is lower than a specific specified threshold, that is, for example, when the maximum probability value output from the softmax function is lower than another specified threshold, the receiving entity 20 gives feedback indicating a request to start sending additional input data among the multiple input data to the sending entity 10. When the sending entity 10 receives the feedback, the sending entity 10 sequentially sends the second and subsequent items of input data among the multiple items of input data to the receiving entity 20. The receiving entity 20 sequentially inputs the second and subsequent items of input data to the AI / ML model to obtain second and subsequent inference values, and sequentially updates the inference results based on the second and subsequent inference values.

[0108] When the accuracy of the first inference result is greater than or equal to a specific specified threshold, that is, for example, when the maximum probability value output from the softmax function is greater than or equal to another specified threshold, the receiving entity 20 gives feedback indicating a request to stop sending additional input data to the sending entity 10. When the sending entity 10 receives the feedback, the sending entity 10 stops sending additional input data.

[0109] (2-1. Configuration of sending entity and receiving entity)

[0110] Figure 8 1 shows the configuration of the sending entity 10 and the receiving entity 20 according to the present disclosure. It should be noted that Figure 8 It shows the logical configuration of the transmission entity 10 and the reception entity 20. The structural unit of each entity may be provided to a single device, or may be provided to a plurality of devices in a distributed manner.

[0111] For example, Fig. 9 The 5G system shown in FIG. 1 includes a user equipment (UE) 31, a radio access network / access network (RAN / AN) 32, a core network 33, and a cloud server 35. The cellular network 34 includes the RAN / AN 32 and the core network 33. In this case, the structural unit of the sending entity 10 may be set to the UE 31 connected to the cellular network 34, for example. In addition, the structural unit of the receiving entity 20 may be set to the cloud server 35 connected to the cellular network 34, for example.

[0112] Alternatively, part or all of the structural units of the sending entity 10 may be set in a distributed manner to UE 31, RAN / AN 32, and core network 33. Similarly, part or all of the structural units of the receiving entity 20 may be set in a distributed manner to RAN / AN 32, core network 33, and cloud server 35.

[0113] exist Fig. 9 In the 5G system shown, UE 31 and RAN / AN 32 are connected to each other via a wireless communication link. RAN / AN 32 and core network 33 are connected to each other via a wireless communication link or a wired communication link. Core network 33 and cloud server 35 are connected to each other via a wireless communication link or a wired communication link.

[0114] For example, the cellular network 34 may be configured in various ways by the RAN / AN 32 or the core network 33 in response to a request issued by the cloud server 35. Such configuration may be provided statically or dynamically. When the UE 31 is a terminal dedicated to AI / ML applications and has limited use, the UE 31 may be pre-configured.

[0115] (2-2, Configuration of the sending entity)

[0116] Back to Figure 8 , the transmitting entity 10 includes an acquisition part 11, a generator 12, a data transmitter 13, and a request receiver 14. In addition, a sensing device (not shown) such as an imaging sensor is connected to the transmitting entity 10. For example, when the transmitting entity 10 is set to the UE 31, the sensing device may be set to the UE 31. Alternatively, when the transmitting entity 10 is set to the RAN / AN 32, the sensing device may be set to the RAN / AN 32.

[0117] The acquisition section 11 acquires raw input data generated by a sensing device (not shown). For example, when the sensing device is an imaging sensor, the raw input data is still image data or moving image data.

[0118] The generator 12 generates multiple items of input data with reduced data amount from the original input data. For example, when the original input data is still image data, multiple items of still image data with reduced data amount, that is, multiple items of still image data with reduced resolution, are generated from the original still image data.

[0119] For example, when three items of still image data with reduced resolution are generated from original still image data, the first item of still image data among the three items of still image data has the lowest resolution, that is, the first still image data has the smallest data amount. The second item of still image data among the three items of still image data has a higher resolution than the first item of still image data, that is, the second still image data has a larger data amount than the first still image data. The third item of still image data among the three items of still image data has a higher resolution than the second item of still image data, that is, the third still image data has a larger data amount than the second still image data. It should be noted that the third still image data can be exactly the same as the original still image data.

[0120] The data transmitter 13 sequentially transmits each item of input data among the multiple items of input data generated by the generator 12 to the receiving entity 20. For example, when the transmitting entity 10 is set to the UE 31 and the receiving entity 20 is set to the cloud server 35, each item of input data among the multiple items of input data is sequentially transmitted from the UE 31 to the cloud server 35 through the cellular network 34.

[0121] The request receiver 14 receives feedback information from the receiving entity 20. For example, when the sending entity 10 is set to the UE 31 and the receiving entity 20 is set to the cloud server 35, the feedback information is received from the cloud server 35 by the UE 31 through the cellular network 34.

[0122] The data transmitter 13 transmits the first input data among the plurality of input data to the receiving entity 20 in the first transmission. After the first transmission, when the request receiver 14 receives a request from the receiving entity 20 to start transmitting the additional input data among the plurality of input data, the data transmitter 13 starts the additional transmission. In the additional transmission, the data transmitter 13 sequentially transmits the second and subsequent input data among the plurality of input data to the receiving entity 20. During the additional transmission, when the request receiver 14 receives a request from the receiving entity 20 to stop transmitting the additional input data, the data transmitter 13 stops the additional transmission.

[0123] It should be noted that after the generation of the first input data, the generator 12 may sequentially generate the second and subsequent input data items after the request receiver 14 receives a request to start sending additional input data from the receiving entity 20. Alternatively, the generator 12 may generate the multiple input data items together.

[0124] (2-3. Configuration of receiving entity)

[0125] The receiving entity 20 includes a data receiver 21 , an inference part 22 including a multi-input AI / ML model, an output part 23 , a determination part 24 , and a request sender 25 .

[0126] The data receiver 21 sequentially receives each item of input data among the multiple items of input data from the sending entity 10. For example, when the sending entity 10 is set to the UE 31 and the receiving entity 20 is set to the cloud server 35, the cloud server 35 sequentially receives each item of input data among the multiple items of input data from the UE 31 through the cellular network 34.

[0127] The inference part 22 includes a multi-input AI / ML model, and each of the multiple input data is sequentially input into the AI / ML model to sequentially obtain each of the multiple inference values. For example, when the original input data is still image data and each of the multiple input data is still image data of different resolutions, the AI / ML model is, for example, a single convolutional neural network (CNN) model for classifying objects appearing in the still image. Here, when the number of pixels varies according to the resolution of each still image data, it is necessary to implement an adjusted preprocessing so that the number of pixels of each still image data is equal to the number of input neurons of the CNN model, that is, for example, it is necessary to apply the scaling function described later. Alternatively, the inference part 22 may include multiple CNN models, each of which has a different number of input neurons corresponding to the number of pixels of each still image data.

[0128] The output section 23 sequentially updates the inference result based on each of the multiple inference values ​​acquired by the inference section 22 to output the updated inference result. The output section 23 compares the inference results and selects the inference result with the highest inference accuracy to set the selected inference result as the final inference result (hard combination), or the output section 23 sequentially updates the inference results by updating the previously obtained inference result with the later obtained inference result to obtain the final inference result (soft combination).

[0129] The determination part 24 determines whether additional input data is needed among the multiple input data. For example, the determination part 24 determines whether additional input data is needed based on the accuracy of the current inference result. For example, if the accuracy of the current estimation result is greater than or equal to a specific specified threshold, that is, for example, when the maximum probability value output from the softmax function is greater than or equal to another specified threshold, the determination part 24 determines that the inference result no longer needs to be updated, and determines that additional input data is not needed. On the other hand, if the accuracy of the current inference result is less than a specific specified threshold, that is, for example, when the maximum probability value output from the softmax function is less than another specified threshold, the determination part 24 determines that the inference result still needs to be updated, and determines that additional input data is needed.

[0130] The request sender 25 sends the feedback information to the sending entity 10. For example, when the sending entity 10 is set to the UE 31 and the receiving entity 20 is set to the cloud server 35, the feedback information is sent from the cloud server 35 to the UE 31 through the cellular network 34.

[0131] The data receiver 21 receives the first input data among the plurality of input data from the transmitting entity 10 in the first reception. After the first reception, when the determination part 24 determines that the additional input data is needed, the request transmitter 25 sends a request to start sending the additional input data to the transmitting entity 10. In the additional reception, the data receiver 21 sequentially receives the second and subsequent input data among the plurality of input data from the transmitting entity 10. During the additional reception, when the determination part 24 determines that the additional input data is not needed, the request transmitter 25 sends a request to stop sending the additional input data to the transmitting entity 10.

[0132] <3. Multi-input AI / ML model and input data according to the present disclosure>

[0133] Next, the configuration of a multi-input AI / ML model and various input data input to the AI / ML model according to the present disclosure are described.

[0134] (3-1. Multi-input AI / ML model)

[0135] As an example of a multi-input AI / ML model, Fig.10 The configuration of a convolutional neural network (CNN) model with an input of 28 pixels × 28 pixels is shown. It should be noted that, for example, another AI / ML model can also be used, which updates the inference result by implementing additional input to the neural network model that has completed inference using a recurrent neural network model.

[0136] (3-2. Method for generating input data)

[0137] Next, various methods for generating multiple items of input data from original input data are described.

[0138] (3-2-1. Generation of input data based on resolution of still image)

[0139] Fig.11 shows that two items of still image data of different resolutions are input to Fig.10 An example of a CNN model shown in . First input data and second input data are generated from original input data, the first input data having a resolution reduced by reducing the number of pixels, and the second input data having a resolution higher than the first input data. It should be noted that here, the second input data has exactly the same resolution as the original input data.

[0140] Before the first input data is input into the CNN model, a scaling function for implementing size enlargement is applied to the first input data. A first inference value is obtained based on the first input data and is set as a first inference result. Subsequently, a second inference value is obtained based on the second input data. Finally, the first inference result is updated based on the first inference value and the second inference value. For example, a hard combination is implemented in which the first inference value and the second inference value are compared, and one of the first inference value and the second inference value that exhibits a higher accuracy is set as the inference result, and a final inference result is output.

[0141] Fig.12 Three methods for generating two items of input data of different resolutions from original still image data are shown.

[0142] First method: low-resolution data (left side in the figure) is generated as first input data, and high-resolution data (right side in the figure) is generated as second input data.

[0143] Method 2: A portion of the low-resolution data (region A on the left side in the figure) is generated as the first input data, and a portion of the high-resolution data (region B on the right side in the figure) is generated as the second input data.

[0144] Method 3: Generate a portion of low-resolution data (area A on the left side of the figure) as the first input data, and generate a portion of the low-resolution data (area A on the left side of the figure) and a portion of the high-resolution data (area B on the right side of the figure) as the second input data.

[0145] (3-2-2. Generation of Input Data Based on Regions in Still Images)

[0146] Fig.13 The two still image data of different regions are input into Fig.10 An example of a CNN model shown in . First input data and second input data are generated from original input data, the first input data only including an area near the central part of the image, and the second input data only including an area near the peripheral edge of the image. A first inference value is obtained based on the first input data and set as a first inference result. Next, the first input data and the second input data are combined, and a second inference value is obtained using the input data obtained by the combination as new second input data. Finally, the first inference result is updated based on the first inference value and the second inference value. For example, a soft combination is implemented in which the latest one of the first inference value and the second inference value is set as the inference result, and the final inference result is output.

[0147] Fig.14 Two methods for generating two items of input data of different regions from original still image data are shown.

[0148] Method 1: Generate data of a specific area (area C on the left side in the figure) as the first input data, and generate data of an area not including the specific area of ​​the first input data as the second input data, or generate data of an area including the specific area of ​​the first input data (area D on the left side in the figure) as the second input data. Here, the area of ​​each item of input data may be determined on the sending entity 10 side, or the area of ​​each item of input data may be determined on the receiving entity 20 side and reported to the sending entity 10. In addition, the area of ​​each item of input data may be statically determined in advance, or may be quasi-statically determined.

[0149] Method 2: Generate data of only the important area (area E on the right side of the figure) as the first input data, and generate data of the area not including the area of ​​the first input data as the second input data, or generate data of the area including the area of ​​the first input data (area F on the right side of the figure) as the second input data. Here, the important area can be determined on the sending entity 10 side, or the important area can be determined on the receiving entity 20 side and reported to the sending entity 10. The important area can be determined dynamically based on the feature quantity. For example, the important area can be determined based on the feature quantity extracted using an algorithm such as a filtering method, a packaging method, or an embedding method. In addition, the important area can be determined by a sensing device that includes an AI / ML function and is connected to the sending entity 10.

[0150] (3-2-3. Generation of input data based on resolution and area in still image)

[0151] Fig.15 An example of a method for generating two items of input data of different resolutions and different regions from original still image data is shown. In addition, when generating each item of input data of different regions and different resolutions from original input data, the resolution-based method in the previously described "3-2-1" and the region-based method in the previously described "3-2-2" can be arbitrarily combined.

[0152] (3-2-4. Generation of input data based on the frame rate of moving images)

[0153] Fig.16 Two methods for generating two items of input data of different frame rates (frames per second: FPS) from raw moving image data are shown.

[0154] Method 1: Generate data including only a portion of frames (the base layer in the figure) as the first input data, and generate data including the remaining frames (the enhanced layer in the figure) as the second input data, wherein the remaining frames correspond to frames that are not frames of the first input data, that is, data including frames that are orthogonal to the frames of the first input data.

[0155] Method 2: Generate data including only a part of the frames (base layer in the figure) as the first input data, and generate data including all frames (base layer and enhancement layer in the figure) including the frames of the first input data as the second input data.

[0156] (3-2-5. Generation of input data by compression coding of image)

[0157] Next, three methods used when a plurality of data sets are generated from original still image data or original moving image data in a process of compression encoding and are used as individual items of input data are described.

[0158] Method 1: In JPEG compression coding, the original still image data is transformed into the frequency domain using discrete cosine transform (DCT), the coefficients of the low-frequency components are set as the first input data, and the coefficients of the high-frequency components are set as the second input data ( Fig.17 ).

[0159] Method 2: In the JPEG progressive scheme, the average value of each block is set as the first input data, and each detailed design information is set as the second or subsequent input data ( Fig.18 ). There can be second and subsequent entries of input data.

[0160] Method 3: When compressing a moving image, the moving image is divided into a portion with a large video change and a portion with a small video change. The portion with a large video change is set as the first input data, and the portion with a small video change is set as the second input data.

[0161] (3-2-6. Generation of input data using error correction code)

[0162] Fig.19 Three methods are shown in which, when packet communication is carried out using an error correction code, first-time transmission data is set as first input data, and retransmission data is set as second or subsequent input data.

[0163] Method 1: Using different ratios of system information to parity information for first-time transmission data and retransmission data (upper part in the figure).

[0164] Method 2: System information is transmitted as first-transmission data, and parity information is transmitted as retransmission data (the middle part in the figure).

[0165] Method 3: Divide the mixture of system information and parity information into blocks, and assign each block to first-time transmission data or retransmission data (lower part in the figure). Here, each block obtained by division can be numbered in the form of redundant version.

[0166] (3-2-7. Generation of input data for separation based on AI / ML models)

[0167] The intermediate layer data generated by the separated AI / ML model may be subjected to a process of generating multiple input data from the original input data. In this case, the intermediate layer data is the original input data, and only specific data is extracted from the intermediate layer data to generate the multiple input data. Here, a limit on the amount of data for each input data may be set in advance. In addition, data compression may be performed after only the specific data is extracted from the intermediate layer data.

[0168] <4. Details of various movements according to the present disclosure>

[0169] Details of various movements according to the present disclosure are described next.

[0170] (4-1. Timing of Switching Method for Sending Input Data)

[0171] When there is performance space in the communication link between the transmitting entity 10 and the receiving entity 20, the transmitting entity 10 may send the original input data acquired by the transmitting entity 10 to the receiving entity 20 without change (single transmission). As described above, for example, when the communication performance is reduced due to the bandwidth limitation of the communication link during the single transmission, the transmitting entity 10 generates a plurality of input data with a reduced data amount from the original input data, and sequentially sends each of the plurality of input data to the receiving entity 20 (multiple transmissions).

[0172] When the transmitting entity 10 switches the method for transmitting input data from single transmission to multiple transmission, for example, the following conditions can be assumed: Also, here, the transmitting entity 10 reports to the receiving entity 20 that the transmission method is switched from single transmission to multiple transmission.

[0173] Condition 1: In a state where the transmission entity 10 is set to the UE 31 , an instruction to implement switching to multiple transmission is received from the RAN / AN 32 .

[0174] Condition 2: An application request is received from the AI / ML application indicating a switch to multiple transmissions.

[0175] Condition 3: In a state where the transmission entity 10 is set to the UE 31, a change or variation of the communication performance is reported by the RAN / AN 32, and it is determined that it is difficult to maintain a single transmission for the communication performance after the change or variation. Here, examples of information used to derive the communication performance include, for example, information on RSRP / RSRQ / RSSI, delay information, delay budget information, information on throughput, and information on allocated wireless communication resources.

[0176] Condition 4: The transmitting entity 10 measures the communication performance, and determines that it is difficult to maintain single transmission with respect to the measured communication performance.

[0177] (4-2, hard combination)

[0178] Fig. 20 2 is a diagram for describing an example in which two items of input data are used for hard combining performed by the output portion 23 of the receiving entity 20.

[0179] In hard combination, a first inference value based on first input data is first obtained and set as the first inference result. Next, a second inference value based on second input data is obtained. The second inference value is compared with the first inference value, and one of the inference values ​​showing higher inference accuracy is set as the final inference result.

[0180] For example, in the case of using a robot, the presence of an obstacle is recognized based on the first inference result, and obstacle avoidance movement is started as the initial movement. Subsequently, the obstacle is recognized as another moving robot based on the final inference result, and the route of the movement for obstacle avoidance is corrected to ensure a sufficient distance from the other robot.

[0181] Fig.21 This is a flowchart for describing the details of the operation performed when the hard combination is performed in the case of using the robot described above. First, first input data is received (S11), a first inference value is obtained based on the first input data (S12), and the first inference value is set as a first inference result (S13). Next, second input data is received (S14), a second inference value is obtained based on the second input data (S15), and the first inference value and the second inference value are compared (S16).

[0182] When the second inference value is more accurate than the first inference value, the second inference value is determined to be true (S17=Yes), and the second inference value is set as the final inference result (S18). On the other hand, when the second inference value is less accurate than the first inference value, the first inference value is determined to be true (S17=No), and the first inference result is maintained (S19).

[0183] (4-3, soft combination)

[0184] Fig. 22 2 is a diagram for describing an example in which two items of input data are used for soft combining performed by the output portion 23 of the receiving entity 20.

[0185] In soft combination, a first inference value based on first input data is first obtained and set as a first inference result. Next, a second inference value based on second input data is obtained and set as a final inference result. Here, the second input data may include a part or all of the first input data. Alternatively, the first input data and the second input data may be combined, and a second inference value may be obtained based on the data obtained by the combination and set as new second input data.

[0186] Fig.23 This is a flowchart for describing the details of the operation performed when the soft combination is performed in the case of using the robot described above. First, first input data is received (S21), a first inference value is obtained based on the first input data (S22), and the first inference value is set as a first inference result (S23). Next, second input data is received (S24), a second inference value is obtained based on the second input data (S25), and the second inference value is set as a final inference result (S26).

[0187] (4-4, start sending a request for additional input data)

[0188] When determination section 24 determines that additional input data is required, request transmitter 25 of receiving entity 20 transmits a request to start transmitting additional input data as feedback information addressed to transmitting entity 10. When it is determined that additional input data is required, for example, the following conditions can be conceived.

[0189] Condition 1: The accuracy of the first inference result (inference accuracy) is lower than a specified threshold. Fig. 22 As shown in , when the accuracy of the first inference result obtained based on the first input data does not reach the desired accuracy, a request to start sending additional input data is sent. It should be noted that in addition to the maximum probability value output from the softmax function described above, for example, the signal-to-noise ratio (SNR), the peak signal-to-noise ratio (PSNR) or the structural similarity (SSIM) can be used as an indicator of inference accuracy. It should be noted that all or part of the original image can be reported periodically as reference information in order to calculate the inference accuracy. This results in, for example, sending a request to start sending additional input data when the inference accuracy does not reach the desired accuracy until the reference information is received next time.

[0190] Condition 2: The delay until the first inference result is obtained (inference delay) is less than a specified threshold. When there is a sufficient period of time from the sending entity 10 sending the first input data to the receiving entity 20 obtaining the first inference result, a request to start sending additional input data is sent.

[0191] (4-5. Stop sending requests for additional input data)

[0192] When determination section 24 determines that additional input data is not required, request transmitter 25 of receiving entity 20 transmits a request to stop transmitting additional input data as feedback information addressed to transmitting entity 10. When it is determined that additional input data is not required, for example, the following conditions can be conceived.

[0193] Condition 1: The accuracy of the inference result (inference accuracy) is greater than or equal to a specified threshold. When the accuracy of the current inference result reaches the desired accuracy, a request to stop sending additional input data is sent. The threshold used in this case may be different from the threshold used for the request to start sending described above.

[0194] Condition 2: The delay until the inference result is obtained (inference delay) is greater than or equal to a specified threshold. When there is not enough time from the sending entity 10 sending the input data to the receiving entity 20 obtaining the inference result, a request to stop sending additional input data is sent. The threshold used in this case may be different from the threshold used for the request to start sending described above.

[0195] Condition 3: The accuracy difference (the amount of improvement in accuracy) between the current inference result and the immediately preceding inference result is less than a specified threshold. When it is determined that improvement in inference accuracy is no longer expected, a request to stop sending additional input data is sent.

[0196] It should be noted that, unlike conditions 1 to 3 described above, the stopping of the sending of additional input data can be determined on the side of the sending entity 10. For example, in the case of use in XR navigation, it is conceivable that the sending entity 10 can be set to the goggles worn by the user and the user's viewpoint can be tracked at all times. In such a use case, when it is determined that the user's line of sight leaves the target object, the sending of additional input data used for inference about the target object can be stopped. In this case, the sending entity 10 reports the stopping of the sending of additional input data to the receiving entity 20.

[0197] (4-6. How to provide feedback)

[0198] For example, ACK / NACK can be used as a method of providing feedback information sent by the request sender 25 of the receiving entity 20, that is, a request to start sending additional input data and a request to stop sending additional input data. For example, when the accuracy of the inference result is greater than or equal to a specified threshold and additional input data is not required, ACK is sent. On the other hand, when the accuracy of the inference result is lower than the specified threshold and additional input data is required, NACK is sent.

[0199] (4-7. Related information serving as feedback information)

[0200] In addition to the request to start sending the additional input data and the request to stop sending the additional input data, the request sender 25 of the receiving entity 20 may send, for example, associated information as indicated below as feedback information.

[0201] Related information 1: the number of times the additional input data is sent. For example, it can be reported whether sending the additional input data once is sufficient or whether the additional input data will be sent multiple times as the related information.

[0202] Related information 2: Specifications of additional input data. For example, various specifications may be specified in the form of related information, such as the amount of additional input data (e.g., xxMB), the resolution of the additional input data, the area of ​​the additional input data, the bit rate of the additional input data, and the proportion of redundant data in the additional input data. In addition, for example, data output from a specific neuron in the neural network model may be specified in the form of management information.

[0203] Related information 3: Delay of additional input data. For example, delay budget information indicating that the additional input data needs to be processed within xx milliseconds may be reported in the form of management information. In addition, time specifications such as requiring the additional input data to be sent within xx milliseconds may be reported in the form of related information.

[0204] Related information 4: the gap between the accuracy of the current inference result and the expected inference accuracy. For example, information about the gap between the accuracy of the current inference result and the expected inference accuracy may be reported in the form of related information.

[0205] Association information 5: Priority of additional input data. For example, the priority of the additional input data (e.g., high, medium, and low) may be reported in the form of association information. The sending entity 10 compares the priority of the requested additional input data with the priority of the additional input data requested by another user, and determines which request from each user will be prioritized.

[0206] (4-8. Changes in Additional Input Data Based on Related Information)

[0207] Based on the aforementioned association information received from the receiving entity 20, the sending entity 10 may change the method for generating the additional input data or the method for sending the additional input data. For example, the following change examples may be envisaged.

[0208] Change example 1: Change the timing of generating additional input data. For example, before receiving a request to start sending the additional input data from the receiving entity 20, the generator 12 of the sending entity 10 may generate each item of additional input data in advance. Alternatively, the generator 12 of the sending entity 10 may sequentially generate each item of additional input data at the timing of receiving the request to start sending. When generating each item of additional input data in advance, the sending entity 10 may report version information about the version of the additional input data to the receiving entity 20 in advance.

[0209] Change Example 2: Changing the resolution of the additional input data. For example, the generator 12 of the sending entity 10 may change the resolution of the additional input data. Alternatively, the generator 12 of the sending entity 10 may change the resolution of the additional input data for each region. Alternatively, the generator 12 of the sending entity 10 may change the resolution of the additional input data to a resolution at which the additional input data has not yet been generated.

[0210] Change Example 3: Changing the area of ​​additional input data. For example, the generator 12 of the sending entity 10 may change the area of ​​additional input data. Alternatively, the generator 12 of the sending entity 10 may pre-generate each item of input data by dividing the original input data into respective areas, and may select input data of a certain area to send the selected input data. Alternatively, the generator 12 of the sending entity 10 may change the area of ​​additional input data to an area where input data has not yet been generated.

[0211] Change Example 3: Transmission of encoding information about encoding of additional input data For example, the data transmitter 13 of the transmitting entity 10 may retransmit system information about the additional input data that has already been transmitted. Alternatively, the data transmitter 13 of the transmitting entity 10 may additionally transmit parity information of the additional input data that has already been transmitted.

[0212] <5. Functional expansion of cellular networks according to the present disclosure>

[0213] In the present disclosure, it is necessary to send multiple different communications (at different times) in an AI / ML application including a multi-input AI / ML model. For example, it is necessary to send input data of different resolutions using different streams.

[0214] (5-1. Network slicing for 5G systems)

[0215] The 5G system makes it possible to provide communication supporting the multi-input AI / ML model described above in the form of network slicing. When the AI / ML application is activated, the UE selects a network slice that provides communication supporting the multi-input AI / ML model and implements a process of requesting to establish a protocol data unit (PDU) session corresponding to the selected network slice.

[0216] S-NSSAI is assigned to each network slice as information (network slice selection assistance information) used to assist in selecting a network slice, and the network slice is identified by S-NSSAI. S-NSSAI includes a pair of a mandatory slice / service type (SST) and an optional slice distinguisher (SD), wherein the SST includes 8 bits used to identify the slice type, and the SD includes 24 bits used to distinguish different slices in a single SST.

[0217] The core network can select a network function (NF) necessary to provide a PDU session corresponding to a network slice based on S-NSSAI. For example, an access and mobility management function (AMF) that receives a message requesting to establish a PDU session selects one of multiple candidates for a session management function (SMF) based on S-NSSAI. The selected SMF selects one of multiple candidates for a user plane function (UPF) based on S-NSSAI. Here, UPF is a network function that processes user plane data for a PDU session.

[0218] When the AI / ML application is executed, the UE selects a data network name (DNN) connected to the cloud server 35 providing the AI / ML application. The message requesting the establishment of a PDU session described above includes the DNN and the S-NSSAI. In addition, the established PDU session is associated with the DNN and the S-NSSAI.

[0219] (5-2. QoS control for 5G systems)

[0220] At the non-access stratum (NAS) level, QoS flow is the finest granularity to distinguish different kinds of QoS in a PDU session. QoS flows are identified by QoS flow ID (QFI) in a PDU session. RAN / AN can establish at least one data radio bearer (DRB) between RAN / AN and each UE along with a PDU session. DRB is a logical path used to transmit data. The 5G QoS model supports guaranteed flow bit rate (GBR) with bandwidth guarantee and non-guaranteed flow bit rate (non-GBR) without bandwidth guarantee.

[0221] RAN / AN and core network ensure the quality of service by mapping each packet to an appropriate QoS flow and an appropriate DRB. In other words, a two-step mapping including mapping IP flows to QoS flows in NAS and QoS flows to DRBs in the access layer (AS) is implemented.

[0222] At the NAS level, QoS flows are characterized by QoS profiles and QoS rules, with the QoS profiles provided by the core network 33 to the RAN / AN and the QoS rules provided by the core network to the UE. The QoS profiles are used by the RAN / AN to determine the method used to implement the processing over the radio interface. The QoS rules are used to instruct the UE to map user plane traffic to QoS flows in the uplink.

[0223] The QoS profile is set to the RAN / AN through the SMF, AMF and reference point N2, or is pre-set in the RAN / AN. In addition, through the AMF and reference point N1, the SMF can provide the UE with at least one QoS rule, and if necessary, can also provide the UE with QoS flow-level QoS parameters associated with the at least one QoS rule.

[0224] In addition or in lieu thereof, reflective QoS control may be applied to the UE. Reflective QoS control is QoS control that monitors the QFI of packets in the downlink and applies the same mapping to packets in the uplink.

[0225] A QoS flow is a GBR QoS flow or a non-GBR QoS flow depending on its QoS profile. The QoS profile of a QoS flow includes QoS parameters such as 5G QoS Identifier (5QI) and Allocation and Retention Priority (ARP).

[0226] ARP includes information about priority, preemption capability, and preemption vulnerability. Priority defines the relative importance of QoS flows, and the minimum value of priority represents the highest priority. Preemption capability is an indicator that defines whether a specific QoS flow can obtain resources that have been allocated to another QoS flow with a lower priority. Preemption vulnerability is an indicator that defines whether a specific QoS flow can give up resources allocated to the specific QoS flow to another QoS flow with a higher priority. Preemption capability and preemption vulnerability are set to "enabled" or "disabled", respectively.

[0227] In a GBR QoS flow, the QoS profile includes, for example, guaranteed flow bit rate (GFBR) in uplink and downlink, maximum flow bit rate (MFBR) in uplink and downlink, maximum packet loss rate in uplink and downlink, delay critical resource type, and notification control.

[0228] In non-GBR QoS flows, the QoS profile includes, for example, Reflective QoS Attributes (RQA) and additional QoS flow information. The Notification Control of QoS parameters indicates whether a request for a report from the RAN / AN is issued when a specific QoS flow does not meet the GFBR. When the Notification Control for a specific GBR QoS flow is "enabled" and it is determined that the GFBR is not met, the RAN / AN sends a report on this matter to the SMF.

[0229] In this case, the RAN / AN must maintain the QoS flow unless the RAN / AN requests to release the RAN resources for the GBR QoS flow, which is a special state such as radio link failure state or RAN internal congestion state. In addition, when it is determined that the GFBR meets the QoS flow again, the RAN / AN sends a new report on this matter to the SMF.

[0230] The aggregate maximum bit rate (AMBR) is related to the session AMBR of each PDU session and the UE-AMBR of the corresponding UE. The session AMBR imposes a limit on the aggregate bit rate that is expected to be provided to all non-GBR QoS flows for a specific PDU session and is managed by the UPF. The UE-AMBR imposes a limit on the aggregate bit rate that is expected to be provided to all non-GBR QoS flows for a specific UE and is managed by the RAN / AN.

[0231] 5QI is related to QoS features and provides policies used to set node-specific parameters for each QoS flow. Pre-standardized or set 5G QoS features are known from 5QI rather than explicitly signaled. Signaled QoS features can be included as part of the QoS profile.

[0232] The QoS characteristics include, for example, information on priority, packet delay budget, packet error rate, average window and maximum data burst size. The packet delay budget may include a packet delay budget in a core network.

[0233] At the AS level, DRB defines the method for processing packets over the Uu interface. DRB provides a single packet forwarding process for any packet.

[0234] The RAN / AN maps the QoS flow to the DRB based on the QFI and the QoS profile set for the QFI. The RAN / AN can establish different DRBs for packets requesting different packet forwarding processes. In addition, the RAN / AN can also multiplex multiple QoS flows belonging to a single PDU session in a single DRB.

[0235] In the uplink, the mapping of QoS flows to DRBs is controlled by one of two different methods of signaling mapping rules. One of the methods is a method called reflective mapping. When radial mapping is implemented, the UE monitors the QFI of downlink packets for each DRB and applies the same mapping to uplink packets. The other method is a method called explicit configuration. When explicit configuration is implemented, the mapping rules for the mapping of QoS flows to DRBs are signaled by RRC.

[0236] In the downlink, the QFI is signaled by the RAN / AN over the Uu interface for reflective quality of service purposes (RQoS). Unless a reflective mapping is used for a QoS flow carried by a particular DRB, the QFI for that DRB is not signaled by the RAN / AN or NAS over the Uu interface.

[0237] In the uplink, the RAN / AN may be configured to signal the QFI to the UE over the Uu interface. In addition, a default DRB may be set for each PDU session. When an uplink packet does not fit the explicit configuration or the reflective mapping, the UE maps the packet to the default DRB for the PDU session.

[0238] The core network may send additional QoS flow information parameters related to any QoS flow to the RAN / AN to give instructions to make specific traffic more frequent with respect to a specific non-GBR QoS flow in a PDU session compared to other non-GBR QoS flows in the PDU session.

[0239] It is up to the RAN / AN to determine how to map multiple QoS flows in a PDU session to a single DRB. For example, the RAN / AN may map a GBR QoS flow and a non-GBR QoS flow to a single DRB or to different DRBs. In addition, the RAN / AN may map each GBR QoS flow among multiple GBR QoS flows to a single DRB or to different DRBs.

[0240] In 5G NR, a new Service Data Adaptation Protocol (SDAP) sublayer is introduced for QoS control implemented through QoS flows. The SDAP sublayer maps QoS flow traffic to appropriate DRBs. The SDAP sublayer may include multiple SDAP entities, and includes an SDAP entity for each PDU session through the Uu interface. The SDAP entity is established or released through RRC.

[0241] The QoS flow is identified by the QFI in the PDU session container included in the GTP-U header. The PDU session is identified by the GTP-U tunnel endpoint ID (TEID). The SDAP sublayer maps each QoS flow to a specific DRB.

[0242] (5-3. Multi-QoS flow control for 5G systems)

[0243] In an AI / ML application including a multi-input AI / ML model, each item of input data is input to the AI / ML model, and different QoS flows are mapped to each item of input data. In other words, different QoS flows corresponding to each item of input data input to the AI / ML model are mapped to a PDU session that processes the input data. Here, the number of QoS flows mapped to a single PDU session may be included in a message requesting establishment of the PDU session, or may be pre-set for each S-NSSAI.

[0244] Through the QoS profile for each QoS flow obtained from the core network, the RAN / AN can determine the 5QI applied to the QoS flow. In addition, through the QoS rules obtained from the RAN / AN, the UE obtains information about the mapping of user plane traffic in the uplink to the QoS flow. The UE can determine the QoS flow to which each input data input to the AI / ML model is mapped based on the information about the mapping. For each QoS flow, the RAN / AN allocates radio resources to the UE, that is, implements scheduling.

[0245] <6. Example>

[0246] As an example of an AI / ML application including an AI / ML model according to the present disclosure, FIG24 shows a sequence when a robot in a factory implements autonomous driving while implementing image recognition. In this example, under the assumption of typical cloud learning, the sending entity 10 is set to a robot (UE) 31, and the receiving entity 20 is set to a cloud server 35.

[0247] At time T1 in FIG. 24 , there is performance space in the communication link between the robot 31 and the cloud server 35. Therefore, when the robot 31 acquires raw image data (S101), the robot 31 sends the raw image data to the cloud server without change in a single transmission (S102). The cloud server 35 inputs the received image data to the multi-input AI / ML model, and sends a movement instruction given based on the inference result to the robot 31 (S104).

[0248] At time T2, the communication performance rapidly decreases due to the bandwidth limitation of the wireless communication link between the robot 31 and the RAN / AN 32. When the RAN / AN 32 detects the decrease, the RAN / AN 32 reports the decrease (change or variation) in the communication performance to the robot 31 (S105). When the robot 31 receives the report, the robot 31 changes the method for transmitting the image data from single transmission to multiple transmissions (S106), and reports the change to the cloud server 35 (S107).

[0249] When the robot 31 acquires the original image data (S108), the robot 31 generates first input data with reduced resolution based on the original image data (S109), and sends the first input data to the cloud server 35 (S110). When the cloud server 35 receives the first input data, the cloud server 35 inputs the first input data to the multi-input AI / ML model to obtain a first inference result (rough recognition result) (S111).

[0250] The cloud server 35 sends an initial movement instruction to the robot 31 based on the rough recognition result (S112). For example, movement for obstacle avoidance is started based on the rough recognition result indicating that an obstacle is located in front of the robot 31. When the robot 31 receives the initial movement instruction from the cloud server 35, the robot 31 starts movement for obstacle avoidance according to the initial movement instruction (S113).

[0251] Next, since the accuracy of the first inference result obtained in the previously described step S111 is lower than the specified threshold, the cloud server 35 sends a request to the robot 31 to start sending additional input data (S114). When the robot 31 receives the request, the robot 31 generates second input data having a higher resolution than the first input data (S115), and sends the second input data to the cloud server 35 (S116). When the cloud server 35 receives the second input data, the cloud server 35 inputs the second input data to the multi-input AI / ML model to obtain a second inference result (detailed recognition result) (S117).

[0252] Based on the detailed recognition result, the cloud server 35 sends an updated movement instruction to the robot 31 (S118). For example, the route of movement for obstacle avoidance is corrected based on the detailed recognition result indicating that the obstacle in front of the robot 31 is a person, so as to ensure sufficient safety. When the robot 31 receives the updated movement instruction from the cloud server 35, the robot 31 corrects the route of movement for obstacle avoidance according to the updated movement instruction (S119).

[0253] Next, since the accuracy of the second inference result obtained in step S117 described above is greater than or equal to the specified threshold, the cloud server 35 determines that additional input data is not required, and sends a request to stop sending additional input data to the robot 31 (S120). When the robot 31 receives the request, the robot 31 stops sending additional input data (S121).

[0254] <7. Conclusion>

[0255] As described above, the transmitting entity 10 according to the present disclosure generates multiple items of input data with reduced data amount from the original input data. When the transmitting entity 10 receives a request to start sending additional input data among the multiple items of input data from the receiving entity 20 after sending the first item of input data among the multiple items of input data, the transmitting entity 10 starts sending the additional input data. When the transmitting entity 10 receives a request to stop sending the additional input data from the receiving entity 20, the transmitting entity 10 stops sending the additional input data.

[0256] The receiving entity 20 sequentially inputs each of the multiple input data received from the sending entity 10 into the multi-input AI / ML model to sequentially update the inference result. When additional input data is required, the receiving entity 20 sends a request to start sending the additional input data to the sending entity 10. Subsequently, when additional input data is not required, the receiving entity 20 sends a request to stop sending the additional input data to the sending entity 10.

[0257] According to the characteristics described above, the transmitting entity 10 and the receiving entity 20 according to the present disclosure make it possible to provide an AI / ML application that updates the inference result in stages until the desired result is obtained. For example, in an environment where the performance in a wireless communication link changes significantly, the inference result can also be updated in stages until an inference result with the desired accuracy is obtained.

[0258] It should be noted that the technology according to the present disclosure is not limited to specific standards, and any modifications can be made to the settings shown where appropriate. It should be noted that the examples described above are examples for specifically implementing the present disclosure, and the present disclosure can be specifically implemented in a variety of other embodiments. For example, various modifications, substitutions, omissions or combinations can be applied without departing from the scope of the present disclosure. For example, the embodiments obtained by applying such modifications, substitutions, omissions or combinations are also included in the scope of the present disclosure and the embodiments of the present disclosure and their equivalent expressions.

[0259] In addition, the processing procedures described in the present disclosure may be regarded as methods including a series of procedures. Alternatively, the processing procedures described in the present disclosure may be regarded as programs for causing a computer to implement a series of procedures, or a recording medium in which the program is stored. In addition, the processing described above is implemented by a CPU such as a computer. In addition, since the examples of the present disclosure are not affected by the type of recording medium, the type of recording medium is not particularly limited.

[0260] It should be noted that the structural units shown in the present disclosure may be provided by software or may be implemented by hardware. For example, each structural unit may be a software module provided by software such as a microprogram, or may be provided by a processor that executes the software module. Alternatively, each structural unit may be implemented by a circuit block (die) on a semiconductor chip, i.e., an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). In addition, the number of structural units does not necessarily have to be equal to the number of hardware used to implement the structural unit. For example, multiple structural units may be implemented by a single processor or circuit. On the contrary, a single structural unit may be implemented by multiple processors or circuits.

[0261] It should be noted that the type of processor described in the present disclosure is not limited. For example, the processor may be a CPU, a microprocessing unit (MPU), or a graphics processing unit (GPU).

[0262] It should be noted that the present disclosure may take the following configurations.

[0263] (1) A sending entity for an AI / ML application, the sending entity comprising:

[0264] Get the acquisition part of the original input data;

[0265] A generator for generating multiple input data with a reduced data amount from original input data;

[0266] a data transmitter for sequentially transmitting items of input data among the plurality of items of input data; and

[0267] A request receiver receives feedback information based on whether additional input data is required among the plurality of input data.

[0268] (2) The sending entity according to (1), wherein

[0269] After the data transmitter transmits the first input data among the plurality of input data, when the request receiver receives a request to start transmitting the additional input data, the data transmitter starts transmitting the additional input data, and

[0270] When the request receiver receives the request to stop sending the additional input data, the data transmitter stops sending the additional input data.

[0271] (3) The sending entity according to (2), wherein

[0272] After the generator generates the first item of input data, when the request receiver receives a request to start sending additional input data, the generator sequentially generates items of additional input data.

[0273] (4) A sending entity according to any one of (1) to (3), wherein

[0274] The original input data is still image data, and

[0275] The plurality of input data are still image data of different resolutions.

[0276] (5) A sending entity according to any one of (1) to (3), wherein

[0277] The original input data is still image data, and

[0278] The plurality of input data are still image data of different parts or all of the areas.

[0279] (6) A sending entity according to any one of (1) to (5), wherein

[0280] The transmitting entity is provided to a terminal device connected to a cellular network.

[0281] (7) A sending entity according to any one of (1) to (5), wherein

[0282] The transmission entity is provided in a distributed manner to a cellular network and a terminal device, the terminal device being connected to the cellular network.

[0283] (8) A receiving entity for an AI / ML application, the receiving entity comprising:

[0284] A data receiver for sequentially receiving each item of input data among a plurality of items of input data;

[0285] An inference part, sequentially inputting each of the plurality of input data into the multi-input AI / ML model to sequentially obtain each of the plurality of inference values;

[0286] an output part, sequentially updating the inference result based on the plurality of inference values ​​to output the updated inference result;

[0287] determining whether a certain portion of the input data needs to be added among the plurality of input data; and

[0288] A request sender that sends feedback information based on a determination as to whether additional input data is required, the determination being performed by a determining portion.

[0289] (9) A receiving entity according to (8), wherein

[0290] When the determining part determines that additional input data is needed, the request sender sends a request to the sending entity to start sending the additional input data, and

[0291] After sending the request to start sending the additional input data, when the determining part determines that the additional input data is not needed, the request sender sends a request to stop sending the additional input data to the sending entity.

[0292] (10) A receiving entity according to (8) or (9), wherein

[0293] The determination section determines whether additional input data is required based on the accuracy of the inference result.

[0294] (11) A receiving entity according to (8) or (9), wherein

[0295] The determination section determines whether additional input data is required based on a delay incurred until an inference result is obtained.

[0296] (12) A receiving entity according to (8) or (9), wherein

[0297] The determination section determines whether additional input data is required based on a difference in accuracy between a current inference result and an immediately preceding inference result.

[0298] (13) A receiving entity according to any one of (8) to (12), wherein

[0299] The output section outputs an inference result based on the comparison of the plurality of inference values.

[0300] (14) A receiving entity according to any one of (8) to (12), wherein

[0301] The output section sets the latest one of the plurality of inferred values ​​as an inference result.

[0302] (15) A receiving entity according to any one of (8) to (14), wherein

[0303] The receiving entity is arranged to a server device connected to a cellular network.

[0304] (16) A receiving entity according to any one of (8) to (14), wherein

[0305] The receiving entity is arranged in a distributed manner to a cellular network and a server device, the server device being connected to the cellular network.

[0306] (17) A data transmission method for AI / ML applications, the data transmission method comprising:

[0307] Get the original input data;

[0308] generating multiple input data with reduced data volume from original input data;

[0309] sequentially sending each item of input data among the plurality of items of input data; and

[0310] Feedback information based on whether additional input data is required among the plurality of input data is received.

[0311] (18) A data processing method for AI / ML applications, the data processing method comprising:

[0312] sequentially receiving each item of input data among the plurality of items of input data;

[0313] sequentially inputting each of the plurality of input data into the multi-input AI / ML model to sequentially obtain each of the plurality of inferred values;

[0314] sequentially updating the inference result based on the plurality of inference values ​​to output an updated inference result;

[0315] determining whether additional input data is required among the plurality of input data; and

[0316] Feedback information is sent based on the determination as to whether additional input data is needed.

[0317] Reference numerals list

[0318] 10 - Sending Entity

[0319] 11 - Get Part

[0320] 12 — Generator

[0321] 13——Data transmitter

[0322] 14 - Request Receiver

[0323] 20 - Receiving Entity

[0324] 21 - Data Receiver

[0325] 22 - Inference

[0326] 23——Output section

[0327] 24 - Determine the part

[0328] 25 - Request sender

[0329] 31——UE (Terminal Equipment)

[0330] 32——RAN / AN

[0331] 33 - Core Network

[0332] 34 - Cellular Network

[0333] 35——Cloud Server (Server Device)

Claims

1. A sending entity for an AI / ML application, the sending entity comprising: Get the acquisition part of the original input data; A generator for generating multiple input data with a reduced data amount from original input data; a data transmitter for sequentially transmitting each item of input data among the plurality of items of input data; as well as A request receiver receives feedback information based on whether additional input data is required among the plurality of input data.

2. The sending entity according to claim 1, wherein After the data transmitter transmits the first input data among the plurality of input data, when the request receiver receives a request to start transmitting the additional input data, the data transmitter starts transmitting the additional input data, and When the request receiver receives the request to stop sending the additional input data, the data transmitter stops sending the additional input data.

3. The sending entity according to claim 2, wherein After the generator generates the first item of input data, when the request receiver receives a request to start sending additional input data, the generator sequentially generates items of additional input data.

4. The sending entity according to claim 1, wherein The original input data is still image data, and The plurality of input data are still image data of different resolutions.

5. The sending entity according to claim 1, wherein The original input data is still image data, and The plurality of input data are still image data of different parts or all of the areas.

6. The sending entity according to claim 1, wherein The transmitting entity is provided to a terminal device connected to a cellular network.

7. The sending entity according to claim 1, wherein The transmission entity is provided in a distributed manner to a cellular network and a terminal device, the terminal device being connected to the cellular network.

8. A receiving entity for an AI / ML application, the receiving entity comprising: A data receiver for sequentially receiving each item of input data among a plurality of items of input data; An inference part, sequentially inputting each of the plurality of input data into the multi-input AI / ML model to sequentially obtain each of the plurality of inference values; an output part, sequentially updating the inference result based on the plurality of inference values ​​to output the updated inference result; determining whether a certain portion of the input data needs to be added among the plurality of input data; as well as A request sender that sends feedback information based on a determination as to whether additional input data is required, the determination being performed by a determining portion.

9. A receiving entity according to claim 8, wherein When the determining part determines that additional input data is needed, the request sender sends a request to the sending entity to start sending the additional input data, and After sending the request to start sending the additional input data, when the determining part determines that the additional input data is not needed, the request sender sends a request to stop sending the additional input data to the sending entity.

10. The receiving entity of claim 8, wherein The determination section determines whether additional input data is required based on the accuracy of the inference result.

11. The receiving entity of claim 8, wherein The determination section determines whether additional input data is required based on a delay incurred until an inference result is obtained.

12. The receiving entity of claim 8, wherein The determination section determines whether additional input data is required based on a difference in accuracy between a current inference result and an immediately preceding inference result.

13. The receiving entity of claim 8, wherein The output section outputs an inference result based on the comparison of the plurality of inference values.

14. The receiving entity of claim 8, wherein The output section sets the latest one of the plurality of inferred values ​​as an inference result.

15. The receiving entity of claim 8, wherein The receiving entity is arranged to a server device connected to a cellular network.

16. The receiving entity of claim 8, wherein The receiving entity is arranged in a distributed manner to a cellular network and a server device, the server device being connected to the cellular network.

17. A data transmission method for AI / ML applications, the data transmission method comprising: Get the original input data; generating multiple input data with reduced data volume from original input data; sequentially sending each item of input data among the plurality of items of input data; as well as Feedback information based on whether additional input data is required among the plurality of input data is received.

18. A data processing method for AI / ML applications, the data processing method comprising: sequentially receiving each item of input data among the plurality of items of input data; sequentially inputting each of the plurality of input data into the multi-input AI / ML model to sequentially obtain each of the plurality of inferred values; sequentially updating the inference result based on the plurality of inference values ​​to output an updated inference result; determining whether additional input data is required among the plurality of input data; as well as Feedback information is sent based on the determination as to whether additional input data is needed.

Images