Federated deep reinforcement learning (FDRL)-based uav(unmanned aerial vehicle) navigation optimization system and operating method for efficient heterogeneous noma (non-orthogonal multiple access) communication signal transmission

Abstract

According to various embodiments, a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission comprises a server, a plurality of UAVs, and a plurality of ground terminals receiving the heterogeneous NOMA communication signals from the plurality of UAVs. Based on a situation in which the plurality of UAVs are operating in the air, the server receives first information corresponding to operation information and environment information of at least one of the plurality of UAVs, converts the first information into second information corresponding to navigation optimization information of the plurality of UAVs through a deep neural network (DNN)-based FDRL algorithm, transmits the second information to each of the plurality of UAVs, and determines navigation information of the plurality of UAVs based on the second information. According to various embodiments, a method of operation of a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous non-orthogonal multiple access (NOMA) communication signal transmission may include: receiving first information corresponding to operation information and environment information of at least one UAV among a plurality of UAVs of the system; converting the first information into second information corresponding to navigation optimization information of a plurality of UAVs of the system through a deep neural network (DNN)-based FDRL algorithm; transmitting the second information to each of the plurality of UAVs of the system; and determining navigation information of the plurality of UAVs of the system based on the second information. Various other embodiments are also possible.

Description

Technology Field

[0001] The present invention relates to an FDRL (federated deep reinforcement learning)-based UAV (unmanned aerial vehicle) navigation optimization system and a method of operation for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission. Specifically, the invention relates to a UAV navigation optimization system and a method of operation for improving network coverage, quality, and energy efficiency based on the distance between a base station (BS) and a UAV by considering the connectivity between a base station and a plurality of UAV devices. Background Technology

[0003] In the case of existing UAV navigation optimization methods, significant computational complexity issues arise during the optimization process, such as the traveling salesman problem (TPS).

[0004] Recently, research on deep learning-based navigation optimization technology has been actively conducted to mitigate these problems, but there are limitations in achieving efficient learning and performance in heterogeneous network and 5G network environments.

[0005] Various deep reinforcement learning (DRL)-based approaches have been introduced to achieve robust energy efficiency during UAV operations in such environments.

[0006] However, since research on DRL-based navigation techniques has primarily been conducted for the purpose of solutions to a single problem, new technologies are required to achieve robust performance in a wider variety of scenarios.

[0007] The present invention proposes a navigation technology for operating multiple UAV-BSs using a federated deep reinforcement learning (FDRL) algorithm in a heterogeneous NOMA system. Prior art literature

[0009] Korean Patent Publication No. 10-2022-0156117 (2022.11.25) The problem to be solved

[0010] Accordingly, the technical problem that the present invention aims to solve is created to solve the aforementioned problem, and the invention aims to provide an FDRL (federated deep reinforcement learning) based UAV (unmanned aerial vehicle) navigation optimization system and a method of operation for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission.

[0011] In addition, the present invention proposes a navigation technology for operating multiple UAV-BSs using a federated deep reinforcement learning (FDRL) algorithm in a heterogeneous NOMA system.

[0012] The technical problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which the present invention belongs from the description below. means of solving the problem

[0014] According to various embodiments, a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission comprises a server, a plurality of UAVs, and a plurality of ground terminals receiving the heterogeneous NOMA communication signals from the plurality of UAVs. Based on a situation in which the plurality of UAVs are operating in the air, the server receives first information corresponding to operation information and environment information of at least one of the plurality of UAVs, converts the first information into second information corresponding to navigation optimization information of the plurality of UAVs through a deep neural network (DNN)-based FDRL algorithm, transmits the second information to each of the plurality of UAVs, and determines navigation information of the plurality of UAVs based on the second information.

[0015] According to various embodiments, a method of operation of a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission may include: receiving first information corresponding to operation information and environment information of at least one UAV among a plurality of UAVs of the system; converting the first information into second information corresponding to navigation optimization information of a plurality of UAVs of the system through a deep neural network (DNN)-based FDRL algorithm; transmitting the second information to each of the plurality of UAVs of the system; and determining navigation information of a plurality of UAVs of the system based on the second information. Effects of the invention

[0017] The present invention has the effect of enabling more efficient learning and faster determination of optimal points by having each configured UAV-BS agent perform learning in each 5G macro cell environment by the FDRL framework, and by having each UAV-BS agent apply information on collected experience and trajectories to the learning of a single global network.

[0018] The FDRL-based UAV-BS proposed in this invention can achieve energy efficiency with actual RE values ​​almost identical to when only GP is used, while also achieving effective performance in CNR and CS indicators.

[0019] The effects of the present invention are not limited to those described above, and potential effects expected from the technical features of the present invention will be clearly understood from the description below. Brief explanation of the drawing

[0021] FIG. 1 illustrates a block diagram of a system according to various embodiments of the present invention. FIG. 2 is a diagram illustrating a multi-UAV supported heterogeneous 5G NOMA system scenario in the present invention according to various embodiments. FIG. 3 is a diagram illustrating the parameters of various formulas used in the present invention according to various embodiments. FIG. 4 is a flowchart illustrating the operation process of the system of the present invention according to various embodiments. FIG. 5 is a diagram showing the structure of a DNN model for UAV-BS navigation proposed in the present invention according to various embodiments. Specific details for implementing the invention

[0022] Hereinafter, various embodiments of this document are described with reference to the accompanying drawings. The embodiments and the terms used therein are not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, and / or substitutions of said embodiments. In relation to the description of the drawings, similar reference numerals may be used for similar components. A singular expression may include a plural expression unless the context clearly indicates otherwise. In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of items listed together. Expressions such as "first," "second," "first," or "second" may modify said components regardless of order or importance and are used only to distinguish one component from another and do not limit said components. When it is mentioned that a certain (e.g., 1st) component is "(functionally or telecommunicationally) connected" or "connected" to another (e.g., 2nd) component, said certain component may be directly connected to said other component or connected through another component (e.g., 3rd component).

[0023] In this document, "configured to" may be used interchangeably with, depending on the context, for example, hardware- or software-wise, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to." In some cases, the expression "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "processor configured to perform A, B, and C" may mean a dedicated processor for performing the corresponding operations (e.g., an embedded processor), or a general-purpose processor capable of performing the corresponding operations by executing one or more software programs stored in a memory device (e.g., a CPU or application processor).

[0024] An electronic device according to various embodiments of the present document may include, for example, at least one of a smartphone, tablet PC, mobile phone, video phone, e-book reader, desktop PC, laptop PC, netbook computer, workstation, server, PDA, PMP (portable multimedia player), MP3 player, medical device, camera, or wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD)), a fabric or clothing integrated type (e.g., electronic clothing), a body-attached type (e.g., skin pad or tattoo), or a bio-implantable circuit. In some embodiments, the electronic device may include at least one of, for example, a television, a DVD (digital video disk) player, an audio device, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync™, Apple TV™, or Google TV™), a game console (e.g., Xbox™, PlayStation™), an electronic dictionary, an electronic key, a camcorder, or a digital photo frame.

[0025] Referring to FIG. 1, an electronic device (101) within a network environment (100) in various embodiments is described. The electronic device (101) may include a bus (110), a processor (120), a memory (130), an input / output interface (150), a display (160), and a communication interface (170). In some embodiments, the electronic device (101) may omit at least one of the components or additionally include other components. The bus (110) may include a circuit that connects the components (110-170) to each other and transmits communication (e.g., control messages or data) between the components. The processor (120) may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor (120) may, for example, perform operations or data processing regarding the control and / or communication of at least one other component of the electronic device (101).

[0026] Memory (130) may include volatile and / or non-volatile memory. Memory (130) may store instructions or data related to at least one other component of the electronic device (101), for example. According to one embodiment, memory (130) may store software and / or programs (140). Programs (140) may include, for example, a kernel (141), middleware (143), an application programming interface (API) (145), and / or application programs (or "applications") (147), etc. At least part of the kernel (141), middleware (143), or API (145) may be referred to as an operating system. The kernel (141) can control or manage system resources (e.g., bus (110), processor (120), or memory (130), etc.) used to execute operations or functions implemented in other programs (e.g., middleware (143), API (145), or application program (147)). Additionally, the kernel (141) can provide an interface that controls or manages system resources by accessing individual components of the electronic device (101) from the middleware (143), API (145), or application program (147).

[0027] Middleware (143) may act as an intermediary to enable, for example, an API (145) or an application program (147) to communicate with the kernel (141) to exchange data. Additionally, middleware (143) may process one or more work requests received from the application program (147) according to priority. For example, middleware (143) may process the one or more work requests by assigning a priority to at least one of the application programs (147) to use the system resources of the electronic device (101) (e.g., bus (110), processor (120), or memory (130), etc.). The API (145) is an interface for the application (147) to control functions provided by the kernel (141) or middleware (143), and may include at least one interface or function (e.g., command) for, for example, file control, window control, image processing, or character control. The input / output interface (150) can, for example, transmit commands or data input from a user or other external device to other component(s) of the electronic device (101), or output commands or data received from other component(s) of the electronic device (101) to the user or other external device.

[0028] The display (160) may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a micro-electromechanical system (MEMS) display, or an electronic paper display. The display (160) may display various content (e.g., text, images, videos, icons, and / or symbols, etc.) to a user, for example. The display (160) may include a touch screen and may receive touch, gesture, proximity, or hovering input using, for example, an electronic pen or a part of the user's body. The communication interface (170) may establish communication between, for example, the electronic device (101) and an external device (e.g., a first external electronic device (102), a second external electronic device (104), or a server (106)). For example, the communication interface (170) can be connected to a network (162) via wireless or wired communication to communicate with an external device (e.g., a second external electronic device (104) or a server (106)).

[0029] Wireless communication may include cellular communication using at least one of, for example, LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). According to one embodiment, wireless communication may include at least one of, for example, WiFi (wireless fidelity), Bluetooth, Bluetooth Low Energy (BLE), Zigbee, NFC (near field communication), Magnetic Secure Transmission, Radio Frequency (RF), or Body Area Network (BAN). According to one embodiment, wireless communication may include GNSS. GNSS may be, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". Wired communication may include at least one of, for example, USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).The network (162) may include at least one of a telecommunications network, for example, a computer network (e.g., LAN or WAN), the Internet, or a telephone network.

[0030] Each of the first and second external electronic devices (102, 104) may be the same or a different type of device as the electronic device (101). According to various embodiments, all or part of the operations performed on the electronic device (101) may be performed on one or more other electronic devices (e.g., electronic devices (102, 104), or a server (106). According to one embodiment, when the electronic device (101) needs to perform a function or service automatically or upon request, the electronic device (101) may request at least some of the associated functions from another device (e.g., electronic devices (102, 104), or a server (106)) instead of performing the function or service itself or additionally. The other electronic device (e.g., electronic devices (102, 104), or a server (106)) may perform the requested function or additional functions and transmit the result to the electronic device (101). The electronic device (101) may provide the requested function or service by processing the received result as is or additionally. For this purpose, for example, cloud computing, distributed computing, or client-server computing technologies may be used.

[0031] FIG. 2 is a diagram illustrating a multi-UAV supported heterogeneous 5G NOMA system scenario in the present invention according to various embodiments.

[0032] FIG. 3 is a diagram illustrating the parameters of various formulas used in the present invention according to various embodiments.

[0033] FIG. 4 is a flowchart illustrating the operation process of the system of the present invention according to various embodiments.

[0034] FIG. 5 is a diagram showing the structure of a DNN model for UAV-BS navigation proposed in the present invention according to various embodiments.

[0036] Before describing the detailed operation, the present invention intends to describe a method for efficient navigation of a UAV (220) for efficient heterogeneous NOMA communication between a server (e.g., server (106) of FIG. 1) acting as a base station (210) with reference to FIG. 2, a plurality of UAVs (220), and a plurality of ground terminals (230) receiving heterogeneous NOMA communication signals from the plurality of UAVs (220). Definitions of terms to be described below may be indicated with reference to FIG. 3, and among the terms used in the present invention, UAV-BS represents a UAV acting as a base station; therefore, in this specification, UAV and UAV-BS will be treated as the same term. Furthermore, all operations and experimental processes to be described below may be described in a situation where at least one UAV (220) is flying in the air and performing heterogeneous NOMA communication with a base station (210) and a ground terminal (230). Furthermore, the operation process described below is merely a process for a single cycle, and communication can be performed by periodically repeating these operation processes. However, this is merely an example, and the same can be applied if there are other applicable communication situations. Additionally, since heterogeneous NOMA communication is a method currently in use, a detailed description is omitted in this invention.

[0038] In operation 401, according to various embodiments, a server (106) of a system (e.g., electronic device (101) of FIG. 1) may receive first information corresponding to operation information and environment information of at least one of a plurality of UAVs.

[0039] According to one embodiment, the operation information includes the movement trajectory and average movement speed of the at least one UAV, and the environment information may include temperature, humidity, wind direction, wind speed, and weather information around the at least one UAV.

[0040] For example, as shown in FIG. 2, in a situation where multiple UAVs (220) are flying in the air and at least one of them is collecting operational information and environmental information, a base station (210) can collect the information in real time. The operational information described in the present invention may include all information regarding the operation of the UAV, and the environmental information may include all information regarding the surrounding environment in which the UAV is flying.

[0042] In operation 403, according to various embodiments, the server (106) of the system (e.g., the electronic device (101) of FIG. 1) can convert the first information into second information corresponding to the navigation optimization information of the plurality of UAVs through a deep neural network (DNN)-based FDRL algorithm. For example, when the base station (210) receives various information (e.g., the first information of operation 401) from the plurality of UAVs (220) in operation 401, since the values ​​contained in this information may differ, the information can be learned through a deep neural network (DNN)-based FDRL algorithm to unify this information for easier communication. The deep learning model based on the DNN mentioned here may include a deep neural network (DNN). For example, artificial intelligence models include, but are not limited to, CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or Deep Q-Networks. Additionally, the first deep learning model and the second deep learning model may be composed of DNN-based deep learning models trained to receive at least one text input and output a motion-related text token. As a specific example, the first deep learning model may be composed of a DNN-based FFN (feed-forward-network), backpropagation, and RNN (recurrent neural network).FFN is a neural network that proceeds in a unidirectional manner from the input layer to the hidden layer and from the hidden layer to the output layer; while it is the simplest and most convenient, it has the limitation that it cannot adjust the error weights. Backpropagation returns the output value from the output layer to the input layer to reduce the error in the result; it is a frequently used DNN modeling method because it effectively inputs weight deviations and offers relatively high learning speeds. RNN generates output nodes through intersections with hidden nodes via the context unit, but since it does not affect the input and output layers, it can be used in a manner similar to a timeline. Furthermore, the first and second deep learning models apply the structure of CNN, and examples regarding this are not limited.

[0043] According to one embodiment, the FDRL algorithm operates based on a deep Q-learning (DQL) algorithm, with state information, behavior information, and compensation information. The state information includes the current location coordinates of the plurality of ground terminals, the behavior information includes the current distance, azimuth, and inclination angle of the plurality of UAVs, and the compensation information includes the channel-per-noise ratio (CNR), coverage score (CS), and residual energy (RE) of the plurality of ground terminals. The RE can optimize a compensation function corresponding to the following [Equation 1] based on the channel-per-noise ratio (CNR), coverage score (CS), and residual energy (RE) of the plurality of ground terminals.

[0044] [Mathematical Formula 1]

[0045]

[0046] Here, is the above compensation function at time t, is the coverage score of the nth UAV among the plurality of UAVs at time t, low energy threshold of the above plurality of UAVs, is the channel-to-noise ratio from the at least one UAV to the nth ground terminal among the plurality of ground terminals at time t, can represent the threshold value of the minimum channel-to-noise ratio for each of the above plurality of ground terminals.

[0047] In addition, the coverage score of the nth UAV among the plurality of UAVs at time t Specifically, it can be expressed by the following [Equation 2], and the low energy threshold of the plurality of UAVs. It can be expressed by the following [Mathematical Equation 3], and such and To maximize [Equation 4], a concave function for the nth UAV-BS can be defined as shown below. The parameters of [Equation 2], [Equation 3], and [Equation 4] can be verified by referring to Fig. 3 and [Equation 5] and [Equation 6] below.

[0048] [Mathematical Formula 2]

[0049]

[0050] [Mathematical Formula 3]

[0051]

[0052] [Mathematical Formula 4]

[0053]

[0054] Here, Indicates the presence or absence of surrounding obstacles when the UAV is flying, and can have a value of 1 if there are obstacles and 0 if there are no obstacles.

[0055] As a result, the basic DNN neural network model in the present invention can be designed as shown in FIG. 5. Through this structure, a plurality of UAVs (220) can perform learning in their respective 5G macro cell environments by means of a DNN-based FDRL model, and information collected from each UAV can be applied to the learning of a single global network (in the present invention, a base station (210)).

[0056] [Mathematical Formula 5]

[0057]

[0058] [Mathematical Formula 6]

[0059]

[0060] Here, represents a value between 0 and 1, and It can represent the number of time steps in which the n-th UAV covers the m-th ground terminal.

[0062] In operation 405, according to various embodiments, the server (106) of the system (e.g., the electronic device (101) of FIG. 1) can convert the first information into second information corresponding to navigation optimization information of the plurality of UAVs through a deep neural network (DNN)-based FDRL algorithm. Specifically, the second information may be information that is converted in a direction that optimizes the reward function described in operation 403 after learning the information collected from the aforementioned plurality of UAVs (220) through a deep neural network (DNN)-based FDRL algorithm, and may be information used for the efficient operation of the plurality of UAVs (220).

[0063] In operation 407, according to various embodiments, the server (106) of the system (e.g., the electronic device (101) of FIG. 1) can determine the navigation information of the plurality of UAVs based on the second information.

[0064] According to one embodiment, the server (106) of the system (e.g., the electronic device (101) of FIG. 1) can receive the heterogeneous NOMA communication signal from the plurality of UAVs based on the navigation information of the plurality of UAVs. According to various embodiments, a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission comprises a server, a plurality of UAVs, and a plurality of ground terminals receiving the heterogeneous NOMA communication signals from the plurality of UAVs. Based on the situation in which the plurality of UAVs are operating in the air, the server receives first information corresponding to operation information and environment information of at least one of the plurality of UAVs, converts the first information into second information corresponding to navigation optimization information of the plurality of UAVs through a deep neural network (DNN)-based FDRL algorithm, transmits the second information to each of the plurality of UAVs, and can determine navigation information of the plurality of UAVs based on the second information.

[0065] According to various embodiments, the FDRL algorithm operates based on a deep Q-learning (DQL) algorithm, with state information, behavior information, and compensation information. The state information includes the current location coordinates of the plurality of ground terminals, the behavior information includes the current distance, azimuth, and inclination angle of the plurality of UAVs, and the compensation information includes the channel-per-noise ratio (CNR), coverage score (CS), and residual energy (RE) of the plurality of ground terminals. The RE can optimize a compensation function corresponding to the following [Equation 1] based on the channel-per-noise ratio (CNR), coverage score (CS), and residual energy (RE) of the plurality of ground terminals.

[0066] [Mathematical Formula 1]

[0067]

[0068] Here, is the above compensation function at time t, is the coverage score of the nth UAV among the plurality of UAVs at time t, low energy threshold of the above plurality of UAVs, is the channel-to-noise ratio from the at least one UAV to the nth ground terminal among the plurality of ground terminals at time t, can represent the threshold value of the minimum channel-to-noise ratio for each of the above plurality of ground terminals.

[0069] According to various embodiments, based on the navigation information of the plurality of UAVs, the server, the plurality of ground terminals can receive the heterogeneous NOMA communication signal from the plurality of UAVs.

[0070] According to various embodiments, a method of operation of a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous NOMA (non-orthogonal multiple access) communication signal transmission may include: receiving first information corresponding to operation information and environment information of at least one UAV among a plurality of UAVs of the system; converting the first information into second information corresponding to navigation optimization information of a plurality of UAVs of the system through a deep neural network (DNN)-based FDRL algorithm; transmitting the second information to each of the plurality of UAVs of the system; and determining navigation information of a plurality of UAVs of the system based on the second information.

[0073] The terms “module” or “part” as used in this document include a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. “Module” or “part” may be a component formed integrally or a minimum unit or part thereof that performs one or more functions. “Module” or “part” may be implemented mechanically or electronically and may include, for example, an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or programmable logic device known or to be developed that performs certain operations, and may be executed by a processor (120). At least part of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments may be implemented as instructions stored in a computer-readable storage medium (e.g., memory (130)) in the form of a program module. When the above instruction is executed by a processor (e.g., processor (120)), the processor may perform a function corresponding to the above instruction. Computer-readable recording media may include a hard disk, a floppy disk, a magnetic medium (e.g., magnetic tape), an optical recording medium (e.g., CD-ROM, DVD, magneto-optical medium (e.g., floptical disk), built-in memory, etc. Instructions may include code generated by a compiler or code that can be executed by an interpreter. A module or program module according to various embodiments may include at least one of the aforementioned components, some of which may be omitted, or additionally include other components. Operations performed by a module, program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0074] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of this disclosure. Accordingly, the scope of this disclosure should be interpreted to include all modifications or various other embodiments based on the technical concept of this disclosure.

Claims

Claim 1 A federated deep reinforcement learning (FDRL)-based multi-UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous non-orthogonal multiple access (NOMA) communication signal transmission comprises a server, a plurality of UAVs, and a plurality of ground terminals receiving heterogeneous NOMA communication signals from the plurality of UAVs, and based on a situation in which the plurality of UAVs are operating in the air, the server receives first information corresponding to operation information and environmental information of the at least one UAV from at least one UAV among the plurality of UAVs, converts the first information into second information corresponding to navigation optimization information of the plurality of UAVs through a deep neural network (DNN)-based FDRL algorithm, transmits the second information to each of the plurality of UAVs, determines navigation information of the plurality of UAVs based on the second information, and the navigation information is control data determined to control the communication signal transmission efficiency and the flight power of the plurality of UAVs in combination, and the FDRL The algorithm operates based on the DQL (deep Q-learning) algorithm, using state information, behavior information, and reward information. The state information includes the current location coordinates of the plurality of ground terminals. The behavior information includes the current distance, azimuth, and inclination angle of the plurality of UAVs. The reward information includes the channel-per-noise ratio (CNR) and coverage score (CS) of the plurality of ground terminals, and the current residual energy (RE) of the plurality of UAVs. The RE includes the channel-per-noise ratio (CNR) and coverage score (CS) of the plurality of ground terminals.A system that optimizes a compensation function corresponding to the following [Equation 1] based on the current remaining energy (RE, residual energy) of the plurality of UAVs above. [Equation 1] Here, is the above compensation function at time t, is the coverage score of the nth UAV among the plurality of UAVs at time t, low energy threshold of the above plurality of UAVs, is the channel-to-noise ratio from the at least one UAV to the nth ground terminal among the plurality of ground terminals at time t, represents the threshold value of the minimum channel-to-noise ratio for each of the above plurality of ground terminals. Claim 2 A system according to claim 1, wherein the operation information includes the movement trajectory and average movement speed of the at least one UAV, and the environment information includes temperature, humidity, wind direction, wind speed, and weather information around the at least one UAV. Claim 3 delete Claim 4 A system according to claim 1, wherein the server is characterized in that the plurality of ground terminals receive the heterogeneous NOMA communication signal from the plurality of UAVs flying according to the navigation information determined by the server. Claim 5 A method of operation of a federated deep reinforcement learning (FDRL)-based multiple UAV (unmanned aerial vehicle) navigation optimization system for efficient heterogeneous non-orthogonal multiple access (NOMA) communication signal transmission, comprising: a step in which a server of the system receives first information corresponding to operation information and environment information of at least one UAV among a plurality of UAVs of the system from at least one UAV; a step in which the server converts the first information into second information corresponding to navigation optimization information of the plurality of UAVs of the system through a deep neural network (DNN)-based FDRL algorithm; a step in which the server transmits the second information to each of the plurality of UAVs of the system; and a step in which the server determines navigation information of the plurality of UAVs of the system based on the second information.A method of operation of a system comprising: [Equation 1]; wherein the navigation information is control data determined to control the communication signal transmission efficiency and the flight power of the plurality of UAVs by combining them; wherein the FDRL algorithm operates based on a DQL (deep Q-learning) algorithm by means of state information, behavior information, and compensation information; wherein the state information includes the current location coordinates of the plurality of ground terminals; wherein the behavior information includes the current distance, azimuth, and inclination angle of the plurality of UAVs; wherein the compensation information includes the channel-per-noise ratio (CNR) of the plurality of ground terminals, the coverage score (CS), and the current residual energy (RE) of the plurality of UAVs; and wherein the RE optimizes a compensation function corresponding to the following [Equation 1] based on the channel-per-noise ratio (CNR) of the plurality of ground terminals, the coverage score (CS), and the current residual energy (RE) of the plurality of UAVs. Here, is the above compensation function at time t, is the coverage score of the nth UAV among the plurality of UAVs at time t, low energy threshold of the above plurality of UAVs, is the channel-to-noise ratio from the at least one UAV to the nth ground terminal among the plurality of ground terminals at time t, represents the threshold value of the minimum channel-to-noise ratio for each of the above plurality of ground terminals.

Images