GPU free-edge nodes and inference systems

Abstract

Traditional AI systems use GPUs, but they consume a lot of power and place a heavy operational burden on edge devices and data centers. Furthermore, their reliance on the cloud leads to communication delays and security risks, posing problems in fields requiring real-time processing, such as autonomous driving and medical diagnosis. While FPGAs and ASICs exist as alternatives to GPUs, they face challenges in terms of implementation costs and flexibility. [Solution] This invention provides a power-saving AI system that performs AI inference and learning without using a GPU. By utilizing knowledge distillation, quantization, and sparsification, and optimizing computation using CPUs, FPGAs, and ASICs, it achieves low-power and high-precision AI inference. Furthermore, it performs real-time inference on edge devices, reducing cloud dependency. In addition, it establishes a highly efficient AI infrastructure by building a distributed AI network and dynamically adjusting the load.

Description

Technical Field

[0001] The present invention belongs to the technical field related to computational processing of artificial intelligence (AI), and particularly relates to a power-saving AI system that utilizes alternative computing devices such as Central Processing Unit (CPU), Field-Programmable Gate Array (FPGA), and Application-Specific Integrated Circuit (ASIC) without using a Graphics Processing Unit (GPU).

[0002] Conventional AI computation is mainly based on parallel processing utilizing GPUs. However, GPUs consume large amounts of power and have high operating costs, so they are not particularly suitable for AI processing in edge devices and low-power environments. The present invention provides a technology that does not require a GPU, maximizes energy efficiency, and optimizes the inference and learning processes of AI.

[0003] The purpose of the present invention is to achieve performance equal to or better than conventional GPU-based AI computation with low power consumption by combining AI model lightweighting (knowledge distillation, quantization, sparsification), computing architecture optimization, and utilization of dedicated hardware (FPGA, ASIC).

[0004] Furthermore, the present invention **includes a system that improves computational efficiency by dynamically managing a plurality of computing nodes and optimally allocating computational resources of each node in a distributed AI network.** This technology enables power-saving and high-efficiency AI computation in various computing environments such as cloud environments, on-premises environments, and edge device environments.

[0005] The technology of the present invention is particularly applicable to fields where energy efficiency is emphasized (such as edge computing, IoT devices, smart phones, smart cities, industrial robotics, autonomous vehicles, and medical equipment), reduces the introduction cost of AI, and enables the construction of sustainable AI infrastructure.

Background Art

[0006] In recent years, the development of artificial intelligence (AI) has led to the increasing adoption of automated systems utilizing machine learning and deep learning. Training and inference of AI models requires enormous computational processing, and GPUs (Graphics Processing Units) are primarily used as high-performance computing resources. GPUs have an architecture suitable for parallel processing with numerous cores, enabling significant acceleration in AI computations.

[0007] However, AI computing using GPUs faces the challenge of extremely high power consumption. For example, training a large-scale AI model can consume hundreds of watts of power on a single GPU, and the entire data center can generate megawatts of power. This leads to increased operating costs and a greater environmental burden, highlighting the need for sustainable AI computing systems.

[0008] Furthermore, it is difficult to equip edge devices (smartphones, IoT devices, industrial robots, autonomous vehicles, etc.) with high-performance GPUs, making efficient AI inference processing a challenge. Generally, when running AI on edge devices, the method used involves sending data to a GPU server in the cloud to perform inference, but this method causes problems such as communication delays, security risks, and increased communication costs.

[0009] Conventional low-power AI techniques include knowledge distillation, model quantization, and pruning. Knowledge distillation is a method of transferring the learning results of a large teacher model to a smaller student model to reduce computational costs. Quantization is a technique that reduces computational load by converting floating-point arithmetic to low-precision integer arithmetic. Pruning is a method of reducing model size and computational costs by removing unnecessary neurons and parameters.

[0010] Furthermore, AI accelerators using FPGAs (Field-Programmable Gate Arrays) and ASICs (Application-Specific Integrated Circuits) are being developed as power-saving technologies at the hardware level. FPGAs are semiconductor devices whose hardware logic can be flexibly reconfigured, allowing for the design of circuits optimized for specific AI computation processes. On the other hand, ASICs are specialized semiconductors designed specifically for particular applications, enabling high computational performance while minimizing power consumption.

[0011] While research into GPU-independent AI computing architectures is progressing with conventional technologies, a technique to optimize the balance between power consumption, computing speed, and cost has yet to be established. For example, FPGAs and ASICs have low power consumption, but they have challenges in programmability (flexibility), making them difficult to adapt to changes and updates of AI models. On the other hand, lightweight AI models that have undergone knowledge distillation or quantization may not perform well in fields that require high-precision AI inference (such as medical diagnosis and autonomous driving).

[0012] Furthermore, AI inference processing often requires real-time processing. In particular, applications such as autonomous driving, medical diagnosis, financial transactions, and industrial robotics demand decision-making in milliseconds, necessitating technologies to optimize AI computations without using GPUs. However, conventional technologies have not yet established a sufficiently practical AI computing architecture as a GPU alternative, thus requiring a new approach.

[0013] To address these challenges, this invention provides a new AI computing architecture that enables power-efficient AI inference and training without the use of GPUs. This invention combines AI model lightweighting techniques such as knowledge distillation, quantization, and sparsification with power-efficient computing devices such as FPGAs and ASICs to achieve high-precision AI inference with low power consumption. Furthermore, it utilizes a distributed AI network to optimally distribute the computing load, thereby constructing an efficient AI computing infrastructure. [Prior art documents] [Patent Documents]

[0014] [Patent Document 1] Patent No. 7054321 [Patent Document 2] Patent No. 6892104 [Overview of the project] [Problems that the invention aims to solve]

[0015] Traditional AI systems commonly utilize parallel computing with GPUs (Graphics Processing Units), contributing to improved processing power for large-scale machine learning and AI inference. However, AI computation using GPUs faces the following significant challenges.

[0016] (1) High power consumption and increased operating costs AI computing using GPUs achieves high processing power, but it also consumes a very large amount of power. For example, in data centers, a GPU cluster consumes hundreds to thousands of watts of power, which in turn increases cooling costs. Such a high-power consumption environment makes it difficult to build a sustainable AI computing infrastructure, and there is a need to reduce energy costs.

[0017] (2) Difficulty in using edge devices and low-power environments Edge devices such as smartphones, IoT devices, industrial robots, and autonomous vehicles are difficult to equip with high-performance GPUs due to power consumption and hardware limitations. Therefore, when performing AI inference on these devices, it is necessary to send data to the cloud and perform AI inference using the GPU. However, this method leads to problems such as communication delays, security risks, and increased communication costs.

[0018] (3) Challenges of GPU alternative technologies While some research is being conducted on low-power AI computing technologies utilizing FPGAs (Field-Programmable Gate Arrays) and ASICs (Application-Specific Integrated Circuits), these technologies lack flexibility in program modification and are optimized for specific applications, making them difficult to apply to a wide range of AI computing tasks. Furthermore, while AI model optimization techniques (knowledge distillation, quantization, sparsification, etc.) reduce computational costs, a technology that achieves power savings while maintaining high accuracy has not yet been established.

[0019] (4) The problem of balancing the real-time capabilities and computational efficiency of AI models Applications requiring real-time AI inference, such as autonomous driving, medical diagnosis, financial transactions, and industrial equipment control, demand low-latency and high-precision inference. However, while GPUs can provide high computing power, they increase power consumption and cost. On the other hand, conventional CPU-based AI inference consumes less power but is inferior to GPUs in terms of processing speed. This invention provides a new method for maximizing the computational efficiency of AI inference and learning without using a GPU. [Means for solving the problem]

[0020] To solve the above problems, the present invention provides a power-saving AI system that can perform AI inference and training with high efficiency without using a GPU. The technology of the present invention combines AI model lightweighting technology, an optimized computing architecture, and the use of power-saving hardware to achieve performance equivalent to or better than conventional GPU-based AI computing with lower power consumption.

[0021] (1) Utilization of AI model optimization technology This invention significantly reduces the computational complexity of AI models by combining techniques such as knowledge distillation, quantization, and pruning, thereby achieving power-efficient and high-speed AI inference. Specifically, the following techniques are employed. Knowledge distillation: The learning results of a large-scale teacher model are inherited by a small student model, maintaining high accuracy while reducing computation cost. Quantization: Converts the numerical operations of an AI model to low precision (e.g., 16-bit or 8-bit) and reduces the load on the arithmetic circuit. Pruning: Removes unnecessary connections in the neural network to reduce computational load.

[0022] (2) Adoption of Power-Saving Computing Architectures In the present invention, by combining CPU-based parallel processing and dedicated computing units utilizing FPGA and ASIC, GPU-independent AI computation is enabled. Optimal utilization of CPU: Optimizes multithreaded processing to achieve high-performance AI inference without a GPU. Utilization of FPGA: Implements custom logic specialized for AI inference on FPGA to optimize energy consumption. Utilization of ASIC: Utilizes a low-power processor specialized for specific AI operations to optimize for edge devices.

[0023] (3) Computational Load Optimization via Decentralized AI Networks The present invention includes a management system for decentralized AI networks, which optimally distributes computational load among nodes of cloud, on-premises, and edge devices. Dynamic scheduling of AI tasks: Allocates optimal computational resources according to the computing capacity and power usage status of each node. Optimization of AI inference on edge devices: Performs a part of AI inference on edge devices and cooperates with cloud AI only when necessary. Real-time optimization: Monitors the load status of the AI model in real time and selects the optimal computation method.

[0024] (4) Specific applications of power-saving AI systems The technology of the present invention can be applied in the following fields: Autonomous driving: Real-time road condition analysis is performed without using a GPU, and the vehicle's operation is controlled. Reduce energy consumption. Medical Diagnosis: Building medical image analysis AI utilizing FPGAs and ASICs to speed up diagnosis Improved accuracy. Smart cities: Utilizing low-power AI cameras and IoT devices to enhance urban life A monitoring system was built.

[0025] As described above, the present invention solves the challenges of energy efficiency, computation speed, and operating costs in AI computing, and builds a sustainable AI infrastructure. This enables the establishment of a new AI architecture that does not depend on GPUs, and expands the scope of application of AI technology. [Effects of the Invention]

[0026] According to the present invention, it is possible to achieve high-precision AI inference and learning while significantly reducing power consumption compared to conventional GPU-based AI computation. Furthermore, the technology of the present invention is applicable to edge devices and cloud environments, and can simultaneously achieve optimization of computing resources and reduction of operating costs.

[0027] (1) Reduce power consumption and enable sustainable AI computing. This invention enables AI inference and training without the use of a GPU, thereby significantly reducing power consumption in data centers and edge devices. Specifically, the following effects can be obtained: Reduced power consumption: Up to 70% power savings compared to traditional GPU-based computing. Possible. Reduced heat generation: Lower power consumption reduces the burden on the cooling system, and data The operating costs of the center will be reduced. Reducing environmental impact: CO2 emissions can be reduced, enabling the construction of a sustainable AI infrastructure.

[0028] (2) Optimization of real-time AI inference on edge devices This invention enables high-precision AI inference to be performed on edge devices (smartphones, IoT devices, industrial equipment, etc.) without the need for a GPU, thereby simultaneously achieving improved real-time processing performance and reduced communication costs. Eliminating communication latency: By performing inference processing locally without relying on the cloud, Improved real-time performance. Improved security: AI model processing can be completed at the edge, eliminating the need for cloud services. Reduces the risk of data transfer to [location]. Extends battery life for edge devices: Because it does not use the GPU, it consumes less power. It allows for long-term operation with low power consumption.

[0029] (3) Faster AI inference and improved energy efficiency This invention utilizes low-power hardware such as CPUs, FPGAs, and ASICs to enable high-speed AI inference and training. This results in faster performance than conventional CPU-based AI inference and achieves performance comparable to that of a GPU. Computational load is reduced through lightweighting techniques (knowledge distillation, quantization, and sparsification). By utilizing hardware acceleration (FPGA / ASIC), energy efficiency can be improved. Maximize. Optimizes CPU-based parallel processing to achieve high-performance AI inference without a GPU.

[0030] (4) Optimization of computing resources using a distributed AI network This invention incorporates a management function for a distributed AI network and improves overall computational efficiency by optimally allocating AI tasks. Dynamic scheduling: AI inference and training tasks in the cloud, on-premises, and Optimal distribution among edge devices. Improved throughput through load balancing: Appropriately distribute high-load processing to improve computational efficiency. Maximize it. Task assignment based on power usage: Prioritize devices with low power consumption. By utilizing this, we can achieve energy optimization.

[0031] (5) Realization of a low-cost AI computing environment Because this invention does not use a GPU, the cost of building and operating an AI computing environment can be significantly reduced. Hardware cost reduction: Reduce the cost of deploying and maintaining GPUs, making it more cost-effective. We provide a high-performance AI computing infrastructure. Reducing cloud usage costs: By enhancing AI processing on edge devices, Reduce reliance on the cloud and lower data transfer costs. Scalable deployment: From small edge devices to large data systems It can be implemented in a wide range of environments, including centers.

[0032] (6) Expanding the scope of application of AI technology The technology of the present invention is particularly effective in the following fields and can expand the scope of application of AI. Autonomous driving: Real-time image recognition and decision-making with low power consumption. Medical Diagnosis: Realizing low-cost, high-precision diagnostic support AI utilizing FPGAs and ASICs. Industrial robots: Optimization of AI inference in battery-powered robots. Smart cities: Utilizing low-power AI cameras and IoT sensors to collect urban data Enhance analysis.

[0033] This invention solves the problems of conventional GPU-dependent AI computation and enables the construction of a sustainable and highly efficient AI infrastructure. This will accelerate the spread of AI technology and contribute to the realization of an energy-efficient AI society. [Brief explanation of the drawing]

[0034] [Figure 1] This is an overall system configuration diagram of the GPU-free AI system according to the present invention.

[0035] [Figure 2] This is a flowchart of the AI ​​inference and learning process in the GPU-free AI system of the present invention. [Modes for carrying out the invention]

[0036] The GPU-free AI system of the present invention is a system that performs AI inference and training without using a GPU, achieving highly efficient AI computation while reducing power consumption. By combining lightweight AI models, an optimized computing architecture, the use of power-saving hardware, and load balancing through a distributed AI network, the present invention achieves performance equivalent to or better than conventional GPU-based AI computation with low power consumption. 1. Methods for optimizing AI models

[0037] This invention aims to reduce the computational load by optimizing AI models, thereby enabling highly accurate inference. Specifically, it combines the following technologies. Knowledge distillation: The learning results of a large-scale teacher model are processed into a small-scale model. By inheriting the student model, computational costs are reduced while maintaining high accuracy. Quantization: Converts an operation to low precision (16-bit or 8-bit), Reduces the load on the arithmetic circuit. Sparsification (Pruning): Removes unnecessary neural network connections and reduces the number of connections. Reduce the computational load.

[0038] By applying these optimization techniques, it is possible to reduce the consumption of computing resources and achieve power savings while maintaining the same inference accuracy as conventional GPU-based AI models. 2. Low-power AI computing architecture

[0039] This invention employs a power-saving computing architecture for optimally processing AI inference and training computations without a GPU. The following techniques are utilized: CPU-based optimization: Multithreaded processing and SIMD (Single Instruction, By utilizing Multiple Data technology, it enables high-speed AI inference even without a GPU. Utilization of FPGA (Field-Programmable Gate Array): Customization specialized for AI inference We designed a unique circuit to achieve processing power equivalent to a GPU with low power consumption. Introduction of ASICs (Application-Specific Integrated Circuits): For specific applications By utilizing a specially designed chip, it enables AI computation with ultra-low power consumption.

[0040] This invention provides an AI processing environment that combines these different computing architectures, allowing for the selection of the optimal computing means according to the system's application and power constraints. 3. Building a Decentralized AI Network

[0041] This invention achieves computational load distribution by optimally managing AI inference and learning within a distributed network. Dynamic scheduling: Real-time information on the computing power and power usage of each node. Analyze the data and assign the optimal processing method. Integration with edge devices: Run AI inference on edge devices and send it to the cloud. Reduce dependence. Real-time optimization: Monitors AI load and automatically allocates optimal computing resources. Select this.

[0042] This enables efficient AI computing across data centers, on-premises environments, and edge devices. 4 Examples

[0043] Examples of the application of the technology of the present invention are shown below. (1) Applications to autonomous driving The AI ​​system of this invention is applied to autonomous vehicles to perform real-time road condition analysis without a GPU. This enables high-speed AI decision-making while reducing power consumption. (2) Application to medical diagnosis By introducing FPGAs into medical image analysis AI, we achieve low power consumption and high-precision diagnosis. We optimize AI systems within hospital data centers to improve energy efficiency. (3) Applications to smart cities By applying the AI ​​technology of this invention to urban surveillance systems and utilizing low-power cameras and IoT devices, large-scale data processing can be performed with minimal power consumption.

[0044] As described above, the present invention can optimize AI computation without using a GPU, reducing power consumption while achieving high-precision inference and learning. This enables the construction of an energy-efficient AI infrastructure and contributes to the development of a sustainable AI society. [Examples]

[0045] The following describes in detail an embodiment of the GPU-free AI system of the present invention. The present invention aims to perform AI inference and training with low power consumption, and in particular provides a method for operating an AI model without using a GPU. The following embodiment describes the specific system configuration, applied technology, and its effects. Example 1: Application to an autonomous driving system

[0046] The AI ​​system of the present invention can be applied to the control system of an autonomous vehicle. Conventional autonomous driving systems commonly use high-performance GPUs for image recognition and processing of sensor data. However, GPUs consume a lot of power, which affects the battery life of electric vehicles. In this invention, low power consumption of the autonomous driving AI is achieved by introducing real-time image processing technology using a CPU, FPGA, or ASIC without using a GPU.

[0047] The operation flow of this system is as follows: 1. Cameras and LiDAR (Light Detection and Ranging) mounted on the vehicle scan the surrounding environment in real time. 1. Image data and sensor data are preprocessed using an FPGA or dedicated ASIC, Remove Iz. 2. The AI ​​model is executed using CPU-optimized parallel processing technology to detect pedestrians and obstacles. Detects harmful substances. 3. The predicted travel route is transmitted to the control unit, and the vehicle's direction of travel is determined.

[0048] This system enables up to 50% power savings compared to conventional GPU-based processing, extending the driving range of electric vehicles. Furthermore, by eliminating the use of GPUs, it contributes to cost reduction and system miniaturization. Example 2: Application to a medical diagnostic support system

[0049] The AI ​​system of the present invention can be applied to image analysis in medical diagnostic support. Conventionally, deep learning models using GPUs have been used for the analysis of CT scans and MRI images. However, medical devices equipped with high-performance GPUs are expensive and have high operating costs, making them difficult for small and medium-sized medical institutions to implement.

[0050] This invention develops a low-power diagnostic support system incorporating an FPGA or a dedicated AI accelerator, enabling high-precision medical image analysis without relying on a GPU.

[0051] The system's operation flow is as follows: 1. Acquire CT scan or MRI images and perform real-time preprocessing (noise reduction, contrast correction) of the image data using an FPGA. 2. Using an optimized quantization model on a CPU or ASIC for the AI ​​model The system analyzes the data to detect tumors and abnormal areas. 3. Provide the diagnostic results to the physician and use them as supplementary diagnostic support information.

[0052] By applying this invention, it is possible to reduce power consumption by 60% while maintaining the same level of accuracy as conventional AI diagnostic systems equipped with GPUs. Furthermore, the cost of introducing the system is significantly reduced, enabling AI-powered diagnostic support even in small and medium-sized medical institutions. Example 3: Application to surveillance camera systems in smart cities

[0053] The AI ​​system of the present invention can also be applied to surveillance cameras and traffic management systems in smart cities. Conventional surveillance camera systems perform AI analysis using GPUs on cloud servers, resulting in large amounts of data communication and processing costs.

[0054] This invention enables AI processing on edge devices and allows for the detection of abnormal behavior in real time.

[0055] The processing flow of this system is as follows: 1. The surveillance camera acquires video data, and the FPGA or a dedicated processor performs video compression and noise reduction. 2. Using an optimized sparsified network on the CPU for the AI ​​model It executes and detects suspicious individuals and abnormal behavior. 3. Send only the necessary data to the cloud server and notify the administrator with an alert.

[0056] By utilizing this invention, the amount of data transferred to the cloud can be reduced by more than 70%, simultaneously achieving reduced communication costs and protection of privacy. Furthermore, because it does not use a GPU, the overall operating costs of the surveillance camera system can be reduced, enabling the deployment of AI surveillance systems in more cities. Example 4: AI optimization of industrial robots

[0057] The AI ​​system of the present invention can also be applied to optimizing the motion of industrial robots. In factory automation lines, it is common to use AI to optimize the motion of robot arms, but AI systems using GPUs are expensive and consume a lot of power.

[0058] This invention achieves real-time operational optimization by introducing an AI inference engine that utilizes FPGAs and ASICs, without using a GPU.

[0059] The flow of this system is as follows: 1. Acquire sensor data from the robot arm and perform motion optimization calculations using an FPGA. implement. 2. Execute the AI ​​model on the CPU to determine the optimal control pattern in real time. do. 3. Update the AI's training data to continuously improve the robot's operational efficiency.

[0060] By applying this invention, power consumption can be reduced by up to 40% compared to GPU-based systems, thereby lowering factory operating costs. Furthermore, the real-time capabilities of AI inference are improved, contributing to increased productivity.

[0061] As described above, the GPU-free AI system of the present invention can be applied to a wide range of fields, including autonomous driving, medical diagnosis, smart cities, and industrial robots, enabling reduced power consumption, lower costs, and high-precision AI inference. This will promote the widespread adoption of energy-efficient AI technology and contribute to the development of a sustainable AI society. [Industrial applicability]

[0062] The GPU-free AI system of the present invention is a technology that efficiently performs AI inference and training without using a GPU, and has applications in a wide range of industrial fields. In particular, it has high utility in the following industrial fields in terms of reducing energy consumption, optimizing costs, and improving real-time processing. 1. Data center and cloud AI services

[0063] Current data centers are deploying GPUs on a large scale for AI inference and machine learning, but GPUs consume a lot of power, leading to increased server operating costs. By introducing the technology of the present invention, low-power AI processing using CPUs, FPGAs, and ASICs becomes possible, improving the energy efficiency of data centers.

[0064] The following advantages can be expected from the application of this invention. Reduce data center power consumption by up to 50%, lowering cooling costs. We built an AI inference platform that does not rely on GPUs, reducing hardware costs. Reduce the operating costs of large-scale cloud AI services and adopt a subscription model. Promote the widespread adoption of AI services. 2. Autonomous Driving and Mobility Industry

[0065] Autonomous vehicles and advanced driver-assistance systems (ADAS) require real-time image analysis, but AI processing using GPUs consumes a lot of power, affecting the driving range of electric vehicles (EVs). By applying the technology of the present invention, AI inference can be achieved without a GPU, enabling power savings and cost reductions for in-vehicle AI systems.

[0066] The following advantages can be expected from the application of this invention. This makes it possible to reduce EV battery consumption and extend the driving range. Real-time AI processing using FPGAs and ASICs enables low-latency autonomous driving. The decision is feasible. Cost reductions will accelerate the spread of autonomous driving technology. 3. Medical and healthcare industry

[0067] In the medical field, AI-powered image diagnostic technologies (such as CT scans and MRI image analysis) are becoming widespread. However, AI systems equipped with GPUs are expensive, making them difficult for some medical institutions to implement. By utilizing the technology of the present invention, it becomes possible to perform highly accurate medical image diagnosis with low power consumption, thereby reducing costs for medical institutions and improving the accuracy of diagnoses.

[0068] The following advantages can be expected from the application of this invention. AI diagnostic systems utilizing FPGAs and ASICs reduce costs for medical institutions. We can develop low-cost AI diagnostic support systems that can be implemented even in small hospitals. By not using a GPU, the processing speed of medical data can be improved, and real-time processing is possible. Providing diagnostic support. 4. Smart City Surveillance Systems

[0069] As cities become smarter, the use of AI is expanding in surveillance camera systems and traffic management systems. However, conventional surveillance systems mainly rely on cloud processing using GPUs, which leads to increased communication costs due to the transfer of large amounts of data. By utilizing the technology of this invention, AI processing can be realized on edge devices, enabling a reduction in communication costs and enhanced real-time monitoring.

[0070] The following advantages can be expected from the application of this invention. Reducing the load on data centers by using edge AI for analyzing surveillance camera footage. Reduce. Reduce cloud dependency and cut communication costs by up to 70%. A real-time anomaly detection system can be implemented at a low cost. 5. Industrial robots and smart factories

[0071] While advanced AI technology is used to control industrial robots, GPU-based AI systems consume a lot of power, contributing to increased factory operating costs. By utilizing the technology of this invention, it becomes possible to introduce power-efficient AI inference technology and improve the efficiency of industrial robots.

[0072] The following advantages can be expected from the application of this invention. AI inference processing for industrial robots is performed without a GPU, reducing power consumption by up to 40%. This will reduce the operating costs of smart factories and lower the barrier to AI implementation. This enables the construction of energy-efficient production lines, leading to sustainable manufacturing. realization. 6. Financial and Securities Trading Systems

[0073] In the financial industry, AI is used for market analysis and risk assessment, but computational processing using GPUs is costly and can affect trading speed. By utilizing the technology of this invention, it is possible to perform high-speed analysis of market data without using GPUs and build a low-cost and power-efficient financial AI system.

[0074] The following advantages can be expected from the application of this invention. Reduce the cost of real-time market analysis and improve the profitability of financial institutions. Enhanced risk management through power-saving AI that minimizes GPU usage. The acceleration of trading algorithms contributes to improving the efficiency of the securities market.

[0075] The technology of the present invention enables high-precision inference and learning while reducing the power consumption of AI in the aforementioned industrial fields. This promotes the widespread adoption of sustainable AI technology and contributes to the construction of energy-efficient social infrastructure.

Claims

1. A low-power AI system that performs artificial intelligence (AI) inference and learning without using a GPU, comprising at least means for reducing the computational load of the AI ​​model, wherein the reduction means is means for reducing the computational load of the AI ​​model by applying one of knowledge distillation, quantization, or sparsification, and comprising computing means for performing AI inference processing, wherein the computing means performs AI inference using one of a CPU, FPGA, or ASIC.

2. A power-saving AI system according to claim 1, wherein the CPU processes AI inference at high speed using multithreading and SIMD (Single Instruction, Multiple Data) technology.

3. A power-saving AI system according to claim 1, characterized in that the FPGA configures hardware logic specialized for AI inference and performs the AI ​​inference processing without using a GPU.

4. The low-power AI system according to claim 1, wherein the ASIC comprises a circuit optimized for a specific AI computation process, and is characterized in that it performs high-speed processing while minimizing the power consumption of AI inference.

5. The low-power AI system according to claim 1, wherein the knowledge distillation transfers the learning results of a large-scale teacher model to a small-scale student model, thereby reducing the computational load.

6. The low-power AI system according to claim 1, wherein the quantization converts the computational precision of the AI ​​model into 16-bit or 8-bit integer arithmetic, thereby reducing computational load and memory usage.

7. The power-saving AI system according to claim 1, wherein the sparsification reduces computational cost by removing unnecessary connections within the neural network.

8. The low-power AI system according to claim 1, wherein the AI ​​inference process enables real-time data processing and performs high-speed processing with reduced latency.

9. A power-saving AI system according to claim 1, wherein the AI ​​system operates on an edge device and performs AI inference locally without relying on the cloud.

10. The power-saving AI system according to claim 9, characterized in that the edge device is a smartphone, an IoT device, an industrial robot, or an autonomous vehicle.

11. A power-saving AI system according to claim 1, wherein the AI ​​system records the results of AI inference on a blockchain to prevent data tampering and ensure transparency.

12. A low-power AI system according to claim 1, wherein the AI ​​system operates on a distributed network and distributes AI inference processing among multiple computing nodes.

13. The power-saving AI system according to claim 12, wherein the distributed network performs dynamic scheduling and optimally distributes computing tasks according to the load status of each node.

14. The low-power AI system according to claim 1, characterized in that the AI ​​system performs real-time video analysis and is applied to anomaly detection in surveillance cameras, traffic management, or facial recognition.

15. The power-saving AI system according to claim 1, wherein the AI ​​system performs analysis of CT scan images and MRI images in medical diagnostic support to improve diagnostic accuracy.

16. The power-saving AI system according to claim 1, wherein the AI ​​system performs real-time object detection and path prediction in an autonomous vehicle and provides driving assistance.

17. The low-power AI system according to claim 1, wherein the AI ​​system is applied to the analysis of financial market data and performs real-time market analysis, risk assessment, and trade optimization.

18. The power-saving AI system according to claim 1, wherein the AI ​​system is applied to traffic management and urban planning in a smart city and performs real-time data collection and analysis.

19. The power-saving AI system according to claim 1, wherein the AI ​​system is used to optimize the operation of an industrial robot and improve work efficiency.

20. The power-saving AI system according to claim 1, wherein the AI ​​system is applied to crop growth monitoring, pest detection, and optimization of harvest time in the agricultural field.

Images