Method for personalized / device-specific fine-tuning of fully trained artificial neural network chipset applicable to XR terminal or like

Abstract

Provided is a method for personalized / device-specific fine-tuning of a fully trained artificial neural network chipset applicable to an XR terminal or the like. A user device according to an embodiment of the present invention includes: a first artificial neural network trained on general-purpose data; and a second artificial neural network trained on personal data to adjust inference results of the first artificial neural network. By applying an artificial neural network that performs personalized fine-tuning based on characteristics of an individual user with minimal processor resources when implementing an artificial neural network chipset, which is advantageous in terms of mass-producibility and processing speed, in the device such as an XR terminal that is heavily influenced by physical, behavioral, and habitual characteristics of an individual, the present invention can prevent service quality degradation and user dissatisfaction caused by differences in the characteristics of individuals.

Description

A method for fine-tuning individual / device-specific, trained artificial neural network chipsets applicable to XR terminals, etc.

[0001] The present invention relates to a technology for utilizing artificial neural networks, and more specifically, to a method for fine-tuning a user device equipped with an artificial neural network chipset that has completed learning for the purpose of motion tracking, vision processing, etc., such as an XR terminal, according to the characteristics of the user.

[0002] Recently, in order to improve computational speed and reduce the weight of hardware, as shown in the upper part of Figure 1, there are increasing cases of using a trained artificial neural network by manufacturing it as a chipset using ASIC, etc., rather than using a general-purpose processor such as GPU or NPU.

[0003] For example, a learned artificial neural network chipset for image processing can be used in a camera module, or an artificial neural network chipset that supports HDR / resolution enhancement can be embedded in a display terminal to process low-quality content in high-quality hardware without consuming the main processor's resources.

[0004] Furthermore, personal XR terminals, such as Meta's Quest series, have also built-in artificial neural network chipsets to process viewpoint tracking, motion / gesture tracking, and vision processing of external cameras, thereby reducing the weight of XR terminals and increasing computational speed.

[0005] However, in the case of personal devices such as XR terminals, artificial neural network chipsets trained on a wide range of subjects may not be optimized for individuals with different physical / behavioral / habitual characteristics, as shown in the lower part of Fig. 1, and thus performance degradation due to individual differences in gaze / motion tracking, etc. may occur.

[0006] Additionally, in the case of vision processing, services provided by fixed artificial neural network chipsets may be unsatisfactory to many individuals due to physical differences in individual vision, binocular distance, and viewpoint.

[0007] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide a method for applying an artificial neural network that performs individual fine-tuning according to the characteristics of each user using only minimal processor resources, as a means of preventing a deterioration in service quality due to differences in individual characteristics when installing an artificial neural network chipset superior in terms of mass production and computational speed in devices that are greatly influenced by the physical / behavioral / habitual characteristics of individuals, such as XR terminals.

[0008] According to one embodiment of the present invention for achieving the above purpose, a user device includes a first artificial neural network trained with general data; and a second artificial neural network trained with personal data for adjusting the inference results of the first artificial neural network.

[0009] The first artificial neural network may have fixed parameters when the user device is shipped.

[0010] The second artificial neural network may be able to adjust its parameters through learning even after the user device is shipped.

[0011] Universal data may be learning data that reflects the common characteristics of a large number of people, and personal data may be learning data that reflects the characteristics of users using user devices.

[0012] User characteristics may include physical characteristics, behavioral characteristics, and habitual characteristics.

[0013] The first artificial neural network may be mounted on an ASIC-based AI chipset, and the second artificial neural network may be mounted on an FPGA-based AI chipset or run on a processor.

[0014] A user device according to the present invention may further include a processor that performs a function based on an inference result of a first artificial neural network adjusted by a second artificial neural network.

[0015] The functions may include sensing functions, processing functions, and output functions.

[0016] The user device according to the present invention may further include a data acquisition unit that acquires data to be processed by the first artificial neural network and the second artificial neural network.

[0017] According to another aspect of the present invention, an artificial neural network processing method is provided, including a step of a first artificial neural network receiving data as input and outputting an inference result; and a step of a second artificial neural network adjusting the inference result of the first artificial neural network; wherein the first artificial neural network is trained using general data, and the second artificial neural network is trained using personal data.

[0018] According to another aspect of the present invention, a user device is provided, characterized in that it includes: a first artificial neural network whose parameters are fixed when the user device is shipped; and a second artificial neural network whose parameters can be adjusted through learning even after the user device is shipped, and for adjusting the inference results of the first artificial neural network.

[0019] According to another aspect of the present invention, an artificial neural network processing method is provided, including a step of a first artificial neural network installed in a user device receiving data as input and outputting an inference result; a step of a second artificial neural network installed in the user device adjusting the inference result of the first artificial neural network; wherein the parameters of the first artificial neural network are fixed when the user device is shipped, and the parameters of the second artificial neural network can be adjusted through learning even after the user device is shipped.

[0020] As described above, according to embodiments of the present invention, when a device, such as an XR terminal, which is greatly influenced by an individual's physical / behavioral / habitual characteristics, is equipped with an artificial neural network chipset that is superior in terms of mass production and computational speed, an artificial neural network that performs individual fine-tuning according to the characteristics of an individual user with only minimal processor resources is applied, thereby preventing service quality degradation and dissatisfaction due to differences in individual characteristics.

[0021] Figure 1 is a general-purpose fixed artificial neural network chipset,

[0022] Figure 2 is a conceptual diagram of a method for fine-tuning an artificial neural network chipset according to an embodiment of the present invention.

[0023] Figure 3 is a configuration of a user device according to one embodiment of the present invention;

[0024] Figures 4 and 5 are configurations of a user device according to another embodiment of the present invention.

[0025] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0026] An embodiment of the present invention presents a user device utilizing a general-purpose fixed artificial neural network chipset and a small, learnable artificial neural network for individual adjustment.

[0027] This is a technology that applies a small, learnable artificial neural network that performs individual fine-tuning according to the characteristics of each user, as shown in Fig. 2, to personal devices that are greatly influenced by an individual's physical / behavioral / habitual characteristics, while installing an artificial neural network chipset superior in terms of mass production and computational speed, and to prevent a decline in service quality due to differences in individual characteristics.

[0028] FIG. 3 is a diagram illustrating the configuration of a user device according to one embodiment of the present invention. As illustrated, the user device according to the embodiment of the present invention is configured to include a data acquisition unit (110), an artificial neural network module (120), a processor (130), and an application unit (140).

[0029] The data acquisition unit (110) is a configuration for acquiring data to be processed in the artificial neural network module (120), and includes a communication interface for receiving data from internal sensors of the user device as well as external sensors, external devices / servers, and an API interface.

[0030] The artificial neural network module (120) is configured to analyze and infer data acquired from the data acquisition unit (110) and output the inference result, and is configured to include a general-purpose fixed artificial neural network (121) and a personal adjustment artificial neural network (122).

[0031] A universal fixed-loop artificial neural network (121) is an AI chipset equipped with an ASIC-based artificial neural network trained using universal data. Therefore, the universal fixed-loop artificial neural network (121) boasts extremely fast computational / processing speeds and does not utilize processor resources on the user's device. Here, universal data refers to training data that reflects common characteristics of many people.

[0032] The parameters of the universal fixed artificial neural network (121) are already fixed when the user device is shipped, so parameter updates through later learning are not possible.

[0033] A personalized artificial neural network (122) is an artificial neural network trained using personal data, implemented using an AI chipset equipped with an FPGA or driven by processor resources of a user device such as a CPU / GPU / NPU. Here, personal data refers to training data that reflects the physical / behavioral / habitual characteristics of an individual user.

[0034] The personal tuning artificial neural network (122) is a small artificial neural network for adding individual fine tuning to the general inference of the general fixed artificial neural network (121), and the processing resource usage by the personal tuning artificial neural network (122) is very small.

[0035] The personalized artificial neural network (122) can adjust its parameters through learning even after the user device is shipped. Accordingly, it is trained to output inference results that reflect the user's individual characteristics from the data acquired from the data acquisition unit (110) and the inference results of the artificial neural network module (120).

[0036] The processor (130) controls the application unit (140) to perform a given function based on the inference results by the artificial neural network module (120). The application unit (140) is configured to perform functions, and the functions to be performed are determined by the type of user device.

[0037] For example, if the user device is an XR terminal, the functions performed by the application unit (140) under the control of the processor (130) may include sensing functions such as eye / pupil tracking, viewpoint tracking, and hand / gesture tracking, image / vision processing functions such as image processing / generation, and conversation output functions.

[0038] So far, we have described in detail preferred embodiments of a user device utilizing a general-purpose, fixed artificial neural network chipset and a small, learnable artificial neural network for individual tuning.

[0039] Meanwhile, in the above embodiment, the interlocking structure of the universal fixed artificial neural network (121) and the personal adjustment artificial neural network (122) within the artificial neural network module (120) is exemplary and can be modified.

[0040] For example, as illustrated in FIG. 4, it is possible to implement a personal adjustment artificial neural network (122) to output a final inference result that reflects the characteristics of the individual user from the data acquired from the data acquisition unit (110) and the inference result of the general fixed artificial neural network (121).

[0041] Furthermore, as illustrated in FIG. 5, when a universal fixed artificial neural network (121) and a personal adjustment artificial neural network (122) each output inference results from data acquired from a data acquisition unit (110), it is possible to generate a final inference result by combining them (e.g., voting, ensemble, etc.) in a processor (130).

[0042] AI chipsets with fixed parameters can be mass-produced, but individual performance is poor. While utilizing general-purpose processors such as GPUs / NPUs can secure the freedom of artificial neural networks, there are disadvantages in terms of computing power / speed / price, etc.

[0043] Accordingly, in the embodiment of the present invention, rather than learning the entire artificial neural network, a low-capacity / small-sized artificial neural network is connected to the artificial neural network chipset according to individual characteristics, so that the inference result is fine-tuned, thereby reflecting individual characteristics by consuming only a small amount of computing resources with the improved speed of the existing AI chipset.

[0044] Due to individual / terminal differences, it has been difficult to mass-produce and implement AI chipsets for general-purpose learned artificial neural networks. However, by introducing the fine-tuning technique presented in the embodiment of the present invention, fine-tuning is possible by consuming only a small amount of computing resources when utilizing a general-purpose learned chipset, thereby contributing to the activation of the technology.

[0045] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0046] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. First artificial neural network trained by general data; A user device, characterized by including a second artificial neural network for adjusting the inference results of the first artificial neural network by learning from personal data.

2. In claim 1, The first artificial neural network is, A user device characterized in that parameters are fixed when the user device is shipped.

3. In claim 2, The second artificial neural network is, A user device characterized in that parameters can be adjusted through learning even after the user device is shipped.

4. In claim 3, General data is, It is learning data that reflects the common characteristics of many people, Personal data, A user device characterized by having learning data that reflects the characteristics of a user using the user device.

5. In claim 4, User characteristics are: A user device characterized by including physical characteristics, behavioral characteristics, and habitual characteristics.

6. In claim 2, The first artificial neural network is, Equipped with ASIC-based AI chipsets, The second artificial neural network is, A user device characterized by being equipped with an FPGA-based AI chipset or driven by a processor.

7. In claim 1. A user device further comprising a processor that performs a function based on the inference results of the first artificial neural network adjusted by the second artificial neural network.

8. In claim 7, The function is, A user device comprising a sensing function, a processing function, and an output function.

9. In claim 1, A user device further comprising a data acquisition unit that acquires data to be processed by a first artificial neural network and a second artificial neural network.

10. The first artificial neural network receives data as input and outputs an inference result; A second artificial neural network comprises a step of adjusting the inference result of the first artificial neural network; The first artificial neural network is, It is learned by general data, The second artificial neural network is, An artificial neural network processing method characterized by learning from personal data.

11. On the user device, A first artificial neural network whose parameters are fixed when the user device is shipped; A user device characterized by including a second artificial neural network for adjusting the inference results of the first artificial neural network, wherein parameters can be adjusted through learning even after the user device is shipped.

12. A step in which the first artificial neural network installed in the user device receives data and outputs an inference result; A step of a second artificial neural network mounted on a user device adjusting the inference results of the first artificial neural network; The first artificial neural network is, The parameters are fixed when the user device is shipped. The second artificial neural network is, An artificial neural network processing method characterized in that parameters can be adjusted through learning even after the user device is shipped.

Images