Program fully automatic system, server, and method linked to computer automatic execution program and 6g technology

Abstract

This computer automatic execution program device automatically outputs an instruction by means of user voice recognition and inputs the instruction as a program control command, so that a user can automatically execute a desired program by means of voice without an input through an icon outputted on a display, thereby shortening execution time and obtaining a more accurate program execution result. In addition, the result obtained through the program device is integrated into a virtual environment platform linked to 6G technology so as to allow more vivid experiences and tests in a non-face-to-face virtual space, thereby enabling safer purchases than in reality, and allowing a user to receive automated services from shopping to delivery. In particular, by introducing various input methods, the blind, deaf, and elderly can all use computers conveniently, and furthermore, work efficiency can be maximized when the program fully automatic system is used in companies.

Description

Fully automatic system, server and method for linking computer automatic execution program and 6G technology

[0001] The present invention relates to a computer automatic execution program device, and more particularly, to a program fully automatic system that links a computer automatic execution program and 6G technology to extract a program desired by a user and automatically execute it by linking voice recognition technology using AI technology and 6G technology.

[0002] Unless otherwise expressly stated herein, the material described in this section is not prior art to the claims of this application, and its inclusion in this section does not constitute an admission that it is prior art.

[0003] A computer system is an organic assembly of hardware, software, and data, a combination of components used to process, store, and transmit information using computer technology. Computer systems are designed for various purposes and applications and are used in diverse fields, including information processing and storage, problem solving, communications, and entertainment. These systems operate through the harmonious interaction of hardware and software, enabling users to utilize computers efficiently. Advances in information and communication technology (ICT) and the proliferation of smartphones have made daily life virtually impossible without the use of computing systems. Conventional computers typically require users to search for desired programs or applications and then execute them manually to perform the desired process. Furthermore, the process of executing conventional computer programs can be extremely cumbersome for visually impaired individuals or the elderly with limited computer skills. Consequently, those who have difficulty operating a computer often rely on others for assistance, resulting in an inconvenient situation.

[0004] A computer automatic execution program device and a program automatic system according to an embodiment are configured to extract a program that the user wants to execute from among several programs stored in a computing system through user voice recognition and execute the extracted program based on the voice recognition result. In the embodiment, the user's voice is recognized, an STT model is used to extract an instruction for executing a voice air program, and this is input as a program control command, thereby enabling the user to automatically execute a desired program through voice without inputting an icon displayed on a display.

[0005] In addition, in the embodiment, the result output by the program based on 6G technology is output as at least one of the metaverse, virtual reality (AR), and extended reality (XR), so that the user can experience the output result more vividly. Furthermore, the result output by the computer automatic execution program is integrated into the virtual environment platform of the 6G-based program automatic system, so that it can be tested as if it were actual reality in a non-face-to-face virtual space, so that if the result is what the user wants, the service is provided automatically from payment to delivery by selecting purchase.

[0006] The problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0007] A computer automatic execution program device according to an embodiment includes a memory storing at least one command and a processor, and at least one command is executed by the processor to convert a user's voice into text through a language model including an STT model, extract a program that the user wants to use from the converted text, detect an instruction for program control extracted from the converted text, input the instruction into the program to control the program, and integrate the output result through the control of the program into a virtual reality platform of a program automatic system linked to 6G technology, so that the projected result in a virtual space of the digital world can be experienced more vividly, and since testing is possible as if it were real, it is safer than reality, and since the entire system is automatic, anyone can easily purchase the result, and when a purchase is selected, an authentication procedure is automatically performed on the integrated payment server to complete the payment. The completion button automatically outputs a delivery address to provide an automated service up to delivery. In this way, a computer can be conveniently used with just voice input.

[0008] The computer automatic execution program device according to the embodiment is installed in various computer systems such as smartphones and smart pads, and automatically executes programs and functions desired by the user by applying various input methods such as direct input by the user, voice input, motion input, and blind typing input, thereby enabling many people such as the blind, the deaf, the elderly, and those who are not familiar with computers to use computers more conveniently.

[0009] In addition, it automatically extracts instructions from voice and inputs them into the program and executes them without the user having to search for the program or enter commands on the touchscreen, thereby shortening the computer execution time and obtaining more accurate program execution results. The obtained execution results are projected into a virtual space of the digital world based on 6G technology, allowing for a vivid experience of the execution results in augmented reality, and tactile interaction and testing are possible just like in reality, making purchases safer than in reality. Since the entire system is automatic, anyone can vividly experience anything on Earth projected into the virtual space as if it were real with just voice input, and conveniently use the computer with just voice input.

[0010] Figure 1 is a drawing showing a program-automatic system linked with 6G technology according to an embodiment.

[0011] Figure 2 is a drawing showing a computer auto-run program.

[0012] Figure 3 is a diagram showing the data processing configuration of a computer automatic execution program according to an embodiment.

[0013] Figure 4 is a drawing showing an example of a module or model stored in memory according to an embodiment.

[0014] Figure 5 is a diagram showing the data processing configuration of the processor.

[0015] Figure 6 is a drawing showing a function that a program (100) can perform through a processor (130).

[0016] Figure 7 is a drawing showing a fully automatic program system that links a computer automatic execution program device and 6G technology.

[0017] Figure 8 is a diagram showing the structure of the BERT model used in the embodiment.

[0018] Figure 9 is a perspective view of a space stereoscopic phone.

[0019] Figure 10 is a spatial stereoscopic imaging display diagram of a space stereoscopic imaging phone.

[0020] Figure 11 is a perspective view of a space stereoscopic video phone manufactured in the form of a brush.

[0021] Figure 12 is a perspective view of a space stereoscopic video phone manufactured in the form of a necklace.

[0022] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0023] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0024] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0025] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0026] In this specification, the term 'unit' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware.

[0027] Some of the operations or functions described herein as being performed by a terminal, apparatus, or device may instead be performed by a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described herein as being performed by a server may also be performed by a terminal, apparatus, or device connected to the server.

[0028] Hereinafter, the present invention will be described in detail with reference to the attached drawings.

[0029] Figure 1 is a drawing showing a program-automatic system linked to 6G technology according to an embodiment.

[0030] Referring to FIG. 1, a program automatic execution system using 6G technology according to an embodiment may be configured to include a program automatic execution device (100) and a program automatic execution server (200). In the embodiment, the program automatic execution server (200) creates an application for program automatic execution and distributes it to the program automatic execution device (100), and in the embodiment, the program automatic execution device (100) installs the program automatic execution application distributed from the server (200), and automatically executes several applications and programs pre-installed in the device (100) using the installed application through the user's voice and motion recognition.

[0031] In an embodiment, the program auto-execution device (100) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a notebook, desktop, or laptop equipped with a navigation system or web browser. In this case, at least one purchaser terminal (100) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one program automatic execution device (100) may include, for example, all kinds of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc., as wireless communication devices that ensure portability and mobility.

[0032] Figure 2 is a drawing showing a computer auto-run program.

[0033] Referring to FIG. 2, a computer auto-execution program may include a communication unit, memory, and a processor. The communication unit converts the received user's voice into text through an STT conversion module, extracts keywords, and generates instructions. In addition, the program control history for each user is accumulated and learned to update a program auto-control model, and an auto-control processor for each user can be generated through the updated program auto-control model. The generation unit of the processor stores instructions for program control that match the recognized voice and text. In addition, instructions matched to each keyword are combined through an instruction generation model to generate commands for program control. The generated commands are input into the program to automatically control the computer auto-execution program. A model and data for generating a program output result generated through the computer auto-execution program auto-control into at least one of a 3D-based metaverse, virtual reality, and augmented reality are output.

[0034] Figure 3 is a diagram showing the data processing configuration of a computer automatic execution program according to an embodiment.

[0035] Referring to FIG. 3, a computer automatic execution program (100) according to an embodiment may include a communication unit (110), a memory (120), and a processor (130).

[0036] The communication unit (110) can be configured regardless of the communication mode, such as wired or wireless, and can be configured with various communication networks, such as a personal area network (PAN) and a wide area network (WAN). In addition, the communication unit (110) can operate based on the known World Wide Web (WWW), and can also use a wireless transmission technology used for short-distance communication, such as infrared (IrDA) or Bluetooth. For example, the communication unit (110) can be responsible for transmitting and receiving data required to perform a technique according to an embodiment of the present disclosure. The memory (120) may refer to any type of storage medium. For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk. The memory (120) may also include a user information database as illustrated in FIG. 1.

[0037] The memory (120) can store at least one instruction that can be executed by the processor (130). In addition, the memory (120) can store any type of information generated or determined by the processor (130) and any type of information received by the program automatic execution system (100). For example, as will be described later, the memory (120) can store a verb evaluation table and a plurality of model answers.

[0038] Various types of modules or models may be stored in the memory (120). This is illustrated in Fig. 4.

[0039] Figure 4 is a drawing showing an example of a module or model stored in memory according to an embodiment.

[0040] Referring to FIG. 4, a memory (120) may store an STT conversion module (121), a user information database (122), a keyword extraction model (123), an instruction generation model (125), an SSL / TLS security model (126), a program automatic control model (127), a 6D output generation model (128), a feedback provision model (129), and an integrated payment server (139). Each module or model may be in the form of an application executable by a processor (130).

[0041] Looking at each module or model, the STT conversion module (121) is designed to convert speech into text. Since the STT conversion module (121) itself is already a known technology, a detailed description thereof will be omitted.

[0042] All models stored in the memory (120) according to the embodiment may be models trained using machine learning or deep learning methods. In this case, the models may be trained using transfer learning. Below, we will briefly examine transfer learning.

[0043] Transfer learning refers to a model acquired by transferring a pre-trained model to a specific task and then further fine-tuning it. BERT is a representative example of a pre-trained model, illustrated in Figure 8. Since the structure and characteristics of BERT itself are already well-known, a detailed description thereof will be omitted.

[0044] In the embodiments, all models employ language models as pre-trained models. These language models are models that undergo semi-supervised learning on a large corpus using at least one of various well-known techniques, such as masked language model (MLM) or next sentence prediction (NSP). Since MLM and NSP are already well-known techniques, detailed descriptions thereof will be omitted.

[0045] The pre-trained model described above is combined with a layer for evaluating response sentences, and training is performed on this combined model. This training is referred to as fine-tuning, as the pre-trained model is fine-tuned to perform a given task.

[0046] Fine-tuning utilizes multiple training data sets. The training input data contains the answer sentences to interview questions, while the training labeling data contains the evaluation results for those answer sentences.

[0047] Meanwhile, the learning, authentication, and input data according to the embodiment may include at least one of the user's gender, age, fingerprint, password, pattern SNS authentication, integrated authentication, various biometric recognition, phone number, address, date of birth, and name, as well as the user's identity information and voice inputted from the user stored in the user information database (122).

[0048] The keyword extraction model (123) is designed to extract keywords from response sentences. Since the technique for extracting keywords from sentences is already well-known, a detailed description thereof will be omitted.

[0049] The instruction generation model (125) matches instructions corresponding to the extracted keywords. To this end, the instruction generation model (125) stores instructions for program control that match the recognized voice and text. What is an instruction in the embodiment? It is a command for controlling a program. In the embodiment, each instruction is stored in various codes, and the instruction generation model (125) generates a command for program control by combining instructions matched to each keyword. For example, if the stored keywords are "shopping," "product," and "order," the instruction generation model (125) can extract instructions such as the home shopping URL and product information that the user wants to access, and generate a command. The SSL / TLS security protocol (126) implements an error handling mechanism to prepare for errors that may occur during communication. In addition, it defines and adapts a processing method for error codes or statuses according to the protocol specifications. The computer autorun program according to the embodiment applies security protocols such as SSL / TLS to ensure data security, or performs additional tasks for data encryption and authentication. Furthermore, the computer autorun program according to the embodiment tests and debugs the linked system to identify and correct potential problems.

[0050] The program automatic control model (127) inputs the generated command into the program to automatically control the program. In an embodiment, the program automatic control model (127) is obtained by fine-tuning a pre-trained model, and the pre-trained model is trained in a semi-supervised learning manner by applying at least one of a masked language model (MLM) and a next sentence prediction (NSP) to a plurality of corpuses. In an embodiment, the fine-tuning is obtained by supervising the pre-trained model with a plurality of training data including training input data and training label data, and the plurality of training input data may include a program extraction and program control process for voice input of a plurality of users and a program control history for each user.

[0051] The output generation model (128) is an artificial neural network model that generates output, which is a result of program automatic control, as a 6G-based 5D output. To this end, the output generation model (128) stores models and data for generating program output results as at least one of a 5D-based metaverse, virtual reality, augmented reality, or mixed reality.

[0052] The feedback model (129) is an artificial neural network model that evaluates and updates a learned artificial neural network model and a deep learning model. In an embodiment, the feedback model (129) can evaluate the artificial neural network model through at least one of accuracy, precision, and recall. Accuracy is an index that measures how well the results predicted by the artificial neural network model match the actual results. Precision is an index that measures the proportion of actual positives among the results predicted as positive. Recall is an index that measures the proportion of actual positives predicted by the model as positive. In an embodiment, the feedback model (129) can calculate the accuracy, precision, and recall of the artificial neural network model, and evaluate the artificial neural network model based on at least one of the calculated indexes.

[0053] In an embodiment, the feedback model (129) can measure the accuracy of an artificial neural network model using an evaluation dataset.

[0054] The evaluation dataset consists of data that the model did not use for training and is used to objectively evaluate the model's performance.

[0055] In the embodiment, the feedback model (129) executes an artificial neural network model using an evaluation dataset, and compares the predicted value of the artificial neural network model for each input data with the actual correct answer value of the corresponding data.

[0056] The accuracy of the model's predictions can then be measured through comparison results. For example, in the Peabody model (129), accuracy can be calculated as the proportion of data correctly predicted by the model among all data.

[0057] In addition, the feedback model (129) calculates the F1 score, which is an indicator of the balance of precision and recall, which is an indicator calculated as the harmonic mean of precision and recall, and calculates the calculated F1 score. Based on this, an artificial neural network model can be evaluated, and an AuC-RoC curve, which is an indicator that visualizes the performance of a classification model in a graph, can be generated, and an artificial neural network model can be evaluated based on the generated AuC-RoC curve. In an embodiment, the feedback model (129) can be evaluated as having better performance as the area under the RoC curve (AuC) is closer to 1.

[0058] In addition, the feedback model (129) can evaluate the interpretability of the artificial neural network model. In an embodiment, the feedback model (129) evaluates the interpretability of the artificial neural network model through SHAP (SHapley Additive explanations) and LIME (Local Interpretable Model-agnostic Explanations) methods. SHAP (SHapley Additive explanations) is a library that provides an interpretation of the results predicted by the model, and the feedback model (129) extracts SHAP values ​​from the library. In an embodiment, the feedback model (129) can predict how much the characteristic information input to the model influenced the model prediction through the extraction of SHAP values.

[0059] The LIME (Local Interpretable Model-agnostic Explanations) method is a method for explaining a model's predictions for individual samples. In an embodiment, the feedback model (129) approximates a sample as an interpretable model through the LIME method and calculates the importance of each characteristic information. In addition, the feedback model (129) can estimate the influence of each characteristic variable by analyzing the internal weights and bias values ​​of the model. The feedback model (129) performs improvement work when the fairness of the artificial neural network model is low or shows discrimination. In an embodiment, the feedback model (129) collects additional data representing a specific group and performs a data preprocessing process when data from a specific group is insufficient by a certain level or more. In an embodiment, the feedback model (129) performs a data preprocessing process including data normalization, outlier removal, and data scaling to prevent the model from learning unnecessary patterns. In addition, specific conditions can be added to the learning algorithm to prevent discrimination or ensure fairness.

[0060] The feedback model (129) evaluates the performance of the model by comparing the model's predicted results with actual results through confusion matrix analysis to ensure fairness. A confusion matrix is ​​a matrix that evaluates the classification performance of a model in supervised learning. The confusion matrix displays the classification results by comparing the model's predicted results with actual results. The feedback model (129) can evaluate the performance of the model by calculating the accuracy and misclassification rate for each class through confusion matrix analysis. In addition, in the embodiment, the feedback model (129) enables the verification of the distribution of data through visual analysis of the learning data. For example, in the case of image data, the diversity and fairness of the data can be evaluated by visualizing image samples for each class. Furthermore, the feedback model (129) verifies the fairness and diversity of the learning data through fairness verification and evaluation index calculation, thereby enabling the improvement of the artificial neural network model. Fairness verification is to check whether the artificial neural network model shows discrimination for specific data attributes with respect to the learning information. The feedback model (129) can compare the number of samples for each attribute or evaluate the classification performance for each attribute to check whether there is discrimination for a specific attribute. In addition, the feedback model (129) calculates various indices to evaluate the performance of the artificial neural network model. For example, the performance of the model can be evaluated by calculating indices such as accuracy, precision, recall, and F1-score. At this time, the fairness and diversity of the model can be evaluated by calculating indices for each class. In addition, the feedback model (129) collects feedback on problems that occur when the artificial neural network model is used in a real environment and continuously improves the artificial neural network model by reflecting the collected feedback in the artificial neural network model. Meanwhile, since each of the above-mentioned models can be an estimation model learned by an artificial neural network, let's briefly look at these estimation models.

[0061] The estimation model in this specification may refer to any form of computer program that operates based on a network function, an artificial neural network, and / or a neural network. Throughout this specification, the terms model, neural network, network function, and neural network may be used interchangeably. A neural network is a network in which one or more nodes are interconnected through one or more links to form input node and output node relationships within the neural network. The characteristics of a neural network may be determined based on the number of nodes and links within the neural network, the correlation between the nodes and links, and the weight value assigned to each link. A neural network may be composed of a set of one or more nodes. A subset of the nodes constituting the neural network may constitute a layer.

[0062] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. A deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, a generative adversarial network (GAN), a transformer, and the like. The description of the above-described deep neural network is merely an example, and the present disclosure is not limited thereto. Neural networks can learn through at least one of the following methods: supervised learning, unsupervised learning, semi-supervised learning, self-supervised learning, or reinforcement learning. Neural network learning can be the process of applying knowledge to the neural network to perform a specific action.

[0063] Neural networks can be trained to minimize output errors. This process involves repeatedly inputting training data into the neural network, calculating the neural network output and target error for the training data, and backpropagating the neural network error from the output layer to the input layer to update the weights of each node in the neural network to reduce the error. In supervised learning, labeled data is used for each training data, while unlabeled data can be used for each training data. The amount of change in the connection weights of each updated node can be determined by the learning rate. The neural network's calculation of the input data and backpropagation of the error can constitute a learning cycle (epoch). The learning rate can vary depending on the number of iterations in the neural network's training cycle. Additionally, methods such as increasing the learning data, regularization, dropout that disables some nodes, and batch normalization layers can be applied to prevent overfitting.

[0064] In one embodiment, the estimation model may borrow at least a portion of a transformer. The transformer may be composed of an encoder that encodes embedded data and a decoder that decodes the encoded data. The transformer may have a structure that receives a series of data, performs encoding and decoding steps, and outputs a series of data of different types. In one embodiment, the series of data may be processed into a form operable by the transformer. The process of processing the series of data into a form operable by the transformer may include an embedding process. Expressions such as data tokens, embedding vectors, and embedding tokens may refer to data embedded into a form operable by the transformer. The transformer may utilize an attention algorithm to process the encoders and decoders within the transformer to encode and decode the series of data. An attention algorithm is an algorithm that calculates the similarity of one or more keys for a given query, reflects the similarity in the values ​​corresponding to each key, and then calculates the attention value by weighting the values ​​with the reflected similarity. Various types of attention algorithms can be classified depending on how the query, key, and value are set. For example, if attention is obtained by setting the query, key, and value all to the same value, this may refer to a self-attention algorithm. If attention is obtained by reducing the dimension of the embedding vector to process a series of input data in parallel and calculating individual attention heads for each divided embedding vector, this may refer to a multi-head attention algorithm. A transformer may be composed of modules that perform multiple multi-head self-attention algorithms or multi-head encoder-decoder algorithms.In one embodiment, the transformer may also include additional components other than attention algorithms, such as embedding, normalization, and softmax. A method for constructing a transformer using an attention algorithm may include the method disclosed in Vaswani et al, Attention Is All You Need, 2017 NIPS, which is incorporated herein by reference. The transformer can be applied to various data domains, such as embedded natural language, segmented image data, and audio waveforms, to transform a series of input data into a series of output data. To transform data with various data domains into a series of data that can be input to the transformer, the transformer can embed the data. The transformer may process additional data that represents the relative positional relationship or phase relationship between the series of input data. Alternatively, the series of input data may be embedded by additionally reflecting vectors that represent the relative positional relationship or phase relationship between the input data. In an example, the relative positional relationship between a series of input data may include, but is not limited to, the word order within a natural language sentence, the relative positional relationship of each segmented image, the temporal order of segmented audio waveforms, etc. The process of adding information expressing the relative positional relationship or phase relationship between a series of input data may be referred to as positional encoding.

[0065] In one embodiment, the estimation model may include a Recurrent Neural Network (RNN), a Long-Short Term Memory (LSTM) network, a Bidirectional Encoder Representations from Transformers (BERT), or a Generative Pre-trained Transformer (GPT). In one embodiment, the estimation model may be a model trained by a transfer learning method. Here, transfer learning refers to a learning method that pre-trains a large amount of unlabeled training data using a semi-supervised learning or self-learning method to obtain a pre-trained model having a first task, and then fine-tunes the pre-trained model to be suitable for a second task and trains the labeled training data using a supervised learning method to implement a target model.

[0066] Let us look at the processor (130) again with reference to FIG. 3. First, the processor (130) according to one embodiment may perform technical features according to embodiments of the present disclosure, which will be described later, by executing at least one instruction stored in the memory (120). In one embodiment, the processor (130) may be configured with at least one core and may include a processor for data analysis and / or processing, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU) of a computer autorun program (100).

[0067] Let us examine data processing in the processor (130) below with reference to FIG. 5.

[0068] Figure 5 is a diagram showing a data processing configuration of a processor according to an embodiment.

[0069] Referring to FIG. 5, a processor (130) according to an embodiment may be composed of a collection unit (131), a preprocessing unit (132), a learning unit (133), a generation unit (134), an automatic control unit (135), an output generation unit (136), a feedback unit (137), and an evaluation unit (140). The term 'unit' used in this specification should be interpreted as including software, hardware, or a combination thereof, depending on the context in which the term is used. For example, software may be machine language, firmware, embedded code, and application software. As another example, hardware may be a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a sensor, a MEMS (Micro-Electro-Mechanical system), a passive device, or a combination thereof.

[0070] The collection unit (131) collects training data for the deep learning model and user information for automatic program execution. The user information may include user voice and motion information, user-specific program control history information, and a user information database. The preprocessing unit (132) preprocesses the collected training data to remove biased or discriminatory data from the collected artificial intelligence training data.

[0071] The preprocessing unit (132) preprocesses the collected data and processes it into a format suitable for artificial intelligence model learning. For example, the preprocessing unit (132) can perform processes such as noise removal, outlier removal, and missing value processing. Furthermore, the preprocessing unit (132) can prevent the model from learning unnecessary patterns by performing data preprocessing, such as normalizing data, removing outliers, or scaling data.

[0072] Learning (133) trains an artificial neural network model using preprocessed learning data. The learning unit (133) selects an algorithm suitable for the field required by each industry or job through analysis of industry, job, and specific metadata, and detects optimal hyper parameters from the selected algorithm. In an embodiment, hyper parameters are parameters that can be adjusted during the artificial intelligence model learning process. Hyper parameters may include, but are not limited to, the learning rate, batch size, and number of epochs of the artificial neural network model. The learning unit (133) extracts keywords included in each industry or job as metadata, compares the extracted metadata with keywords describing the algorithm, and selects an algorithm that is similar to a certain level or higher based on the comparison result. In addition, the learning unit (133) trains the artificial neural network model using input data and correct answer data. In an embodiment, the learning unit (133) inputs input data to the model, and the model outputs a predicted value for the input data. Thereafter, the learning unit (133) compares the predicted value with the correct answer data to calculate a prediction error. The learning unit (133) performs learning by adjusting the weights and biases of the model to minimize the prediction error. In this process, the speed and accuracy of learning can be controlled by adjusting hyperparameters including the learning rate. The learning unit (133) repeats the artificial neural network model learning process described above, and allows the model to learn the relationship between input data and correct data. The learning unit can perform hyperparameter tuning to detect the lowest values ​​of hyperparameters through experiments and verification. In addition, the learning unit samples the collected learning data and learns the sampled data. Sampling is the process of extracting some data from the dataset. In the embodiment, when the dataset is larger than a certain capacity and is diverse, some data is extracted through sampling and used for model learning.This reduces the data required for model learning and speeds up learning. Furthermore, in cases where the dataset is unbalanced, sampling can be used to balance classes. Furthermore, the learning unit generates learning data through data augmentation and trains an artificial neural network model with the generated data. Data augmentation is the process of transforming collected learning data to generate new data. For example, the learning unit (133) applies left-right flipping, rotation, resizing, color conversion, etc. to image data among the collected learning data to generate new image data and uses the new image data as learning data. This allows, in the embodiment, to increase the size of the dataset and improve model performance in various situations. In the embodiment, data augmentation can be applied not only to image data but also to text data. For example, the learning unit (132) can generate new learning data by applying word substitution, synonym substitution, sentence reversal, etc. to text data.

[0073] The generation unit (134) matches instructions corresponding to the extracted keywords. To this end, the generation unit (134) stores instructions for program control that match the recognized voice and text. An instruction is a command for controlling a program, and in the embodiment, each instruction is stored in various codes, and the generation unit (134) generates instructions for program control by combining instructions matched to each keyword through an instruction generation model. For example, if the stored keywords are "home shopping," "product," and "order," the generation unit (134) can extract instructions such as the home shopping URL and product information that the user wants to access and generate instructions.

[0074] The automatic control unit (135) inputs the generated commands into the program and automatically controls the program. In a subsequent embodiment, the output generation unit (136) outputs a model and data for generating the program output result as at least one of a 3D-based metaverse, virtual reality, and augmented reality. The output generation unit (136) generates a model and data to be output to a virtual environment based on the execution result of the program. The model may include 3D objects, environments, characters, etc., and the data may include texture animation, physical characteristics, etc. Thereafter, the output generation unit (136) analyzes the program execution result to determine what type of output should be generated. For example, when the program outputs a simulation result, the simulated object and environment are projected onto the virtual environment. Thereafter, the output generation unit (136) designs a 3D model to be used in an actual virtual environment using a 3D modeling tool or design software based on the generated model and data. At this time, the shape, color, material, etc. of the model may be defined and animation may be added. Afterwards, the output generation unit (136) integrates into the metaverse, virtual reality, and augmented reality platforms: It integrates the generated 3D model and data into the selected virtual environment platform (metaverse, virtual reality, augmented reality). Since each platform may have its own format and technology, the model and data are converted or formatted to fit the specifications of the corresponding platform. Afterwards, the output generation unit (136) outputs the generated model and data to the selected virtual environment. Through this, the user can view or interact with the 3D model. In addition, the output generation unit (136) allows the user to manipulate and interact with the generated output. When the user moves or manipulates the model within the virtual environment, the output generation unit detects and reflects this.

[0075] The feedback unit (137) evaluates the artificial neural network model learned through the feedback model. In an embodiment, the feedback unit (137) can evaluate the artificial neural network model through at least one of accuracy, precision, and recall. Accuracy is an index that measures how well the results predicted by the artificial neural network model match the actual results. Precision is an index that measures the proportion of actual positives among the results predicted as positive. Recall is an index that measures the proportion of actual positives predicted by the model as positive. In an embodiment, the feedback unit (137) can calculate the accuracy, precision, and recall of the artificial neural network model and evaluate the artificial neural network model based on at least one of the calculated indices. The feedback unit (137) can measure the accuracy of the artificial neural network model using an evaluation dataset. The evaluation dataset is composed of data that was not used to train the model and is used to objectively evaluate the performance of the model.

[0076] In an embodiment, the evaluation unit (140) executes an artificial neural network model using an evaluation dataset and compares the predicted value of the artificial neural network model for each input data with the actual correct answer value of the corresponding data. Thereafter, the degree of accuracy of the model's predictions can be measured through the comparison results. For example, the accuracy in the evaluation unit (140) can be calculated as the ratio of data correctly predicted by the model among the entire data. In addition, the feedback unit (137) can calculate the F1 score, which is an index indicating the balance of precision and recall, which is an index calculated as the harmonic mean of precision and recall, and evaluate the artificial neural network model based on the calculated F1 score, generate an AUC-ROC curve, which is an index that visualizes the performance of the classification model in a graph, and evaluate the artificial neural network model based on the generated AUC-ROC curve. In an embodiment, the evaluation unit (140) can evaluate that the performance of the model is better as the area under the ROC curve (AUC) is closer to 1. In addition, the feedback unit (137) can evaluate the interpretability of the artificial neural network model. The feedback unit (137) evaluates the interpretability of the artificial neural network model through the SHAP and LIME methods. SHAP is a library that provides an interpretation of the results predicted by the model, and the evaluation unit (140) extracts SHAP values ​​from the library. The evaluation unit (140) can predict how much the characteristic information input to the model influenced the model prediction through the extraction of SHAP values. The LIME method is a method for explaining the model's prediction for individual samples. The evaluation unit (140) approximates the sample to an interpretable model through the LIME method and calculates the importance of each characteristic information. In addition, the evaluation unit can estimate the influence of each characteristic variable by analyzing the internal weight and bias values ​​of the model.

[0077] Figure 6 is a drawing showing the functions that a computer automatic execution program (100) can perform through a processor (130).

[0078] Fig. 6 is merely an example, and the idea of ​​the present invention is not limited to what is shown in Fig. 6.

[0079] Referring to FIG. 6, in step S100, the user's voice is converted into text through a language model including an STT model, and in step S100, a program that the user wants to use is extracted from the converted text. In step S200, instructions for program control extracted from the converted text are detected. In step S300, the instructions are input into a program to control the program, and in step S400, the output results through the control of the program are generated as virtual reality, augmented reality, and metaverse based on 6G technology. In an embodiment, a STT (speech to text) conversion module is stored in the memory, and the instructions are generated by converting a control command spoken by the user into a voice by the STT conversion module and extracting keywords. In an embodiment, a user-specific program control history can be accumulated, a program automatic control model can be updated through learning, and a user-specific automatic control process can be generated through the updated program automatic control model. A program automatic control model is obtained by fine-tuning a pre-trained model, and the pre-trained model is trained by applying at least one of a masked language model (MLM) and a next sentence prediction (NSP) to a plurality of corpora using a semi-supervised learning method. In this case, the fine-tuning is obtained by supervising the pre-trained model on a plurality of training data including training input data and training label data, wherein the plurality of training input data includes a program extraction and program control process for a plurality of users' voice inputs and a program control history for each user. Instructions for program control matching the recognized voice and text are stored in the memory, and a model and data for generating a program output result as at least one of a 6D-based metaverse, virtual reality, and augmented reality are stored in the memory.Below, we will examine the method and changes in which the 5D-based model and data stored in the memory are applied to a fully automatic program system linked to 6G technology, with reference to Figure 7.

[0080] Figure 7 is a drawing showing a fully automatic program system linked to 6G technology.

[0081] A fully automatic program system linked to 6G technology relates to a method of applying 6G technology to the output of the above program (100). 6G is an intelligent communication infrastructure that connects the virtual and the real without spatial and temporal constraints through enhanced 5G performance, artificial intelligence-based network optimization, and expanded coverage at sea, in the air, and in space. The 6G era network is a communication that integrates ground communication, satellite communication, and maritime communication, and can connect signals even in blind spots such as deserts, uninhabited areas, and the sea, and will be widely used in various fields such as spatial communication, intelligent interconnection, emotional and tactile exchange, multi-sensory and mixed reality, cooperation between devices, and fully automatic transportation.

[0082] Referring to FIG. 7, a program-integrated automatic system linked to 6G technology may be configured to include a computer automatic execution program device (100), a program automatic execution server (200), a virtual environment platform (600), and an integrated payment server or (BRICS PAY BRICS international integrated payment server).

[0083] In a program-automatic system linked with 6G technology, the program automatic execution server (200) may store a website, software, a neutrino dataset, a spatial stereoscopic image system, and various AI navigations.

[0084] For example, a blind shopping guide (a blind shopping guide that helps a blind person purchase a desired product by providing AI functions to shopping malls to explain the product's color, shape, performance, material, etc. in language or blind characters because a blind person cannot see many products in a shopping mall) may be stored, and an AI mute conversation commentator (an AI mute conversation commentator that allows a mute person's inputted text or sign language to be conveyed to the other person as speech or text, and the other person's words to be conveyed to the mute person as text or sign language) may be stored.

[0085] There may be an AI painter that automatically draws various pictures (oil paintings, watercolors, cartoons, etc.), an AI counselor that automatically provides consultations, an AI designer that automatically designs various designs, an AI cinematographer that can automatically film and produce a drama by inputting a story, an AI doctor of oriental medicine that can automatically take the pulse and treat patients, an AI teacher that teaches various subjects,

[0086] In addition, since one keycap of a computer keyboard can be replaced with one instrument and various instruments can be embedded, one keyboard can become an orchestra, and many AI navigations, such as an AI music orchestra, AI composer, AI lyricist, AI nurse, AI lawyer, etc., can be invented and embedded. This is a program automatic execution server (200).

[0087] In an embodiment, a program auto-execution server (200) creates an application for program auto-execution and distributes it to a computer auto-execution program device (100), and in an embodiment, the computer auto-execution program device (100) installs the program application distributed from the server (200), and automatically executes several applications and programs pre-installed on the device (100) using the installed application through the user's voice and motion recognition.

[0088] The program device (100) capable of automatically executing the user's voice and motion recognition converts the user's voice into text through a language model including an STT conversion model, extracts a program that the user wants to use from the converted text, and also detects an instruction for program control from the program automatic execution server (200) from the converted text. The detected instruction is input into the automatic control program (100) to control the program, and a model and data for generating the output result of the program control in a 6G-based virtual reality are stored in the memory.

[0089] A processor may perform the technical features according to embodiments of the present disclosure by executing at least one instruction stored in a memory.

[0090] In the above processor, the collection unit collects training data for a deep learning model and user information for automatic program execution.

[0091] The preprocessing unit preprocesses the collected learning data to remove data with bias or discrimination from the collected artificial intelligence learning data.

[0092] The learning unit trains an artificial neural network model using preprocessed learning data, selects an algorithm suitable for the field required by each industry or job through analysis of metadata for each industry or job, detects the optimal hyperparameters from the selected algorithm, and trains the artificial neural network model using input data and correct answer data.

[0093] Also, the collected learning data is sampled and the sampled data is used for learning. Sampling is the process of extracting some data from the dataset.

[0094] Data augmentation creates learning data and trains an artificial neural network model with the generated data. Data augmentation is the process of transforming collected learning data to create new data.

[0095] The learning unit generates image data from the collected training data and uses the new image data as training data. This allows the example to increase the dataset size and improve model performance in various situations.

[0096] The generation unit matches the extracted keywords with corresponding instructions. To this end, it stores instructions for program control that match the recognized speech and text. In the embodiment, an instruction is a command for program control. In the embodiment, each instruction is stored in various codes, and the generation unit generates commands for program control by combining instructions that match each keyword through an instruction generation model.

[0097] The control unit inputs the generated commands into the program and automatically controls the program. The generation unit outputs a model and data for generating at least one of 6D-based space imaging, metaverse, virtual reality, and augmented reality as the program output result. The output generation unit generates a model and data to be output to a virtual environment based on the program execution result. The model may include Model-based and Model-free, CAD Model, Seen / Unseen, etc., and the data may include Objavers-Levels, LLM, Diffusion, etc. After that, the output generation unit analyzes the program execution result to determine what type of output should be generated. For example, if the program outputs a simulation result, the simulated objects and environment are projected onto the virtual environment. After that, the output generation unit designs a 6D model to be used in an actual virtual environment using BEXEL software based on the generated model and data. At this time, the model's shape, color, material, etc. can be defined and animation can be added. Afterwards, the output generation unit integrates into space imaging, metaverse, virtual reality, and augmented reality platforms: The generated 6D model and data are integrated into the selected virtual environment platform (space imaging, metaverse, augmented reality). Since each platform may have its own format and technology, the model and data are converted or formatted according to the specifications of the corresponding platform. The output generation unit outputs the generated model and data to the selected virtual environment. Through this, the user can view or interact with the 6D model. In the case of the metaverse, it can also be shared or interacted with with other users. In addition, the output generation unit allows the user to manipulate and interact with the generated output.When a user moves or manipulates a model within a virtual environment, the output generation unit detects and reflects this movement, and provides a function to collect and share sensor data and location information using 6G technology in real time. This allows the user's location and surrounding environment to be accurately identified and appropriate virtual information provided based on the user's movements. Furthermore, a system is implemented that projects virtual information into the user's field of vision through an AR device or glasses. This system can arrange and manipulate virtual information based on the user's location and gaze direction.

[0098] To achieve this, a fully automated program system utilizing 6G technology visually displays relevant information to the user on the screen of an AR device when the program is executed. This information can be superimposed on the user's field of view or positioned to allow interaction with the user.

[0099] Furthermore, 6G technology, which is used in a fully automated program system that integrates 6G technology, supports advanced interactivity, allowing users to operate and control programs in an augmented reality environment through motion or voice commands. This allows for an experience that is convenient for the user and suitable for the environment. For example, in a virtual space such as a company projected into the digital world, all employees can work on computers remotely together without face-to-face contact, and one person can operate multiple computers from a distance, which can significantly improve system control efficiency. Furthermore, using the embodiment, when purchasing a vehicle, the entire driving interface of the vehicle can be reproduced in a non-face-to-face virtual space, allowing the user to experience the driving state realistically, and fully autonomous driving can be realized through the embodiment. Also, using the embodiment, it is possible to text-type a vehicle to be purchased in augmented reality. For this purpose, the user can view or interact with the vehicle in the 6D model, and in the case of the metaverse, it can be shared with other users, or the user can manipulate and interact with the output (car) in the virtual space. When the model is moved or manipulated within the virtual environment, the generation unit detects and reflects this. In other words, if you move or transform an image in the virtual space as if it were real, it will be transformed and moved. In other words, in order to select a real object that you like 100% in a virtual space that is like reality, you must use the AI ​​function. To this end, the learned artificial neural network model is evaluated through the feedback model, and the feedback unit can evaluate the artificial neural network model through at least one of accuracy, precision, and recall.

[0100] Accuracy is a measure of how well the predicted results of an artificial neural network model match actual results. Precision measures the proportion of true positives among predicted positives. Recall measures the proportion of true positives predicted by the model.

[0101] The feedback unit can measure the accuracy of the artificial neural network model using an evaluation dataset. It can also assess the interpretability of the artificial neural network model. The evaluation unit uses the Local Interpretable Model-Aghostic Explanations (LIME) method to approximate the sample with an interpretable model and calculate the importance of each characteristic. Furthermore, the evaluation unit can estimate the influence of each characteristic variable by analyzing the model's internal weights and biases. The extracted output (car) can be text-based and allows users to experience riding in the vehicle they plan to purchase.

[0102] If you click on the purchase selection (141) after testing and it is the desired product, the integrated payment server (BRICS PAY BRICS international integrated payment server) stored in the computer auto-execution program (100) automatically connects to the user information database (122), automatically performs an authentication procedure to complete the payment, and when the purchase is completed (143), it automatically connects to the user information database (122) and receives the address, name, phone number, etc. from the user's identity information, and the desired product (printed product) is automatically delivered to the delivery location, and when the purchase is completed (143), it automatically withdraws from the product (printed product) purchase site to maintain the correct security status.

[0103] Purchase completion (143) switches the user information database (122) to a power saving state, and the user information database (122) is automatically powered on after user authentication (facial recognition, etc.) from the integrated payment server, thereby safely protecting the user's identity information.

[0104] In addition, when data security is required, the program (100) implements an error handling mechanism, applies a security protocol such as SSL / TLS, or performs additional tasks for data encryption and authentication, thereby maintaining the perfect security of the integrated payment server (BRICS PAY BRICS ultra-low-cost integrated payment server).

[0105] (BRICS PAY BRICS International Integrated Payment Server) can function as an integrated payment server and is adopted in this system to ensure joint economic development and stable currency circulation in each country around the world.)

[0106] Figure 9 is a perspective view of a space stereoscopic phone.

[0107] Referring to FIG. 9, the space stereoscopic image phone connected to 6G technology installs the computer automatic execution program device (100) of the present invention in the body of the "iPhone" (see FIG. 9-a), and can run the phone with voice and motion input without touch screen input. In addition, the battery, which is the biggest problem of the "iPhone", is eliminated, and an "atomic battery" with good performance, a volume smaller than a coin, and a lifespan of 50 years is installed, so that the volume of the phone can be minimized. Therefore, the phone can be manufactured in various forms such as a brush, a necklace, an bracelet, a ring (see FIG. 11, FIG. 12), and there is no need to worry about the battery of the phone exploding. In addition, the performance has been further upgraded, so that the mute, the blind, the elderly, and children can all use it easily. It also uses the existing display, and an LED lens with a diameter of about 3 cm is installed on the back of the phone (see FIG. 9-b), and when an image is sent to the controller via wired or wireless connection to the control tablet, the image is transmitted from the LED lens, and the light of the screen emitted by each module is transmitted from the LED lens. The light a1, a2, a3, a4 emitted by any module on the LED lens is projected onto the "projection system on the photoresist" installed on the outer surface (Fig. 9-c lens side view) (see Fig. 10), that is, the light a1, a2, a3, a4 emitted by any module on the LED lens is projected onto the center point P of the entrance pupil, and is reflected from the center point P' of the exit pupil through the "projection system on the photoresist", and the reflected lines a1", a2", a3", a4" intersect the space imaging plane A'B' at points a1', a2', a3', a4' in the image space, thereby generating a space stereoscopic image, and a space stereoscopic image phone that also enables space travel is proposed.

[0108] Additionally, computer autorun programs test and debug linked systems to find and fix potential problems.

[0109] Testing can be conducted in various scenarios using simulators or mobile environments. Furthermore, according to the embodiment, the computer autorun program can provide program output in augmented reality (AR) using 6G technology. 6G technology provides ultra-high-speed data transmission and ultra-low-latency communication, enabling the rapid transmission and reception of large amounts of data. The computer autorun program utilizes this to rapidly receive data, images, videos, and streams necessary for program execution. Furthermore, 6G technology provides the ability to collect and share sensor data and location information in real time, enabling the computer autorun program to accurately identify the user's location and surrounding environment and provide appropriate virtual information based on the user's movements. The computer autorun program also provides linkage functions. For example, a system can be implemented that projects virtual information into the user's field of view using AR devices or glasses. This fully automated program system can position and manipulate virtual information based on the user's location and gaze direction. To achieve this, the fully automated program analyzes the surrounding environment in real time to detect physical objects or images around the user. Based on this, it automatically recognizes and executes virtual information or programs related to the objects or images. Once the program is executed, the fully automated program system visually displays the information to the user on the AR device's screen. This information can be displayed in a way that allows interaction with the user by overlaying it on the user's field of view or adjusting its position within the screen. Furthermore, 6G technology, which operates in fully automated program systems, supports advanced interactivity, allowing users to organize and control programs in an AR environment through gestures or voice commands. This allows for an experience tailored to the user's convenience and environment.For example, in a virtual space such as a company projected into the digital world, all employees can work remotely on computers together, allowing one person to operate multiple computers remotely, significantly improving system control efficiency. Furthermore, using this embodiment, when purchasing a vehicle, the entire driving interface of the vehicle can be recreated in a non-face-to-face virtual space, allowing the user to experience the driving conditions realistically and implementing full autonomous driving through this embodiment. Furthermore, the entire driving interface of the vehicle can be recreated in a non-face-to-face virtual space on the user's terminal. Specifically, by implementing full autonomous driving through augmented reality in a virtual space projected into the digital world on a phone, the user can experience the same as riding in the vehicle they are purchasing, enabling them to test the vehicle as if it were real. In other words, the output (vehicle) produced when the user's desired voice command passes through various stages of the computer's automatic execution program, including the learning section, generation section, automatic control section, output generation section, feedback section, and evaluation section, can be the output (vehicle) that most closely matches the user's desired voice command. Because this process is tailored to the user's preferences, materials, colors, etc., the output (vehicle) can be the closest match to the user's desired voice command. If the printout (car) is the desired product in the test, selecting the purchase button automatically connects to the integrated payment server of the computer auto-run program device. That is, when the purchase button is selected, the printout generation unit detects and reflects the selection. The automatic control unit, upon receiving the detection and reflection from the printout generation unit, automatically performs an authentication process on the integrated payment server based on the user's identity information in the user information database to complete the payment. Once payment is complete, the completion button automatically withdraws the user from the printout (car) purchase site, maintaining a secure and safe environment.

[0110] Additionally, the "Complete" button is detected and reflected in the output generation unit. The automatic control unit automatically connects to the user information database based on the detection and reflection of the output generation unit. Based on the user's identity information, including their delivery address, phone number, and name, the output (car) is automatically delivered to the delivery address.

[0111] Additionally, the auto-control button simultaneously enters the delivery address from the user information database and switches the user information database to power-saving mode. The user information database is automatically powered on only upon user authentication (facial recognition), ensuring complete security of the user's identity.

[0112] In addition, the fully automatic program system linked to the 6G technology according to the embodiment introduces a new electromagnetic wave blocking technology (i.e., (Al) aluminum is plated on the thinnest possible vinyl, a thin vinyl is attached thereon, and (Al) aluminum is plated on it for a second time. The thinnest possible vinyl is attached thereon, (Al) aluminum is plated on it for a third time, and the thinnest possible vinyl is attached thereon to make (Al) aluminum paper for electromagnetic wave blocking. The (Al) aluminum paper for electromagnetic wave blocking can be used as a protective film by blocking wind, rain, cold, and ultraviolet rays outdoors, and the (Al) aluminum paper for electromagnetic wave blocking can be sealed to block electromagnetic waves, enabling remote wireless charging of the terminal. The fully automatic program system linked to the 6G technology is characterized by including a thin aluminum paper that enables remote wireless charging of the terminal by sealing the generator of the terminal with the (Al) aluminum paper for electromagnetic wave blocking.

[0113] In addition, the embodiment applies the blind typing function to a fully automated program system, allowing the blind or the mute to use the computer through simple automatic search. Furthermore, the embodiment allows the deaf person's fingerprint input and motion input using sign language to be verbally reproduced to the other party through the fully automated program system, and the other party's language is also reproduced to the deaf person in written or sign language, allowing the deaf (mute) person to live and communicate freely.

[0114] Referring to Figure 8, BERT is an example of a pre-trained model, and the specific configuration and operation are described in Figure 4.

[0115] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware (referred to herein, for convenience, as software), various forms of program or design code, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0116] The various embodiments presented herein can be implemented as a method, system, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage system. For example, computer-readable storage media include, but are not limited to, magnetic storage systems (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, OPPs, DISCs, etc.), smart cards, and flash memory systems (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, the various storage media presented herein include one or more systems and / or other machine-readable media for storing information.

[0117] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0118] [Explanation of symbols]

[0119] Computer Autorun Program ---------------------- 100

[0120] Memory ----------------------------------------- 120

[0121] Processor --------------------------------------- 130

[0122] Program Autorun Server ------------------------ 200

[0123] How to Apply 6G Technology to Program Output ------ 300

[0124] Fully automated program system incorporating 6G technology --------- 500

[0125] Virtual Environment Platform --------------------------------- 600

Claims

The fully automatic program system that links 6G technology is It is composed of a computer automatic execution program device (100), a program automatic execution server (200), a virtual environment platform (600), and an integrated payment server (142). In a fully automatic program system that links 6G technology, The program auto-execution server (200) may have a website and various software, AI navigation, and may have built-in AI blind shopping guide, AI mute conversation commentator, AI painter, AI counselor, AI designer, AI oriental doctor, AI music band, AI composer, lyricist, etc., and many AI navigations may be invented and stored. The program auto-execution server (200) creates an application for program auto-execution and distributes it to a computer auto-execution program device (100), and in the embodiment, the computer auto-execution program device (100) installs the program auto-execution application distributed from the server (200), and automatically executes various applications and programs pre-installed in the device (100) through user's voice and motion recognition using the installed application. A device (100) capable of automatically executing by a user's voice signal converts a received voice signal into text through a language model including an STT model, applies the converted text to a program automatic execution server (200) to detect an instruction for program control, inputs the detected instruction into an automatic control program to control the program, and generates a model and data for generating an output result through the program control as at least one of a 6D-based metaverse, virtual reality, and augmented reality, and stores the same in memory. The processor can perform the technical specifications according to the embodiments of the present disclosure by executing at least one instruction stored in the memory, that is, the processor stores instructions for program control matching the voice and text recognized by the processor, the generation unit generates instructions for program control by combining instructions matched to each keyword through an instruction generation model, the automatic control unit inputs the generated instructions into the program to automatically control the program, and the result output by the automatic control outputs a model and data for generating at least one of a 6D-based metaverse, virtual reality, and augmented reality by the output generation unit, and the output generation unit generates a model and data to be output to a virtual environment based on the execution result of the program, and the model can include Model-based and Model-free, CAD Model, Seen / Linseen, environment, character, etc., and the data can include Objavers-IVIS, LLM, Diffusion, texture, animator, physical characteristics, etc., and the output generation unit analyzes the program execution result to determine what kind of output should be generated, and the output generation unit generates a 3D modeling tool or Using BEXEL software, you can design 6D models used in actual virtual environments, define the shape, color, material, etc. of the model, and add animations. The output generation section outputs models and data to the selected virtual environment on space imaging, virtual reality, and augmented reality platforms, so that users can view or interact with the 6D model. In the case of the metaverse, it can be shared or interacted with other users, and since users can manipulate and interact, when the model is moved or manipulated within the virtual environment, the output generation section detects and reflects this, and collects, shares, and links sensor data and real-time location information.A fully automatic program system that links 6G technology to enable testing of outputs in a virtual environment and purchasing and payment of desired outputs, based on the user's location and viewing direction, and can place and adjust virtual information. In claim 1, Computer auto-run programs, It consists of a communication unit, memory, and processor, A user's voice signal received in a communication unit is converted into text through a language model including an STT conversion model, the converted text is applied to a program automatic execution server (200) to detect an instruction for program control, the detected instruction is input to an automatic control program to control the program, and a model and data for generating an output result through the program control into at least one of a 6D-based metaverse, virtual reality, and space imaging are generated and stored in memory. A computer-automatic execution program device characterized in that the processor executes at least one instruction stored in a memory to perform the technical specifications according to the embodiments of the present disclosure, that is, stores instructions for program control matching voice and text recognized by the processor, the generation unit generates instructions for program control by combining instructions matched to each keyword through an instruction generation model, the automatic control unit inputs the generated instructions into a program to automatically control the program, and the result output by the automatic control outputs a model and data for generating at least one of a 3D-based metaverse, virtual reality, and augmented reality by the output generation unit, and the output generation unit generates a model and data to be output to a virtual environment based on the execution result of the program. In the second paragraph, a computer automatic execution program device characterized in that a STT (Speech to text) conversion model is stored in the memory, and the instruction is generated by converting a control command spoken by a user by voice by the STT conversion model and extracting a keyword from the converted text. . In the second paragraph, the computer automatic execution program, A computer automatic execution program device characterized by accumulating and learning user-specific program usage history to update a program automatic control model and generating a user-specific automatic control processor through the updated program automatic control model. In the fourth paragraph, the program automatic control model, A computer automatic execution program device, characterized in that the pre-learned model is acquired by fine-tuning, and the pre-learned model is learned in a semi-supervised learning manner by applying at least one of a masked language model (MLM) and a next sentence prediction (NSP) to a plurality of corpuses, and the fine-tuning is acquired by supervising learning of the pre-learned model on a plurality of learning data including learning input data and learning labeling data, wherein the plurality of learning input data include program extraction and program control processes for voice inputs of a plurality of users and program control histories for each user. In the second paragraph, a computer automatic execution program device characterized in that the memory stores instructions for program control matching the recognized voice and text, and the memory stores models and data for generating the program output result as at least one of a 5D-based metaverse, virtual reality, and augmented reality. . In the first paragraph, The program auto-run server (200) is Websites, software, neutrino datasets, spatial stereoscopic imaging systems, and AI navigation are stored. The above server (200) creates an application for automatic program execution and distributes it to a program automatic execution device (100), and the device (100) installs the program automatic execution application distributed from the server (200), and automatically executes several applications and programs pre-installed on the device (100) through user's voice and motion recognition using the installed application. In the first paragraph, The virtual environment platform allows users to operate and control programs in the virtual environment of the virtual environment platform through motion or voice commands due to the advanced interactivity of 6G technology, so that products can be tested in a non-face-to-face virtual space connected to the virtual environment platform, and vehicle performance tests can be performed by reproducing all Internet driving interfaces in the virtual space, allowing realistic driving experiences, and fully autonomous driving can be implemented. In addition, if it is a desired product, it can be purchased by selecting the desired product through a performance test of the vehicle to be purchased in augmented reality, and in addition, all employees of the company can work on computers together in the non-face-to-face virtual space of the company connected to the virtual environment platform, and since one person can work on multiple computers from a distance, it creates new work systems and work methods for each sector of the company and society, and is a program fully automatic system linked to 6G technology that features a non-face-to-face shopping mall and shopping method in a virtual space where all things coexist. In Article 8, the integrated payment server (BRICS PAY BRICS International Integrated Payment) in a non-face-to-face space connected to the virtual environment platform Integrated Payment Server (BRICS PAY BRICS International Integrated Payment), User information database, Includes SSL / TLS security protocols. In the virtual environment platform, users can operate and control programs in an augmented reality environment through gestures or voice commands, so that the entire driving interface can be reproduced in the virtual space of the Internet to test vehicle performance, enable realistic driving experiences, and implement fully autonomous driving. If you choose to purchase the product you want by testing the performance of the vehicle you want to purchase in augmented reality, Automatically connects to the integrated payment server (BRICS PAY BRICS international integrated payment server) stored in the computer automatic execution program (100) and automatically completes payment using personal information stored in the user information database. If data security is required, the program implements error handling mechanisms, applies security protocols such as SSL / TLS, or performs additional tasks to encrypt and authenticate data. A fully automated program system that links 6G technology and features an automatic payment method and a system that automatically sets the address based on the identity information stored in the user information database and automatically delivers to the customer. The multi-functional input function Collect input information from all users, including blind and mute users, and generate results on a virtual space platform based on the collected input information and provide them to users. Recognizes the user's voice or motion input and integrates multiple programs to automatically control the program. A fully automatic programmable system that links 6G technology, featuring a multi-functional input function that enables all operations including driving, home purchase, commuting, etc., that the user (disabled person) desires. In the first paragraph, The computer automatic execution program device (100) connects 6G technology. "It can be used after being installed on a space stereoscopic image phone or computer, and converts a command (neutrino) that the user wants to execute into a voice signal in the communication section into text through a language model including an STT conversion model, and applies the converted text to a program automatic execution server (200) to detect an instruction for program control, and inputs the detected instruction into an automatic control program to control the program, and generates data for outputting the output (neutrino) to a 6D-based space stereoscopic image system through the program control and stores it in memory, and the output generation model stored in the memory converts the output (neutrino) which is the result of the program automatic control into 6D-based 6-dimensional or 10-dimensional space by adding "line" and "gravity". n The artificial neural network model that generates an output (neutrino) of a different dimension, an endlessly extending universe connected by infinite extension and vibration of dimensions, and for this purpose, the processor stores instructions for program control matching the voice and test recognized by the processor to execute the data (neutrino) stored in the memory, and the generation unit generates instructions (neutrino) for program control by combining instructions matched to each keyword through the instruction generation model, and the automatic control unit inputs the generated instructions into the program to automatically control the program, and the result (neutrino) output by the automatic control generates data for output to a 6D-based spatial stereoscopic image system by the output generation unit, and the output generation unit designs a 6D model used in an actual virtual environment using BEXEL software based on the data generated based on the execution results of the program, and the output generation unit outputs the data converted by the neutrino to the spatial stereoscopic image system, so that the user can view anywhere on Earth at will and observe the universe, The above-mentioned neutrinos are as fast as light, and the 5D technology has 360 terabytes of storage space and 6G technology with a storage period of 13.8 billion years. The program-integrated fully automatic system is capable of space travel. Space travel is 10 years away for us. n It will enlighten you to the dimension of space and time. n The space-time of the dimension is a form in which the universe is made up of an infinite number of light-like things that start from one point and vibrate and extend infinitely, and this is a form like gravity, and the two coexist and extend endlessly to form the universe, and it is a form that fills the universe and extends endlessly like a bomb exploding. Since neutrinos are made of the same form, space travel that shows us the data of neutrinos by connecting them to a fully automated system of programs is 10 n A fully automated program system that links 6G technology and is characterized by being able to prove that it is a dimension of space and time. In the first paragraph, The above programmable automatic system linked to the 6G technology. A fully automatic programmable system that combines 6G technology with electromagnetic wave blocking capabilities, harvesting capabilities, and remote wireless charging using wireless power transmission technology. In Article 12, The above program fully automatic system introduces a new electromagnetic wave blocking technology into the terminal to block electromagnetic waves during wireless charging, thereby enabling remote wireless charging. That is, (Al) aluminum is plated on the thinnest possible vinyl, a thin vinyl is attached thereon, (Al) aluminum is plated a second time on top of that, the thinnest possible vinyl is attached thereon, (Al) aluminum is plated a third time on top of that, and a thin vinyl is attached thereon to create a thin (Al) aluminum paper that blocks electromagnetic waves of the terminal. This electromagnetic wave blocking paper can be used as a protective film to block wind, rain, and cold outdoors, and when the generator of the terminal is sealed with this electromagnetic wave blocking paper, electromagnetic waves can be blocked, enabling remote wireless charging of the terminal. This is a programmable fully automatic system that links 6G technology and features a manufacturing method. In Article 10 A fully automatic program system that links 6G technology A fully automated program system that features a built-in blind guide that helps blind people who have difficulty with daily life to purchase desired products by providing AI functions to shopping malls to explain the color, shape, material, etc. of each product in the shopping mall using language, blind characters, or sign language, as blind people cannot see many products when shopping. In Article 10 A fully automatic program system that links 6G technology A fully automatic program system that uses 6G technology to enable a mute person who has difficulty in daily life to have a built-in mute conversation function that allows the text input by the mute person to be conveyed to the other party as speech and the other party's text or sign language to be conveyed to the mute person. In Article 9 The integrated payment server (BRICS PAY BRICS international integrated payment) in a non-face-to-face space connected to a virtual environment platform requires perfect security. In a non-face-to-face shopping mall in a virtual space where all things coexist, when a user clicks on the purchase selection (141) for a desired product, the integrated payment server (142) automatically connects to the user information database (122), automatically performs an authentication procedure to complete payment, and upon completion of purchase (143), the integrated payment server automatically connects to the user information database (122) and automatically receives the user's identity information such as address, phone number, name, etc., and automatically delivers the desired product (printed goods) to the delivery location, and upon completion of purchase (143), the user automatically withdraws from the desired product (printed goods) purchase site or shopping mall to maintain a safe and secure state, and upon completion of purchase (143), the user information database (122) is switched to a power-saving state, and the user information database (122) can be powered on only after passing the user authentication (facial recognition, etc.) from the integrated payment server, thereby safely protecting the user's identity information, and in cases where data security is required, the program implements an error handling mechanism, applies a security protocol such as SSL / TLS, or performs additional tasks for data encryption and authentication, thereby enabling the integrated payment server (BRICS PAY BRICS) to operate. A fully automated program system that links 6G technology and features perfect security (the international integrated payment server has the same configuration as the integrated payment server, so its description is omitted). In Article 11, The space stereoscopic image phone connected to 6G technology can run the phone with voice and motion input without touchscreen input by installing the computer automatic execution program device (100) of the present invention in the "iPhone" body, and also eliminates the battery, which is the biggest problem of the "iPhone", and minimizes the volume of the phone by installing an "atomic battery" that has good performance, is smaller than a coin, and can be used for 50 years, so it can be manufactured in various forms such as a brush, necklace, bracelet, ring, etc., and there is no need to worry about the battery of the phone exploding, and it also uses the existing display and installs a "projection system on photoresist" including LED and lens on the back of the phone, and sends an image to the controller via wired or wireless to the control tablet, and the image is transmitted from the LED, and the light of the screen emitted by each module is projected onto the convex lens installed on the outside of the LED to enlarge the image, and the light of the enlarged image is projected onto the center point of the entrance pupil of the "projection system" installed on the outside of the lens, and is reflected from the center point of the exit pupil through the "projection system", and the reflected line in the image space is A space stereoscopic imaging phone and its structure and manufacturing method characterized in that the space stereoscopic imaging phone intersects with a space imaging plane and forms an intersection point to obtain a space stereoscopic imaging that is significantly larger than the projected object, and when the user moves or adjusts the model within a virtual environment, the output generating unit detects and reflects this, so that the user can move or transform the space stereoscopic imaging as if it were real, can take images, can travel through space, and can automatically execute the phone by installing a program device (100). The program's fully automatic system is a program's fully automatic system that links 6G technology, featuring (neutrinos) that can be applied in practice. In Article 11, The space stereoscopic image phone automatically controls the program device (100) with the user's voice (motion) command (space stereoscopic image) and connects to the "space stereoscopic image system" stored in the program automatic execution server, and in the space stereoscopic image, the display source is used to send the light of the content to be displayed to the LED controller, and when the image is sent to the controller, the LED transmits the image, and the light of the screen emitted by each module is projected onto the convex lens connected to the LED to enlarge the image, and the light (a1, a2, a3, a4) of the enlarged image is projected onto the center point p of the entrance pupil of the "projection system" connected to the lens, and is reflected from the center point p' of the exit pupil through the "projection system", and the reflected line (a1", a2", a3", a4") in the image space intersects the space imaging plane and forms the intersection point (a1', a2', a3', a4'), so that a space imaging significantly larger than the projected object can be obtained, and the light emitted continuously from the LED screen The light of the different screens creates a new intersection point on another spatial imaging plane according to the time of emission, and thus the spatial imaging that appears at the continuously generated different positions can produce a stereoscopic image in the image space, and the size of the image can be adjusted using the lens focal length, and at the same time while watching the spatial stereoscopic image, if you call "neutrino" to the program device (100) of the phone, it can automatically connect to the "neutrino" data set" stored in the program automatic execution server, and through the neutrino data connection, the extremely small neutrino can travel through space from the 6th dimension to the infinite 10n-th power space-time, and this is a spatial stereoscopic image that can be executed by the program device characterized by already entering the neutrino era in the 6G era.

Images