Automated construction system for generating learning data based on robot movements in a manufacturing environment
The automated system addresses the challenge of generating standardized training data for robot motion control by preprocessing and processing data into a multi-purpose format using generative AI, ensuring high-quality data for AI-based robot control across diverse manufacturing environments.
Patent Information
- Application Number
- KR1020250009357
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2045-01-22
AI Technical Summary
Current manufacturing environments face challenges in generating standardized, high-volume training data for AI-based robot motion control due to non-standardized data collection, high costs, and difficulty in adapting data across different industries and environments.
An automated construction system comprising a robot motion extractor, analyzer, and repositories to collect, preprocess, and process data into a multi-purpose format, utilizing generative AI to generate high-quality training data for robot motion control.
The system effectively collects, preprocesses, and processes data into a usable format, providing high-quality training data for robot motion control across various environments, enhancing AI model performance and adaptability.
Smart Images

Figure 112025008682645-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to an automated construction system for producing high-quality and high-volume training data for AI-based robot motion control in a manufacturing environment. It relates to a construction system for automatically identifying and classifying situations according to the work environment, generating commands according to the situation as explicit / ambiguous based on sLLM, and automatically generating training data for controlling robot motion, such as detailed reasoning processes for robot behavior, responses to behaviors, recognition of objects, relationships between objects, and segment areas of objects. Background Technology
[0003] Robot automation technology is establishing itself as a core technology for improving productivity and quality in manufacturing environments. Robots used in manufacturing perform various processes such as assembly, inspection, and packaging, providing both precision and efficiency. Recently, with the advancement of collaborative robot (Cobot) technology, environments are being realized where human workers and robots can safely collaborate in the same space. This robotic technology maximizes motion precision and operational efficiency by integrally utilizing advanced sensors, path optimization algorithms, and real-time data processing technologies.
[0004] AI technology is further evolving the role of robots in manufacturing environments. According to a 2023 McKinsey report, AI-based manufacturing systems are estimated to have the potential to increase productivity by up to 30%. AI analyzes manufacturing data to automatically optimize work paths, predict equipment failures in advance, and automate quality control processes. AI supports robots in performing complex quality inspection tasks with precision, leading to reduced defect rates and lower production costs. In particular, generative AI, which emerged in 2022, is being applied to enhance process efficiency across various industries, and research necessary for this is progressing rapidly.
[0005] In the future, robotics and AI technologies in manufacturing environments hold the potential for continuous development and are expected to become the core driving force of smart factories. Establishing a smart factory requires sophisticated robot movements and the learning and decision-making capabilities of AI; to achieve this, securing and utilizing large-scale training data is crucial. For instance, technology is needed to standardize unstructured data generated during manufacturing processes and convert it into AI training data. This training data plays a vital role in enhancing AI model performance and providing solutions optimized for the manufacturing environment.
[0006] However, training AI models in current manufacturing environments is difficult because the collected data is often non-standardized, making it hard to utilize directly without preprocessing. Furthermore, since the majority of training data is generated to suit specific domains, it poses a problem that makes it unusable in other industries. The high cost of constructing training data also poses a challenge for small-scale manufacturing plants in building AI models, and there is the issue of difficulty in generating large volumes of training data for various situations and environments. Prior art literature
[0008] Korean Registered Patent No. 10-1944087 Method for generating training data for training an artificial intelligence model for task matching The problem to be solved
[0009] Therefore, the objective of the present invention is to provide an automation system capable of collecting training data for robot motion control based on generative AI in a manufacturing environment, preprocessing and processing the data into a multi-purpose data format, and additionally generating data necessary for robot motion control using generative AI to provide high-quality training data. means of solving the problem
[0011] An automated construction system for generating learning data based on robot motion in a manufacturing environment according to the present invention for achieving the above objective comprises: a robot motion extractor that manages the processing of collecting and analyzing manufacturing robot-related learning data constructed from robot motion-related videos or other environments; a robot motion analyzer that analyzes work environments, work execution commands, object information, etc., from videos and images related to robot motion; a work management repository that stores overall process state information for generating learning data; and an analysis result repository that stores standardized processed learning data.
[0012] Here, it is desirable that the robot motion extractor collects source data through a robot data collector, stores the history of related tasks in a task management repository, extracts information related to robot motion control within a video or image through a robot image processor and a robot image processor, extracts the Segmentation Mask and BBox of an object within the image through an object segment processor, and transmits them to an analysis result repository in accordance with the standardized format of the training data.
[0013] In addition, it is desirable for the robot motion analyzer to transmit video or images to an image processing model composed of sLLM to generate command and task inference information for robot motion, and to transmit images to an object detection model composed of VLM to generate Segmentation Mask and BBox information of the object and transmit them to a robot motion extractor. Effects of the invention
[0015] According to the present invention, training data for robot motion control based on generative AI in a manufacturing environment is collected, preprocessed and processed into a data format that can be used for multiple purposes, and additional data necessary for robot motion control is generated using generative AI, thereby providing high-quality training data. Brief explanation of the drawing
[0017] FIG. 1 is an overall block diagram of an automation construction system for generating learning data based on robot operation in a manufacturing environment according to the present invention. Figure 2 is a block diagram of a robot motion extractor. Figure 3 is a block diagram of a robot motion analyzer. Figure 4 is the overall flowchart of the robot motion extractor. Figure 5 is a flowchart of the robot image processor in the robot motion extractor. Figure 6 is a flowchart of the robot image processor in the robot motion extractor. Figure 7 is a flowchart of the object segment processor in the robot motion extractor. Specific details for implementing the invention
[0018] Hereinafter, an automated construction system (1) for generating learning data according to robot operation in a manufacturing environment according to a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.
[0019] FIG. 1 is an overall block diagram of an automation construction system (1) for generating learning data according to robot motion in a manufacturing environment according to the present invention, FIG. 2 is a block diagram of a robot motion extractor (10), FIG. 3 is a block diagram of a robot motion analyzer (20), FIG. 4 is an overall flowchart of a robot motion extractor (10), FIG. 5 is a flowchart of a robot image processor (12) of a robot motion extractor (10), FIG. 6 is a flowchart of a robot image processor (13) of a robot motion extractor (10), and FIG. 7 is a flowchart of an object segment processor (14) of a robot motion extractor (10).
[0020] With reference to FIGS. 1 to 7, the configuration of an automation system (1) for generating learning data according to robot operation in a manufacturing environment is described in detail.
[0021] An automated construction system (1) for generating learning data based on robot motion in a manufacturing environment includes a robot motion extractor (10), a robot motion analyzer (20), a work management repository (30), and an analysis result repository (40).
[0022] The robot motion extractor (10) manages the processing of robot motion-related video or manufacturing robot-related learning data built in other environments, and analyzes the data.
[0023] The robot motion extractor (10) includes a robot data collector (11), a robot image processor (12), a robot image processor (13), and an object segment processor (14).
[0024] The robot data collector (11) collects source data.
[0025] The robot image processor (12) extracts information related to robot motion control within the image. The robot image processor (12) includes an image frame splitter (121) and a robot action and command extractor (122).
[0026] The video frame splitter (121) extracts the robot image.
[0027] The robot behavior and command extractor (122) extracts robot images.
[0028] The robot image processor (13) extracts information related to robot motion control within an image. The robot image processor (13) includes an explicit / ambiguous command classifier (131) and a detailed operation extractor (132).
[0029] The explicit / ambiguous command classifier (131) extracts command information for robot movements.
[0030] The detailed operation extractor (132) extracts information about detailed operation actions.
[0031] The object segment processor (14) extracts the Segmentation Mask and BBox of an object within an image. The object segment processor (14) includes an object attribute extractor (141) and a target object selector (142).
[0032] The object attribute extractor (141) identifies an object present in the image and extracts the attributes of the object.
[0033] The target object selector (142) extracts Segmentation Mask, BBox information, etc. present in the image.
[0034] The robot motion analyzer (20) analyzes the work environment, work execution commands, object information, etc., from the video and images related to the robot motion. The robot motion analyzer (20) transmits the robot motion video and the image extracted from the robot motion video to the image processing model from the robot image processor and the robot image processor of the robot motion processor, and extracts data according to the robot motion through the AI model. The robot motion analyzer (20) includes an image processing model (21) and an object detection model (22).
[0035] The image processing model (21) is configured with an image or image as sLLM. The image processing model (21) generates command and task inference information for robot movements.
[0036] The object detection model (22) is composed of an image as a VLM. The object detection model (22) generates the object's Segmentation Mask and BBox information.
[0037] The task management storage (30) stores the entire process state information of the training data generation.
[0038] The analysis result repository (40) stores standardized processed training data.
[0039] FIG. 1 is an overall block diagram of an automation construction system (1) for generating learning data according to robot operation in a manufacturing environment according to the present invention.
[0040] The robot motion analyzer and robot motion extractor within the system operate as services. Separately, the job management repository and analysis result repository operate within the system.
[0041] FIG. 2 is a block diagram of a robot motion extractor (10).
[0042] The robot motion extractor collects video data containing robot motions to be generated as training data from existing training data and robot video data, gathering information along with related metadata. The collected video data is processed by a robot image processor, which operates an image frame splitter and robot behavior and command extractors to extract information from the robot video. Subsequently, a robot image processor operates an explicit / ambiguous command classifier and a detailed operation extractor to extract information regarding commands and detailed action behaviors related to robot movements. Finally, an object segment processor operates an object attribute extractor and a target object selector to identify objects present in the image and extract their attributes, Segmentation Mask, and BBox information. The extracted information is stored in an analysis result repository in accordance with a pre-standardized training data structure. When processing large volumes of data, the robot motion extractor is configured as a system with multiple servers or a cloud environment to secure additional computing resources through an auto-scaling policy.
[0043] FIG. 3 is a block diagram of a robot motion analyzer (20).
[0044] The robot motion analyzer transmits robot motion videos and images extracted from them from the robot motion processor's robot image processor to the image processing model, extracting data based on robot motion through the AI model. It utilizes a service approach capable of handling a large volume of analysis requests by being configured with an AI Model Handler API to efficiently process analysis requests and responses from the AI model. The AI Model Handler API loads AI models used for video extraction sLLM, image extraction sLLM, and object detection / extraction VLM into memory, and upon receiving an analysis request, it performs real-time analysis processing and transmits the results.
[0045] FIG. 4 is an overall flowchart of a robot motion extractor (10).
[0046] The overall flowchart of the robot motion extractor outlines the process of generating image labeling data, starting with the collection of robot motion video data. The collected robot motion video data is divided into image frames to extract robot motion information and perform labeling. If a problem occurs during this process, the analysis of that video is halted, and the process switches to collecting other robot motion videos. If information is extracted successfully, the image labeling data is saved, and detailed operation plan information is extracted based on explicit or ambiguous commands associated with the robot motion in each image. Upon successful extraction, object-specific attributes (color, size, texture, etc.), positional relationships between objects, and Segmentation Mask and Bbox information are extracted and saved in a labeled format.
[0047] FIG. 5 is a flowchart of the robot image processor (12) of the robot motion extractor (10).
[0048] The flowchart of the robot image processor of the robot motion extractor selects motion episodes based on the collected robot motion video, and divides and saves the video into frames. It infers robot tasks from the robot motion video using sLLM and generates explicit or ambiguous commands based on this. The generated commands are classified again, and in the case of ambiguous commands, detailed operation plans are extracted by separating them into behavioral units. Again, object information within the video is extracted through sLLM to identify the object on which the robot performs a target task, and the corresponding data is saved in a labeled format.
[0049] FIG. 6 is a flowchart of the robot image processor (13) of the robot motion extractor (10).
[0050] The flowchart of the robot image processor of the robot motion extractor analyzes images extracted from robot motion video using sLLM to extract information related to actions and commands within the image. If the command for the robot operation is ambiguous, detailed operation plan information is additionally extracted, and object information within the image, positional relationships between objects, and attribute information (color, size, texture, etc.) are extracted, and the corresponding data is saved in a labeled format.
[0051] FIG. 7 is a flowchart of the object segment processor (14) of the robot motion extractor (10).
[0052] The flowchart of the object segment processor in the robot motion extractor loads object list information from labeling data stored in robot motion videos and images. The robot motion video is divided into frames, and a selection is made based on the starting frame of the video. For the selected image frames, Segmentation Mask and Bbox information for the objects are extracted from the VLM. Objects that serve as targets for the robot motion are additionally identified, and the corresponding data is saved in a labeled format.
[0053] Due to the automated system (1) for generating learning data based on robot motion in the above manufacturing environment, learning data for robot motion control based on generative AI in the manufacturing environment is collected, preprocessed and processed into a data format that can be used for multiple purposes, and additional data required for robot motion control is generated using generative AI to provide high-quality learning data. Explanation of the symbols
[0055] 1: Automation system for generating training data based on robot motion in a manufacturing environment 10: Robot motion extractor 11: Robot data collector 12: Robotic Image Processor 121: Image Frame Splitter 122: Robot Behavior and Command Extractor 13: Robot Image Processor 131: Explicit / Ambiguous Command Classifier 132: Detailed Operation Extractor 14: Object Segment Handler 141: Object Attribute Extractor 142: Target Object Separator 20: Robot motion analyzer 21: Image processing model 22: Object Detection Model 30: Task Management Repository 40: Analysis Result Repository
Claims
Claim 1 In an automation system for generating training data based on robot motion in a manufacturing environment, a robot motion extractor that manages the processing of collecting and analyzing training data related to manufacturing robots, such as videos related to robot motion or training data built in other environments; a robot motion analyzer that analyzes work environments, work execution commands, object information, etc., from videos and images related to robot motion; and a work management repository that stores overall process state information for training data generation. An automated construction system for generating training data based on robot motion in a manufacturing environment, comprising an analysis result repository that stores standardized and processed training data, wherein the robot motion extractor collects source data through a robot data collector and stores the history of related tasks in a task management repository, extracts information related to robot motion control within a video or image through a robot image processor and a robot image processor, extracts the Segmentation Mask and BBox of an object within the image through an object segment processor and transmits it to the analysis result repository in accordance with the standardized format of the training data, and wherein the robot motion analyzer transmits the video or image to an image processing model composed of sLLM to generate command and task inference information regarding robot motion, and transmits the image to an object detection model composed of VLM to generate Segmentation Mask and BBox information of the object and transmits it to the robot motion extractor. Claim 2 delete Claim 3 delete
Citation Information
Patent Citations
Robot control device, robot control method, and robot control program
JP2020075337A
Artificial intelligence-based automated robotic device
KR1020220125922A
Apparatus and method for controlling robot
KR102220304B1
Open-vocabulary robotic control using multi-modal language models
WO2024178241A1
