System for providing integrated control-based digital twin development tools to reduce deployment and operational costs

Abstract

The present invention relates to a system for providing an integrated control-based digital twin development tool for reducing introduction and operation costs by providing an essential 3D model automatic generation function required for building a digital twin system, thereby reducing 3D development effort. To this end, the present invention provides a system for providing an integrated control-based digital twin development tool for reducing introduction and operation costs, comprising: a video upload unit for uploading a video file captured in a 360-degree direction; a first process unit for automatically extracting and augmenting the video file uploaded through the video upload unit into frame-unit image files; a second process unit for extracting a 3D model through an image deep learning model of the source file augmented through the first process unit; and a simulation unit for creating a 3D model object through the second process unit and simulating it through a viewer.

Description

Technology Field

[0001] The present invention relates to a development tool provision system. More specifically, it relates to an integrated control-based digital twin development tool provision system for reducing implementation and operation costs. Background Technology

[0002] The manufacturing industry urgently needs to adopt digital twin solutions to improve productivity and continuously resolve various on-site issues.

[0003] However, the unit cost and operating expenses for adopting digital twin solutions act as barriers to entry, which limits the widespread adoption of digital twins.

[0004] The reasons for the high implementation cost of digital twin solutions include the following:

[0005] First, it is difficult to lower development costs for digital twins and XR content because there are no alternatives to reduce the labor costs required for 3D model development.

[0006] Second, due to the lack of standardization for the large amount of data generated in the manufacturing industry, new development infrastructure for data interfaces is required for every project.

[0007] The reason for the high operating costs of digital twin solutions is that significant investment is required to apply a new digital twin system whenever equipment is added or lines are rearranged from the facilities at the time of system introduction.

[0008] As such, there is a growing need to develop practical and efficient solutions to lower adoption barriers for clients seeking to implement digital twin solutions, as well as solutions that reduce maintenance costs and enable effective operation after implementation.

[0009] Prior Art: KR Published Patent Application 10-2024-0043354 (2024.04.03) The problem to be solved

[0010] The present invention has been devised to solve the aforementioned problems, and in particular, aims to provide a standard development tool for a solution for the development and operation of a digital twin. means of solving the problem

[0011] The integrated control-based digital twin development tool providing system for reducing introduction and operation costs according to the present invention, devised to achieve the above objective, includes a video upload unit that uploads a video file captured in a 360-degree direction; a first process unit that automatically extracts the video file uploaded through the video upload unit into frame-unit image files and then augments them; a second process unit that extracts a 3D model through an image deep learning model of the source file augmented through the first process unit; and a simulation unit that generates a 3D model object through the second process unit and then simulates it through a viewer.

[0012] In addition, one embodiment of the present invention may further include a JenX-DT comprising: a 3D model upload unit that uploads 3D model data automatically generated through Jeni3D; a data interface unit that provides a standardized data interface connection function; and an equipment placement unit that provides an equipment placement function by drag and drop by equipping equipment 3D asset modules by manufacturer.

[0013] Additionally, the first processing unit checks the extension of a video file uploaded through the video upload unit and checks the number of frames per second, and the second processing unit inputs the frames extracted from the first processing unit into an image deep learning model to generate point cloud data and normalizes it, and the generated point cloud data is stored in an internal standard format and the AI ​​inference result is stored as a cache in a temporary directory as an intermediate result, and may include metadata so as to be linked with subsequent format conversion and preview functions.

[0014] Additionally, GenEye3D may further include a precision evaluation unit that evaluates whether point cloud data generated by an AI model in a second processing unit has a value greater than or equal to a set precision by comparing it with an object augmented in a first processing unit.

[0015] Additionally, the data interface section may include: a standard data format definition module that defines and documents the structure of JSON (JavaScript Object Notation) data including coordinates, attributes, and state between Unity and Unreal and the server; an external system integration module that designs REST (Representational State Transfer) APIs (Application Programming Interfaces) for integration with external systems including sensors and facility control systems, and applies a security authentication token-based calling method; a hierarchy tag registration module that assigns hierarchical tags to Unity objects and provides a UI (User Interface) for entering tag names; a tag deletion module that deletes hierarchy tags registered by the hierarchy tag registration module; a spreadsheet form download module that provides a function to download a predefined tag form as a spreadsheet file, having columns including ID, Name, Parent, and Level; and a spreadsheet batch upload module that registers multiple tag information in bulk through a spreadsheet file and reflects it in the tree structure after validating the file.

[0016] Additionally, the equipment placement unit may include a drag-and-drop placement module that places equipment icons by dragging and dropping them by applying placement rules and saves coordinate information; a collision detection module that checks the minimum spacing between equipment placed by the drag-and-drop placement module and displays a warning message in case of a collision; and a placement data storage module that saves the current equipment placement status in a level format, taking into account user account-based server storage integration and version management. Effects of the invention

[0017] According to the present invention, the introduction of a digital twin can have the effect of improving a company's competitiveness through system optimization, cost reduction, and risk management.

[0018] In addition, according to the present invention, there is an effect of reducing 3D development effort by providing an essential 3D model automatic generation function required for building a digital twin system.

[0019] In addition, according to the present invention, by providing a standard data interface function for field data connection, there is an effect of reducing the effort required for data connection, which occupies a significant amount of effort during the development of a digital twin system, and ensuring connection stability.

[0020] In addition, according to the present invention, the cost of 3D development can be reduced by providing common assets at the manufacturing site required for the implementation of a digital twin system. Brief explanation of the drawing

[0021] FIG. 1 is a conceptual diagram of an integrated control-based digital twin development tool providing system for reducing introduction and operation costs according to an embodiment of the present invention. Figure 2 is a conceptual diagram of GenEye3D in Figure 1. FIG. 3 is an exemplary diagram of an integrated control-based digital twin system developed by an integrated control-based digital twin development tool providing system for reducing introduction and operation costs according to an embodiment of the present invention. FIGS. 4 to 7 are drawings for explaining examples of screen designs of GenX DT. Specific details for implementing the invention

[0022] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, it should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present invention, if it is determined that a detailed description of related known components or functions may obscure the essence of the present invention, such detailed description is omitted. Additionally, while preferred embodiments of the present invention will be described below, the technical concept of the present invention is not limited or restricted thereto and can be modified and implemented in various ways by those skilled in the art.

[0023] FIG. 1 is a conceptual diagram of a system providing an integrated control-based digital twin development tool for reducing introduction costs and operating costs according to an embodiment of the present invention, FIG. 2 is a conceptual diagram of GenEye3D in FIG. 1, and FIG. 3 is an example diagram of an integrated control-based digital twin system developed by the system providing an integrated control-based digital twin development tool for reducing introduction costs and operating costs according to an embodiment of the present invention.

[0024] A digital twin development tool provision system based on integrated control for reducing introduction and operation costs according to one embodiment of the present invention is comprised of JenAI-3D (10) and JenX-DT (20), with reference to FIGS. 1 to 3.

[0025] GenEye3D enables users to capture and upload 360-degree videos using smartphone cameras, etc., allowing them to secure high-quality 3D model content even with videos of this quality level.

[0026] Through this, GenEye3D can drastically reduce the development period and unit cost of projects requiring 3D model development, and, for example, can be provided as a development tool to digital twin development companies.

[0027] GenX DT is a solution that performs digital twin operations by receiving 3D models created through Geni3D. It features scalable standard data interface connections and distribution functions, providing a UI / UX that ensures development convenience through simple operation.

[0028] GenX DT configuration provides connectivity features for IoT data interfaces, such as smart factory data, and allows the application of standard connection tags from the existing manufacturing industry.

[0029] GenX DT provides functions for uploading 3D models generated from Geni3D solutions and connecting data interfaces, and is equipped with manufacturer-specific equipment 3D asset modules to provide standard 3D asset equipment and lines.

[0030] In addition, GenX DT provides convenient facility placement features through 3D asset drag-and-drop and offers an integrated control-based monitoring UI / UX.

[0031] GenEye3D can be developed as a client and can also be expanded and deployed as a server system.

[0032] The Geni3D client includes application development and distribution system setup, login and authentication request functions, video upload functions, remote server interface API, and final 3D model object download functions.

[0033] The Geni3D server system includes, for example, the establishment of an Apache Tomcat-based web server environment, control functions for existing Geni3D solutions, member management and authentication functions, the ability to run a server simulation viewer, and anti-theft features such as time limits based on pricing policies.

[0034] To this end, GenEye3D includes a video upload unit, a first processing unit, a second processing unit, an accuracy evaluation unit, a training data linkage unit, and a simulation unit.

[0035] The video upload section uploads video files filmed from a 360-degree direction.

[0036] The first processing unit automatically extracts the video file uploaded through the video upload unit into frame-by-frame image files and then augments them.

[0037] Image file augmentation is the process of generating additional image files based on frame image files collected from a video to secure sufficient data for training.

[0038] The first processing unit checks the extension of the video file uploaded through the video upload unit and checks the frames per second.

[0039] The second processing unit extracts a 3D model through an image deep learning model of the source file augmented through the first processing unit.

[0040] The second processing unit inputs the frames extracted from the first processing unit into an image deep learning model to generate point cloud data and then normalizes it.

[0041] The generated point cloud data is stored in an internal standard format, and AI inference results are cached in a temporary directory as intermediate results, including metadata to enable linkage with subsequent format conversion and preview functions.

[0042] The precision evaluation unit compares the point cloud data generated by the AI ​​model in the second process unit with the object augmented in the first process unit and evaluates whether it has a value greater than or equal to the set precision.

[0043] The simulation unit creates a 3D model object through the second process unit and then simulates it through a viewer.

[0044] When a user uploads a video captured in 360 degrees to the GenEye3D solution, it is possible to automatically acquire a high-quality 3D model through image deep learning.

[0045] GenEye3D can drastically shorten the development period by building 3D data using AI, and enables faster rendering by expressing complex 3D scenes with relatively less data, making it possible to achieve more natural rendering than existing polygon-based modeling.

[0046] In addition, GenEye3D uses blobs of various sizes and shapes to easily represent complex structures, enabling the maintenance of high quality even in scenes requiring intricate detail.

[0047] The following is an example of the response between the user side and the server side of GenEye3D.

[0048] User side Server side Upload the video you want to turn into 3D. The server generates the video using the point cloud method. Preview the 3D converted file with a viewer. The server shows the conversion progress to the user. The user downloads the 3D converted file. The server provides files created for the user. Check who created what in the creation list. The server provides a generated list.

[0049] Referring to FIG. 1, GenX DT (20) automatically receives 3D model objects created by GenAI3D and distributes them to a web-based integrated control solution or a C / S-based integrated control solution by automatically uploading 3D models, providing basic 3D assets, and providing a standard API for data connection, thereby enabling a reduction of the total workload of the integrated control system by about 50%.

[0050] Previously, creating 3D models required significant labor, as it involved obtaining 3D drawings, utilizing video 3D model software, utilizing 3D scanners, utilizing LiDAR for drones, and utilizing actual measurement data to create 3D model objects through LowPoly 3D modeling, UV Map, Texture, and Shader work.

[0051] When using GenEye3D, the process is simple and no expensive equipment is required, resulting in a 70% reduction in labor compared to existing work.

[0052] In addition, previously, creating a digital twin required a significant amount of work because it was necessary to receive 3D model objects, upload the 3D models, perform animation work, and perform data connection work, and then deploy them to a web-based integrated control solution or a C / S-based integrated control solution.

[0053] Referring to Figure 1, when using GenX DT (20), 3D models are automatically uploaded, basic 3D assets are provided, and a standard API for data connection is provided, so there is a 30% reduction in effort compared to existing work.

[0054] The GenX DT digital twin solution provides an open interface compatible with UNITY and Unreal-based data interface standards and automation systems such as Siemens and Rockwell, allows for configuring specific settings by directly dragging systems and modules into a 3D world, and vividly implements systems and machines as digital twins for simulation and virtual commissioning using real-time 3D HMI, and can be utilized on various platforms such as Windows, Linux, Mac, and iOS.

[0055] In addition, GenX DT detects and corrects errors early by simulating and testing all relevant processes in the digital twin in conjunction with the control system.

[0056] In addition, GenX DT is supported by the open Unity and Unreal engines and provides customized business models embedded in an extensive ecosystem.

[0057] To this end, GenX DT includes a 3D model upload unit, a data interface unit, and an equipment placement unit.

[0058] The 3D model upload section uploads 3D model data automatically generated through Geni3D.

[0059] The data interface section provides standardized data interface connection functions.

[0060] To this end, the data interface section includes a standard data format definition module, an external system integration module, a hierarchy tag registration module, a tag deletion module, a spreadsheet form download module, and a spreadsheet batch upload module.

[0061] The Standard Data Format Definition module defines and documents the structure of JSON data, including coordinates, properties, and state, between Unity and Unreal and the server.

[0062] JSON (JavaScript Object Notation) is a method of representing data as key-value pairs and is used in web API responses, configuration files, data exchange formats, and more.

[0063] The external system integration module designs REST (Representational State Transfer) APIs (Application Programming Interfaces) for integration with external systems, including sensors and equipment control systems, and applies a security authentication token-based calling method.

[0064] The Hierarchy Tag Registration Module assigns hierarchical tags to Unity objects and provides a User Interface (UI) for entering tag names.

[0065] The tag deletion module deletes hierarchy tags registered by the hierarchy tag registration module.

[0066] The spreadsheet form download module provides a function to download a predefined tag form as a spreadsheet file that has columns including ID, Name, Parent, and Level.

[0067] The spreadsheet batch upload module registers multiple tag information in bulk via a spreadsheet file, and reflects it in the tree structure after validating the file.

[0068] The equipment placement section is equipped with equipment 3D asset modules by manufacturer and provides a drag-and-drop equipment placement function. This function enables the resolution of the operational cost issues mentioned in the technology forming the background of the invention.

[0069] To this end, the facility placement unit includes a drag-and-drop placement module, a collision detection module, and a placement data storage module.

[0070] The drag-and-drop placement module applies placement rules to position equipment icons via drag-and-drop and saves coordinate information.

[0071] The collision detection module checks the minimum spacing between facilities placed by the drag-and-drop placement module and displays a warning message in the event of a collision.

[0072] The batch data storage module stores the current facility batch status in a level format, taking into account user account-based server storage integration and version management.

[0073] The present invention, including GenEye3D and GenXDT, can be expanded and applied in various forms such as web-based integrated control systems and application program integrated control systems, and can be utilized in manufacturing industries, smart cities, smart farms, etc. by expanding its application to various industrial sectors.

[0074] Figure 3 illustrates an example of an integrated control-based digital twin system developed with GenAI3D and GenXDT.

[0075] Referring to Figure 3, it can be seen that despite the drastically reduced labor and equipment, it demonstrates a quality suitable for using digital twin functions such as mirroring, monitoring, and simulation.

[0076] Figures 4 to 7 are drawings for explaining examples of screen designs of GenX DT.

[0077] The digital twin scene (1) can place the model in the factory scene and allows the user to see how it is currently placed.

[0078] The hierarchical structure (2) shows how the models are hierarchically structured.

[0079] Drag and drop (3) allows you to place the model in the scene by dragging and dropping.

[0080] The model (4) currently placed in the scene shows the currently placed model and displays the clicked model when clicked.

[0081] PLC-MQTT (Message Queuing Telemetry Transport) gateway web navigation (5) allows the gateway to be installed if the gateway is not installed, and if the gateway is installed, it allows the gateway to be configured.

[0082] Referring to Fig. 5, the location of the model and the model's name tag can be added and deleted (1) by pressing the + button to set the tag name and tag value, by pressing the v button to delete the tag, and by pressing the trash can button to delete all tags.

[0083] Adding and deleting value tags (2) that the model has can be done by pressing the + button to set the tag name and tag value, pressing the v button to delete the tag, and pressing the trash can button to delete all tags.

[0084] The animation speed setting (3) can set the speed at which the animation moves.

[0085] Referring to Fig. 6, MQTT configuration (1) can set the Host, port, username, and password.

[0086] Referring to Fig. 7, the list of models placed in the scene (1) shows the names of the models placed in the scene as a list, and the value UI (2) below appears.

[0087] The value UI (2) sets the value tags in a graph form and changes over time, allowing users to visually check them.

[0088] The foregoing description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications, changes, and substitutions within the scope of the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments and accompanying drawings. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0089] 10 : GenEye3D 20 : GenX DT

Claims

Claim 1 A system for providing an integrated control-based digital twin development tool for reducing introduction and operation costs, comprising: a video upload unit for uploading video files captured from a 360-degree direction; a first processing unit for automatically extracting and augmenting the video files uploaded through the video upload unit into frame-unit image files; a second processing unit for extracting a 3D model through an image deep learning model of the source file augmented through the first processing unit; and a simulation unit for creating a 3D model object through the second processing unit and simulating it through a viewer. Claim 2 A system for providing an integrated control-based digital twin development tool for reducing introduction and operation costs, comprising: a 3D model upload unit for uploading 3D model data automatically generated through Geni3D; a data interface unit for providing a standardized data interface connection function; and a facility placement unit equipped with manufacturer-specific facility 3D asset modules to provide a facility placement function via drag and drop, further comprising a JenX-DT. Claim 3 A system for providing an integrated control-based digital twin development tool for reducing introduction and operation costs, wherein, in claim 1, the first processing unit checks the extension of a video file uploaded through a video upload unit and checks the number of frames per second, the second processing unit inputs the frames extracted from the first processing unit into an image deep learning model to generate point cloud data and normalizes it, the generated point cloud data is stored in an internal standard format, and the AI ​​inference result is stored as a cache in a temporary directory as an intermediate result, and includes metadata to enable linkage with subsequent format conversion and preview functions. Claim 4 In paragraph 3, GenEye3D further comprises a precision evaluation unit that evaluates whether point cloud data generated by an AI model in a second processing unit is compared with an augmented object in a first processing unit and evaluates whether it has a value greater than or equal to a set precision, thereby providing a digital twin development tool based on integrated control for reducing introduction and operation costs. Claim 5 In paragraph 2, the data interface unit comprises: a standard data format definition module that defines and documents the structure of JSON (JavaScript Object Notation) data including coordinates, attributes, and state between Unity and Unreal and the server; an external system integration module that designs REST (Representational State Transfer) API (Application Programming Interface) for integration with external systems including sensors and facility control systems and applies a security authentication token-based calling method; a hierarchy tag registration module that assigns hierarchical tags to Unity objects and provides a UI (User Interface) for entering tag names; a tag deletion module that deletes hierarchy tags registered by the hierarchy tag registration module; a spreadsheet form download module that provides a function to download a predefined tag form as a spreadsheet file having columns including ID, Name, Parent, and Level; and a spreadsheet batch upload module that registers multiple tag information in bulk through a spreadsheet file and reflects it in a tree structure after file validation, thereby providing a system for providing a digital twin development tool based on integrated control for reducing implementation and operation costs. Claim 6 In paragraph 2, the equipment placement unit comprises: a drag-and-drop placement module that places equipment icons by dragging and dropping them by applying placement rules and stores coordinate information; a collision detection module that checks the minimum spacing between equipment placed by the drag-and-drop placement module and displays a warning message in case of a collision; and a placement data storage module that stores the current equipment placement status in a level format, taking into account user account-based server storage linkage and version management, thereby providing a system for providing an integrated control-based digital twin development tool for reducing introduction and operation costs.

Images