Method, device, and system for providing 3D modeling and AR motion simulation of user-photographed objects

Abstract

An apparatus according to one embodiment receives shooting input data including one or more object images or object video data obtained by shooting a real object from a user's terminal, generates a 3D object model corresponding to the real object based on the shooting input data, places the generated 3D object model in an augmented reality-based 3D space corresponding to the actual shot image or video, applies at least one spatial transformation among position translation, rotation, or size deformation to the 3D object model placed in the augmented reality-based 3D space, determines a motion applicable to the 3D object model based on structural parameters of the generated 3D object model, and applies the determined motion to the 3D object model placed in the augmented reality-based 3D space to simulate and provide the motion state of the 3D object model.

Description

Technology Field

[0001] The following embodiments relate to technology that provides a method, device, and system for providing 3D modeling and AR motion simulation of a user-captured object. Background Technology

[0003] With the recent improvement in mobile device performance and advancements in image processing technology, technologies that utilize images or video data captured directly by users to visualize real-world objects in virtual spaces are being utilized in various fields. In particular, there is a continuously increasing demand for 3D modeling and augmented reality technologies to provide virtual environments similar to real-world environments in fields such as interior design, product placement simulation, virtual exhibitions, and education and training.

[0004] However, conventional 3D modeling technology has often relied on methods of loading and using pre-made 3D models, or presupposed specialized equipment or complex modeling processes. As a result, there have been limitations in allowing general users to easily create and utilize 3D models of real-world objects through everyday photography alone. Furthermore, 3D object models generated from user-captured data often remain at the level of simply reproducing the external appearance, lacking the ability to consider the structural characteristics or operational capabilities of the object.

[0005] Meanwhile, although the technology of placing virtual objects in real space using augmented reality is also widely known, conventional technologies generally provided only basic spatial transformation functions for virtual objects, such as translation, rotation, and scaling. Technologies that apply realistic actions to objects or differentiate applicable actions based on the structural characteristics of the objects have not been sufficiently presented. Consequently, problems have existed where actions unrelated to the object's structure were applied, or where deformations or interference that cannot realistically occur took place.

[0006] Furthermore, conventional technologies often lacked clear criteria for determining actions applicable to an object, or were limited to simply reproducing the action selected by the user. Consequently, structural constraints or permissible ranges for modification were not sufficiently considered, leading to unrealistic results during the application of actions. In particular, the application of actions without considering the relationships or relative positions between the constituent parts of an object acted as a factor that could undermine the reliability of the user experience.

[0007] In addition, conventional technology lacked a systematic response to cases where the quality or shooting range of the input data captured by the user was insufficient for generating a 3D model. Even when the captured data was partially insufficient or information regarding a specific direction or part was not sufficiently secured, problems arose where model generation simply failed or the accuracy of the results was reduced. At such times, there was a limitation in that specific guidance was not provided to the user on which parts to additionally capture and how, leading to repeated shooting or trial and error.

[0008] Therefore, there is a need for technology that can generate a more precise 3D model of a real object based on input data captured by a user, and rationally determine applicable actions by considering the structural characteristics of the generated model. Furthermore, there is a need for technology that allows users to realistically verify the operational state of an object by intuitively simulating and providing the results of action application in an augmented reality-based 3D space. In addition, if the captured input data is insufficient, guidance technology is also required to identify the missing information and induce additional shooting corresponding to it.

[0009] The present invention was proposed in recognition of the limitations of such conventional technology and due to the technical need to provide 3D modeling based on user-captured objects and augmented reality-based motion simulation more accurately and reliably.

[0010] Therefore, technology is required to provide a method, device, and system for providing 3D modeling and AR motion simulation of user-captured objects. Prior art literature

[0012] Republic of Korea Registered Patent No. 10-2580961 (Published Sep. 20, 2023) Republic of Korea Registered Patent No. 10-2561210 (Published July 31, 2023) Republic of Korea Registered Patent No. 10-2413517 (Published June 28, 2022) Republic of Korea Registered Patent No. 10-1980261 (Published May 21, 2019) The problem to be solved

[0013] The embodiments aim to provide a method for creating a real object as a 3D object model based on image or video data captured by a user, and for simulating the placement and operation of the model in an augmented reality-based 3D space.

[0014] The embodiments aim to provide a method for analyzing the external shape and structural characteristics of an object from captured input data to establish relationships between structural regions and to rationally determine applicable motions by considering the corresponding structural constraints.

[0015] The embodiments aim to provide a method for automatically generating a new motion that minimizes structural interference while satisfying a user-specified target state for a generated 3D object model, and simulating and providing the motion.

[0016] The embodiments aim to provide a method for simulating a construction or placement sequence that is actually applicable step-by-step by considering the relationship between structural regions constituting a three-dimensional object model, and to provide this intuitively to the user.

[0017] The embodiments aim to provide a method for identifying insufficient structural information and generating and providing additional shooting directions or shooting guides for specific parts corresponding thereto when shooting input data is insufficient for creating a three-dimensional object model.

[0018] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood from the description below. means of solving the problem

[0020] According to one embodiment, a method for providing 3D modeling and AR motion simulation of a user-captured object may include: receiving shooting input data including one or more object images or object video data obtained by capturing a real object from a user's terminal; generating a 3D object model corresponding to the real object based on the shooting input data; placing the generated 3D object model in an augmented reality-based 3D space corresponding to the actual captured image or video; applying at least one spatial transformation among position translation, rotation, or size deformation to the 3D object model placed in the augmented reality-based 3D space; determining a motion applicable to the 3D object model based on structural parameters of the generated 3D object model; and applying the determined motion to the 3D object model placed in the augmented reality-based 3D space to simulate and provide the motion state of the 3D object model.

[0021] The step of generating a 3D object model corresponding to the real object based on the above-mentioned shooting input data may include: extracting object shape information including contours, feature points, and surface texture features of the real object from the above-mentioned shooting input data; dividing structural regions constituting the real object based on at least one of shape discontinuity sections, curvature change sections, or joint candidate sections based on the object shape information; analyzing at least one of connection relationships, relative positional relationships, or rotatable relationships between the divided structural regions to derive motion relationship information between the structural regions; setting the structure of the 3D object model by setting structural parameter information including a structural reference axis, structural degrees of freedom, and a structural parameter allowable range based on the motion relationship information between the structural regions; and configuring the 3D object model to include predefined motion application parameters linked to the structural parameter information.

[0022] The step of determining motions applicable to the 3D object model may include: querying candidate motions stored in a motion motion database based on motion relationship information between the structural regions; comparing motion parameter information included in each candidate motion with structural parameter information set in the 3D object model by matching parameter items of the same type; determining whether motion parameter information of each candidate motion is included within an allowable range defined by the structural parameter information based on the comparison result; determining whether interference occurs between structural regions when applying motion by referring to the relative positional relationship between the structural regions for the candidate motion determined to be included within the allowable range; comparing the amount of deformation occurring in the structural region when applying motion with predefined deformation allowable range data for the candidate motion; classifying each candidate motion into applicable motions, motions requiring restriction conditions when applied, or inapplicable motions based on the comparison result and whether interference occurs between the structural regions; and determining a set of motions applicable to the 3D object model in the augmented reality-based 3D space according to the classification result.

[0023] The method further includes the step of generating a new motion motion of the generated 3D object model; wherein the step of generating a new motion motion of the 3D object model may include: providing a motion goal input UI to a user's terminal for specifying a target position, target angle, or target state of the 3D object model in the augmented reality-based 3D space; converting an input received through the motion goal input UI into target state data corresponding to structural parameter information set in the 3D object model; comparing the target state data with the current state of the 3D object model to calculate a plurality of motion path candidates that satisfy an allowable condition defined by the structural parameter information; for each of the motion path candidates, evaluating the possibility of interference based on whether overlap or collision occurs between structural regions on the motion path; selecting the motion path with the lowest possibility of interference between structural regions based on the evaluation result to generate a new motion motion corresponding to the target state data; and applying the generated new motion motion to the 3D object model in the augmented reality-based 3D space to simulate and provide a motion process toward the target state.

[0024] The method further includes the step of simulating, step by step, the construction sequence to be performed to place the generated 3D object model and providing it to the user's terminal;

[0025] The step of simulating and providing the construction sequence of the above 3D object model step by step may include: identifying a construction target structural area where placement, fixing, or joining is performed for each structural area constituting the above 3D object model; establishing a successor-successor construction relationship based on positional relationships, joining relationships, or space occupancy relationships between the construction target structural areas for each of the identified construction target structural areas; defining a construction stage in which placement or fixing of the construction target structural areas is performed step by step based on the established successor-successor construction relationship; distinguishing between a structural area where placement or fixing is completed at each stage and a structural area that has not yet been constructed according to the defined construction stage and displaying them on the augmented reality-based 3D space; and sequentially providing a construction sequence simulation result to a user's terminal in which the completion status is updated step by step according to the progress of the construction stage.

[0026] After the step of receiving shooting input data from the user's terminal, the method may further include: a step of generating acquired structural data including structural data items used for generating the 3D object model based on the shooting input data; a step of comparing the acquired structural data with a predefined reference structural data item used for generating the 3D object model to identify data items among the reference structural data items that have not been acquired or do not meet the criteria of the predefined reference structural data item as insufficient data items; a step of generating shooting guide information to induce additional shooting corresponding to the insufficient data items when the insufficient data items are identified; and a step of providing the generated shooting guide information to the user's terminal.

[0027] The step of generating the above shooting guide information may include: identifying direction information or specific part information for which additional shooting of the physical object is required based on the above missing data item; generating a first request including direction information for which additional shooting is required or a second request including specific part information for which additional shooting is required based on the identified information; and generating shooting guide information including the first request or the second request.

[0028] A device according to one embodiment may be combined with hardware and controlled by a computer program stored on a medium to execute the method of any one of the methods described above. Effects of the invention

[0030] The embodiments can provide a method to intuitively check the spatial placement and operational status of a real object by placing a 3D object model generated based on an object captured by a user into an augmented reality-based space and simulating operational motion.

[0031] The embodiments can provide highly reliable motion simulation that does not violate structural constraints by selecting applicable motions based on the structural characteristics of an object and the operational relationship between structural regions.

[0032] The embodiments can automatically generate new motions that minimize interference between structures while satisfying a user-specified target state, thereby providing simulations even for previously undefined motions.

[0033] The embodiments can improve work understanding and planning efficiency by visualizing the actual applicable construction or placement sequence step-by-step based on the relationship between structural regions constituting a three-dimensional object model.

[0034] The embodiments can effectively supplement the completeness of the data required for creating a 3D object model by identifying structural deficiencies in the shooting input data and providing additional shooting guides corresponding thereto.

[0035] Meanwhile, the effects according to the embodiments are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0037] FIG. 1 is a schematic diagram showing the configuration of a system according to one embodiment. FIG. 2 is a flowchart illustrating the process of providing 3D modeling and AR motion simulation of a user-captured object according to one embodiment. FIG. 3 is a flowchart illustrating the process of creating a 3D object model corresponding to a real object according to one embodiment. FIG. 4 is a flowchart illustrating the process of determining a motion applicable to a 3D object model according to one embodiment. FIG. 5 is a flowchart illustrating the process of generating a new motion of a 3D object model according to one embodiment. FIG. 6 is a flowchart illustrating the process of simulating and providing the construction sequence of a 3D object model step by step according to one embodiment. FIG. 7 is a flowchart illustrating the process of generating shooting guide information to induce additional shooting based on shooting input data according to one embodiment. FIG. 8 is a flowchart illustrating the process of generating shooting guide information according to one embodiment. FIG. 9 is a drawing illustrating an example screen in which a display is output to guide a user that there is an applicable motion motion for a 3D object model placed in an augmented reality-based 3D space according to one embodiment. FIG. 10 is a drawing illustrating an example screen in which an object selection interface is provided for selecting a 3D object model to be placed in an augmented reality-based 3D space according to one embodiment. FIG. 11 is a drawing illustrating an example screen in which a 3D object model is created based on shooting input data captured by a user according to one embodiment, and shape and size information of the created 3D object model is displayed. FIG. 12 is a diagram illustrating an example screen in which a generated 3D object model according to one embodiment is placed in an augmented reality-based 3D space, positional relationship and distance information of the object are displayed, and spatial transformation is applied. FIG. 13 is an example diagram of the configuration of a device according to one embodiment. Specific details for implementing the invention

[0038] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, various modifications may be made to the embodiments, and thus the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.

[0039] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes modifications, equivalents, or substitutions that fall within the technical concept.

[0040] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0041] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0042] The terms used in the embodiments are for illustrative purposes only and should not be interpreted as intended to be limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0043] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments pertain. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0044] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0045] The embodiments can be implemented in various forms of products, such as personal computers, laptop computers, tablet computers, smartphones, smart home appliances, intelligent automobiles, kiosks, and wearable devices.

[0046] FIG. 1 is a schematic diagram showing the configuration of a system according to one embodiment.

[0047] Referring to FIG. 1, a system according to one embodiment may include a user terminal (100) and a device (200) capable of communicating with each other through a communication network.

[0048] First, the communication network can be configured regardless of the mode of communication, such as wired or wireless, and can be implemented in various forms to enable communication between servers and between servers and terminals.

[0049] The user's terminal (100) is a terminal directly operated by a user who uses the method for providing 3D modeling and AR motion simulation of a user-shot object according to the present invention, and is a device used to take a picture of a real object and to check a 3D object model and an augmented reality-based simulation result generated based on the captured data.

[0050] The user's terminal (100) may be, for example, a smartphone, a tablet computer, a laptop computer, a desktop computer, an augmented reality device, a virtual reality device, or a similar computing device, but is not limited thereto.

[0051] The user's terminal (100) can capture an image or video of a real object including a camera, and can provide the captured input data obtained through the capture to the device (200).

[0052] Additionally, the user's terminal (100) can visually output an augmented reality-based 3D space through a display and can display the placement status, positional relationship, distance information, applicable motion, simulation results, and shooting guide information of the generated 3D object model on the screen.

[0053] At this time, the user's terminal (100) can receive inputs such as object selection, viewpoint movement, zoom in or out, and target state designation through touch input, gesture input, button input, or other user input means.

[0054] That is, the user's terminal (100) can function as a user interface device responsible for capturing a real object, receiving user input, and providing a visual representation of a 3D object model and simulation results processed by the device (200).

[0055] The user's terminal (100) may be configured to perform all or part of the computational functions, storage / reference functions, input / output functions, and control functions of a conventional computer, and the user's terminal (100) may be configured to communicate with the device (200) via wired or wireless communication.

[0056] The user's terminal (100) may be connected to a website operated by a person or organization providing a service using the device (200), or may have an application developed and distributed by a person or organization providing a service using the device (200) installed. The user's terminal (100) may be linked with the device (200) through the website or application.

[0057] The user's terminal (100) can access the device (200) through a web page, application, etc. provided by the device (200).

[0058] The device (200) may be a private server owned by a person or organization providing a service using the device (200), a cloud server, or a peer-to-peer (P2P) set of distributed nodes. The device (200) may be configured to perform all or part of the computational functions, storage / reference functions, input / output functions, and control functions that a conventional computer possesses.

[0059] The device (200) can be configured to communicate with the user's terminal (100) via wired or wireless means.

[0060] The device (200) is a core processing device for performing a method of providing 3D modeling and AR motion simulation of a user-captured object, and receives shooting input data provided from the user's terminal (100), and can perform 3D object model creation, structural analysis, motion determination, and simulation provision based on the data.

[0061] The device (200) can generate a 3D object model corresponding to a real object by analyzing object image or object video data received from the user's terminal (100), and can set structural regions, structural parameters, and operation relationship information between structural regions based on the external shape information and structural characteristics of the generated 3D object model. In addition, the device (200) can determine applicable motion based on this structural information or generate a new motion that satisfies a target state specified by the user.

[0062] Additionally, the device (200) can place the generated 3D object model in an augmented reality-based 3D space and generate motion motion or construction sequence simulation results while applying spatial transformations such as positioning, rotation, or size deformation, and provide them to the user's terminal (100). Along with this, if the shooting input data is insufficient for generating the 3D object model, the device (200) can identify the missing structural data items and generate additional shooting directions or shooting guide information for specific parts corresponding thereto and provide them to the user's terminal (100).

[0063] That is, the device (200) can intuitively and reliably provide the user with the spatial characteristics and operational possibilities of a real object by integrally performing the entire process from receiving user shooting data to creating a 3D object model, determining structure-based motion, providing augmented reality-based simulation, and inducing additional shooting.

[0064] The process of providing 3D modeling and AR motion simulation of user-captured objects can be carried out through a server that includes a processor for collecting and processing such information. The service can be provided via a web-based platform or a smartphone application, and in some cases, processing can be performed by applying artificial neural networks or machine learning.

[0065] Additionally, the device (200) can communicate wirelessly or via wired connection with websites including social media platforms such as blogs, cafes, Instagram, Facebook, Twitter, and YouTube, and web pages including articles, and the device (200) can access the websites to obtain information.

[0066] Meanwhile, for convenience of explanation, only one user terminal (100) is shown in FIG. 1 and the following description, but the number of terminals can vary depending on the embodiment. As long as the processing capacity of the device (200) allows, there is no particular limit to the number of terminals.

[0067] In the present invention, Artificial Intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, and perceptual ability, and implements them on a computer, and may include concepts such as machine learning and symbolic logic. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. AI technology can analyze input data as a machine learning algorithm, learn from the results of the analysis, and make judgments or predictions based on the results of the learning. Furthermore, technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms can also be understood as falling within the category of AI. For example, technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control may be included.

[0068] Machine learning can refer to the process of training neural network models using experience in processing data. It implies that through machine learning, computer software improves its own data processing capabilities. A neural network model is constructed by modeling the correlations between data, and these correlations can be expressed by multiple parameters. A neural network model extracts and analyzes features from given data to derive correlations between them; machine learning can be defined as the process of optimizing the model's parameters by repeating this process. For example, a neural network model can learn the mapping (correlation) between inputs and outputs for data given as input-output pairs. Alternatively, even when only input data is provided, a neural network model can derive regularities between the given data and learn those relationships.

[0069] An artificial intelligence learning model or neural network model can be designed to implement the structure of the human brain on a computer and may include multiple network nodes that have weights and simulate neurons of a human neural network. The multiple network nodes may have interconnected relationships by simulating the synaptic activity of neurons, where neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes may be located in layers of different depths and exchange data according to convolutional connections. The artificial intelligence learning model may be, for example, an Artificial Neural Network (ANN) or a Convolutional Neural Network (CNN). As an embodiment, the artificial intelligence learning model may be machine learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning may include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost, Perceptron, Genetic Programming, and Clustering.

[0070] Among these, CNNs are a type of multilayer perceptron designed to use minimal preprocessing. CNNs consist of one or more convolutional layers and standard artificial neural network layers stacked on top, additionally utilizing weights and pooling layers. Thanks to this structure, CNNs can fully utilize two-dimensional input data. Compared to other deep learning architectures, CNNs demonstrate good performance in both image and audio fields. CNNs can also be trained using standard backpropagation. CNNs have the advantage of being easier to train than other feedforward artificial neural network techniques and using a small number of parameters.

[0071] Convolutional networks are neural networks comprising sets of nodes with bounded parameters. Many computer vision tasks have been significantly improved, driven by the increased size of available training data and the availability of computational power, combined with algorithmic advancements such as discriminative linear units and dropout training. In the case of massive datasets, such as those available for many tasks today, outfitting is not critical, and increasing the network size improves test accuracy. Optimal utilization of computing resources becomes a limiting factor. To address this, distributed, scalable implementations of deep neural networks can be employed.

[0072] FIG. 2 is a flowchart illustrating the process of providing 3D modeling and AR motion simulation of a user-captured object according to one embodiment.

[0073] Referring to FIG. 2, first, in step S201, the device (200) may receive shooting input data including one or more object images or object image data obtained by shooting a real object from a user's terminal.

[0074] That is, the device (200) can receive shooting input data including one or more object images or object video data obtained by shooting a real object from a user's terminal (100).

[0075] The user's terminal (100) can perform at least one of autofocus, auto exposure, and hand shake correction during the shooting process, and can generate an object image or object image data as a result of shooting.

[0076] The shooting input data is input data generated by the user's terminal (100) shooting a real object, and means a data set including one or more object images or object video data.

[0077] The shooting input data may be a set of still image files, a video file containing multiple frames, or a sequence of continuously captured frame images.

[0078] In addition to object images or object video data, the shooting input data may include additional information indicating the shooting conditions at the time of shooting.

[0079] Additional information may include, for example, at least one of the resolution of an image or video, the number of frames, the frame rate, the time of shooting, the exposure time, the focal length, the ISO value, camera internal parameters, and terminal attitude information.

[0080] Camera intrinsic parameters may be information including focal length, principal point coordinates, distortion coefficients, etc., and terminal attitude information may be information indicating the tilt, rotation direction, or gravity reference direction of the terminal at the time of shooting.

[0081] This additional information can be automatically generated by the user's terminal (100) when shooting and included as file metadata, and may also be provided optionally depending on the settings of the terminal or application.

[0082] The device (200) can store the received shooting input data and verify the integrity and basic quality of the data for use in subsequent processing.

[0083] The device (200) can check whether the file format of the shooting input data is a supported format, whether the file is corrupted, and whether it is decodingable.

[0084] When object image data is received, the device (200) can perform actual decoding to check whether the frame is restored normally.

[0085] The device (200) can determine whether the shooting input data satisfies the minimum shooting conditions for creating a 3D object model.

[0086] At this time, the minimum shooting conditions may include, for example, at least one of the total number of frames of the object image data, whether there is a change in viewpoint between frames, and whether the object is identified at a certain ratio or higher within the frame.

[0087] Whether a viewpoint change exists between frames can be determined based on whether the average displacement distance of feature points between consecutive frames or the amount of change in terminal attitude is greater than or equal to a preset standard.

[0088] These criteria may be set differently depending on the object size, shooting distance, or target 3D model precision.

[0089] The device (200) can normalize the components of the captured input data into a form that facilitates subsequent processing.

[0090] When object image data is received, the device (200) can extract frames at regular intervals by referring to the frame rate of the image.

[0091] For example, in a video captured at 30 frames, one frame can be extracted every 5 frames to generate a set of frame images with a uniform viewpoint distribution across the entire video.

[0092] When multiple object image data are received, the image resolution can be converted to a preset standard resolution or the image orientation information can be unified.

[0093] The device (200) can select different processing paths depending on whether there is additional information included in the shooting input data.

[0094] For example, if camera internal parameters or terminal attitude information are included, the device (200) can directly reference such information during subsequent depth estimation or coordinate matching processes.

[0095] Conversely, even if such information is not included, the device (200) can perform subsequent processing by estimating the relative structure using the change in feature points between frames and the change in relative position.

[0096] As a specific embodiment, a case in which a user photographs a folding chair using a user's terminal (100) is described.

[0097] The user can start shooting from the front of the chair, then move the device to the left to shoot the side, continuously shoot the rear and right sides, and then perform additional shooting from an angle looking down at the top.

[0098] This shooting process can generate object video data approximately 25 seconds in length, and the video can be recorded at 30 frames per second with a resolution of 1920×1080.

[0099] After receiving the object image data, the device (200) can calculate the total number of frames and the amount of feature point movement between frames to confirm that there is a sufficient change in viewpoint.

[0100] Subsequently, the device (200) extracts one frame every 6 frames from the entire video to generate a set of approximately 125 frame images, which can be used as input data for subsequent processing.

[0101] With this configuration, the device (200) can reliably secure shooting input data including various viewpoint information and shooting condition information of a real object.

[0102] In addition, by performing quality verification and normalization processing of the captured input data in advance, errors that may occur during the subsequent 3D object model generation process due to differences in the shooting environment or terminal performance can be reduced in advance.

[0103] As a result, the step of receiving shooting input data can function as a basic data preparation process to ensure processing accuracy and reproducibility throughout the present invention.

[0104] In step S202, the device (200) can generate a 3D object model corresponding to a real object based on the captured input data.

[0105] That is, the device (200) can generate a 3D object model corresponding to a real object based on the received shooting input data.

[0106] A 3D object model refers to a three-dimensional data structure that reproduces the external appearance and three-dimensional structure of a real object in digital space by analyzing object images or object video data included in the captured input data.

[0107] A 3D object model may include multiple vertex coordinates defined in a coordinate system, mesh information representing the connection relationships between vertices, information on the overall size and ratio of the object, and texture information for expressing surface textures.

[0108] These 3D object models are configured in a geometrically consistent form so that they can be placed in an augmented reality-based 3D space or have motion applied to them.

[0109] Here, a product object means an object in which the same 3D object model is mapped to and managed with at least one product attribute information among product identifier, product name, brand, category, product image, product specification, or external link information.

[0110] The device (200) may be configured so that it does not manage the 3D object model merely as simple modeling data, but can be utilized in conjunction with external product information in the form of the product object.

[0111] The device (200) can store identification information corresponding to a 3D object model and product attribute information in the form of a mapping table, or can refer to product attribute information stored in an external product management system.

[0112] The external product management system may include a management page operated in a PC environment, and product attribute information may be provided to the device (200) and the user's terminal (100) after being registered or modified in the said management system.

[0113] For example, the management page may include a product registration menu, a 3D object model file upload menu, and a product attribute information editing menu, and the administrator can input or modify 3D object model data corresponding to each product along with product name, brand, category, and product specification information in a PC environment. The results of such registration or modification are reflected by the device (200) and can be automatically synchronized and displayed on the library screen or product list screen provided to the user's terminal (100).

[0114] Product attribute information may include commercial-related items such as price, currency unit, discount rate, minimum order quantity, supply price, or cost price, but the device (200) may be configured to provide such items as reference information for product identification and comparison rather than using them directly for payment processing.

[0115] For example, product attribute information can be utilized as reference information in business-to-consumer (B2C), business-to-business (B2B), or platform-mediated B2B2C transaction structures. Additionally, in some implementations, commission information or reference price information for quotation calculation is provided, allowing it to be used as data for comparison and review in quotation-based purchasing flows.

[0116] For example, price information may be displayed together on a product detail screen or object information display area provided to the user's terminal (100) so that the user can determine whether to purchase while simultaneously checking the spatial placement result of the 3D object model and the price information. However, the method of creating, storing, and updating product attribute information may vary depending on the implementation environment and may be provided independently of the creation of the 3D object model and the placement and motion simulation functions in an augmented reality-based 3D space.

[0117] Through this, 3D object models are used for interior design and simulation, and can also be utilized as a medium for providing product information to assist in purchasing decisions as needed.

[0118] The device (200) can extract external shape information of a real object from an object image or object image data included in the shooting input data.

[0119] Appearance information refers to information including at least one of the object's contour location, major corner location, surface curvature variation range, brightness distribution, and color pattern distribution.

[0120] The device (200) can identify an object region by using at least one of color contrast, boundary line change, and frame-to-frame change amount to distinguish between an object and a background.

[0121] The device (200) can perform preprocessing on the captured input data prior to extracting external shape information.

[0122] Preprocessing may include removing noise contained in the image, correcting brightness deviations, or correcting lens distortion.

[0123] For example, if some areas of an object appear excessively dark because the lighting direction is not consistent during shooting, the device (200) can equalize the brightness by analyzing the brightness distribution of the entire image.

[0124] In addition, if edge distortion is detected in an image captured by a wide-angle camera, the device (200) can correct the distortion by referring to camera intrinsic parameters included in the shooting input data or by analyzing the change in curvature of the image edges.

[0125] The device (200) can extract feature points of an object from a preprocessed image or set of frames.

[0126] A feature point is a point that represents the shape of an object and refers to a point where the identity of the same object can be reliably tracked through movement between frames.

[0127] Feature points can be selected based on at least one of the intersection points of contour lines, points of abrupt change in curvature, and the center points of repetitive surface patterns.

[0128] The device (200) can select only feature points that are repeatedly detected in a plurality of images or frames, thereby preventing transient noise or background elements from being mistaken for feature points.

[0129] The device (200) can calculate depth information of an object by matching the same feature points between multiple frames.

[0130] Depth information refers to the relative distance information of an object's surface, calculated by analyzing how the position of the same feature point on the screen changes according to changes in the shooting time.

[0131] The device (200) calculates the movement vector and movement amount of a feature point between consecutive frames, and can adopt it as a feature point to be used for depth calculation only if the movement amount is greater than or equal to a certain standard.

[0132] If camera internal parameters or terminal attitude information are provided, the device (200) can use them to correct the depth information to be closer to the actual distance ratio.

[0133] Conversely, even if such information is not provided, the device (200) can construct the three-dimensional structure of the object in a manner that maintains relative depth relationships.

[0134] The device (200) can generate a basic three-dimensional shape of an object by combining the calculated depth information and the shape information.

[0135] The basic geometry generated at this stage is a mesh structure that encloses the entire outline of the object, reflecting the approximate shape and proportions of the real-world object.

[0136] The device (200) can check the distance between adjacent vertices and the vertex connection angle to adjust the shape so that the mesh is not abnormally twisted or self-intersection does not occur.

[0137] The device (200) can perform mesh refinement processing on the basic shape.

[0138] Mesh refinement may include processing that increases vertex density in sections with large changes in curvature and reduces unnecessary vertices in flat sections.

[0139] Through this, the 3D object model can sufficiently reflect the morphological characteristics of the real object without being excessively complex.

[0140] The device (200) can map a texture onto the surface of a 3D object model using color information or brightness information included in the shooting input data.

[0141] Texture is image information applied to the surface of a 3D object model and is used to express the materiality and appearance of a real object.

[0142] The device (200) corresponds color information extracted from each image or frame to mesh coordinates to connect the textures so that they are not broken, and can perform averaging or correction processing when there is a large color difference between frames.

[0143] As a specific embodiment, a case is described in which the device (200) generates a 3D object model using the shooting input data of a folding chair.

[0144] The device (200) can identify the outlines of the legs, seat, and backrest of the chair in the image frame, respectively, and extract the points where these outlines are connected as feature points.

[0145] The length and slope of the bridge can be estimated by analyzing the change in position of the bridge end feature points between frames.

[0146] In addition, the flat and curved areas of the seat plate can be distinguished based on the repetitive surface pattern appearing on the upper surface of the seat plate.

[0147] By synthesizing these analysis results, the device (200) can generate a 3D object model having a structure in which a seat, legs, and a backrest are connected.

[0148] Unlike a single-image-based planar model, the 3D object model generated in this way reflects the object's three-dimensional structure and relative size ratio.

[0149] As a result, spatial alignment with the real environment is improved when subsequently placed in an augmented reality-based 3D space, and the possibility of structurally unnatural deformation occurring even when motion is applied can be reduced.

[0150] For a detailed explanation regarding this, refer to Fig. 3.

[0151] In step S203, the device (200) can place the generated 3D object model in an augmented reality-based 3D space corresponding to an actual captured image or video.

[0152] That is, the device (200) can place the generated 3D object model in an augmented reality-based 3D space corresponding to an actual captured image or object image.

[0153] At this time, the augmented reality-based 3D space includes an AR placement screen in which a 3D object model is superimposed and displayed on a real environment video captured through the user's terminal (100), and the same motion data is applied to the screen as well so that the product operation state in the real space can be visually reproduced.

[0154] An augmented reality-based 3D space refers to a space configured such that a virtual 3D object is superimposed and displayed on an image or video of a real environment captured through a user's terminal (100).

[0155] This space includes a coordinate system that reflects the viewpoint, orientation, and distance relationships of the actual filming scene, and is configured so that virtual objects are perceived as existing within the real environment.

[0156] The device (200) can analyze object images or object image data included in the shooting input data to set a reference coordinate system of an augmented reality-based 3D space.

[0157] The reference coordinate system can be established based on environmental elements that have a relatively stable position within the shooting scene.

[0158] Environmental elements can be flat elements, such as floor surfaces, walls, and desk tops, or visual features with little positional variation, such as floor tiles with repeating patterns, wall corners, and straight edges of furniture.

[0159] The device (200) can identify a flat candidate in a captured image or video.

[0160] Plane candidates can be detected based on a constant region of changes in pixel gradient within a frame, feature point distribution, or relative position changes between frames.

[0161] For example, if multiple feature points move in the same direction and depth changes appear similar at the bottom of the captured image, the device (200) can determine that area as a candidate for a bottom plane.

[0162] If the shooting input data includes depth information or terminal attitude information, the device (200) can use this to calculate the direction vector and reference height of the planar candidate more accurately.

[0163] The device (200) can select a plane with high reliability among the identified plane candidates as a reference plane.

[0164] Reliability can be evaluated based on at least one of the number of feature points included in the plane candidate, the dispersion of distances between feature points, and the stability of the plane position between frames.

[0165] The selected reference plane is used as the reference coordinate system of the augmented reality-based 3D space.

[0166] The device (200) can position the generated 3D object model so as to correspond to a reference coordinate system.

[0167] Placement includes processing to set the reference position, reference direction, and reference height of the 3D object model.

[0168] The device (200) can identify an area corresponding to the bottom part or contact surface of a 3D object model and adjust the position so that the area contacts a reference plane.

[0169] At this time, the position can be calculated so that the center coordinates or reference contact surface coordinates of the 3D object model are naturally aligned within the reference coordinate system.

[0170] The device (200) can set the initial orientation of the 3D object model based on the captured input data.

[0171] For example, the front direction of a 3D object model can be set based on the direction in which the object is facing the camera in the initial frame of the captured video.

[0172] Alternatively, if terminal attitude information is provided, the orientation of the object may be set by referring to the terminal's direction vector during shooting.

[0173] The device (200) can update coordinate transformations so that a 3D object model in an augmented reality-based 3D space remains in the same position according to changes in the viewpoint of the captured image.

[0174] To this end, the device (200) can estimate the position and orientation of a virtual camera corresponding to each frame of a captured image and calculate the projection position of a 3D object model according to the virtual camera parameters.

[0175] Through this process, even if the user moves or rotates the terminal while shooting, the 3D object model can be displayed as if it were fixed in the actual environment.

[0176] The device (200) can reflect a placement correction input input from the user's terminal (100).

[0177] Users can fine-tune the position of a 3D object model through touch or drag input on the screen.

[0178] The device (200) can convert the input into a positional displacement or direction change amount in a reference coordinate system and reflect it in a 3D object model.

[0179] At this time, the device (200) can apply position constraints so that the object does not penetrate below the reference plane or remain excessively suspended in the air.

[0180] As a specific example, a case is described where a user places a folding chair on the living room floor and uses a video.

[0181] The device (200) can identify a floor plane as a reference plane by analyzing the repeating pattern of floor tiles and the pattern of feature point movement between frames in the captured image.

[0182] Based on the lower leg area of ​​the 3D object model created thereafter, the position can be set so that the chair is placed naturally on the floor plane.

[0183] When a user performs an input to move a chair toward a wall on the screen, the device (200) can update the position of the 3D object model by converting the input amount of movement into a change in position in the reference coordinate system.

[0184] Even if the user's terminal (100) moves during shooting, the device (200) can update virtual camera parameters to maintain the 3D object model in a fixed state at the actual floor position.

[0185] With this configuration, the device (200) can stably place a 3D object model in a position that is spatially aligned with the actual shooting environment.

[0186] As a result, visual discrepancies caused by inconsistencies with the actual environment can be minimized during subsequent spatial transformations or motion simulations.

[0187] In step S204, the device (200) may apply at least one spatial transformation among position translation, rotation, or size deformation to a 3D object model placed in an augmented reality-based 3D space.

[0188] That is, the device (200) can apply at least one spatial transformation among positional translation, rotation, or size deformation to a 3D object model placed in an augmented reality-based 3D space.

[0189] Spatial transformation refers to the process of changing the spatial state of a 3D object model in an augmented reality-based 3D space.

[0190] Spatial state refers to a state that includes the position coordinates, orientation coordinates, and size ratio of a 3D object model.

[0191] Space conversion can be performed by user input, or the device (200) can perform it automatically according to predefined rules.

[0192] The device (200) can interpret user operation information received from the user's terminal (100) as a space conversion request.

[0193] User operation information may be provided in the form of screen touch input, drag input, pinch input, rotation gesture, slider operation, or button selection.

[0194] The device (200) can distinguish the type of conversion according to the input type and quantify the direction, amount of movement, or amount of rotation of the input into a spatial conversion parameter.

[0195] The device (200) can convert the drag direction and drag distance generated from the user's terminal (100) into a movement vector of the reference coordinate system in order to apply positional movement.

[0196] At this time, the drag distance can be measured as a pixel movement amount in the screen coordinate system, and the device (200) can convert the pixel movement amount into an actual coordinate movement amount in an augmented reality-based 3D space.

[0197] To convert pixel displacement into actual coordinate displacement, the device (200) may refer to at least one of the focal length of a virtual camera, screen resolution, and current depth value of a 3D object model.

[0198] For example, the conversion factor can be determined so that the further a 3D object model is located from the virtual camera, the greater the same pixel displacement is converted into a larger actual displacement.

[0199] The device (200) can check whether the position change result maintains contact with the reference plane.

[0200] To this end, the device (200) can automatically correct the height coordinates of the 3D object model after movement so that the reference contact surface of the 3D object model touches the reference plane.

[0201] For example, if the bottom part of the 3D object model penetrates below the reference plane after positioning, the device (200) can adjust the height coordinate by increasing it by the penetration depth so that it is placed back on the reference plane.

[0202] Conversely, if the model is floating more than a certain distance from the reference plane after moving, the device (200) can restore the contact state by reducing the height coordinate by the floating distance.

[0203] The device (200) can calculate the rotation direction and rotation angle of a rotation gesture generated at the user's terminal (100) to apply rotation.

[0204] The rotation angle can be calculated from the relative angle change of two finger gestures or the slider input value.

[0205] The device (200) can set the reference axis of rotation to a vertical axis, a horizontal axis, or any axis specified by the user.

[0206] For example, when rotating a furniture object placed on a floor, the device (200) can set the vertical axis as the reference axis to control the change in direction without changing the tilt relative to the floor surface.

[0207] The device (200) can apply angle snapping so that it is aligned to an angle desired by the user even when the rotation input is fine.

[0208] Angle snapping refers to the process of aligning by rounding or correcting the rotation angle to a fixed unit.

[0209] For example, the device (200) can provide rotation angles aligned in 1-degree or 5-degree increments.

[0210] These angle units can be set differently depending on user settings or model type.

[0211] The device (200) can convert the enlargement or reduction ratio of the pinch input into a scale factor to apply a size variation.

[0212] The scale factor refers to the scaling value multiplied by the overall size of the 3D object model.

[0213] The device (200) can apply the scale factor as a continuous value, or it can apply it by changing the scale value in a fixed unit.

[0214] The device (200) can limit unrealistic enlargement or reduction by applying an allowable range of size deformation.

[0215] The allowable range can be defined as the minimum and maximum scales relative to the base size of the 3D object model.

[0216] For example, the device (200) can limit the scale factor to a range of 0.2 to 3.0.

[0217] This scope may be set differently depending on the object type, purpose of use, or user settings, and is not limited to the scope described as an example.

[0218] The device (200) can check the spatial consistency of the 3D object model while the spatial transformation is applied.

[0219] Spatial consistency refers to whether a 3D object model is physically positioned with environmental elements such as reference planes and walls without appearing awkward.

[0220] The device (200) can check whether it intersects with a reference plane and a wall surface by using the outer bounding box or outer surface of the 3D object model.

[0221] When an intersection occurs, the device (200) can calculate the depth of the intersection and correct the position or limit the rotation angle in the direction in which the intersection is removed.

[0222] As a specific example, the state in which a user places a 3D object model of a folding chair in an augmented reality-based 3D space on a living room floor is described.

[0223] The user can drag the chair on the screen to move it next to the sofa.

[0224] The device (200) can calculate the actual coordinate movement amount using the drag pixel movement amount on the screen and the current depth value of the chair, and move the chair on the floor plane.

[0225] Afterward, when the user performs a rotation gesture to change the direction of the chair, the device (200) can rotate the chair 45 degrees relative to the vertical axis.

[0226] Additionally, when the user performs a pinch-in input to see the chair reduced and placed in a narrow space, the device (200) can apply a scale factor of 0.8 times.

[0227] In this case, if a condition occurs where a part of the chair leg penetrates below the floor plane, the device (200) can calculate the penetration depth and automatically correct the height coordinates.

[0228] By applying such spatial transformations, users can examine the same 3D object model under various position, orientation, and size conditions.

[0229] As a result, it is possible to perform a layout review of the actual installation environment, verify the possibility of interference with the surrounding environment during object operation, and evaluate usability based on spatial conditions in advance.

[0230] In step S205, the device (200) can determine a motion applicable to the 3D object model based on the structural parameters of the generated 3D object model.

[0231] That is, the device (200) can determine a motion applicable to the 3D object model based on the structural parameters of the generated 3D object model.

[0232] Structural parameters refer to information that expresses the structural characteristics of a 3D object model in the form of numerical values ​​or conditions.

[0233] Structural parameters are reference information used to restrict or allow how each part constituting a 3D object model can move, and are used to determine the suitability of motion.

[0234] Structural parameters may include at least one of the connection relationship between structural regions, rotatableness, rotatable angle range, movable distance range, deformableness, and deformable tolerance range.

[0235] The structural area refers to the partial structure that constitutes the 3D object model.

[0236] Structural regions can be classified based on sections of shape discontinuity, sections with large changes in curvature, or sections where connections with other parts are concentrated.

[0237] For example, in the case of a folding chair, the seat, backrest, legs, and the connecting part between the seat and backrest can each be classified as a structural area.

[0238] The device (200) can identify structural regions by analyzing boundaries where angle changes between vertices occur rapidly in the mesh of a 3D object model, or sets of vertices that are repeatedly connected.

[0239] The device (200) can analyze the connection relationships between structural regions.

[0240] The connection relationship indicates whether two structural regions are connected in a fixed state, a state that allows rotation, or a state that allows movement.

[0241] For example, if two structural regions share one or more common vertices and can rotate relative to each other around a specific axis, the device (200) can determine that the connection is a rotational connection.

[0242] Conversely, if no change in relative position occurs between structural regions, it can be determined as a fixed connection.

[0243] The device (200) can set a structural reference axis for each structural area.

[0244] A structural reference axis refers to the axis that serves as a reference when a structural area rotates or moves.

[0245] The structural reference axis can be set based on the main direction of the mesh constituting the structural area, the direction of the vertex arrangement of the connection part, or the orientation information of the actual object observed in the captured input data.

[0246] For example, the backrest of a folding chair can have a structural reference axis set so that it can rotate forward and backward relative to the hinge direction connected to the seat.

[0247] The device (200) can set the structural degrees of freedom for each structural region based on the structural reference axis.

[0248] Structural degrees of freedom refer to the types and dimensions of movement allowed within a structural domain.

[0249] When the structural degrees of freedom are 1, only rotation around a single axis may be allowed, and when the structural degrees of freedom are 2 or more, movement combining rotation and translation may be allowed.

[0250] The device (200) can set structural degrees of freedom by combining connection relationships by structural area and reference axis information.

[0251] The device (200) can set the allowable range of structural parameters for each structural area.

[0252] The allowable range of structural parameters may include minimum and maximum values ​​of rotation angle, minimum and maximum values ​​of travel distance, or the maximum allowable ratio of strain.

[0253] For example, the rotation angle of the backrest of a folding chair can be set to a range of 0 to 110 degrees, and rotation exceeding this range may be determined as an unallowed movement.

[0254] These tolerance ranges may be established by reflecting the actual structural shape observed in the shooting input data, the possibility of interference between structural areas, or empirical criteria considering user safety.

[0255] Motion refers to a motion pattern in which at least one of the structural regions of a 3D object model changes its position, angle, or shape over time.

[0256] Motion data can be stored as predefined motion data, and each motion data can include one or more motion parameters.

[0257] Motion parameters may include at least one of the amount of change in rotation angle, rotation speed, amount of change in travel distance, direction of movement, and motion sequence information.

[0258] The device (200) can compare the structural parameters and the motion parameters included in the motion by item.

[0259] For example, if a specific motion includes a motion parameter that rotates the backrest from 0 degrees to 130 degrees, the device (200) can compare whether the rotation angle range exceeds the structural parameter allowable range of 0 degrees to 110 degrees.

[0260] If the comparison result exceeds the allowable range, the device (200) may classify the motion as an inapplicable motion.

[0261] The device (200) can examine whether interference occurs between structural regions even for motions included within the allowable range.

[0262] Interference refers to the phenomenon where one structural region encroaches upon or overlaps the outer boundary of another structural region during the application of motion.

[0263] The device (200) can divide the progression section of the motion of operation into a plurality of intermediate states and calculate the minimum distance between the outer bounding box or outer surface of the structural area in each intermediate state.

[0264] If the calculated minimum distance is less than a preset interference tolerance standard, the device (200) may determine that the motion is an interference-prone motion.

[0265] The device (200) can calculate the amount of deformation that occurs when motion is applied.

[0266] The amount of deformation can be defined as at least one of the rate of change in distance between mesh vertices of the structural area, the rate of change in length relative to the reference shape, or the rate of change in area.

[0267] The device (200) can determine whether there is a possibility of visual distortion or structural collapse by comparing the calculated amount of deformation with the deformation allowable range included in the structural parameters.

[0268] The device (200) can classify the operation motion by combining the results of comparing the allowable range of structural parameters, determining whether interference occurs, and comparing the amount of deformation.

[0269] Classification can consist of applicable motions, motions requiring constraints upon application, or non-applicable motions.

[0270] Motions requiring constraints upon application refer to motions where conditions are applied, such as limiting the rotation angle to the maximum allowable angle or reducing the motion speed within specific ranges.

[0271] As a specific example, a case in which motion is determined for a 3D object model of a folding chair is described.

[0272] The device (200) identifies the space between the seat and the backrest as a rotational connection and can set the rotatable angle from 0 degrees to 110 degrees.

[0273] The connection between the chair legs and the seat can be set as a fixed connection, and structural parameters can be configured so that movement or rotation is not allowed.

[0274] In this state, the motion of rotating the backrest from 0 to 90 degrees falls within the allowable range and can therefore be classified as an applicable motion.

[0275] On the other hand, the motion of rotating the backrest from 0 to 150 degrees exceeds the allowable range and may be determined as an inapplicable motion.

[0276] Through this judgment process, the device (200) can prevent unrealistic motions that do not reflect the actual structural constraints of the 3D object model from being selected.

[0277] As a result, subsequent motion simulations can provide only structurally valid and realistic motions, thereby improving the reliability and usability of the simulation results.

[0278] For a detailed explanation regarding this, please refer to Fig. 4.

[0279] In step S206, the device (200) can apply the determined motion motion to a 3D object model placed in an augmented reality-based 3D space to simulate and provide the motion state of the 3D object model.

[0280] That is, the device (200) can apply a determined motion to a 3D object model placed in an augmented reality-based 3D space to simulate and provide the motion state of the 3D object model.

[0281] The operational state refers to a state that includes the position, orientation, rotation angles for each structural region, and relative positional relationships between structural regions of a 3D object model at a specific point in time.

[0282] Motion state simulation refers to the process of continuously calculating and visually reproducing how these states change over time according to motion.

[0283] The device (200) can select motion motions classified as applicable motions or motions with constraints and use them for simulation.

[0284] Motion parameters included in the selected motion are applied in chronological order, and the device (200) can divide the entire period from the start time of the motion to the end time into multiple time intervals.

[0285] Each time interval can be set based on a fixed time interval or a fixed angle change amount, and intermediate operation states corresponding to each time interval are generated sequentially.

[0286] When calculating each intermediate operation state, the device (200) can refer back to the structural parameters to check whether the connection relationship and allowable range between structural regions are continuously maintained.

[0287] For example, if the rotation angle approaches the upper limit of the allowable range of structural parameters during the process of gradually increasing the rotation angle, the device (200) recognizes that point in time as an operation limit state and can restrict the operation progress so that the rotation does not exceed the allowable range thereafter.

[0288] This ensures that structural constraints are consistently maintained throughout the entire motion range.

[0289] The device (200) can repeatedly check whether interference occurs between structural regions in each intermediate operation state where the operation motion is applied.

[0290] To this end, the device (200) can calculate the outer bounding box or outer surface of the structural area in each intermediate operation state and calculate the minimum distance or overlap with other structural areas or environmental elements.

[0291] If the calculated minimum distance is less than a preset interference tolerance standard, the device (200) may determine the corresponding intermediate operating state as an interference risk state.

[0292] The device (200) can reflect this in the simulation results if an interference risk condition occurs.

[0293] For example, the device (200) can reduce the speed of the motion in the interference risk section or indicate that the motion is paused in that section.

[0294] Alternatively, a modified operation state can be generated and provided as a simulation result by automatically adjusting the rotation angle or travel distance to a range where interference does not occur.

[0295] This control is intended to allow users to intuitively understand the structural limitations of motion.

[0296] The device (200) can calculate the amount of deformation of the structural area in each intermediate operation state.

[0297] The amount of deformation can be defined as at least one of the rate of change in distance between mesh vertices constituting the structural region, the ratio of change in length relative to the reference shape, or the ratio of change in area.

[0298] The device (200) can determine whether there is a possibility of visual distortion or structural collapse by comparing the calculated amount of deformation with the deformation allowable range included in the structural parameters.

[0299] If the deformation exceeds the allowable range, the device (200) classifies the intermediate operating state as abnormal and may include a warning indication in the simulation result.

[0300] The device (200) can continuously output the results of the operation state simulation in an augmented reality-based 3D space.

[0301] The output is provided through the display of the user's terminal (100), and the operation of the 3D object model is superimposed on the actual captured image or object image.

[0302] The device (200) can synchronize the frame time of the captured video with the time interval of the operation state simulation so that even if the terminal moves or rotates, the operation of the 3D object model appears to be fixed in the actual environment.

[0303] The device (200) can provide a paused state at a specific point in time during the simulation through the user's terminal (100).

[0304] In this case, the user can visually check the distance between structural areas, the rotation angle, and whether contact is made at that point.

[0305] In addition, the device (200) can also provide a visual indication of whether it approaches or exceeds any of the structural parameter tolerance range, interference tolerance standard, or deformation tolerance range.

[0306] As a specific example, a case is described in which a motion of folding the backrest is simulated for a 3D object model of a folding chair.

[0307] The device (200) can select a motion in which the backrest rotates from 0 degrees to 90 degrees and divide it into 1-degree units to create 90 intermediate motion states.

[0308] In each intermediate operation state, the device (200) can calculate the rotation angle between the backrest and the seat and calculate the minimum distance with the leg structure area.

[0309] If the minimum distance to the bridge structure area decreases rapidly in a section where the rotation angle is 85 degrees or more, the device (200) may indicate the section as being in a state of interference risk.

[0310] This state can be provided in the form of a color change, a warning icon, or a message on the user's terminal (100) screen.

[0311] By providing such operation state simulations, users can verify the entire operation process step-by-step without actually manipulating the object.

[0312] In addition, potential structural interference, deformation limits, and tolerances during operation can be identified in advance, allowing for the prevention of problems that may arise during actual use or the manufacturing phase.

[0313] Through this, the device (200) can provide a highly reliable AR motion simulation that reflects the structural constraints of a real object by placing a 3D object model generated based on shooting input data captured from a user's terminal (100) in alignment with an augmented reality-based 3D space corresponding to an actual captured image or video, and then selecting and applying a motion motion according to structural parameters.

[0314] FIG. 3 is a flowchart illustrating the process of creating a 3D object model corresponding to a real object according to one embodiment.

[0315] Referring to FIG. 3, first, in step S301, the device (200) can extract object shape information including the contour, feature points, and surface texture features of a real object from the captured input data.

[0316] That is, the device (200) can extract object shape information including the contour, feature points, and surface texture features of a real object based on the captured input data.

[0317] Object appearance information refers to information that expresses the morphological characteristics of a real object as basic data for two-dimensional and three-dimensional analysis.

[0318] Object appearance information includes contour information defining the entire outer boundary of the object, feature point information representing changes in the object's shape, and surface texture feature information indicating the materiality or pattern of the object's surface.

[0319] This object shape information is subsequently used as reference data for structural area partitioning, structural relationship analysis, and structural parameter setting.

[0320] The device (200) can identify an object region based on object image or object image data included in the shooting input data.

[0321] Object region identification is a process for extracting object regions distinguished from the background within an image or video, and can be performed based on at least one of color distribution difference, brightness contrast, boundary line change, or amount of movement between frames.

[0322] For example, if the same area moves together between consecutive frames in object image data, the device (200) can determine that area as an object area.

[0323] The device (200) can extract contour information based on the identified object region.

[0324] Contour information refers to a set of boundary coordinates formed along the outer edge of an object area.

[0325] The device (200) can form a contour along a point where a rapid change in pixel values ​​occurs between an object area and a background area, and if the contour is broken or distorted by noise, it can correct it into a continuous contour by interpolating adjacent contour sections.

[0326] In this case, the contour coordinates can be stored in the form of a pixel coordinate system or a normalized coordinate system.

[0327] The device (200) can extract feature points by analyzing contour information and pixel distribution within the object area.

[0328] A feature point is a point that represents a change in an object's shape and refers to a point that can be reliably tracked as the same location when comparing frames.

[0329] Feature points can be selected based on points of abrupt change in the direction of the contour, branching points of the contour, or points within the object where there is a distinct change in brightness or color.

[0330] The device (200) can prevent temporary noise or background elements from being mistaken for feature points by selecting not only feature points detected in a single frame but also feature points that are repeatedly detected in multiple frames.

[0331] The device (200) can extract texture features of the object surface.

[0332] Surface texture characteristics refer to patterns that appear repeatedly on the surface of an object, color distribution, changes in brightness or contrast, or characteristics of brightness changes caused by fine irregularities on the surface.

[0333] The device (200) can divide the interior of an object region into blocks of a certain size and then analyze a color histogram, brightness distribution, or pattern repeatability within each block to produce texture features.

[0334] These texture features can be used later to distinguish areas of the same material when dividing structural regions, or to secondarily determine the boundaries between different structural regions.

[0335] As a specific example, a case is described where a user photographs a folding chair and provides the photograph input data.

[0336] The device (200) can extract a contour line along the entire outer edge of the chair in the image frame and extract feature points at locations where the shape changes abruptly, such as the point where the legs and the seat meet, and the point where the seat and the backrest are connected.

[0337] In addition, different surface texture features can be extracted by analyzing the differences in color and light / dark distribution appearing in the fabric pattern on the upper part of the seat and the metal surface of the leg part.

[0338] The contours, feature points, and surface texture features extracted in this way can be integrated and stored as object shape information.

[0339] By extracting object shape information in this way, the device (200) can obtain basic data that reflects the overall shape and detailed structure of the actual object.

[0340] As a result, it becomes possible to make judgments that fully reflect the actual form of the object during the subsequent process of dividing structural areas or analyzing structural relationships.

[0341] In step S302, the device (200) can divide the structural region constituting the real object based on at least one of the shape discontinuity section, the curvature change section, or the joint candidate section, based on the object shape information.

[0342] That is, the device (200) can divide the structural region constituting the real object based on at least one of the shape discontinuity section, the curvature change section, or the joint candidate section, based on the extracted object shape information.

[0343] A structural area refers to a partial structure constituting a physical object, meaning a geometric or functional unit distinct from other parts.

[0344] The structure area is subsequently used as the basic unit for analyzing structural relationships, setting structural parameters, and determining whether to apply motion.

[0345] Therefore, structural domain partitioning is performed to reasonably determine operability without oversimplifying the actual structural characteristics of the object.

[0346] A shape discontinuity refers to a section in the external shape of an object where a continuous shape is suddenly interrupted or changes abruptly.

[0347] For example, sections where thickness changes abruptly, boundaries where one plane ends and another begins, or boundaries where different parts are joined may correspond to sections of shape discontinuity.

[0348] The device (200) can determine a point where the angle of change of direction between adjacent contour sections in the contour information is greater than or equal to a preset standard as a shape discontinuity section.

[0349] A curvature change section refers to a section on an object's surface where the curvature value changes abruptly.

[0350] Curvature is a numerical value representing the degree of bending of an object's surface; it is low in flat areas and high in sharply curved areas.

[0351] The device (200) calculates local curvature in a contour or surface mesh candidate included in object shape information, and if the difference in curvature between adjacent regions is greater than a certain standard, it can identify the boundary as a curvature change section.

[0352] At this time, the curvature calculation may be a 2D curvature calculated based on contour information, or a 3D curvature estimated using object shape information obtained from multiple viewpoints, and this may be set differently for each embodiment depending on the object's shooting method or analysis environment.

[0353] These sections of curvature change can be used as boundaries where different structural regions meet.

[0354] A joint candidate region refers to a connection site where relative rotation or movement between structural regions is highly likely to occur.

[0355] Joint candidate segments can be identified based on relative position changes repeatedly observed in the captured input data, or the pattern of rotational center shift of feature points.

[0356] For example, if two specific regions in the object image data move in different directions or the relative angle changes around a certain axis, the device (200) can determine the corresponding connection part as a joint candidate section.

[0357] The device (200) can establish structural region division criteria by combining one or more of shape discontinuous sections, curvature change sections, and joint candidate sections.

[0358] Since a mis-division may occur when dividing a structural area based on only a single criterion, the device (200) can increase the reliability of the division by considering multiple criteria together.

[0359] For example, a location where shape discontinuity sections and curvature change sections are detected simultaneously can be set as a priority structural area boundary.

[0360] The device (200) can divide object shape information based on the set structural area boundaries to create multiple structural areas.

[0361] Each structural region may include contour segments, feature point sets, and surface texture features contained within that region.

[0362] The device (200) can adjust area boundaries so that no overlap occurs between structural areas and apply minimum area size criteria so that the interior of the structural area is not excessively segmented.

[0363] At this time, the minimum area size criterion is merely an example to prevent the structural area from being excessively subdivided, and may be set differently for each embodiment depending on the number of structural areas, the total size of the object, the shooting resolution, or the purpose of analysis.

[0364] As a specific example, a case in which the object shape information of a folding chair is divided into structural regions is described.

[0365] The device (200) can identify a section where the contour direction changes abruptly at the point where the legs and seat of the chair meet as a discontinuous section of shape.

[0366] In addition, the area where changes in curvature occur intensively at the connection point between the seat and the backrest can be identified as the section of curvature change.

[0367] If the backrest repeatedly rotates relative to the seat plate in the captured video, the device (200) can determine the corresponding connection part as a joint candidate section.

[0368] Based on the results of this analysis, the device (200) can divide the leg structure area, the seat structure area, and the backrest structure area into independent structure areas.

[0369] By such structural area division, the device (200) can divide the physical object into units that reflect functional and structural characteristics.

[0370] As a result, when analyzing the connection relationships or operational possibilities between structural domains thereafter, it becomes possible to make realistic operational judgments without distorting the actual structure of the object.

[0371] In step S303, the device (200) can derive operational relationship information between structural regions by analyzing at least one of the connection relationship, relative position relationship, or rotatable relationship between the divided structural regions.

[0372] That is, the device (200) can derive operational relationship information between structural regions by analyzing at least one of the connection relationship, relative position relationship, or rotatable relationship between divided structural regions.

[0373] Information on operational relationships between structural regions refers to information indicating how different structural regions are connected and what relative operations are possible as a result of that connection.

[0374] Information on the operational relationships between structural regions is subsequently used as a criterion for setting structural parameter information and determining the suitability of motion.

[0375] Accordingly, the device (200) quantifies and analyzes the external shape information of the structural regions and the connection points so that the method of connection and the possibility of relative movement between the structural regions can be objectively calculated.

[0376] A connection relationship refers to the type of way in which two structural regions are combined with each other.

[0377] The connection relationship can be classified into at least one of, for example, a fixed connection, a rotary connection, a movable connection, or a disengageable connection.

[0378] A fixed connection refers to a connection where changes in relative position between structural regions are not permitted, a rotational connection refers to a connection where relative rotation with respect to a specific axis is permitted, and a movable connection refers to a connection where relative movement in a specific direction is permitted.

[0379] The device (200) can identify the section where the boundaries of two structural regions meet as a connecting section.

[0380] A connection interval can be defined as an area where the contours of two structural regions are close to or touch each other, or an area where a set of feature points shared by two structural regions exists.

[0381] The device (200) can determine that a connection section exists by comparing the contour coordinates and feature point coordinates for each structural region, and if the minimum distance value between two structural regions is less than or equal to a preset standard.

[0382] This standard may vary depending on the shooting resolution, object size, and model precision.

[0383] The device (200) can classify connection relationships by analyzing geometric characteristics in the connection section.

[0384] For example, if two structural regions share a large area in the connection section and the normal direction of the boundary surface is similar, the device (200) can be determined as a fixed connection.

[0385] Conversely, if the connection section is relatively narrow and a curvature distribution appears in which one structural region rotates relative to another structural region around the connection section, the device (200) can be determined to have the possibility of rotational connection.

[0386] In addition, if a boundary pattern is confirmed in which a connecting section is formed long and a structural area can move as if sliding in a specific direction, the device (200) can be determined to have the possibility of moving connection.

[0387] Relative positional relationship refers to information that numerically represents the relative positions of two structural regions.

[0388] The relative positional relationship can be expressed as at least one of the distance between the center coordinates of two structural regions, a relative direction vector, the direction of the contact surface of the structural regions, or the height difference with respect to a reference plane.

[0389] The device (200) can calculate the center coordinates of each structural region and calculate a vector between the center coordinates to define the relative direction.

[0390] In addition, the representative direction of the contour or the representative normal vector of the surface for each structural region can be calculated to additionally record the directional relationships between them.

[0391] The rotatable relationship refers to a relationship indicating whether relative rotation is possible between two structural regions, and if rotation is possible, the position and direction of the axis of rotation.

[0392] The device (200) can estimate the center of rotation by analyzing the distribution of feature points around the connection section and the pattern of feature point movement between frames.

[0393] For example, if the feature points of a structural region based on a specific connection segment in object image data repeatedly show an arc-shaped trajectory, the device (200) can estimate the centerline of the trajectory and calculate a candidate rotation axis.

[0394] If terminal pose information or camera pose information is provided, the device (200) can improve the accuracy of rotation axis estimation by correcting the change in viewpoint between frames.

[0395] The device (200) can generate motion relationship information between structural regions by integrating the results of analysis of connection relationships, relative positional relationships, and rotatable relationships.

[0396] Information on operational relationships between structural regions can be stored for each pair of structural regions, and for each pair of structural regions, connection type, connection segment location, rotation axis candidates, and relative direction vector may be included.

[0397] In addition, the device (200) can store the reliability of the analysis results together.

[0398] Reliability can be calculated based on at least one of, for example, the number of feature points detected in the connection interval, the number of frames used to estimate the rotation axis, and the consistency of the rotation trajectory.

[0399] As a specific example, a case in which operational relationship information between structural regions is derived for a folding chair is described.

[0400] The device (200) can repeatedly detect a narrow connection section between the seat structure area and the backrest structure area, and can confirm that the feature point of the backrest area in the captured image shows an arc trajectory based on the seat.

[0401] Accordingly, the device (200) classifies the connection relationship between the seat and the backrest as a rotational connection and can derive the connection section location and the rotation axis candidate.

[0402] On the other hand, if a wide area of ​​boundary sharing is confirmed between the seat structure area and the leg structure area and no change in relative position is observed, the device (200) can classify the connection as a fixed connection.

[0403] By deriving information on the operational relationships between such structural regions, the device (200) can obtain a structural analysis result that reflects the actual coupling method and relative motion characteristics of the real object.

[0404] As a result, when setting structural parameter information thereafter, the rotation axis, allowable range, and degrees of freedom can be realistically defined, thereby improving the accuracy and feasibility of determining motion.

[0405] In step S304, the device (200) can set the structure of a 3D object model by setting structural parameter information including a structural reference axis, structural degrees of freedom, and a structural parameter allowable range based on operational relationship information between structural regions.

[0406] That is, the device (200) can set the structure of a 3D object model by setting structural parameter information including a structural reference axis, structural degrees of freedom, and a structural parameter allowable range based on operational relationship information between structural regions.

[0407] Structural parameter information refers to a set of information that defines the range and form of movement structurally allowed by a 3D object model.

[0408] Structural parameter information can be set independently for each structural area, and is set in conjunction with each other based on operational relationship information between structural areas.

[0409] Through this, the 3D object model is composed not of simple shape data, but of a operable model that includes structural constraints.

[0410] The device (200) can set a structural reference axis by referring to a rotatable relationship or a movable relationship included in the motion relationship information between structural regions.

[0411] A structural reference axis refers to the axis that serves as a reference when a structural region rotates or moves relative to another structural region.

[0412] The structural reference axis can be set based on the position coordinates of the connection segment, the direction of the vertex array constituting the connection segment, or the direction vector of the rotation axis candidate.

[0413] At this time, the structural reference axis may be set as a single axis or as a combination of multiple structural reference axes, and this may be set differently for each embodiment depending on the combination form of the structural area.

[0414] For example, in the case of a pair of structural regions classified as rotational connections, the device (200) can set the average direction vector of the rotational axis candidates as the structural reference axis.

[0415] The device (200) can define structural degrees of freedom based on a set structural reference axis.

[0416] Structural degrees of freedom refer to the dimensions of independent movement allowed by the structural domain.

[0417] When the structural degrees of freedom are 0, the corresponding structural region remains fixed relative to other structural regions.

[0418] When the structural degrees of freedom are 1, only rotation around a single axis is allowed.

[0419] In cases where the structural degrees of freedom are 2 or more, movement combining rotation and translation or multiple axis rotation may be allowed.

[0420] The device (200) can set structural degrees of freedom by combining the type of connection relationship, the stability of the relative positional relationship, and the actual movement pattern observed in the shooting input data.

[0421] The device (200) can set the allowable range of structural parameters for each structural area.

[0422] The allowable range of structural parameters may include the minimum and maximum values ​​of the allowable rotation angle relative to the structural reference axis, the minimum and maximum values ​​of the travel distance, or the maximum ratio of the allowable deformation.

[0423] In this case, the permissible range may vary depending on the type of object, purpose of use, or shooting environment, and is not limited to specific values ​​or ranges.

[0424] This allowable range can be set by reflecting empirical criteria that consider the actual range of movement observed in the shooting input data, the possibility of interference between structural regions, and structural stability.

[0425] For example, if a rotatable relationship between structural regions is confirmed, the device (200) can set an upper limit of the rotatable range based on the maximum rotation angle observed in the captured image.

[0426] In addition, for directions where no rotation is observed in the captured video, the rotation allowance range can be set to 0 to limit rotation in that direction.

[0427] The allowable range set in this way is used as a comparison standard when applying motion later.

[0428] When setting the allowable range of structural parameters, the device (200) can also consider the possibility of interference occurring between structural regions.

[0429] To this end, the device (200) can predict the minimum distance between structural regions within an allowable range by simulation by referring to the relative positional relationship included in the operation relationship information between structural regions.

[0430] If the prediction result indicates a high probability of overlap between structural regions at a specific rotation angle or travel distance, the device (200) may set only the range excluding that section as the allowable range.

[0431] The device (200) can set deformation tolerance criteria along with setting structural parameter tolerance ranges.

[0432] The deformation tolerance criterion is a standard for determining whether the degree of deformation of a structural area during operation is visually or structurally acceptable.

[0433] The deformation tolerance criterion can be defined as at least one of the rate of change in distance between mesh vertices, the ratio of change in length relative to the reference shape, or the ratio of change in area.

[0434] For example, if the amount of deformation in a specific structural area exceeds 5 percent of the reference shape, the operation can be set to be judged as unrealistic.

[0435] This value may be set differently depending on the object type, material properties, or purpose of use, and is not limited to a specific value.

[0436] The device (200) can integrate the set structural reference axis, structural degrees of freedom, and structural parameter allowable range into structural parameter information and apply it to a 3D object model.

[0437] This structural parameter information is stored in association with structural area identifiers and is directly used to determine the operational feasibility of each structural area.

[0438] As a specific example, the case of setting structural parameter information of a folding chair is described.

[0439] The device (200) determines the connection between the seat and the backrest as a rotational connection and can set the hinge direction connecting the seat and the backrest as a structural reference axis.

[0440] Based on the backrest rotation range observed in the captured video, the allowable rotation range can be set from 0 degrees to 110 degrees.

[0441] In addition, the connection between the seat and the legs is determined to be a fixed connection, so the structural degrees of freedom can be set to 0.

[0442] In addition, by setting the deformation tolerance standard for the leg structure area very low, it is possible to restrict bending or deformation of the leg during operation.

[0443] By setting structural parameter information in this way, the device (200) can be configured so that the 3D object model reflects actual structural constraints.

[0444] As a result, it is possible to prevent structurally impossible or unrealistic movements from being selected during the subsequent motion determination and simulation stages.

[0445] In step S305, the device (200) can configure a 3D object model to include predefined motion application parameters linked to structural parameter information.

[0446] That is, the device (200) can configure a 3D object model to include predefined motion application parameters linked to structural parameter information.

[0447] Motion application parameters refer to a set of parameters that define the temporal and spatial application method when motion is actually applied to a 3D object model.

[0448] Motion application parameters are set in direct linkage with structural parameter information and are intended to control the application of motion so as not to exceed the structural reference axis, structural degrees of freedom, and the allowable range of structural parameters.

[0449] This parameter is distinct from the definition of the motion itself and serves to adjust the application results so that even the same motion produces different results depending on the object structure.

[0450] The device (200) can set motion application direction parameters by referring to the structural reference axis included in the structural parameter information.

[0451] Motion application direction parameters refer to information indicating which axis the rotation or translation included in the motion is applied relative to.

[0452] At this time, the motion application direction parameter may be set based on a single structural reference axis, or it may be set to correspond to multiple structural reference axes.

[0453] For example, for a structural area set as a rotational connection, the device (200) can set a motion application direction parameter so that rotational motion is applied only with respect to the structural reference axis of the structural area.

[0454] This eliminates unnecessary directional components included in the motion, allowing only structurally valid movements.

[0455] The device (200) can set motion application dimension parameters by referring to structural degrees of freedom.

[0456] The motion application dimension parameter is a parameter used to limit the types and number of movements to which motion is applied.

[0457] For example, for a structural area where the structural degrees of freedom are set to 1, only rotational motion is allowed, and motion parameters including translation or complex rotation can be automatically disabled.

[0458] In the case of a structural region where the structural degrees of freedom are 0, the motion application dimension parameter can be set so that all motion applications to that structural region are restricted.

[0459] The device (200) can set motion application limit parameters by referring to the allowable range of structural parameters.

[0460] Motion application limit parameters refer to parameters that numerically define the maximum rotation angle, maximum travel distance, or maximum deformation ratio to which motion can be applied.

[0461] The device (200) can limit the application range by cutting off the excess section or advancing the end time of the motion motion when the motion parameters included in the motion motion exceed the corresponding limit.

[0462] For example, for a structural area where the rotation allowance range is set from 0 to 110 degrees, if a motion motion including a 130-degree rotation is selected, the device (200) can set a motion application limit parameter so that rotation is applied only up to 110 degrees.

[0463] The device (200) can set motion application correction parameters by referring to deformation allowance criteria included in structural parameter information.

[0464] Motion application correction parameters are parameters used to adjust the operation speed, application sequence, or intermediate state so that the amount of deformation occurring in the structural area during the application of motion does not exceed the allowable range.

[0465] Such motion application correction may be performed in a pre-calculation manner before the motion application, or it may be performed in real-time during the simulation process.

[0466] For example, if the amount of mesh deformation increases rapidly as the rotation speed increases in a specific structural area, the device (200) can set a motion application correction parameter to reduce the rotation speed in that rotation section.

[0467] The device (200) can include motion application parameters in a 3D object model in conjunction with a structural area identifier.

[0468] Through this, each structural area comes to possess constraint information regarding the motions and application methods applicable to it.

[0469] This configuration enables the consistent maintenance of different application rules for each structural area during the motion simulation phase.

[0470] As a specific example, a case in which motion application parameters are included in a 3D object model of a folding chair is described.

[0471] The device (200) can set motion application direction parameters so that only rotational motion is allowed based on the structural reference axis between the seat and the backrest.

[0472] In addition, for a backrest structural area where the structural degree of freedom is set to 1, motion application dimension parameters can be set so that motion parameters including movement components are automatically removed.

[0473] If the rotation allowable range is set from 0 degrees to 110 degrees, the device (200) can set a motion application limit parameter so as not to apply a range where the rotation angle exceeds 110 degrees.

[0474] In addition, if it is predicted that the amount of mesh deformation increases in the rotation end section, the device (200) can set a motion application correction parameter to gradually reduce the rotation speed in that section.

[0475] In this way, by including motion application parameters in the 3D object model in conjunction with structural parameter information, the device (200) can be controlled so that even if the motion is the same, it is applied only in a manner that conforms to the object structure.

[0476] As a result, the possibility of structural inconsistencies, excessive deformation, or unrealistic movements occurring during the motion simulation process can be effectively reduced.

[0477] Through this, the device (200) can stably generate a movable 3D object model that reflects the actual structural constraints of a real object by configuring a 3D object model that includes object shape information, structural regions, motion relationships between structural regions, and structural parameters in stages from shooting input data, and motion application parameters linked thereto.

[0478] FIG. 4 is a flowchart illustrating the process of determining a motion applicable to a 3D object model according to one embodiment.

[0479] Referring to FIG. 4, first, in step S401, the device (200) can query candidate motions stored in the motion motion database based on motion relationship information between structural regions.

[0480] That is, the device (200) can query candidate motions stored in the motion motion database based on motion relationship information between structural regions.

[0481] A motion database refers to a data repository in which various motions applicable to 3D object models are predefined and stored.

[0482] Motion refers to a motion pattern defined such that at least one of the structural regions of a 3D object model performs rotation, translation, or shape change over time, and each motion includes one or more motion parameter information.

[0483] Motion parameter information may include at least one of a change in rotation angle, rotation axis information, direction of movement, distance of movement, sequence of movements, speed of movement, or time division information.

[0484] The device (200) can search for candidate motions by using motion relationship information between structural regions as a lookup criterion.

[0485] The motion relationship information between structural regions includes the connection type for each pair of structural regions, whether rotation is possible, whether movement is possible, candidates for structural reference axes, and relative positional relationship information.

[0486] The device (200) can use this information to exclude motions including rotation or movement for structural regions composed only of fixed connections, and to preferentially look up motions including rotational movement when there are pairs of structural regions including rotational connections.

[0487] For each motion stored in the motion motion database, the device (200) can refer together to the target structural area, the required structural degrees of freedom, the required rotation axis type, or the direction of movement information.

[0488] Through this, the device (200) can select only motion motions that are basically compatible with motion relationship information between structural regions as candidate motion motions.

[0489] For example, if a rotational connection does not exist in the motion relationship information between structural regions, motions that require a rotation axis as a prerequisite may be excluded from the candidates.

[0490] When the device (200) queries candidate motions, it can also consider the reliability of motion relationship information between structural regions.

[0491] For structural regions where a highly reliable rotational connection relationship has been confirmed, motions defined based on that rotational connection can be preferentially included as candidates.

[0492] Conversely, if only low-reliability connection relationships are identified, only conservatively defined motions for the corresponding structural area can be included as candidates to prevent excessive motions from being selected in subsequent steps.

[0493] The device (200) can store the candidate motion lookup results in conjunction with the structural area identifier.

[0494] This allows for a clear distinction of which motion corresponds to which structural area during the subsequent process of comparing motion parameters and determining the allowable range.

[0495] As a specific example, a case is described in which candidate motions are retrieved based on motion relationship information between structural regions of a folding chair.

[0496] The device (200) can identify a pair of structural regions in which a rotational connection relationship is confirmed between the seat and the backrest.

[0497] Accordingly, motions that rotate the backrest relative to the seat, such as 'backrest folding motion' and 'backrest unfolding motion', can be searched as candidate motions.

[0498] On the other hand, if the connection between the seat and the legs is identified as a fixed connection, motions involving rotation or movement of the legs may be excluded from candidate motions.

[0499] In this way, by selecting candidate motions based on motion relationship information between structural regions, the device (200) can prevent motions that are structurally unapplicable from being transmitted to a subsequent judgment stage.

[0500] As a result, the efficiency and judgment accuracy of the subsequent motion parameter comparison and classification process can be improved.

[0501] In step S402, the device (200) can compare motion parameter information included in each candidate motion and structural parameter information set in the 3D object model by matching parameter items of the same type.

[0502] That is, the device (200) can compare motion parameter information included in each candidate motion and structural parameter information set in the 3D object model by matching parameter items of the same type.

[0503] Motion parameter information refers to information that expresses specific operation conditions assigned to a structural area in the form of numerical values ​​or rules when motion is applied.

[0504] Motion parameter information may include at least one of rotation axis information, amount of change in rotation angle, rotation direction, direction of movement, distance of movement, sequence of movements, speed of movements, and time division interval information.

[0505] In the present invention, “motion” means an animation sequence defined by binding to a bone structure that constitutes a 3D object model.

[0506] In addition, in the present invention, “motion” refers to one or more individual motions or a combination thereof that are applied and executed on an augmented reality-based 3D space or a 3D object model, which are defined to cause actual spatial changes in the structural area of ​​a 3D object model among the motions.

[0507] This motion is constructed by arranging multiple keyframe data on a time axis, and each keyframe may include information on position change, rotation change, size change, or joint angle change of a structural area or bone unit.

[0508] The device (200) can manage each motion by mapping it to a structural unit so that it corresponds to the structural area information and structural parameter information of the 3D object model.

[0509] Through this, multiple types of motion can be defined for the same 3D object model, or the same motion can be reused to fit different object model structures.

[0510] When this motion parameter information is stored in the motion database, it is explicitly separated and stored by parameter item.

[0511] The device (200) may not impose a separate systemic limit on the number of motion executions or repetitions.

[0512] In other words, a single motion can be applied in a single-execution manner, or it can be set to run repeatedly depending on the user's selection or simulation settings.

[0513] The repetition, pause, or resumption points of a motion can be defined by the time axis configuration of the keyframe data.

[0514] For example, you can designate a keyframe range of a specific section within the entire motion sequence as a repeating playback section, or set keyframes after a specific point in time to remain in a static state.

[0515] In addition, the total duration of the motion sequence is determined by the length of the time axis where the keyframe data is placed.

[0516] The device (200) can configure the motion sequence according to general animation production standards for implementing natural physical movement and may not set a pre-set upper limit standard for the total duration or quality of the motion.

[0517] Through this, various types of motions, ranging from motions performed in a short time such as simple opening and closing movements to complex motions involving multiple joint changes, can be managed with the same structure.

[0518] Structural parameter information is information that defines the structural constraints of a 3D object model and includes structural reference axes, structural degrees of freedom, structural parameter tolerance ranges, and deformation tolerance criteria.

[0519] Structural parameter information is stored in association with structural area identifiers and clearly defines acceptable operating conditions for each structural area.

[0520] The device (200) can correspond motion parameter items included in the candidate motion to items of structural parameter information by type.

[0521] For example, if rotation axis information is included in the candidate motion, the device (200) can compare the rotation axis information with the structural reference axis included in the structural parameter information.

[0522] In addition, if the candidate motion motion includes a change in rotation angle, the device (200) can compare the change in angle with the allowable rotation angle range included in the allowable range of structural parameters.

[0523] The device (200) can determine the type suitability of motion parameters based on structural degrees of freedom.

[0524] For example, for a structural region where the structural degrees of freedom are set to 0, a candidate motion containing translation or rotation parameters may be determined not to match the type of the structural parameter information.

[0525] For structural regions where the structural degrees of freedom are set to 1, only rotation parameters are used for comparison, and items containing translation parameters may be excluded from comparison or classified as inconsistent items.

[0526] The device (200) can compare whether the direction of the rotation axis matches when comparing rotation parameters.

[0527] Rotation axis comparison can be performed by calculating the angular difference between the rotation axis vector defined in the candidate motion and the structural reference axis vector.

[0528] If the angle difference is less than or equal to a preset allowable standard, the device (200) can determine that the rotation axis is structurally aligned.

[0529] This acceptance criterion may be set differently depending on the object type, structural area size, or shooting precision.

[0530] The device (200) can compare whether the direction of movement and the direction of movement allowance match when comparing movement parameters.

[0531] If a movable direction is set in the structural parameter information, the device (200) can compare whether the movement direction vector of the candidate motion is aligned with that direction.

[0532] Alignment status can be determined based on the inner product value or angle difference between direction vectors.

[0533] The device (200) can compare the consistency with the structural parameter information with respect to the operation sequence and time division information.

[0534] For example, if a structural constraint is set such that the operation of a specific structural area must be performed only after the operation of another structural area, the device (200) can compare whether the sequence of operations defined in the candidate operation motion satisfies the constraint.

[0535] In addition, if the time division information is excessively subdivided and there is a possibility that it may cause an increase in the amount of deformation of the structural area, the device (200) may mark the item as a subject for additional review in a subsequent judgment step.

[0536] As a specific example, a case is described in which candidate motion and structural parameter information of a folding chair are compared.

[0537] The device (200) can compare whether the rotation axis information included in the 'backrest folding motion' matches the hinge direction connecting the seat and the backrest.

[0538] In addition, it is possible to check whether the amount of change in rotation angle included in the motion falls within the range of 0 to 110 degrees set as the structural parameter tolerance range by matching each item.

[0539] On the other hand, candidate motions that include movement parameters, such as 'full chair movement motion', may be classified as unsuitable candidates in the comparison step because they do not match the type of the leg structure region where the structural degrees of freedom are set to 0.

[0540] By matching and comparing parameter items of the same type in this way, the device (200) can systematically verify the suitability of candidate motions based on structural constraints and numerical conditions, without relying on simple motion names or classifications.

[0541] In step S403, the device (200) can determine, based on the comparison result, whether the motion parameter information of each candidate motion is included within the allowable range defined by the structural parameter information.

[0542] That is, the device (200) can determine whether the motion parameter information of each candidate motion is included within the allowable range defined by the structural parameter information based on the comparison results for each parameter item.

[0543] The allowable range refers to a standard included in the structural parameter information, encompassing the minimum and maximum values ​​of the allowable rotation angle, the minimum and maximum values ​​of the travel distance, or the upper and lower limits of the operating speed for each structural area.

[0544] This tolerance range is a value established by synthesizing the structural reference axes, structural degrees of freedom, relative positional relationships between structural regions, and potential for interference derived from the preceding structural setup process.

[0545] Therefore, determining the acceptable range functions not as a simple numerical comparison, but as a key judgment procedure to determine whether the operation is structurally safe and realistic.

[0546] The device (200) can perform an allowable range determination for rotation parameters.

[0547] When determining rotation parameters, the device (200) can compare the start and end values ​​of the rotation angle change amount included in the candidate motion with the rotation allowable range included in the structural parameter information.

[0548] If the entire range of the change in rotation angle is included within the allowable range, the device (200) can determine that the rotation parameter satisfies the allowable range.

[0549] Conversely, if even a portion of the change in rotation angle exceeds the allowable range, the device (200) may determine that the rotation parameter exceeds the allowable range.

[0550] The device (200) can perform an allowable range determination for movement parameters in the same way.

[0551] It is possible to compare whether the amount of change in movement distance included in the candidate motion falls within the allowable movement distance range set in the structural parameter information.

[0552] In this case, the amount of change in travel distance can be determined based on the cumulative travel distance, or it can be determined based only on movement in a specific direction by considering the relative positional relationship between structural regions.

[0553] The device (200) can also perform an allowable range determination for the operation speed or time division parameter.

[0554] If the structural parameter information includes an upper or lower limit for the operation speed, the device (200) can determine whether the operation speed of the candidate operation motion exceeds or falls short of the corresponding range.

[0555] For example, if the rotational speed is excessively fast, the possibility of increased deformation or interference between structural regions may increase, so the device (200) may determine this as exceeding the allowable range.

[0556] The device (200) can determine whether the allowable range of a candidate motion unit is satisfied by synthesizing the results of the allowable range determination for each parameter item.

[0557] If all key motion parameter items are included within the allowable range, the device (200) can determine that the candidate motion is a candidate that satisfies the allowable range.

[0558] Conversely, if one or more key motion parameter items exceed the allowed range, the device (200) may determine that the candidate motion is a candidate that does not meet the allowed range.

[0559] As a specific example, the determination of the allowable range for the 'backrest unfolding motion' of a folding chair is explained.

[0560] The device (200) can check whether the amount of change in rotation angle included in the motion is set from 0 degrees to 100 degrees.

[0561] If the rotational allowable range of the backrest structure area in the structural parameter information is set from 0 degrees to 110 degrees, the device (200) can determine that the rotational parameter satisfies the allowable range.

[0562] On the other hand, in the case of a candidate motion where the rotation angle change amount is set from 0 degrees to 130 degrees, the device (200) can determine that the rotation parameter exceeds the allowable range and classify it as a candidate that does not meet the allowable range.

[0563] By determining the allowable range in this way, the device (200) can prevent motion that deviates from numerical and structural constraints defined in structural parameter information from being transmitted to subsequent stages.

[0564] As a result, the processing burden of the subsequent interference judgment and deformation amount judgment steps can be reduced, while improving the reliability and stability of the overall motion determination process.

[0565] In step S404, the device (200) can determine whether interference occurs between structural regions when applying a motion by referring to the relative positional relationship between structural regions for a candidate motion that is determined to be included within an allowable range.

[0566] That is, for candidate motions determined to be included within the allowable range, the device (200) can determine whether interference occurs between structural regions when applying the motion by referring to the relative positional relationship between structural regions.

[0567] Interference refers to the phenomenon in which one structural region encroaches upon or overlaps the space of another structural region during the process of applying motion.

[0568] The determination of whether interference occurs is performed as a separate procedure from the determination of the allowable range of structural parameters, and is a procedure to verify whether a collision occurs in the actual structural layout even if the operation is numerically permissible.

[0569] The device (200) can use information on the relative positional relationship between structural regions as reference data for interference judgment.

[0570] Relative positional relationship information includes the center coordinates of the structural area, outer boundary coordinates, contact surface direction, and relative placement information with respect to the structural reference axis.

[0571] The device (200) can use this information to determine the basic spatial arrangement between structural regions in an initial state before motion is applied.

[0572] The device (200) can determine whether there is interference by dividing the operation process when a candidate operation motion is applied into a plurality of intermediate states.

[0573] An intermediate state refers to a state in which the entire application interval of a motion is divided into fixed time intervals or fixed angle change intervals.

[0574] For example, in the case of rotational motion, the total rotation angle can be divided into fixed angle units to calculate the position and orientation of the structural area at each division point.

[0575] The device (200) can calculate outer boundary information of the structural area in each intermediate state.

[0576] The outer boundary information can be represented as a minimum circumscribing bounding box surrounding the structural region or a set of vertices constituting the outer surface of the structural region.

[0577] The device (200) can determine whether interference occurs between structural regions by calculating whether there is overlap between outer bounding boxes or the minimum distance between outer surfaces.

[0578] The device (200) can make a final determination of whether interference occurs by referring to the interference tolerance criteria.

[0579] The interference tolerance criterion is a reference value used to determine the minimum allowable distance between structural areas or whether overlap is permitted.

[0580] For example, the case where the minimum distance between structural areas becomes 0 or less can be defined as interference occurrence, or the case where the distance between structural areas decreases below a certain standard can be defined as an interference risk state.

[0581] This standard may be set differently depending on the object type, the functional importance of the structural area, or user safety requirements.

[0582] The device (200) can record the result of determining whether interference occurs in units of candidate motion movements.

[0583] If interference is detected in one or more intermediate states during the application of motion motion, the device (200) may indicate the candidate motion motion as an interference-causing motion.

[0584] Conversely, if no interference occurs between structural regions in any intermediate state, the device (200) can determine that the candidate motion is an interference-free motion.

[0585] As a specific example, a case is described in which interference is determined regarding the rotation motion of the backrest of a folding chair.

[0586] The device (200) can generate intermediate states by dividing the motion of the backrest rotating from 0 degrees to 110 degrees into 5-degree increments.

[0587] In each intermediate state, the device (200) can calculate the distance between the outer bounding boxes between the backrest structure area, the seat structure area, and the leg structure area.

[0588] In a section where the rotation angle is 90 degrees or more, if the outer boundary box of the backrest structure area overlaps with the outer boundary box of the leg structure area, the device (200) can determine that interference has occurred in that section.

[0589] In this case, the device (200) can display the corresponding candidate motion as an interference-generating motion.

[0590] In this way, by determining whether interference occurs based on the relative positional relationship between structural regions, the device (200) can identify in advance motions in which collisions occur in the actual arrangement relationship, even if the motion is within the structurally permissible range.

[0591] As a result, it is possible to effectively prevent unrealistic or dangerous motions from being selected as the final application target in subsequent stages.

[0592] In step S405, the device (200) can compare the amount of deformation occurring in the structural area when the motion is applied with the predefined deformation tolerance range data for the candidate motion.

[0593] That is, for each candidate motion, the device (200) can compare the amount of deformation occurring in the structural area when the motion is applied with predefined deformation tolerance range data.

[0594] Deformation refers to a value that quantitatively expresses how much the shape of a structural region changes from its original shape during the application of motion.

[0595] The amount of deformation may include at least one of the rate of change in length of the structural region, the rate of change in angle, the amount of change in surface curvature, the amount of change in distance between mesh vertices, or the amount of deviation from the structural reference axis.

[0596] Since such deformation amounts must be evaluated by considering the stiffness, material properties, or functional role of the structural area, they are judgment items distinct from simple positional changes.

[0597] Deformation tolerance range data is reference data set in conjunction with structural parameter information, defining the maximum degree of deformation that each structural area can tolerate.

[0598] The deformation tolerance range data may include the ratio of maximum length change per structural region, the maximum angular distortion tolerance value, the surface deformation tolerance limit value, or the mesh distortion tolerance standard.

[0599] This data may be set differently depending on the object type, the functional importance of the structural area, or the purpose of use, and is not limited to specific numerical values.

[0600] The device (200) can calculate the amount of deformation after applying candidate motions to each structural region.

[0601] In this case, the deformation amount can be calculated by comparing the start and end states of the motion, or, similar to the preceding interference judgment process, it can be calculated based on the maximum deformation amount by including intermediate states during the motion process.

[0602] The device (200) can numerically calculate the amount of deformation using the change in the vertex coordinates of the outer mesh of the structural area or the deviation relative to the reference axis.

[0603] The device (200) can compare the calculated amount of deformation with the deformation allowable range data corresponding to each structural region.

[0604] If the amount of deformation falls within the deformation allowable range data, the device (200) can determine that deformation has occurred to an allowable level for the corresponding structural area.

[0605] Conversely, if the amount of deformation exceeds the deformation allowable range data, the device (200) may determine that excessive deformation has occurred in the corresponding structural area.

[0606] The device (200) can synthesize the results of the deformation tolerance range comparison into candidate motion units.

[0607] If the amount of deformation in all relevant structural regions is within the allowable range, the device (200) can determine the candidate motion as a deformation-allowable motion.

[0608] On the other hand, if the amount of deformation in one or more core structural regions exceeds the allowable range, the device (200) may determine the corresponding candidate motion as a deformation-excess motion.

[0609] As a specific example, a case in which the amount of deformation is determined for the backrest structure area of ​​a folding chair is described.

[0610] The device (200) can calculate the maximum rate of change in distance between backrest mesh vertices when backrest rotation motion is applied.

[0611] If the maximum allowable strain of the backrest structure area in the deformation allowable range data is set to, for example, 5 percent, and the calculated strain is 3 percent, the device (200) can determine that the deformation is included within the allowable range.

[0612] On the other hand, if the strain is calculated to be 8 percent, the device (200) can determine that the motion exceeds the structural deformation limit.

[0613] By comparing the deformation amount and the deformation allowable range data in this way, the device (200) can further verify whether excessive deformation occurs that impairs the shape stability of the structural region, even if the motion is structurally free of interference and numerically within the allowable range.

[0614] As a result, it is possible to effectively prevent motions that appear visually feasible but lack actual structural stability from being selected as the final application target.

[0615] In step S406, the device (200) can classify each candidate motion as an applicable motion, a motion requiring restrictions when applied, or an inapplicable motion based on the comparison result and whether interference occurs between structural regions.

[0616] That is, the device (200) can classify each candidate motion into an applicable motion, a motion requiring restriction conditions when applied, or an inapplicable motion by combining the results of the preceding parameter tolerance range determination, the results of the determination of whether interference occurs between structural regions, and the results of the deformation amount comparison.

[0617] Applicable motion refers to a motion that satisfies all allowable ranges defined by structural parameter information, does not cause interference between structural regions, and has a deformation amount that falls within the deformation allowable range data.

[0618] In this case, the device (200) can determine that the motion satisfies structural stability, shape retention, and actual applicability.

[0619] Motions requiring constraints for application refer to motions that satisfy the allowable range of basic structural parameters but can be reliably applied only under specific conditions.

[0620] For example, if the amount of change in rotation angle approaches the upper limit of the allowable range or the amount of deformation approaches the threshold of the allowable range, the device (200) can classify the motion as a motion requiring a restriction condition.

[0621] In this case, constraints can be defined in ways such as reducing the maximum rotation angle, decreasing the operating speed, changing the order of operations, or adding intermediate stop points.

[0622] Inapplicable motion refers to motions where the structural parameter tolerance is exceeded, interference occurs between structural regions, or the amount of deformation exceeds the deformation tolerance range data.

[0623] The device (200) may exclude such motion from application if it determines that applying such motion in an actual structural state could cause structural damage, visual distortion, or user confusion.

[0624] The device (200) can record the reason for classification for each candidate motion.

[0625] Reasons for classification may include items exceeding the allowable range, information on interference occurrence sections, and identifiers for structural areas exceeding deformation amounts.

[0626] This information may be used later to explain simulation results or provide guidance on constraints to the user.

[0627] The device (200) can generate applicable condition information for motions that require limiting conditions when applied.

[0628] Applicable condition information may include the maximum allowable rotation angle, the recommended operating speed range, or information on structural areas requiring caution during operation.

[0629] Through this, the device (200) can selectively utilize the motion within a safe and realistic range without completely excluding the motion.

[0630] As a specific example, a case of classifying the backrest motion of a folding chair is described.

[0631] The device (200) can classify motions in which the backrest rotation angle is from 0 to 90 degrees as applicable motions.

[0632] If a motion with a rotation angle of 0 to 110 degrees approaches the upper limit of the allowable range and the amount of deformation is near the threshold, the device (200) can classify the motion as a motion requiring a limiting condition when applied by applying a condition that limits the maximum rotation angle to 100 degrees.

[0633] On the other hand, if the rotation angle is set from 0 to 130 degrees and exceeds the allowable range of the structural parameter, the device (200) may classify the motion as an inapplicable motion.

[0634] In this way, by subdividing and classifying candidate motions according to multi-stage judgment results, the device (200) can go beyond simply determining applicability and derive a motion determination result that considers stability, realism, and user understanding in an actual usage environment.

[0635] In step S407, the device (200) can determine a set of motions applicable to a 3D object model in an augmented reality-based 3D space based on the classification result.

[0636] That is, the device (200) can determine a set of motions applicable to a 3D object model in an augmented reality-based 3D space based on the classification result.

[0637] A motion set refers to a set that includes motions classified as applicable motions in the preceding classification process, and motions that are permitted for application only if they satisfy the constraints among motions requiring constraints for application.

[0638] This set is used as result data to clearly define the range of action motions that can actually be selected and executed in the simulation provided to the user.

[0639] The device (200) can add motion motions classified as applicable motions to the motion motion set as basic inclusion items.

[0640] Since these motions have passed the structural parameter tolerance range, interference judgment, and deformation amount judgment, they can be configured to be immediately applied in an augmented reality-based 3D space without any additional constraints.

[0641] For motions that require a restriction condition when applied, the device (200) can optionally include them in the motion motion set after checking whether the restriction condition is satisfied.

[0642] For example, if a maximum rotation angle limit, a motion speed limit, or a pre-placement condition for a specific structural area is set, the device (200) can be set so that the motion is executed only within the range that satisfies the condition.

[0643] At this time, the constraint information is saved as an attribute value associated with the motion, and can be automatically reflected when the simulation is executed later.

[0644] The device (200) can exclude motion motions classified as inapplicable motions from the motion motion set.

[0645] In addition, the reason for non-application is stored together and can be used to explain why the corresponding action motion is not selected in the interface provided to the user.

[0646] Through this, users can not only recognize a state where selection is impossible but also understand structural constraints or safety reasons.

[0647] The device (200) can configure a set of motion movements in conjunction with a structural area identifier.

[0648] This allows for the management of motions applicable only to specific structural areas and motions that affect the entire object.

[0649] For example, if rotation is allowed only in certain structural regions, only the motion corresponding to those structural regions can be selectively included.

[0650] The device (200) can provide a determined set of motion motions to the user's terminal (100).

[0651] In this case, the motion set may be provided in a form that displays a motion list, applicability for each motion, and constraint information together.

[0652] The user's terminal (100) can display only the motions that the user can select based on the information on the interface.

[0653] As a specific example, a case for determining the motion set of a folding chair is described.

[0654] The device (200) may include a 'backrest folding motion' and a 'backrest unfolding motion' as applicable motions.

[0655] If the ‘chair part folding motion’ is allowed only at a specific angle or lower, the device (200) may include the motion in the motion motion set by setting a maximum rotation angle limit condition together.

[0656] On the other hand, motions that exceed the allowable range of structural parameters, such as 'total chair twisting motion', may be excluded from the motion set.

[0657] By finally determining the set of motion motions by reflecting the classification results in this way, the device (200) can provide the user with only motion motions that are actually structurally applicable and stable in an augmented reality-based 3D space.

[0658] In addition, the device (200) can provide this motion simulation not only in an augmented reality-based 3D space but also in a project space where interior design is performed.

[0659] At this time, motions executed in the project space can be played back based on the same motion data structure and motion parameter information as motions used in the augmented reality-based 3D space, and the same motion data can be commonly applied to multiple environments without separate motion redefinition.

[0660] For example, if a user places a 3D object model on a space included in a specific project, the device (200) can execute a motion set on the 3D object model on the project design screen to visually provide a motion process assuming an actual installed state.

[0661] This allows users to check the product's opening and closing method, movement path, interlocking operations between structural areas, and usage patterns after installation in advance during the pre-construction phase.

[0662] As a result, the motion simulation of the user-captured object can be visually persuasive while simultaneously securing structural validity and feasibility.

[0663] Through this, the device (200) can improve the realism, reliability, and feasibility of motion simulation for user-captured objects by comprehensively determining the operational relationship between structural regions, the allowable range of structural parameters, the possibility of interference, and the allowable deformation criteria, and by selecting only the stable motions that are actually structurally applicable in an augmented reality-based 3D space.

[0664] FIG. 5 is a flowchart illustrating the process of generating a new motion of a 3D object model according to one embodiment.

[0665] Specifically, the device (200) can generate new motion of a 3D object model.

[0666] In addition, the device (200) can perform and display the same motion on the detailed information screen of the 3D object model provided to the user's terminal (100), in addition to the project space.

[0667] In this case, motions executed on the product detail screen can also be played back based on the same motion data as motions used in the project space or augmented reality-based 3D space, and the structural motion characteristics of the product can be consistently provided in various user interface environments.

[0668] For example, when a user selects a specific 3D object model on a mobile application to view a detailed screen, the device (200) can visually provide the structural features, actual operation method, and usage method of the product by playing a predefined or newly created motion along with static shape information of the object model.

[0669] This allows users to intuitively understand the behavioral characteristics of an object not only during the design phase but also during the product review or purchase decision phases.

[0670] At this time, the newly generated motion can also be configured in the form of a keyframe data sequence bound to this structure, just like the existing motion.

[0671] The device (200) can generate a new motion sequence by generating each intermediate state for reaching a user-specified target state as a plurality of keyframes and arranging the keyframes on a time axis.

[0672] The newly generated motion can be stored in a motion database for reuse, repeated execution, or used as a reference motion for other 3D object models.

[0673] Referring to FIG. 5, first, in step S501, the device (200) may provide a motion goal input UI to the user's terminal for specifying a target position, target angle, or target state of a 3D object model in an augmented reality-based 3D space.

[0674] That is, the device (200) can provide a motion goal input UI to the user's terminal (100) for specifying a target position, target angle, or target state of a 3D object model in an augmented reality-based 3D space.

[0675] Here, the action goal input UI refers to an input interface configured to allow the user to intuitively specify the final state of the 3D object model.

[0676] The operation goal input UI includes operation elements that are visually displayed in an augmented reality-based 3D space and can be operated through at least one of touch input, drag input, rotation gesture, or slider input of the user's terminal (100).

[0677] The device (200) can provide reference coordinate system and reference angle information for specifying a target state together in the operation target input UI based on the current state of the augmented reality-based 3D space in which the 3D object model is placed.

[0678] This reference coordinate system is established based on the position and orientation occupied by a 3D object model in an augmented reality-based 3D space, and can be configured by reflecting spatial alignment information corresponding to actual captured images or videos.

[0679] The action goal input UI may include a positioning element to specify the target location.

[0680] Positioning elements may be provided in the form of virtual handles movable within 3D space, axial movement guides, or grid-based position selection interfaces.

[0681] Users can manipulate these positioning elements to specify the target location where the 3D object model should move.

[0682] The motion goal input UI may include a rotation specification element for specifying the target angle.

[0683] The rotation designation element may be provided in the form of a circular guide rotatable around a structural reference axis, an angle scale display, or a step rotation button.

[0684] This allows the user to specify the target angle at which a 3D object model or a specific structural area should be rotated.

[0685] The action goal input UI may include a state selection element for specifying the goal state.

[0686] The target state refers to a structurally defined state, such as a relative arrangement state between structural regions, a folded state, an unfolded state, or a combined state.

[0687] The user can select one of the predefined target states through a state selection element, and in this case, the device (200) can internally link and set target position and target angle information corresponding to the state.

[0688] As a specific example, a case of generating a new motion of a folding chair is described.

[0689] The device (200) can display an augmented reality-based 3D space in which a folding chair is placed on the user's terminal (100) and provide a motion goal input UI including a rotation guide in the backrest structure area.

[0690] The user can drag the rotation guide to set the backrest to a specific angle as the target state.

[0691] At this time, the device (200) can recognize the target angle and the target placement state of the corresponding structural area together through user input.

[0692] By providing a motion goal input UI in this way, the device (200) can support the user in intuitively specifying the target state of a new motion in an augmented reality-based 3D space without abstract commands or numerical inputs.

[0693] In step S502, the device (200) can convert the input received through the operation goal input UI into goal state data corresponding to structural parameter information set in the 3D object model.

[0694] That is, the device (200) can convert the input received through the operation goal input UI into goal state data corresponding to the structural parameter information set in the 3D object model.

[0695] Target state data refers to a data set quantified by structurally interpreting the target position, target angle, or target state specified by the user through the motion target input UI.

[0696] This data can be stored separately by structural region of the 3D object model and is configured to clearly represent the final state that each structural region must reach.

[0697] The device (200) can map input received from the user's terminal (100) into structural area units.

[0698] For example, if a user performs a rotation input while a specific structural area is selected, the device (200) can recognize that the input is a target state designation for that structural area rather than the entire object.

[0699] To this end, structural area identifier information can be provided in conjunction with the operation goal input UI.

[0700] The device (200) can configure target state data by matching the received input to the item type defined in the structural parameter information.

[0701] For example, if a user inputs an angle change through a rotation guide, the device (200) can interpret this as a rotation parameter item and generate target rotation angle data in conjunction with structural reference axis information.

[0702] When movement input is received, target location data can be generated in conjunction with the movable direction included in the structural parameter information.

[0703] When generating target state data, the device (200) can refer to the allowable conditions defined by the structural parameter information.

[0704] Allowable conditions include structural degrees of freedom, rotation tolerance range, and movement tolerance range, and are used as criteria to prevent target state data from being set to structurally impossible values.

[0705] For example, if a rotation input is received for a structural area where the structural degrees of freedom are set to 0, the device (200) may invalidate the input or correct it to the nearest allowed state.

[0706] The device (200) can include temporal conditions in the target state data.

[0707] When a user specifies a target state, if the user also specifies the duration of the operation or the time of arrival, the device (200) can convert this into a time parameter of the target state data and store it.

[0708] This time parameter can be used to calculate path segmentation density or operation speed in the subsequent operation path candidate calculation process.

[0709] As a specific example, a case is described in which target state data is generated for the backrest structure area of ​​a folding chair.

[0710] When a user performs an input to raise the backrest to a 90-degree angle through the motion goal input UI, the device (200) can generate a rotation angle of 90 degrees with respect to the structural reference axis of the backrest structural area as target state data.

[0711] At this time, if the allowable rotation range of the backrest is set from 0 degrees to 110 degrees in the structural parameter information, the device (200) can determine that the target state data satisfies the allowable condition and store it as is.

[0712] In this way, by converting the user's input into target state data corresponding to structural parameter information, the device (200) converts the intuitive user input into structurally interpretable data, thereby stably establishing the basis for the subsequent process of calculating the motion path and generating new motion motions.

[0713] In step S503, the device (200) can compare the target state data with the current state of the 3D object model to produce a plurality of motion path candidates that satisfy the allowable conditions defined by the structural parameter information.

[0714] That is, the device (200) can compare the target state data with the current state of the 3D object model to produce a plurality of motion path candidates that satisfy the allowable conditions defined by the structural parameter information.

[0715] Here, the current state refers to the position, angle, and combination state of each structural region that a 3D object model placed in an augmented reality-based 3D space has before the target state data is applied.

[0716] The current state is configured to include location coordinates associated with the structural area identifier, rotation angle, and connection status information between structural areas.

[0717] A motion path candidate refers to a sequence of state changes in which a structural region moves or rotates over time during the transition from the current state to the target state data.

[0718] Each motion path candidate is defined as a continuous path including a starting state, multiple intermediate states, and a target state.

[0719] The device (200) can set path generation rules that can be used to calculate operation path candidates by referring to structural parameter information.

[0720] This path generation rule may include whether movement or rotation is possible based on structural degrees of freedom, restrictions on the direction of rotation based on the structural reference axis, and motion sequence constraints.

[0721] This allows structurally unacceptable paths to be excluded at the initial stage.

[0722] The device (200) can generate operation path candidates by applying a plurality of path generation methods based on the difference between current state and target state data.

[0723] For example, a path that rotates continuously in a single direction to a target angle, a path that divides the rotation into multiple segments and performs it step by step, or a path that moves a specific structural area first and then performs the rotation can each be generated as a candidate motion path.

[0724] This path generation method is merely an example and can be extended in various ways depending on the combination form of structural areas, object types, or implementation methods.

[0725] The device (200) can primarily verify whether each motion path candidate satisfies the allowable conditions defined in the structural parameter information.

[0726] In this verification process, it can be checked whether structural degrees of freedom are violated, whether the rotation axis deviates, or whether the allowable range of movement is exceeded at the intermediate state of each path.

[0727] Paths that do not meet the allowance conditions may be excluded from the operation path candidates.

[0728] The device (200) can apply different time division densities or path decomposition methods to the same target state data to ensure diversity of motion path candidates.

[0729] For example, a 5-step and a 10-step split path can be generated for the same rotational goal, respectively, allowing the more stable path to be selected in terms of the possibility of interference or deformation amount in subsequent steps.

[0730] In this case, the number of intermediate states or the division criteria are not limited to a fixed value and can be set differently depending on structural complexity or required precision.

[0731] As a specific example, a case is described in which candidate motion paths are calculated to move the backrest of a folding chair from a folded state to an upright state.

[0732] The device (200) can recognize the case where the backrest is folded at a 0-degree angle in the current state and the target state data is set at a 90-degree angle.

[0733] Accordingly, the device (200) can generate multiple motion path candidates, such as a single path that rotates continuously from 0 to 90 degrees, a divided path that rotates from 0 to 45 degrees, stops briefly, and then rotates from 45 degrees to 90 degrees.

[0734] After verifying whether each path satisfies the rotation tolerance range and rotation axis conditions defined in the structural parameter information, only paths that satisfy the tolerance conditions can be maintained as operation path candidates.

[0735] In this way, by comparing the target state data and the current state to generate multiple motion path candidates, the device (200) can establish a basis for selecting an optimal new motion among various transition paths that consider structural constraints and stability, without relying on a single path.

[0736] In step S504, the device (200) can evaluate the possibility of interference for each candidate operation path based on whether there is an overlap or collision between structural regions on the operation path.

[0737] That is, for each calculated motion path candidate, the device (200) can evaluate the possibility of interference based on whether there is overlap or collision between structural regions on the motion path.

[0738] Here, the possibility of interference refers to the possibility that different structural regions may occupy the same space or approach within an acceptable distance during the process of a structural region moving or rotating along a candidate motion path.

[0739] This evaluation is not based solely on a single final state, but is performed step-by-step across the entire operation path.

[0740] The device (200) can subdivide each motion path candidate into multiple path splitting points.

[0741] Path splitting points can be set based on time intervals, changes in rotation angle, or changes in travel distance, and are defined as the minimum unit capable of interference evaluation while maintaining the continuity of the motion path.

[0742] In this case, the interval or number of path splitting points is not limited to a fixed value and can be set differently depending on structural complexity or required evaluation precision.

[0743] This allows for the precise detection of interference that may occur during the intermediate stages of operation.

[0744] The device (200) can calculate the spatial occupancy status of the structural area at each division point.

[0745] The spatial occupancy status is calculated by reflecting the location coordinates, rotation status, and shape information of the structural area, and can be represented using an outer bounding box or an outer surface model of the structural area.

[0746] The method of representing such space occupancy status is not limited thereto and can be set using various geometric approximation models or collision detection techniques depending on the embodiment.

[0747] This method of representation is commonly used in actual 3D modeling and simulation environments, ensuring feasibility of implementation.

[0748] The device (200) can evaluate whether interference occurs by calculating the distance or overlap between structural areas at each division point.

[0749] For example, if the outer bounding boxes of two structural regions overlap, the device (200) may determine that a collision has occurred at the corresponding division point.

[0750] Alternatively, if the minimum distance between structural regions decreases to below a predefined interference judgment criterion distance, it can be evaluated as a state where interference is possible.

[0751] The device (200) can quantify and evaluate the possibility of interference occurring.

[0752] To this end, at least one of the number of interference occurrences, the ratio of the division points where interference occurs, and the continuous length of the interference interval can be calculated.

[0753] These quantified indicators can be used as a standard to compare the relative stability of operation path candidates in subsequent stages.

[0754] The device (200) can reflect priority information between structural regions included in structural parameter information in the interference evaluation.

[0755] For example, if interference occurs in a structural region of high functional importance, the likelihood of interference in the corresponding motion path candidate can be evaluated as higher.

[0756] This enables interference assessment that considers not only the simple presence of collision but also structural importance.

[0757] As a specific example, the case of evaluating a candidate backrest rotation path of a folding chair is described.

[0758] The device (200) can divide the path of the backrest structure area rotating from 0 to 90 degrees into 10-degree units.

[0759] By calculating the distance between the outer bounding boxes of the backrest structure area and the leg structure area at each division point, it is possible to check whether overlap occurs in specific angle intervals.

[0760] If overlap occurs in the range between 70 and 80 degrees, the device (200) can evaluate the likelihood of interference occurring in the corresponding motion path candidate as high.

[0761] In this way, by systematically evaluating the possibility of overlap or collision between structural regions for each motion path candidate, the device (200) can identify in advance structurally dangerous or unrealistic paths during the transition process to the target state.

[0762] In step S505, the device (200) can generate a new motion corresponding to the target state data by selecting the motion path with the lowest probability of interference between structural regions based on the evaluation results.

[0763] That is, the device (200) can generate a new motion corresponding to the target state data by selecting the motion path with the lowest probability of interference between structural regions based on the results of the interference probability assessment.

[0764] Here, the new motion refers not to a motion simply selected from the existing motion database, but to a motion sequence that defines a specific motion path in chronological order for transitioning from the current state to the target state data.

[0765] The new motion is configured to include position changes, rotation changes, motion sequence, and time division information by structural region.

[0766] The device (200) can compare the interference probability indicators calculated for each motion path candidate.

[0767] At this time, the interference probability indicator may include at least one of the number of interference occurrences, the length of the interference occurrence section, the continuity of the interference occurrence section, or whether interference occurs in a structurally important area.

[0768] The device (200) can calculate the relative stability of each motion path candidate by combining these indicators.

[0769] The device (200) can prioritize selecting the motion path candidate with the highest relative stability.

[0770] These path selection criteria are merely examples and can be extended and applied with various combinations of evaluation metrics depending on object type, structural complexity, or implementation method.

[0771] If multiple motion path candidates have similar stability, the device (200) can determine the final path by applying additional criteria.

[0772] Additional criteria may include at least one of the operation path length, total operation time, average deformation amount of the structural area, or similarity with user input.

[0773] The device (200) can define a new motion based on a selected motion path candidate.

[0774] To this end, the device (200) can connect each segmented point constituting the motion path in chronological order and convert the position and rotation changes of each structural area into continuous motion data.

[0775] At this time, the configuration method of the new motion may be set differently for each embodiment depending on the method of connecting the division points, the interpolation method, or the method of assigning time parameters.

[0776] In this process, motion parameters can be realigned so that the rotation axis, movement direction, and allowable conditions defined in the structural parameter information are maintained.

[0777] The device (200) can assign a time parameter to the new motion.

[0778] Time parameters may include the total operation time, time intervals between division points, or operation speeds per structural region.

[0779] This allows new motions to be configured to be performed naturally without abrupt changes.

[0780] As a specific example, a case of generating a new motion for the backrest of a folding chair is described.

[0781] The device (200) can select the path with the lowest probability of interference with the bridge structure area from among a plurality of motion path candidates rotating from 0 to 90 degrees.

[0782] If the selected path is a rotation path divided into 10-degree units, the device (200) can define a single continuous rotation motion by connecting each divided angle in chronological order.

[0783] At this time, by setting the time parameter to maintain a constant rotation speed, it can be recognized as a natural motion even in actual usage environments.

[0784] In this way, by selecting the motion path with the lowest probability of interference to generate a new motion, the device (200) can automatically derive a structurally safe and realistic motion sequence and can reliably provide a new motion that accurately corresponds to a target state specified by the user.

[0785] In step S506, the device (200) can apply the generated new motion to a 3D object model in an augmented reality-based 3D space to simulate and provide the motion process to the target state.

[0786] That is, the device (200) can apply the generated new motion to a 3D object model in an augmented reality-based 3D space to simulate the motion process toward a target state and provide it to the user's terminal (100).

[0787] Here, simulating the motion process means applying time parameters included in the new motion and position and rotation changes for each structural region in chronological order to continuously reproduce the entire process of the 3D object model moving from its current state to a target state.

[0788] This simulation is provided not as a single result screen, but in a manner where intermediate states from the start to the end of the operation are displayed sequentially.

[0789] When applying a new motion, the device (200) can maintain a spatial alignment relationship between the augmented reality-based 3D space and the actual captured image or video.

[0790] To this end, the device (200) can control the positional and rotational changes of the 3D object model so that they are reflected in the reference coordinate system of the actual shooting environment.

[0791] As a result, users can perceive the 3D object model as moving naturally in real space.

[0792] The device (200) can visually distinguish and provide the operating status of each structural region during the simulation process.

[0793] For example, methods may be used to highlight structural areas where an action is currently being performed, or to display structural areas where an action has not yet been applied semi-transparently.

[0794] This allows users to intuitively understand the relationship between the sequence of operations and the structural areas.

[0795] The device (200) can control the new motion so that if a restriction condition is included in the motion, the restriction condition is reflected in the simulation process.

[0796] For example, if a maximum rotation angle limit or a motion speed limit is set, the device (200) can automatically adjust the motion so as not to exceed the corresponding conditions during simulation.

[0797] This ensures that the simulation results do not exceed the range of actual applicability.

[0798] The device (200) can provide simulation results to the user's terminal (100) in real time or in stages.

[0799] The method of providing such simulation is not limited thereto and, depending on the embodiment, can be extended to playback control based on user interaction, repeat playback, or playback by selecting a specific section.

[0800] In the case of real-time delivery, new motion can be output in the form of a continuous animation as soon as it is applied.

[0801] In the case of step-by-step provision, it can be configured to display a paused state at each split point of the operation path so that the user can check each intermediate state.

[0802] As a specific example, a case of simulating a new motion of the backrest of a folding chair is described.

[0803] The device (200) can apply the generated backrest rotation motion to a chair model in an augmented reality-based 3D space to continuously display the process of the backrest being raised to a target angle from a folded state.

[0804] At this time, since the chair model remains aligned on the actual captured floor video, the user can experience a visual effect similar to the chair moving in the actual space.

[0805] By simulating and providing the motion process with the new motion applied in this way in an augmented reality-based 3D space, the device (200) can intuitively and realistically verify the transition process to a target state specified by the user, and can support verifying the structural validity and applicability of the new motion in advance.

[0806] Through this, the device (200) can intuitively and realistically provide a new motion that is structurally valid and actually applicable to the user's captured object by interpreting the user-specified target state based on structural parameter information, evaluating the possibility of interference occurring for multiple motion path candidates, generating the most stable path as a new motion motion, and then simulating it in an augmented reality-based 3D space.

[0807] FIG. 6 is a flowchart illustrating the process of simulating and providing the construction sequence of a 3D object model step by step according to one embodiment.

[0808] Specifically, the device (200) can simulate the construction sequence that must be performed to place a 3D object model step by step and provide it to the user's terminal.

[0809] Referring to FIG. 6, first, in step S601, the device (200) can identify a construction target structural area where placement, fixing, or joining is performed for each structural area constituting the 3D object model.

[0810] That is, the device (200) can identify a construction target structural area where placement, fixing, or joining is performed for each structural area constituting the 3D object model.

[0811] Here, a structural region refers to a unit component that constitutes a 3D object model, meaning a region that is geometrically or functionally separable.

[0812] Structural regions can utilize the region information divided during the 3D object model creation process as is, and each structural region is defined to include a unique structural region identifier.

[0813] The structural area subject to construction refers to the area within the structural area where actual placement, fixing, or combination with other structural areas is required.

[0814] In other words, structural areas that are merely decorative elements or are considered to be in a fixed state may be excluded from the structural area subject to construction.

[0815] The device (200) can determine construction properties for each structural area.

[0816] Construction attributes are information indicating whether the corresponding structural area is a placement target, a fixed target, or a target for combination with other structural areas, and can be set based on structural parameter information or connection relationship information between structural areas.

[0817] These criteria for judging construction attributes are not limited to this, and various criteria may be applied depending on the object type, construction purpose, or implementation method.

[0818] For example, if a specific structural area is connected to another structural area in a hinge manner, the device (200) can determine that structural area as a structural area to be combined.

[0819] The device (200) can distinguish between structural areas that can be placed independently and structural areas that require prior placement by referring to information on the connection relationships between structural areas.

[0820] For example, a structural area in direct contact with the floor can be identified as a structural area requiring priority placement, and a structural area coupled on top of that structural area can be identified as a structural area requiring subsequent placement.

[0821] The device (200) can identify a structural area to be constructed by considering the spatial occupancy characteristics of the structural area.

[0822] Space occupancy characteristics include the spatial range occupied by the structural area, the possibility of overlap with other structural areas, or the impact on surrounding space upon placement.

[0823] The evaluation of these spatial occupancy characteristics can be performed using at least one of geometric calculations, collision probability analysis, or empirical criteria.

[0824] This allows structural areas where spatial collisions may occur depending on the placement order to be prioritized and identified as structural areas subject to construction.

[0825] As a specific example, a case is described in which a 3D object model of prefabricated furniture is provided as a construction sequence simulation.

[0826] The device (200) can identify a frame structure area placed on the floor as a target structure area for placement.

[0827] In addition, the side plate structural area and the upper plate structural area that are joined to the frame can be identified as the structural area to be joined.

[0828] On the other hand, areas for simple decorative finishing structures can be set to be excluded from the construction sequence simulation or optionally included in the final stage.

[0829] By identifying the structural area to be constructed for each structural area in this way, the device (200) can configure a construction sequence simulation based only on the components that require actions to be performed during the actual construction process, and can lay the foundation for the setting of precedence relationships and the definition of construction stages to lead to a realistic flow in subsequent stages.

[0830] The device (200) can generate a list of construction items for each space included in the project based on the identified construction target structural area.

[0831] Here, construction items may refer to information expressing placement, fixing, or joining activities performed in a specific space as work units.

[0832] For example, if a floor frame structure area is identified as a structure area to be placed, the device (200) can generate a construction item corresponding to “frame placement” for the space, and if a side plate structure area is identified as a structure area to be joined, it can generate a construction item corresponding to “side plate joining”.

[0833] Additionally, the user's terminal (100) may be configured to allow selecting or excluding construction items by space, and the selection results may be stored by project unit or space unit.

[0834] Through this, structural area unit information used in construction sequence simulation can be naturally extended to the selection and management of construction items at the spatial unit level.

[0835] In step S602, the device (200) can establish a successor-successor construction relationship for each identified construction target structural area based on the positional relationship, connection relationship, or space occupancy relationship between the construction target structural areas.

[0836] That is, the device (200) can establish a successor-successor construction relationship for each identified construction target structural area based on the positional relationship, connection relationship, or space occupancy relationship between construction target structural areas.

[0837] Here, the precedence construction relationship refers to relationship information indicating which structural area among multiple construction target structural areas must be placed, fixed, or joined first, and which structural area must be performed subsequently.

[0838] This sequential construction relationship is not arbitrarily determined, but is established according to objective criteria by reflecting structural constraints and actual construction feasibility.

[0839] The device (200) can determine the preceding and succeeding construction relationship based on the positional relationship between the construction target structural areas.

[0840] Positional relationships include the height of the space occupied by the structural area, whether it is in contact with the floor, the center position, or the relative position to other structural areas.

[0841] For example, a structural area in direct contact with the floor may be determined to be a structural area that must be placed prior to a structural area located above it.

[0842] The device (200) can establish a successor-failure construction relationship based on the connection relationship between structural areas.

[0843] A combination relationship refers to a relationship indicating whether a preceding structural region must necessarily exist for a specific structural region to be combined with another structural region.

[0844] For example, if a joining method such as a hinge, bolt, or slot is defined, the device (200) can set a structural area that serves as the standard for joining as a pre-construction target and a structural area that is joined as a post-construction target.

[0845] The device (200) can establish a successor-follower construction relationship based on the space occupancy relationship.

[0846] The space occupancy relationship is information indicating whether the space occupied by another structural area must be empty for one structural area to be placed.

[0847] For example, in the case of a structural area that is joined by inserting it inside, if the external structural area is placed first, the placement of the internal structural area may become impossible, so the device (200) can set the internal structural area as a priority construction target.

[0848] The device (200) can establish a successor-follower construction relationship by combining multiple criteria.

[0849] In this case, the order of application or importance of each criterion may be set differently depending on the combination form of the structural area, the purpose of construction, or the type of object.

[0850] In other words, rather than using only one of positional relationships, coupling relationships, and spatial occupancy relationships, the criterion that best reflects the actual constraints between structural areas is applied first, and other criteria can be applied supplementarily as needed.

[0851] Through this, it is possible to derive a structurally rational construction sequence relationship rather than a simple rule-based one.

[0852] The device (200) can store the established preceding and succeeding construction relationships as relationship data between structural area identifiers.

[0853] These precedence and succession construction relationships can be represented in the form of directed graphs, ordered lists, or sets of constraints.

[0854] This relationship data is subsequently referenced during the construction phase definition and construction sequence simulation processes, and is used as a criterion to determine the feasibility of each phase.

[0855] As a specific example, the case of setting the construction sequence of prefabricated furniture is described.

[0856] The device (200) can set the floor frame structure area to be placed before the upper plate structure area as a prior construction relationship according to the positional relationship.

[0857] In addition, if the side plate structure area is combined with the frame structure area, the frame structure area can be set as the target for prior construction and the side plate structure area as the target for subsequent construction.

[0858] In addition, if the internal reinforcement structure area must be inserted before the external plate structure area, the internal reinforcement structure area can be set as the priority construction target based on the space occupancy relationship.

[0859] In this way, by establishing a sequential construction relationship based on the positional relationship, connection relationship, and space occupancy relationship between the structural areas to be constructed, the device (200) can establish a reasonable construction sequence model that reflects physical constraints and work flow at the actual construction site.

[0860] In step S603, the device (200) can define construction steps in which placement or fixing of the construction target structural area is performed based on the established preceding and succeeding construction relationship.

[0861] That is, the device (200) can define construction stages in which placement or fixing of the construction target structural area is performed based on the established preceding and succeeding construction relationship.

[0862] Here, a construction phase refers to a group of placement, fixing, or joining operations that can be performed together among multiple structural areas subject to construction by sharing the same precedence constraints.

[0863] Each construction phase is defined to include only structural areas where all prior construction relationships are satisfied, and structural areas where prerequisite conditions are not satisfied are not included in that phase.

[0864] The device (200) can align the construction target structural area in a phase alignment manner by referring to the preceding and succeeding construction relationship.

[0865] This alignment method is merely one example of topological alignment, and other alignment or stepping methods satisfying precedence constraints may also be applied.

[0866] Through this, structural areas that must be performed first and structural areas that can be performed subsequently can be distinguished based on objective criteria.

[0867] Based on the results of phase alignment, structural regions with the same prerequisite conditions can be grouped into a single construction phase.

[0868] In this case, the grouping criteria for structural areas may be separated or integrated depending on the possibility of spatial conflict, work difficulty, or construction stability.

[0869] The device (200) can define the construction type of the structural area included in each construction stage.

[0870] The construction type is information indicating which action—placement, fixing, or joining—is performed at that stage.

[0871] For example, a placement action can be defined for a floor frame structure area, a fixing action can be defined for a structure area requiring bolt fastening, and a joining action can be defined for a structure area requiring hinge connection.

[0872] The device (200) can consider the possibility of space occupancy conflicts when defining the construction phase.

[0873] If there is a possibility of spatial conflict occurring when structural areas included in the same stage are placed or combined simultaneously, the device (200) can separate the structural areas into different construction stages.

[0874] This prevents the definition of construction phases that are logically possible but physically difficult.

[0875] The device (200) can assign a step identifier and a step order for each construction step.

[0876] Phase identifiers are used as information to distinguish each phase when managing and providing construction sequence simulation results to users.

[0877] The step sequence indicates the relative position where the corresponding step is performed within the overall construction process.

[0878] As a specific example, a case defining the construction steps of prefabricated furniture is described.

[0879] The device (200) can define the arrangement of the floor frame structure area as the first construction stage.

[0880] The combination of the side plate structural areas that are joined to the next frame can be defined as the second construction phase.

[0881] Subsequently, the joining of the upper plate structure area can be defined as the third construction stage, and finally, the fixing of the finishing structure area can be defined as the fourth construction stage.

[0882] At this time, each step is defined to be performed on the premise of the completion of the previous step.

[0883] By defining the construction stages step by step based on the preceding and succeeding construction relationships in this way, the device (200) can generate a systematic construction sequence model that corresponds to the actual construction flow even for 3D object models with complex structures, and can secure a foundation to accurately provide a visual construction sequence simulation in subsequent stages.

[0884] In step S604, the device (200) can distinguish between structural areas that have been placed or fixed and structural areas that have not yet been constructed, according to the defined construction stages, and display them in an augmented reality-based 3D space.

[0885] That is, the device (200) can distinguish between structural areas where placement or fixing is completed and structural areas that have not yet been constructed at each stage, according to the defined construction stages, and display them in an augmented reality-based 3D space.

[0886] Here, the structural area where placement or fixing is completed refers to the structural area where placement, fixing, or joining is considered to have been actually completed up to the relevant construction stage.

[0887] Structural areas that have not yet been constructed refer to structural areas that are scheduled to be placed or fixed in a subsequent construction phase but have not been carried out at the current stage.

[0888] The device (200) can determine the completion status by referring to the list of structural areas completed up to the corresponding stage at the start of each construction stage.

[0889] This completion status can be automatically updated based on the stage sequence information and structural area identifier assigned when defining the construction phase.

[0890] Through this, the device (200) can clearly distinguish which structural areas have already been placed or fixed during the current construction phase.

[0891] The device (200) can visually distinguish and display the completed structural area and the unconstructed structural area.

[0892] For example, completed structural areas can be displayed while maintaining their actual material, color, and opacity, while uncompleted structural areas can be distinguished through translucency, outline display, or color highlighting.

[0893] This display method is intended to help users intuitively recognize the current construction progress status.

[0894] Such display methods are not limited thereto, and various visual distinction methods or combinations of visual effects may be applied depending on the embodiment.

[0895] The device (200) can apply different marking standards depending on the construction stage.

[0896] For example, structural areas included in the currently ongoing construction phase can be indicated with accent colors or flashing effects, allowing users to easily recognize structural areas requiring attention during that phase.

[0897] On the other hand, for structural areas that have already been completed, the emphasis effect can be removed to clearly indicate that they have been excluded from construction.

[0898] The device (200) can additionally indicate the relationship between structural regions.

[0899] For example, an unconstructed structural area can be output together with the target structural area to be combined using arrows, guidelines, or connection marks, allowing the user to understand in advance where the structural area will be combined.

[0900] This is an auxiliary display function designed to enhance understanding of the construction sequence simulation.

[0901] As a specific example, a case simulating the construction steps of prefabricated furniture is described.

[0902] The device (200) can be made to display the floor frame structure area in a state where placement is completed during the first construction phase.

[0903] At this time, the bottom frame is displayed in an opaque actual color, and the side plate structure area and the top plate structure area that are not yet joined may be displayed in a translucent state.

[0904] When transitioning to the second construction phase, the side plate structure area is highlighted to indicate that it is currently under construction, and the floor frame can be maintained as a completed structure area.

[0905] In this way, by distinguishing between completed structural areas and uncompleted structural areas at each construction stage and displaying them in an augmented reality-based 3D space, the device (200) can support the user in intuitively understanding the current construction progress status and the next task to be performed.

[0906] In step S605, the device (200) can sequentially provide construction sequence simulation results to the user's terminal, in which the completion status is updated step by step as the construction phase progresses.

[0907] That is, the device (200) can sequentially provide construction sequence simulation results, in which the completion status is updated step by step according to the progress of the construction phase, to the user's terminal (100).

[0908] Here, the construction sequence simulation result refers to visual and status-based output information that reflects the completion status of each stage and the current progress status as defined construction stages are applied sequentially over time.

[0909] The device (200) can manage the construction sequence simulation results on a project basis.

[0910] Here, the project may refer to a unit in which spatial information selected from the user's terminal (100), 3D object model identification information placed in the space, and construction sequence information for each structural area of ​​the 3D object model are linked and managed.

[0911] For example, if a user saves an interior design configuration including a living room, bedroom, and kitchen as a single project, the device (200) can manage 3D object models arranged by space and construction sequence information by structural area by mapping them to the same project identifier.

[0912] This allows users to view construction information for various spaces in an integrated manner at the project level, rather than managing it in a dispersed manner.

[0913] The construction sequence simulation results may include the current stage number, a list of structural areas completed in that stage, and information on structural areas scheduled to be performed in the next stage.

[0914] The device (200) can automatically update the completion status at each construction phase transition point.

[0915] The completion status update is performed after verifying that all placement, fixing, or joining actions performed in the previous step have been reflected, and is carried out based on the structural area identifier and step sequence information.

[0916] This allows the results of the construction sequence simulation to be maintained consistently with the actual construction flow.

[0917] The device (200) can provide updated construction sequence simulation results separated into steps.

[0918] The method of providing the results of such construction sequence simulations is not limited thereto and can be implemented in various ways depending on the embodiment, such as continuous playback, comparison display between steps, or summary step display.

[0919] The step-by-step provision can be performed by outputting a simulation screen of the corresponding step whenever each construction step begins, and the user's terminal (100) can clearly recognize the difference between the previous step and the current step.

[0920] Additionally, simulation navigation features may be provided to allow returning to previous steps or re-examining specific steps as needed.

[0921] When a list of construction items is generated, the device (200) can receive construction amount and memo information corresponding to each construction item from the user's terminal (100) and store them in a linked manner.

[0922] Here, construction cost may refer to the cost required to perform a specific construction item, and memo information may refer to text information attached to the item, such as material specifications, construction conditions, precautions, or matters for consultation.

[0923] For example, the user can enter an amount for the “floor tile installation” item and also enter a note such as “tile size 600×600, grout color gray”, and the device (200) can store the information by mapping it to a space identifier and a installation item identifier.

[0924] Additionally, the device (200) can automatically aggregate the input construction amount to calculate the total amount for a space unit, a floor unit, or a project-wide unit, and convert it into construction estimate data in the form of a table and provide it to the user's terminal (100).

[0925] This allows users to generate estimate data that can be modified according to project conditions, rather than fixed estimates, and immediately check the changes in the aggregated results.

[0926] The device (200) can control the speed of progress of the construction phase.

[0927] For example, the simulation of each step can be automatically switched at regular time intervals, or controlled to proceed to the next step according to a confirmation signal input from the user's terminal (100).

[0928] This allows users to adjust the simulation progress speed according to the difficulty or level of understanding of the construction.

[0929] The device (200) can provide status guidance information along with the construction sequence simulation results.

[0930] Status guidance information may include the purpose of the current construction phase, the types of placement or joining activities performed at that phase, and information on structural areas requiring attention.

[0931] This information can be displayed on the user's terminal (100) in the form of text, icons, or simple graphics.

[0932] As a specific example, a case of simulating the construction sequence of prefabricated furniture is described.

[0933] The device (200) can provide a simulation result in a state where the floor frame structure area is placed when the first construction phase is completed.

[0934] Subsequently, when transitioning to the second construction phase, simulation screens reflecting the completed joining of the side plate structure areas can be provided sequentially.

[0935] At each stage transition, the user's terminal (100) may display the current stage number and next stage guidance information together.

[0936] By sequentially providing construction sequence simulation results in which the completion status is updated according to the progress of the construction stages, the device (200) can provide an intuitive and reliable construction guide so that the user can clearly understand the entire construction process step by step and refer to the same sequence during actual construction.

[0937] In addition, the device (200) can generate reference information in the form of a link to share construction estimate data generated for a project with an external user and provide it to the user's terminal (100).

[0938] For example, the device (200) can generate a shared identifier including a project identifier and quotation data version information, and generate link information that can call a viewing page corresponding to the identifier.

[0939] External users can view the quotation table via a link in a web environment without installing a separate application, and users can perform cost negotiations and share changes among construction participants via the link.

[0940] This ensures that estimation information is shared on the same basis rather than being dispersed among project participants, thereby reducing omissions or discrepancies during the consultation process.

[0941] Through this, the device (200) can provide a highly reliable construction guide that supports the user in intuitively understanding the actual construction flow and performing tasks without errors by defining the construction sequence step by step by reflecting the arrangement and combination relationships and spatial constraints of the structural areas constituting the 3D object model and simulating it sequentially in an augmented reality-based 3D space.

[0942] FIG. 7 is a flowchart illustrating the process of generating shooting guide information to induce additional shooting based on shooting input data according to one embodiment.

[0943] In step S701, the device (200) receives shooting input data from a user's terminal and, based on the shooting input data, can generate acquired structure data including structure data items used for generating a 3D object model.

[0944] That is, the device (200) can generate acquired structural data including structural data items used for generating a 3D object model based on the captured input data.

[0945] Here, the acquired structure data refers to a data set that systematically organizes structural information that can be directly utilized to create a 3D object model, extracted from shooting input data received from the user's terminal (100).

[0946] Acquired structure data is not a simple set of images, but consists of minimum unit data items to describe the shape and structure of an object.

[0947] The device (200) can analyze object images or object image data included in the shooting input data to extract structural data items.

[0948] Structural data items may include shape information in which the entire outline of an object is identifiable, shape information including major face or edge information of an object, feature point distribution information required for structural area segmentation, and visual pattern information that can be used for surface texture analysis.

[0949] Structural data items are not limited to this, and additional structural feature information may be included depending on the type of object or the shooting environment.

[0950] These structural data items are subsequently used as basic data for structural region partitioning, structural parameter setting, and structural relationship analysis during the 3D object model creation stage.

[0951] The device (200) can determine whether a structural data item is included based on the shooting angle, shooting distance, and shooting range of the shooting input data.

[0952] For example, if a specific face of an object is not included in the captured input data, shape information or texture information corresponding to that face may not be included in the acquired structure data.

[0953] In this case, the device (200) can display the corresponding structure data item as unacquired.

[0954] The device (200) can organize acquired structure data by classifying structure data items by item type.

[0955] Item types may include overall shape identification items, structural area boundary identification items, joint location identification items, surface pattern identification items, etc.

[0956] By organizing structural data by item type in this way, it is possible to clearly determine whether any items are missing during the subsequent comparison process with reference structural data items.

[0957] The device (200) can store the reliability of each structure data item together with the acquired structure data.

[0958] Reliability is an indicator of how clearly the corresponding structural data item is identified in the shooting input data, and can be calculated based on at least one of resolution, focus clarity, presence of obstruction of view, or number of repeated shots.

[0959] This reliability information can be used as a supplementary criterion to determine whether the criteria are met during the subsequent step of identifying insufficient data items.

[0960] As a specific example, a case in which furniture objects are photographed to generate acquired structural data is described.

[0961] The device (200) can analyze a frame that clearly includes the front and side of a captured image of a furniture object received from a user's terminal (100) and generate a structural data item for the overall contour shape.

[0962] On the other hand, if the lower joint is obscured and not captured, the structural data item corresponding to that joint is displayed as unacquired and can be reflected in the acquired structural data.

[0963] In this way, the device (200) generates acquired structural data that systematically organizes structural data items necessary for creating a 3D object model based on shooting input data, thereby providing a basis for objectively determining whether structural information is satisfied in a subsequent step and deciding whether additional shooting is necessary.

[0964] In step S702, the device (200) can compare the acquired structure data with the predefined reference structure data items used for creating a 3D object model, and identify the data items among the reference structure data items that have not been acquired or do not meet the criteria of the predefined reference structure data items as insufficient data items.

[0965] That is, the device (200) can compare the predefined reference structure data items used for creating a 3D object model with the acquired structure data, and identify the data items among the reference structure data items that have not been acquired or do not meet the criteria as insufficient data items.

[0966] Here, the reference structure data item refers to a set of predefined minimum required structure information to reliably generate a 3D object model.

[0967] Standard structural data items can be set differently depending on the object type; for example, the configuration of required items may vary depending on the use and structural complexity of objects such as furniture, home appliances, and machine parts.

[0968] This standard structural data item is not limited to a specific implementation and is merely an example of a standard item to ensure the structural reproducibility of an object.

[0969] The device (200) can manage reference structure data items by defining them according to item type.

[0970] Item types may include full contour identification items, major face identification items, structural area boundary identification items, joint location identification items, surface texture representative items, etc.

[0971] For each item type, judgment criteria can be established to determine whether the item is satisfied.

[0972] The device (200) can compare the structure data items included in the acquired structure data with the reference structure data items by matching them by item type.

[0973] For example, if the front, rear, left side, and right side shape information of an object is required in the reference structure data item, the device (200) can compare whether all shape information corresponding to the corresponding sides is included in the acquired structure data.

[0974] This comparison can be performed by referring to reliability information in addition to simple inclusion status.

[0975] The device (200) can determine whether the criteria for a reference structure data item are met quantitatively or on a rule basis.

[0976] For example, if the reliability of a specific structural data item is less than a preset threshold value, the device (200) may determine that the item does not meet the threshold.

[0977] Alternatively, even if the shooting angle is not included within a specific range and the boundaries of the structural area are not clearly identified, it may be determined that the criteria are not met.

[0978] The device (200) can distinguish and process cases where no reference structure data items are acquired at all, and cases where some are acquired but do not meet the criteria.

[0979] If it is not acquired at all, the item can be classified as an unacquired item, and if it is partially acquired but does not meet the criteria, it can be classified as an incompletely acquired item.

[0980] This distinction is intended to reflect the differences in shooting methods required when generating shooting guide information later.

[0981] The device (200) can record the status of satisfaction of acquired structural data compared to reference structural data items for each structural data item.

[0982] This record may include whether each item was acquired, whether criteria were met, and the reliability level, and is used as supporting data to determine whether to induce additional filming later.

[0983] As a specific example, a case is described in which the reference structural data item of a furniture object is compared with the acquired structural data.

[0984] The device (200) can identify as an unacquired item if the lower joint shape information is included in the reference structure data item but the corresponding information is not included in the acquired structure data.

[0985] In addition, if upper surface shape information is obtained but the boundary is unclear due to a limited shooting angle, the device (200) may classify this as an item that does not meet the criteria.

[0986] In this way, the device (200) can systematically compare reference structural data items and acquired structural data to identify insufficient data items, thereby objectively determining whether there is a lack of structural information required for creating a 3D object model, and clearly specifying specific targets and ranges that require additional shooting in subsequent steps.

[0987] In step S703, if a missing data item is identified, the device (200) can generate shooting guide information to induce additional shooting corresponding to the missing data item.

[0988] That is, when a deficient data item is identified, the device (200) can generate shooting guide information to induce additional shooting corresponding to the deficient data item.

[0989] Here, the shooting guide information refers to a set of information intended to specifically guide users on which direction, range, and method of shooting should be used to supplement insufficient data items when performing additional shooting through the user's terminal (100).

[0990] The shooting guide information consists not of simple text instructions, but of structured guidance information generated by reflecting the types of missing data items and the causes of the deficiencies.

[0991] The method of configuring such shooting guide information is not limited to this, and may be provided in various modified forms depending on the object type, shooting environment, or implementation method.

[0992] The device (200) can determine the content of the shooting guide information based on the item type of the identified missing data item.

[0993] For example, if there is a lack of items for identifying the entire outline, a guide may be generated to instruct the user to expand the shooting range so that the entire object is included on the screen.

[0994] If there is a lack of identification items for the joint location, a guide can be generated to specify shooting angle and distance conditions so that a specific joint is exposed.

[0995] The device (200) can analyze the cause of the shortage of data items and subdivide the shooting guide information.

[0996] If the cause of the deficiency is determined to be a limitation of the shooting angle, the device (200) may include direction information in the shooting guide information to induce the terminal to move or rotate toward a specific side of the object.

[0997] If the cause of the deficiency is determined to be a lack of shooting distance or resolution, the device (200) may generate a guide to reduce the shooting distance or perform enlarged shooting.

[0998] The device (200) can be configured to include shooting guide information in a form that includes visual elements.

[0999] For example, virtual arrows, translucent guide areas, and recommended shooting area frames can be displayed in an augmented reality-based 3D space so that the user can intuitively understand where to take a shot when taking additional shots.

[1000] These visual guides can be generated based on the location of structural areas corresponding to the missing data items.

[1001] The device (200) can include shooting condition information along with shooting guide information.

[1002] Shooting condition information may include a recommended shooting angle range, a recommended shooting distance range, minimum resolution standards, or guidance on lighting conditions during shooting.

[1003] This condition information is set to meet the criteria required in the reference structure data item and can be provided in the form of a range, not limited to specific numerical values.

[1004] When multiple missing data items are identified, the device (200) can generate shooting guide information by separating it by item.

[1005] Alternatively, to reduce the burden of shooting for the user, an integrated shooting guide can be generated to simultaneously supplement multiple missing data items with a single shot.

[1006] In this case, the integrated shooting guide can be configured based on the common shooting direction or common shooting range of the insufficient data items.

[1007] As a specific example, a case is described where the shape information of the lower joint of a furniture object is identified as an insufficient data item.

[1008] The device (200) can generate an arrow-shaped shooting guide that guides the user's terminal (100) to move to the bottom of the object based on the direction in which the lower coupling part is located.

[1009] In addition, a frame guide can be displayed to position the lower joint at the center of the screen, and the shooting distance can be reduced to guide the joint to be captured at sufficient resolution.

[1010] In this way, the device (200) generates shooting guide information corresponding to the insufficient data item, thereby enabling the user to clearly recognize what structural information needs to be supplemented through additional shooting, and can effectively improve the completeness of the structural data required for the creation of a 3D object model thereafter.

[1011] For a detailed explanation regarding this, refer to Fig. 8.

[1012] In step S704, the device (200) can provide the generated shooting guide information to the user's terminal.

[1013] That is, the device (200) can provide the generated shooting guide information to the user's terminal (100).

[1014] Here, the provision of shooting guide information means that the information is delivered to the user in a form that is recognizable and immediately usable, so that the user can actually perform additional shooting.

[1015] The shooting guide information is not a simple data transmission, but functions as guidance information that directly induces shooting behavior on the user's terminal (100).

[1016] The device (200) can provide shooting guide information to be visually displayed on the display of the user's terminal (100).

[1017] For example, by overlaying virtual arrows, translucent guide frames, and recommended shooting area indicator lines on the shooting screen, users can intuitively recognize the shooting direction and range.

[1018] These visual elements can be displayed in real-time corresponding to the current shooting screen of the physical object.

[1019] The device (200) can provide shooting guide information step by step.

[1020] If there are multiple missing data items, the device (200) can determine whether one shot has been completed and then sequentially provide the next shot guide.

[1021] This allows users to proceed with shooting step-by-step without receiving complex shooting instructions all at once.

[1022] While the device (200) provides shooting guide information, it can also provide the shooting progress status.

[1023] For example, it can indicate that the structural data item corresponding to the current shooting guide has not yet been satisfied, or update and display it as completed if the shooting meets the criteria.

[1024] This status information helps the user clearly understand whether additional shooting is necessary.

[1025] The device (200) can operate by providing shooting guide information and receiving shooting input data in conjunction.

[1026] When additional shooting is performed at the user's terminal (100), the device (200) can analyze the newly received shooting input data again to determine whether the missing data item has been resolved.

[1027] If missing data items are not resolved, the shooting guide information can be supplemented or updated and provided again.

[1028] The device (200) can adjust the method of providing shooting guide information according to the characteristics of the user's terminal (100).

[1029] For example, for devices with small screen sizes or resolutions, a simplified guide can be provided, while for devices with augmented reality features enabled, a more detailed AR-based guide can be provided.

[1030] Adjusting this method of provision does not change the content of the shooting guide information itself, but rather corresponds to adjusting the method of expression and the level of display.

[1031] As a specific embodiment, the case where the user's terminal (100) is a smartphone is described.

[1032] The device (200) can display an arrow indicating the direction of the lower coupling part and a recommended shooting area frame in real time on the camera shooting screen of a smartphone.

[1033] When the user performs a shot following the guide, the device (200) determines that additional shooting is completed and may provide a shooting guide corresponding to the next missing data item or stop providing the shooting guide.

[1034] In this way, the device (200) provides shooting guide information to the user's terminal (100), thereby inducing the user to effectively perform additional shooting and supporting the stable securing of structural data items necessary for creating a 3D object model.

[1035] Through this, the device (200) objectively determines whether structural data for generating a 3D object model is satisfied based on shooting input data, and provides a shooting guide corresponding to the insufficient structural information to the user's terminal (100), thereby improving the completeness of the structural data through additional shooting and enabling stable generation of a 3D object model.

[1036] FIG. 8 is a flowchart illustrating the process of generating shooting guide information according to one embodiment.

[1037] Referring to FIG. 8, first, in step S801, the device (200) can identify direction information or specific part information for which additional shooting of a real object is required based on the missing data item.

[1038] That is, the device (200) can identify direction information or specific part information for which additional shooting of a real object is required based on the missing data item.

[1039] Here, the direction information refers to the shooting direction in which the user's terminal (100) is positioned or facing the real object, and the specific part information refers to a specific area among the structural areas constituting the real object where additional shooting is required.

[1040] Directional information and specific part information are identified by different criteria depending on the type of missing data item and the cause of the deficiency.

[1041] These identification criteria can be extended to various standards depending on the object type, shooting environment, or implementation method.

[1042] The device (200) can first determine the item type of the insufficient data item to determine whether the item is due to insufficient shape of the entire object or due to insufficient information of a specific structural area.

[1043] For example, if the entire contour identification item or the main face identification item is identified as an insufficient data item, the device (200) determines that it is necessary to photograph the entire object from various viewpoints and can perform direction information identification.

[1044] On the other hand, if there is a lack of identification items for the location of the joint part or boundary information for a specific structural area, the device (200) can perform identification of specific part information corresponding to the structural area.

[1045] When identifying direction information, the device (200) can analyze the distribution of shooting times of already acquired structural data items.

[1046] For example, if the shooting input data is mainly concentrated on the front and left sides, the device (200) determines that the right side or rear direction has not been sufficiently captured and can identify the direction as direction information requiring additional shooting.

[1047] In this case, the direction information is not an absolute orientation, but can be defined based on the reference coordinate system of the real object or the camera coordinate system.

[1048] When identifying specific part information, the device (200) can refer to location information of a structural area corresponding to a missing data item.

[1049] Location information of a structural area may include a structural area identifier defined during the structural area partitioning process and the spatial scope of the area.

[1050] The device (200) can determine where the structural area is located among the upper, lower, internal joint, or outer part of the object to identify specific part information requiring additional shooting.

[1051] The device (200) can distinguish and process cases where direction information and specific part information are required simultaneously.

[1052] For example, if a specific structural area is located at the bottom of an object and the corresponding bottom area is hardly included in the current shooting input data, the device (200) can identify shooting direction information in the downward direction and specific part information called the bottom coupling part together.

[1053] This complex identification is intended to provide more precise guidance when generating shooting guide information later.

[1054] As a specific example, a case in which a chair-shaped furniture object is photographed is described.

[1055] If the device (200) has sufficient structural data for the seat top and backrest area, but structural data for the legs and lower joint is identified as an insufficient data item, the lower direction of the chair can be identified as direction information requiring additional shooting.

[1056] In addition, the joint area where the leg and the seat top are joined can be identified as specific part information requiring additional shooting.

[1057] In this way, the device (200) can analyze the type and cause of the insufficient data item and specifically identify direction information or specific part information requiring additional shooting, thereby enabling the shooting request and shooting guide information generated in a subsequent step to directly contribute to supplementing the actually insufficient structural data.

[1058] In step S802, the device (200) may generate a first request including direction information requiring additional shooting or a second request including specific part information requiring additional shooting based on identified information.

[1059] That is, the device (200) can generate, based on the identified information, a first request including direction information requiring additional shooting or a second request including specific part information requiring additional shooting.

[1060] Here, the first request refers to a request to induce supplementation of the shooting direction for the entire physical object, and the second request refers to a request to induce shooting of a specific structural area or specific part constituting the physical object.

[1061] The first and second requests are core elements constituting the shooting guide information, and are generated in different content and formats depending on the type of missing data item.

[1062] If direction information is identified, the device (200) can generate a first request including the direction information.

[1063] The first request may include the direction in which the user's terminal (100) must move or rotate, the time range at which shooting is required, and the reason why shooting in that direction is required.

[1064] For example, if only shooting data for the front and left side is obtained, the device (200) may include in the first request that shooting is required in the direction of the right side or rear of the object.

[1065] The device (200) can express direction information based on relative direction when the first request is generated.

[1066] For example, direction information can be configured in a form such as “move clockwise relative to the object from the current shooting location” or “move to the opposite side of the object and shoot.”

[1067] Such relative expressions can be interpreted and provided in real time depending on the location or direction of the user's terminal (100).

[1068] If specific part information is identified, the device (200) can generate a second request including said specific part information.

[1069] The second request may include the location of a structural area requiring additional shooting, the relative location of said structural area within the object, and shooting range information to clearly identify said structural area.

[1070] For example, if the joint area is identified as an insufficient data item, the device (200) may include content such as “close-up shot of the lower joint area” in the second request.

[1071] The device (200) can associate specific part information with a visual reference when generating a second request.

[1072] By referring to the structural area identifier defined in the structural area partitioning step, it is possible to clearly indicate which location on the physical object the corresponding structural area corresponds to.

[1073] This allows users to easily recognize which parts to focus on when taking additional shots.

[1074] The device (200) may generate only one of the first request and the second request, or generate both requests together, depending on the characteristics of the insufficient data item.

[1075] For example, if both shooting in a specific direction of the entire object and shooting in a specific structural area are required simultaneously, the device (200) can configure the first request and the second request into a single shooting guide flow.

[1076] In this case, the two requests can be provided sequentially or as a single integrated request.

[1077] As a specific example, a case where information on the lower joint of a furniture object is insufficient is described.

[1078] The device (200) can first generate a first request based on direction information that shooting in the downward direction is required.

[1079] At the same time, a second request can be generated based on specific part information indicating that the joint area between the lower leg and the seat part needs to be photographed in close range.

[1080] These two requests are provided sequentially to the user's terminal (100) to induce close-up shooting of the joint part after full lower body shooting.

[1081] In this way, the device (200) can clearly specify the location and range where additional shooting is structurally necessary by generating a first request and a second request based on direction information or specific part information corresponding to the missing data item, and then provide specific and actionable guidance in the subsequent shooting guide information generation stage.

[1082] In step S803, the device (200) can generate shooting guide information including a first request or a second request.

[1083] That is, the device (200) can generate shooting guide information including a first request or a second request.

[1084] Here, shooting guide information refers to a set of information that concretizes the content of a request into visual and procedural guidance information in order to connect the first or second request to the user's actual shooting action.

[1085] The shooting guide information is not merely a list of request phrases, but is configured to be referenced in real time while shooting is being performed on the user's terminal (100).

[1086] When the first request is generated, the device (200) can configure shooting guide information centered on direction information.

[1087] In this case, the shooting guide information may include the direction in which additional shooting is required, the recommended shooting movement path, and reference information for determining whether shooting in that direction has been completed.

[1088] For example, if rear shooting of an object is required, the device (200) can generate shooting guide information including an arrow-shaped guide element that induces movement in the opposite direction relative to the object at the current shooting point.

[1089] When a second request is generated, the device (200) can configure shooting guide information centered on specific part information.

[1090] In this case, the shooting guide information may include the location of the structural area requiring additional shooting, the recommended position where the said structural area should occupy within the screen, and conditions for determining whether the structural area has been sufficiently shot.

[1091] For example, if a joint area is identified by specific part information, the device (200) can generate a shooting guide that guides the joint area to be positioned in the center of the screen or within a designated guide frame.

[1092] The device (200) can configure shooting guide information together with visual elements.

[1093] Visual elements may include arrows indicating direction, a translucent frame indicating the recommended shooting area, and status indicator elements indicating whether shooting is complete.

[1094] These visual elements can be displayed superimposed on the current shooting screen of the real object and can be reflected in real time on the camera screen of the user's terminal (100).

[1095] The device (200) can include shooting condition information in the shooting guide information.

[1096] Shooting condition information may include reference information such as the recommended shooting distance range, recommended shooting angle range, minimum resolution, or minimum number of frames.

[1097] This reference information is set to satisfy the conditions required in the reference structure data item and can be provided in the form of a range rather than being fixed to a specific value.

[1098] The device (200) can configure shooting guide information step by step when both the first request and the second request are generated.

[1099] For example, after first providing a direction shooting guide corresponding to the first request, if it is determined that the shooting in that direction is completed, a specific part shooting guide corresponding to the second request may be provided.

[1100] This step-by-step structure contributes to increasing the shooting success rate by preventing users from receiving complex shooting instructions all at once.

[1101] As a specific example, a case where shooting the bottom of a furniture object is required is described.

[1102] The device (200) can generate shooting guide information that guides the lower part of an object to be included in the bottom guide frame of the screen, along with an arrow that guides it to move in a downward direction.

[1103] After the bottom shot is completed, a shooting guide can be continuously provided to guide close-up shooting so that the bottom joint is positioned in the center of the screen.

[1104] In this way, the device (200) can provide a shooting environment that supports the user in accurately performing additional shooting and effectively supplements missing data items by configuring shooting guide information including a first request or a second request into specific visual and conditional guidance.

[1105] By doing so, the device (200) can identify a direction or specific structural area requiring additional shooting according to the type and cause of the insufficient structural data item, and generate a shooting guide including a first request or a second request corresponding thereto, thereby inducing the user to accurately supplement the necessary structural information and substantially improve the accuracy and completeness of the 3D object model creation.

[1106] FIG. 9 is a drawing illustrating an example screen in which a display is output to guide a user that there is an applicable motion motion for a 3D object model placed in an augmented reality-based 3D space according to one embodiment.

[1107] The screen illustrated in FIG. 9 is an example of an augmented reality-based 3D space output to a user's terminal (100), showing a state in which a 3D object model generated by the device (200) is placed within the space with the actual indoor space as the background.

[1108] In the drawing, an augmented reality-based 3D space with a kitchen space as the background is displayed, and 3D object models in the form of furniture are placed within the space.

[1109] The device (200) can output a motion guide display for an object model that has an applicable motion based on information regarding the motion relationship between structural parameters and structural regions among 3D object models placed in an augmented reality-based 3D space.

[1110] Figure 9 illustrates an example in which such motion guide indicators are displayed in the form of icons superimposed on specific locations of a 3D object model.

[1111] The motion guide indication shown in the drawing is a visual indication provided to allow the user to intuitively recognize that there is an actionable motion for the corresponding 3D object model.

[1112] For example, the icon shown in the drawing can be used as an indicator to inform the user that motions such as opening, closing, moving, rotating, or changes in the connection state between structural regions can be applied to the 3D object model.

[1113] The device (200) can output a motion guide display only when there exists a structural area or a set of structural areas to which motion is applicable.

[1114] That is, if there is no motion within the allowable range according to the structural parameter information, a motion guide display such as that shown in Fig. 9 may not be output.

[1115] The motion guidance display shown in FIG. 9 can be associated with touch input, selection input, or gesture input on the user's terminal (100).

[1116] When the user selects the corresponding display, the device (200) may provide a list of motions applicable to the selected 3D object model, or provide a specific motion by simulating it directly in an augmented reality-based 3D space.

[1117] Additionally, the device (200) can control the device to move to an external web-based product page when a purchase selection input occurs on the user's terminal (100), provided that the selected 3D object model is linked to a product object and external link information exists.

[1118] For example, a button for inputting a purchase selection may be provided together with a motion guide display or an object detail screen, and when the user selects the button, the device (200) can call a purchase page in the form of a browser or webview using stored external link information.

[1119] At this time, the button for inputting the purchase selection may be displayed as text or an icon in the form of “Shop,” “Buy Now,” or a similar form, and depending on the selection of the button, the device (200) may control to call URL information of an external web-based product page and display it on the user’s terminal (100).

[1120] This allows users to naturally move to the product information screen associated with the same object immediately after verifying the product's operational structure through motion simulation, enabling them to check additional information necessary for purchasing decisions.

[1121] As such, FIG. 9 illustrates an exemplary screen configuration that supports the user in intuitively recognizing and selecting a motion motion simulation function by visually guiding whether there is a motion motion applicable to a 3D object model placed in an augmented reality-based 3D space by the device (200).

[1122] In addition, the motion can be played based on the same motion data not only in mobile application environments but also on external web-based product pages linked to product information, and can be utilized as dynamic visual information to explain the functional characteristics and operation structure of the product.

[1123] FIG. 10 is a drawing illustrating an example screen in which an object selection interface is provided for selecting a 3D object model to be placed in an augmented reality-based 3D space according to one embodiment.

[1124] The screen illustrated in FIG. 10 is an example of an object selection interface output to a user's terminal (100), and represents a screen configured to allow the user to select an object to be placed in an augmented reality-based 3D space from among a plurality of 3D object models managed by the device (200).

[1125] Spatial information in the form of a floor plan is displayed at the top of the drawing, and an interface providing a list of selectable objects is positioned at the bottom.

[1126] The device (200) can provide an object selection interface to the user's terminal (100), thereby enabling the user to explicitly select a 3D object model to be placed in an augmented reality-based 3D space.

[1127] Figure 10 illustrates an example in which multiple object images belonging to the bed category are arranged and displayed in a grid format.

[1128] Each object image displayed in the object selection interface represents a selection item corresponding to a 3D object model that is stored in the device (200) or can be provided from an external storage.

[1129] When a selected item is associated with a product object, the device (200) can provide a product status display along with the selected item in the object selection interface.

[1130] For example, a badge, tag, icon, or text indicating whether a product is linked may be displayed around the preview image of the selected item, and at least one of the product name, brand, or category may be displayed together.

[1131] In addition, the device (200) can configure the category menu not only as an object category but also as a brand category, so that the classification results can be reflected to enable object search by brand on the user's terminal (100).

[1132] Through this, users can intuitively select design objects and purchase-linked objects together while selecting objects to place in a project or augmented reality-based 3D space.

[1133] Each selection item is provided in the form of an image or preview that visually represents the appearance of the object, allowing the user to intuitively identify the object to be placed.

[1134] When a user selects a specific object image on an object selection interface, the device (200) can set a 3D object model corresponding to that selection as a target object for subsequently placing in an augmented reality-based 3D space.

[1135] The selected 3D object model is placed in an augmented reality-based 3D space as described in claim 1, and spatial transformations such as positioning, rotation, or scaling may be applied.

[1136] The object selection interface illustrated in FIG. 10 may include at least one of an object category, an object list, and a selection status display, and may be provided in various forms depending on the screen size or interface configuration of the user's terminal (100).

[1137] The interface configuration illustrated in the drawing is merely an example, and the object selection method is not limited thereto.

[1138] As such, FIG. 10 exemplarily illustrates a screen configuration that provides an object selection interface so that the device (200) can select a 3D object model to be placed in an augmented reality-based 3D space.

[1139] FIG. 11 is a drawing illustrating an example screen in which a 3D object model is created based on shooting input data captured by a user according to one embodiment, and shape and size information of the created 3D object model is displayed.

[1140] The screen illustrated in FIG. 11 is an example of a 3D object model generation result screen output to a user's terminal (100), showing the state in which the device (200) analyzes captured input data and generates a 3D object model corresponding to a real object.

[1141] The 3D object model displayed in the drawing is expressed three-dimensionally by reflecting the external appearance of the actual object, and the entire shape and structure of the object are displayed within a single screen.

[1142] The device (200) can calculate size information along with shape information of an object and display it on a screen for a 3D object model generated from shooting input data.

[1143] Figure 11 illustrates an example in which size information corresponding to the width, length, and height of a 3D object model is displayed in numerical form.

[1144] When a 3D object model is linked to a product object, the device (200) can additionally display some of the product specifications and attribute information along with shape and size information on the user's terminal (100).

[1145] For example, size information can be provided as dimension items directly used for product placement, such as length, width, and height, and material information can be provided as attribute items that affect appearance and usage characteristics, such as wood, metal, fabric, or plastic.

[1146] These specification and attribute information may be displayed by referencing values ​​entered from an external product management system, or size information calculated during the 3D object model creation process may be converted into specification information and displayed.

[1147] This allows users to view actual or ratio-based dimensions of objects and product specification information together on the same screen, thereby improving the accuracy of subsequent augmented reality-based 3D spatial placement and space occupancy judgment.

[1148] The size information shown in the drawing refers to dimensional information calculated based on the outer shape or structural area of ​​the 3D object model.

[1149] For example, numerical values ​​corresponding to the width, depth, and height of an object can be displayed distinguished by different colors or positions, which is intended to enable intuitive recognition of size information for each direction.

[1150] The device (200) can calculate size information that reflects the actual scale of the 3D object model based on depth information, structural region division results, or structural parameter information extracted during the 3D object model creation process.

[1151] Additionally, if there is product specification information set in the product object, the device (200) can compare or correct the specification information with the calculated size information to adjust the scale so that the width, length, and height of the 3D object model placed in the augmented reality-based 3D space correspond to the actual product size.

[1152] At this time, size information may be provided in the form of at least one of absolute actual values ​​or relative ratio values.

[1153] The bottom of the screen illustrated in Fig. 11 may include an information area in which size information of the generated 3D object model is summarized and displayed.

[1154] An example is illustrated in which the information area displays the modeling name or status information for identifying the 3D object model, along with the size values ​​for each direction.

[1155] The device (200) can support the user in verifying whether the generated 3D object model properly reflects the shape and size of the real object through the display of such shape and size information.

[1156] In addition, based on the information, the user can determine the suitability of the object model to be placed in the augmented reality-based 3D space later.

[1157] As such, FIG. 11 exemplarily illustrates a screen configuration in which a device (200) generates a 3D object model based on shooting input data and visually provides shape and size information of the generated 3D object model.

[1158] FIG. 12 is a diagram illustrating an example screen in which a generated 3D object model according to one embodiment is placed in an augmented reality-based 3D space, positional relationship and distance information of the object are displayed, and spatial transformation is applied.

[1159] The screen illustrated in FIG. 12 is an example of an augmented reality-based 3D space output to a user's terminal (100), showing a state in which a 3D object model generated by the device (200) is placed on top of floor plan-based spatial information representing an indoor space.

[1160] The top of the drawing displays the overall floor plan structure and dimension information of the indoor space, and a 3D object model is placed inside the space.

[1161] The device (200) can place the generated 3D object model at a specific location in an augmented reality-based 3D space.

[1162] The device (200) may selectively execute motions set on an object for a 3D object model placed in an augmented reality-based 3D space, thereby providing a simulation of the operation process based on the actual environment.

[1163] The motion executed at this time can be played based on the same motion data as the motion used in the project space or product detail screen.

[1164] Here, the project space refers to a virtual design environment where a user places multiple 3D object models for interior design and manages the spatial configuration and construction sequence, and the same motion data can be applied to be reproduced within the space by assuming the installation state of the object.

[1165] In addition, the product detail screen refers to an interface screen for checking the shape, specifications, and external appearance information of a product corresponding to a 3D object model, and the same motion data as in the project space can be reused on the screen to allow for checking the operational structure of the product in advance.

[1166] Figure 12 illustrates an example in which a 3D object model is placed within the floor area of ​​an indoor space, and a display indicating a selection state is output on the outer edge of the object.

[1167] The device (200) can calculate and display positional relationship and distance information between a 3D object model placed in an augmented reality-based 3D space and a reference element in the space.

[1168] In the drawing, an example is shown in which distance information between a 3D object model and a wall, doorway, or space boundary is output together in the form of linear display and numerical information.

[1169] This distance information can be used as reference information to determine the appropriateness of object placement.

[1170] The device (200) can apply spatial transformation to a 3D object model.

[1171] Figure 12 illustrates an example in which an icon indicating the direction of rotation and a mark indicating the path of movement are output together during the process of applying spatial transformations, such as translation or rotation, to a 3D object model.

[1172] This allows users to intuitively adjust the placement position and orientation of 3D object models.

[1173] In addition, the device (200) can display the size information of the 3D object model and the placement result in space together.

[1174] An example is shown at the top of the drawing in which size information corresponding to the width, length, and height of the placed 3D object model is displayed in a summary form.

[1175] This size information indicates that the shape and size information of the 3D object model calculated in Fig. 11 is reflected in the augmented reality-based 3D space.

[1176] At the bottom of the screen illustrated in Fig. 12, an interface for selecting other 3D object models that can be placed in an augmented reality-based 3D space may be provided.

[1177] The device (200) can provide multiple product images in a preview form for a 3D object model linked to a product object in the selection interface.

[1178] For example, if there are representative images and additional images registered in an external product management system, the device (200) can display multiple images for the same 3D object model selection item in a manner such as an image slide, a thumbnail list, or an enlarged preview.

[1179] In this case, the representative image and additional images may be image data registered on the product management page in a PC environment.

[1180] Users can check the material texture, color composition, or detailed structure of the actual product through multiple product images, and by comparing this with the placement results in an augmented reality-based 3D space, they can review the product's appearance and spatial suitability together.

[1181] Through this, users can view the appearance of objects from various angles or with various color options while maintaining the augmented reality-based 3D spatial placement screen, and the accuracy of selecting placement targets can be improved.

[1182] This is intended to provide a selection function for the user to place or replace another object model while maintaining the currently placed 3D object model.

[1183] As such, FIG. 12 illustrates an exemplary screen configuration that supports a user in intuitively checking and adjusting the object placement status based on the actual space by placing a generated 3D object model in an augmented reality-based 3D space and visually providing the positional relationship, distance information, and spatial transformation results of the object.

[1184] At this time, the device (200) can reproduce the motion of an object using the same motion data as the project space or detail screen even in an augmented reality-based 3D space, thereby enabling verification of the range of motion, ease of use, and suitability for installation based on actual environment standards.

[1185] FIG. 13 is an example diagram of the configuration of a device according to one embodiment.

[1186] A device (200) according to one embodiment includes a processor (210) and a memory (220). A device (200) according to one embodiment may be the server or terminal described above. The processor (210) may include at least one device described above through FIGS. 1 to 12 or perform at least one method described above through FIGS. 1 to 12. The memory (220) may store information related to the method described above or store a program in which the method described above is implemented. The memory (220) may be volatile memory or non-volatile memory.

[1187] The processor (210) can execute a program and control the device (200). The code of the program executed by the processor (210) can be stored in memory (220). The device (200) can be connected to an external device (e.g., a personal computer or a network) through an input / output device (not shown in the drawing) and exchange data.

[1188] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[1189] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[1190] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[1191] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[1192] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[1194] 100: User's terminal 200: Device 210: Processor 220: Memory

Claims

Claim 1 A method for providing 3D modeling and AR motion simulation of a user-captured object, performed by a device, comprising: receiving shooting input data including one or more object images or object video data obtained by capturing a real object from a user's terminal; generating a 3D object model corresponding to the real object based on the shooting input data; placing the generated 3D object model in an augmented reality-based 3D space corresponding to the actual captured image or video; applying at least one spatial transformation among position translation, rotation, or size deformation to the 3D object model placed in the augmented reality-based 3D space; and determining a motion applicable to the 3D object model based on structural parameters of the generated 3D object model. The method includes the step of applying the determined motion motion to a 3D object model placed in the augmented reality-based 3D space to simulate and provide the motion state of the 3D object model; and the step of generating a 3D object model corresponding to the real object based on the captured input data comprises: the step of extracting object shape information including the contour, feature points, and surface texture features of the real object from the captured input data; the step of dividing a structural region constituting the real object based on at least one of a shape discontinuity section, a curvature change section, or a joint candidate section based on the object shape information; the step of deriving motion relationship information between structural regions by analyzing at least one of a connection relationship, a relative position relationship, or a rotatable relationship between the divided structural regions; the step of setting the structure of the 3D object model by setting structural parameter information including a structural reference axis, a structural degrees of freedom, and a structural parameter allowable range based on the motion relationship information between structural regions; and the step of configuring the 3D object model to include a predefined motion application parameter linked to the structural parameter information.The step of determining motions applicable to the 3D object model comprises: querying candidate motions stored in a motion motion database based on motion relationship information between the structural regions; comparing motion parameter information included in each candidate motion with structural parameter information set in the 3D object model by matching parameter items of the same type; determining, based on the comparison result, whether the motion parameter information of each candidate motion falls within an allowable range defined by the structural parameter information; for candidate motions determined to fall within the allowable range, determining whether interference occurs between structural regions when applying the motion by referring to the relative positional relationship between the structural regions; for candidate motions determined to fall within the allowable range, comparing the amount of deformation occurring in the structural regions when applying the motion with predefined deformation allowable range data; classifying each candidate motion into applicable motions, motions requiring restriction conditions upon application, or inapplicable motions based on the comparison result and whether interference occurs between the structural regions; and determining a set of motions applicable to the 3D object model in the augmented reality-based 3D space according to the classification result. method., Claim 2 delete Claim 3 In claim 1, the method further comprises the step of generating a new motion of the generated 3D object model; wherein the step of generating the new motion of the 3D object model comprises: providing a motion target input UI to a user terminal for specifying a target position, target angle, or target state of the 3D object model in the augmented reality-based 3D space; converting an input received through the motion target input UI into target state data corresponding to structural parameter information set in the 3D object model; comparing the target state data with the current state of the 3D object model to calculate a plurality of motion path candidates that satisfy an allowable condition defined by the structural parameter information; for each of the motion path candidates, evaluating the possibility of interference based on whether overlap or collision occurs between structural regions on the motion path; based on the evaluation result, selecting the motion path with the lowest possibility of interference between structural regions to generate a new motion of the target state data; and applying the generated new motion of the motion to the 3D object model in the augmented reality-based 3D space to simulate and provide the motion process toward the target state. Method of provision.

Images