Object synchronization device and object synchronization method

Abstract

The invention relates to an object synchronization device and an object synchronization method. The object synchronization apparatus includes a memory storing instructions executable by a computer and at least one processor accessing the memory and executing the instructions. The at least one processor: creates a virtual object corresponding to a source code in a virtual space in response to a compilation request based on the source code of a user; performing verification on the created virtual object; in response to a control request for the validated virtual object, performing synchronization between the validated virtual object and the real object; and when the synchronized real object detects the target object, creating a sub-virtual object corresponding to the detected target object in the virtual space.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority from Korean Patent Application No. 10-2023-0093163 filed on July 18, 2023, in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to an object synchronization device and method thereof, and more particularly, to a technology that enables multiple students to simultaneously participate in robot coding education using a metaverse space. Background Art

[0004] Coding is the process of creating a code that is a language used to execute a program for a specific purpose. Since coding inspires creativity, coding education is increasing in elementary and middle schools. In addition, recently, as coding education has been determined as compulsory education in elementary and middle schools, coding education for students has also increased.

[0005] However, in the process of applying the source code for controlling the robot to the robot, the number of students should be consistent with the number of robots. It is still difficult for students to obtain the source code for controlling the robot. Therefore, it is necessary to develop a technology to solve this problem. Summary of the invention

[0006] The present disclosure aims to solve the above-mentioned problems occurring in the prior art while maintaining the advantages achieved by the prior art unchanged.

[0007] One aspect of the present disclosure provides an object synchronization device for creating a virtual object corresponding to a source code in a virtual space in response to a compilation request based on a user's source code, and performing verification on the created virtual object. Another aspect of the present disclosure provides a related method.

[0008] Another aspect of the present disclosure provides an object synchronization device for performing synchronization between a verified virtual object and a real object in response to a control request for the verified virtual object. Another aspect of the present disclosure provides a related method.

[0009] Another aspect of the present disclosure provides an object synchronization device, which is used to track the state of the detected target object when the synchronized real object detects the target object, and to update the state of the detected object to perform synchronization between the updated target object and the child virtual object. Another aspect of the present disclosure provides a related method.

[0010] The technical problems to be solved by the present disclosure are not limited to the above problems. A person skilled in the art to which the present disclosure belongs should more clearly understand any other technical problems not mentioned in this document through the following description.

[0011] According to one aspect of the present disclosure, an object synchronization device may include a memory storing instructions executable by a computer and at least one processor accessing the memory and executing the instructions. The at least one processor may: create a virtual object corresponding to the source code in a virtual space in response to a compilation request based on the source code of a user; perform verification on the created virtual object; perform synchronization between the verified virtual object and the real object in response to a control request for the verified virtual object; and create a child virtual object corresponding to the detected target object in the virtual space when the synchronized real object detects the target object.

[0012] In an embodiment, at least one processor may provide a user with a development environment for a virtual object to be created in a virtual space, and may create a virtual object corresponding to a source code input into the development environment by the user.

[0013] In an embodiment, at least one processor may: when the number of users is multiple, provide an independent development environment and an independent virtual space for each of the multiple users; create a virtual object corresponding to the source code of a first user among the multiple users in the independent virtual space of the first user; and create a virtual object corresponding to the source code of a second user among the multiple users who is different from the first user in the independent virtual space of the second user.

[0014] In an embodiment, at least one processor may perform software verification for a plurality of call functions included in the source code, may perform verification for movement and action of a virtual object in a virtual space, and may perform verification for compatibility of hardware components. Compatibility may be associated with movement and action of a real object to be synchronized with the virtual object.

[0015] In an embodiment, at least one processor may perform verification for at least one of movement of the virtual object in a first direction, movement in a second direction different from the first direction, or an angle between the first direction and the second direction, or any combination thereof, in the virtual space.

[0016] In an embodiment, when the first verification is performed, at least one processor may perform a second verification different from the first verification including software verification, movement verification, and action verification. The second verification may include verification for compatibility of hardware components, which compatibility is associated with at least one of power consumption of a real object, an allocation of a central processing unit (CPU) and a memory, or an amount of movement corresponding to movement, or any combination thereof.

[0017] In an embodiment, at least one processor may: change a virtual object to a control mode in response to a user's control request; create a control command protocol capable of controlling a real object based on the user's control command for the virtual object; perform synchronization by applying the control command protocol to the real object; and provide a control approval notification for the control request to the user when synchronization is performed.

[0018] In an embodiment, at least one processor may set the control request for the second user to a waiting mode when there is a control request for the virtual object of the second user while the virtual object of the first user is changed to the control mode in response to the control request for the virtual object of the first user. The at least one processor may change the virtual object of the second user to the control mode after changing the virtual object of the first user to the control mode.

[0019] In an embodiment, the at least one processor may perform verification on the movement and action of the child virtual object based on the detected target object being a robot object different from the real object.

[0020] In an embodiment, at least one processor may track a state of the detected target object and may perform synchronization between the updated target object and the child virtual object based on the state of the detected object being updated. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings:

[0022] Figure 1 is a diagram showing an object synchronization device according to an embodiment of the present disclosure;

[0023] Figure 2 is a flowchart for describing an object synchronization method according to an embodiment of the present disclosure;

[0024] Figure 3 is a diagram illustrating a connection between an object synchronization device and a user's personal device according to an embodiment of the present disclosure;

[0025] Figure 4 is a diagram showing a connection between an object synchronization device and a real object according to an embodiment of the present disclosure;

[0026] Figure 5 is a diagram illustrating a method for performing synchronization between a virtual object and a real object in an object synchronization device according to an embodiment of the present disclosure;

[0027] Figure 6is a diagram illustrating a method for performing synchronization between a detected target object and a child virtual object in an object synchronization apparatus according to an embodiment of the present disclosure;

[0028] Figure 7 is a diagram illustrating a connection between an object synchronization device and a user's personal device according to an embodiment of the present disclosure;

[0029] Figure 8 is a diagram illustrating a connection between an object synchronization device and a plurality of user personal devices according to an embodiment of the present disclosure;

[0030] Fig. 9 is a diagram illustrating a method for verifying a virtual object and applying the verified virtual object to a real object in an object synchronization apparatus according to an embodiment of the present disclosure; and

[0031] Fig.10 A computing system related to an object synchronization method according to an embodiment of the present disclosure is shown.

[0032] Regarding the description of the drawings, the same or similar reference numerals may be used for the same or similar components. DETAILED DESCRIPTION

[0033] Hereinafter, some embodiments of the present disclosure are described in detail with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals will be used throughout to mark the same or equivalent components. In addition, detailed descriptions of well-known features and functions are omitted to avoid unnecessary confusion of the main points of the present disclosure. Hereinafter, various embodiments of the present disclosure may be described with reference to the accompanying drawings. However, it should be understood that this is not intended to limit the present disclosure to a specific implementation form. The present disclosure includes various modifications, equivalents and / or alternatives of the embodiments of the present disclosure.

[0034] As used in the present disclosure, terms such as "first", "second", "first", "second", A, B, (a), (b), etc. may be used to refer to various components without regard to order and / or priority, and to distinguish one component from another component, but not to limit the components. For example, a first user device and a second user device represent different user devices, regardless of order and / or priority. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component without departing from the scope of the present disclosure.

[0035] In the present disclosure, the expressions “have”, “may have”, “include” and “comprise”, or “may include” and “may include” indicate the existence of corresponding features (for example, components such as values, functions, operations or parts), but do not exclude the existence of additional features.

[0036] It should be understood that when a component (e.g., a first component) is referred to as being “(operably or communicatively) coupled” / “(operably or communicatively) coupled to” or “connected to” another component (e.g., a second component), the component may be directly coupled to / directly coupled to or directly connected to the other component, or an intermediate component (e.g., a third component) may exist. Conversely, when a component (e.g., a first component) is referred to as being “directly coupled” / “directly coupled to” or “directly connected to” another component (e.g., a second component), it may be understood that there is no intermediate component (e.g., a third component).

[0037] As used in the present disclosure, the expression “configured to” may be used interchangeably with, for example, the expressions “suitable to,” “capable of . . . ,” “designed to,” “suitable for,” “manufactured to,” or “capable of,” depending on the circumstances. When a component, device, element, etc. of the present disclosure is described as having a purpose or performing an operation, function, etc., the component, device, or element shall be considered herein as “configured to” satisfy the purpose or perform the operation or function.

[0038] When it comes to hardware, the term "configured to" means more than just "specially designed to". For example, "a processor configured to perform A, B, and C" may mean that a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) can perform the corresponding operations by running one or more software programs stored in a dedicated processor (e.g., an embedded processor) or a memory device for performing the corresponding operations. Terms in the singular form may include plural forms unless the context clearly indicates otherwise. All terms used herein, including technical or scientific terms, may have the same meanings commonly understood by those skilled in the art described in the present disclosure. It should be further understood that terms defined in dictionaries and commonly used should also be interpreted in accordance with the customs of the relevant art, rather than in an idealized or overly formalized meaning, unless explicitly defined in this manner in various embodiments of the present disclosure. In some cases, even if terms are defined in the specification, they should not be interpreted as excluding embodiments of the present disclosure.

[0039] In the present disclosure, expressions such as "A or B", "at least one of A or / and B", or "one or more of A or / and B", etc. may include any and all combinations of the relevant listed items. For example, the term "A or B", "at least one of A and B", or "at least one of A or B" may refer to all of the following cases: case (1) includes at least one A, case (2) includes at least one B, or case (3) includes at least one A and at least one B at the same time. In addition, when describing the embodiments of the present disclosure, expressions such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", "at least one of A, B or C", and "at least one of A, B or C, or any combination thereof" may include any one or all possible combinations of the items listed together in the corresponding one phrase. In particular, a phrase such as "at least one of A, B or C, or any combination thereof" may include "A", "B" or "C" or "AB" or "ABC" as a combination.

[0040] In the following, reference will be made to Figures 1 to 10 Embodiments of the present disclosure are described in detail.

[0041] Figure 1 is a diagram illustrating an object synchronization apparatus according to an embodiment of the present disclosure.

[0042] The object synchronization apparatus 100 according to the embodiment may include a processor 110 , a memory 120 storing instructions 122 , and an interface 130 .

[0043] The object synchronization apparatus 100 may refer to a real-time object synchronization apparatus that creates a virtual object corresponding to a user's source code and performs synchronization between the created virtual object and a real object 140 to control the real object 140 through a virtual environment. For example, the object synchronization apparatus 100 may create a virtual object corresponding to the source code in a virtual space in response to a compilation request based on the user's source code to control the real object 140 by controlling the virtual object created in a virtual environment (e.g., a virtual space).

[0044] The object synchronization device 100 may perform compilation for converting a text file (e.g., source code) describing a computer program in a human-readable programming language into a machine language that can be interpreted and run by an interpreter of the object synchronization device. For example, the object synchronization device 100 may create a virtual object expressed in a virtual space based on a machine language corresponding to a user's source code. In this article, the user's source code may include at least one function for executing a control function such as movement and action of a virtual object.

[0045] The object synchronization device 100 can provide a user with a scene in which a virtual object exists. For example, a scene can be composed of one or more frames representing a virtual object and a virtual space in which the virtual object exists and is created. Multiple frames can constitute a continuous image. The virtual space can represent a two-dimensional (2D) or three-dimensional (3D) space in which the virtual object exists. For the convenience of description in the specification, the virtual space is mainly described as a 3D space.

[0046] A virtual object may represent an object, fluid, gas, solid, rigid body, soft body, deformable object (e.g., an object including a rigid body and a soft body), etc., formed by a group of one or more particles. For example, the position, shape, size, state, etc. of each particle may be changed according to any motion model in the scene or the virtual space of each particle. In particular, as an example of a solid as a virtual object in this specification, the virtual object is mainly described as being able to be created in a virtual space by a function written by a user, which represents a function associated with controlling a robot.

[0047] The object synchronization device 100 can represent a target in the real world (e.g., a real object 140 such as a robot, hereinafter referred to as "real object 140") as a virtual object in a virtual space. For example, the object synchronization device 100 can create a virtual object corresponding to a source code in a virtual space, and can express and / or provide the created virtual object to the user through the virtual space. Although described below, the user can also control (e.g., make the robot move forward) the real object 140 through the virtual object.

[0048] The object synchronization device 100 can perform verification on the created virtual object. For example, before controlling the real object 140, the object synchronization device 100 can perform verification on whether to apply the virtual object or synchronize the virtual object with the real object 140. In detail, the verification may include software interoperability verification and hardware compatibility verification. The object synchronization device 100 can apply a function associated with the control of the robot written by the user (for example, a function included in the source code) to the virtual object to first verify the virtual object, thereby synchronizing the virtual object with the real object 140. For reference, the object synchronization device 100 can first perform software interoperability verification of the virtual object, and then can perform verification on hardware compatibility. This will be referred to below. Fig. 9 Describe the verification in detail.

[0049] The object synchronization device 100 may perform synchronization between the verified virtual object and the real object 140 in response to a control request for the verified virtual object. For example, the object synchronization device 100 may perform synchronization based on a control request, a control command, and a control command protocol. However, the synchronization process is not limited thereto. The synchronization between the virtual object and the real object 140 may be performed based on a protocol other than the above-mentioned synchronization process.

[0050] When the synchronized real object 140 detects the target object, the object synchronization device 100 can create a sub-virtual object corresponding to the detected target object in the virtual space. For example, unlike the process in which the object synchronization device 100 creates a virtual object corresponding to the user's source code, the object synchronization device 100 can create a sub-virtual object corresponding to the target object (for example, a robot object, a wall object, a stair object, a road object, a human object, etc. that are different from the real object 140). In particular, in the specification, the sub-virtual object is mainly described as an object detected by the virtual object. However, the sub-virtual object is not limited to this. For example, the sub-virtual object can represent a virtual object corresponding to the source code of a user different from the above-mentioned user. Therefore, even if the virtual object does not detect the sub-virtual object, the sub-virtual object can be created in the virtual space.

[0051] The object synchronization device 100 may create a sub-virtual object corresponding to the target object detected by the real object 140 to provide the user with interaction between virtual objects. In particular, the object synchronization device 100 may create a sub-virtual object so as to provide the user with a target object included in a space different from the space where the user actually exists (e.g., the space between the user's current position and a different position with a predetermined length from the user's current position) through the expression of the sub-virtual object.

[0052] The processor 110 may run software and may control at least one other component (e.g., hardware or software component) connected to the processor 110. In addition, the processor 110 may perform various data processing or calculations. For example, the processor 110 may store source code and control requests in the memory 120.

[0053] For reference, the processor 110 may perform all operations performed by the object synchronization apparatus 100. Therefore, for the convenience of description in the specification, the operations performed by the object synchronization apparatus 100 are mainly described as operations performed by the processor 110.

[0054] The memory 120 may temporarily and / or permanently store various data and / or information required to perform object synchronization.

[0055] The interface 130 may represent an interface capable of assisting in performing communication between the object synchronization apparatus 100 and the real object 140. In detail, the interface 130 may represent a shared boundary for exchanging information or signals between two different apparatuses (eg, the object synchronization apparatus 100 and the real object 140).

[0056] The interface 130 may provide the object synchronization apparatus 100 with a shared boundary for performing synchronization between the virtual object and the real object 140. In addition, the real object 140 may detect a target object, and may transmit information about the target object to the object synchronization apparatus 100 through the interface 130. Finally, the object synchronization apparatus 100 may create a child virtual object corresponding to the detected target object in the virtual space based on the information about the target object transmitted through the interface 130.

[0057] Figure 2 is a flowchart for describing an object synchronization method according to an embodiment of the present disclosure.

[0058] In S210, a device (eg, Figure 1 The object synchronization device 100 may create a virtual object corresponding to the source code in a virtual space in response to a compilation request based on the source code of a user.

[0059] The device can provide a development environment for the user. For example, the user can create source code through the development environment of the user's personal device (e.g., the user's personal computer (PC)). However, the performance of the user's personal device (e.g., the computing speed of the central processing unit (CPU), the network speed, etc.) may be different for each of the multiple users. In order to solve this difference, the device can provide the same development environment to the user.

[0060] In detail, the device can provide a user with a development environment for a virtual object, thereby generating a virtual object in response to a compilation request based on source code. The device can provide a virtual object to an interface (e.g., Figure 1 A development environment including the same performance and environment settings is provided to a user's personal device connected to the interface 130).

[0061] When the number of users is multiple, the device can provide an independent development environment and an independent virtual space for each of the multiple users. For example, the independent virtual space can be a space paired with an independent development environment provided to each of the multiple users, in which a virtual object is created or expressed. In detail, in the case where the number of users who wish to create a virtual object through the device is multiple, the device can provide an independent development environment and an independent virtual space for each of the multiple users, thereby generating virtual objects for multiple users in parallel in response to compilation requests for multiple source codes.

[0062] The device may create a virtual object corresponding to a source code of a first user among multiple users in an independent virtual space of the first user, and may create a virtual object corresponding to a source code of a second user among multiple users in an independent virtual space of the second user different from the first user. In this document, the first user and the second user may refer to different users. However, the source code of the first user and the source code of the second user are mainly described as, but not limited to, different source codes.

[0063] In addition, the number of users being multiple is not limited to the above case. For example, based on the number of users being multiple, the device can provide an independent development environment and a subordinate virtual space for each of the multiple users. For example, the subordinate virtual space can be a space bound to an independent development environment provided to each of the multiple users, in which multiple virtual objects are created or expressed. Therefore, the device can provide users with a virtual space in which multiple users can interact. For example, the device can create a virtual object corresponding to the source code of a first user and a virtual object corresponding to the source code of a second user in one virtual space.

[0064] In S220 , the device may perform verification on the created virtual object.

[0065] The device may perform a first verification and a second verification on the created virtual object. For example, the first verification may refer to a software interoperability verification on the created virtual object. In addition, the second verification may refer to a hardware compatibility verification on the created virtual object.

[0066] The first verification may include verification of a plurality of call functions included in the source code and verification of movement and action of the virtual object in the virtual space.

[0067] For example, when source code written by a user is compiled by software integration, software verification may be performed on the processor (e.g., Figure 1 In addition, the source code may include a plurality of calling functions, and in detail, may include a function in which a plurality of functions related to robot movement and action are written in a text form according to a predetermined form. The device may run the software for executing the functions related to the above-mentioned robot movement and action by compiling the source code. In this article, the device may perform verification on the grammatical elements and semantic elements of the text in which the functions related to the above-mentioned robot movement and action are written, thereby performing software verification.

[0068] The device may perform verification for the movement and action of the virtual object in the virtual space. For example, the device may move (e.g., control) the created virtual object in a first direction (e.g., the positive direction of the x-axis in the 3D space represented by the x-axis, the y-axis, and the z-axis) in the virtual space (e.g., the space on the 3D plane). Thereafter, the device may move the virtual object in a second direction different from the first direction (e.g., the positive direction of the y-axis in the 3D space represented by the x-axis, the y-axis, and the z-axis).

[0069] In this article, the device can measure the angle between the first direction and the second direction to perform verification of the movement and action of the virtual object. For example, when the angle between the first direction and the second direction is different from a predetermined angle (e.g., 90 degrees), the device can provide the user with information that the verification of the movement and action of the virtual object has failed.

[0070] However, the verification of the movement and action of the virtual object is not limited thereto. For example, the device may perform at least one of the movement speed of the virtual object in a specified direction, the movement of the wheels, or the movement of the components included in the virtual object, or any combination thereof, as verification of the movement and action of the virtual object.

[0071] The second verification may refer to verification of compatibility of the hardware component with the real object to be synchronized with the virtual object (eg, Figure 1 The second verification may include verification of compatibility of hardware components, which is associated with at least one of power consumption of the real object, allocation of a central processing unit (CPU) and memory, or movement amount corresponding to the movement, or any combination thereof.

[0072] In S230 , the apparatus may perform synchronization between the verified virtual object and the real object in response to the control request for the verified virtual object.

[0073] The device can change the virtual object to a control mode in response to a user's control request for synchronization between the virtual object and the real object. For example, the mode of the virtual object can include a waiting mode, a control mode, and an error mode.

[0074] The waiting mode may include a waiting state in which the virtual object waits in the virtual space from the time point when the virtual object is created to the time point when the user's control request is received. In particular, the waiting mode may include a mode of maintaining the waiting state of the virtual object's control request to the user. The waiting state and waiting mode of the virtual object's control request to the user will be described below.

[0075] The control mode may include a state in which the virtual object is controlled in the virtual space from a time point when a control request of the user is received to a time point when the control of the virtual object ends.

[0076] The error mode may include a state in which the virtual object cannot be controlled due to an error of the virtual object from a time point when the virtual object is created to a time point when the control of the virtual object ends.

[0077] According to the user's control command for the virtual object, the device can create a control command protocol capable of controlling the real object. The control command protocol can include instructions related to the specified format and protocol to send the user's control command to the real object and execute the control command on the real object side.

[0078] The device can apply the control command protocol to the real object, thereby performing synchronization between the virtual object and the real object. For example, the device can synchronize the state of the virtual object with the state of the real object through synchronization between the virtual object and the real object. In addition, the device can limit access to shared data between the virtual object and the real object through synchronization between the virtual object and the real object. In addition, the device can prevent deadlock of the virtual object and the real object through synchronization between the virtual object and the real object.

[0079] When synchronization is performed, the device may provide the user with a control approval notification for the control request. When the control approval notification is received from the device, the user may control the virtual object to control the real object synchronized with the virtual object.

[0080] The device may perform synchronization between multiple virtual objects and real objects in response to control requests for multiple virtual objects. For example, the device may change the first user's virtual object to a control mode in response to a control request for the first user's virtual object. While changing the first user's virtual object to the control mode, the device may receive a control request for a second user's virtual object. When receiving a control request from a second user while changing the first user's virtual object to the control mode, the device may set the second user's control request to a waiting mode.

[0081] The device can separate and synchronize the control request of the first user and the control request of the second user for a real object. For example, after changing the virtual object of the first user to the control mode, the device can change the virtual object of the second user to the control mode. However, the synchronization method for the control requests of multiple users is not limited to this. For example, after the control mode of the virtual object of the first user ends, the device can change the waiting mode of the virtual object of the second user to the control mode. In this article, the virtual object of the first user and the virtual object of the second user can be the same virtual object.

[0082] In S240 , when the synchronized real object detects the target object, the device may create a child virtual object corresponding to the detected target object in the virtual space.

[0083] When the detected target object is a robot object (hereinafter referred to as a "robot") different from the real object, the device can perform verification for the movement and action of the child virtual object. For example, when the target object is a robot, the device can create a child virtual object (e.g., a robot in the virtual space) corresponding to the target object (e.g., a robot in the real world space). The device can express and create the movement and action of the target object in the virtual space by verifying the movement and action of the created child virtual object.

[0084] However, the verification of the child virtual object is not limited thereto. For example, the verification of the child virtual object may include verification of interaction with a virtual space previously created in the virtual space, and verification of movement and action. In detail, the verification of interaction may include verification of the distance between the child virtual object and the virtual object, and verification of the movement line between the child virtual object and the virtual object.

[0085] Figure 3 is a diagram illustrating a connection between an object synchronization device and a user's personal device according to an embodiment of the present disclosure.

[0086] The apparatus 300 may include access to a memory (eg, Figure 1 memory 120) and executes instructions (e.g., Figure 1 For example, the apparatus 300 may include a virtual space creation processor 320, a development environment provision processor 330, a virtual object creation processor 340, and a virtual object verification processor 350.

[0087] For example, the device 300 may be connected to the user's personal device 360 ​​through the interface 310. The device 300 may enable the user to access the virtual space creation processor 320 and access the development environment providing processor 330 through the interface 310.

[0088] The virtual space creation processor 320 may create a robot component 322 and an environment component 324. For example, the robot component 322 may include a virtual object, and may include a virtual object corresponding to a user's source code and a sub-virtual object corresponding to a detected target object.

[0089] The environment component 324 may include a human object, a wall object, a non-device object, and a device object. The human object, the wall object, the non-device object, and the device object may correspond to a person, a wall, a non-device, and a device, respectively.

[0090] The device 300 can express and create the interaction between the robot component 322 and the environment component 324 through the virtual space creation processor 320. In detail, the device 300 can express the interaction between the robot component 322 and the environment component 324 in the virtual space created by the virtual space creation processor 320 through the interface 310 using the user's personal device 360.

[0091] The interaction may include a relationship between the robot component 322 and the environment component 324. For example, when the robot component 322 hits a wall object included in the environment component 324 in the virtual space, the interaction may include an action in which the state of the robot component 322 is changed by the environment component 324.

[0092] The development environment providing processor 330 can provide a modular virtual environment (env.) and a compiled source code library. For example, the modular virtual environment (env.) can refer to a module for creating and managing a virtual environment, and the compiled source code library can include uncompiled source code and source code compiled by a user's compilation request.

[0093] The virtual object creation processor 340 can create a virtual object based on the source code included in the compiled source code library. For example, the virtual object creation processor 340 can create a virtual object and can deploy the created virtual object. In particular, the virtual object creation processor 340 can create and deploy objects included in the robot component 322 and objects included in the environment component 324 in the virtual space.

[0094] The virtual object verification processor 350 may perform verification for the virtual object created by the virtual object creation processor 340. For example, the virtual object verification processor 350 may perform software (SW) integration for a plurality of call functions included in the source code for the virtual object, verification for movement and action of the virtual object in the virtual space, and verification for hardware (HW) compatibility of hardware components associated with movement and action of a real object to be synchronized with the virtual object.

[0095] The user personal device 360 ​​may include a coding module, a robot control module, and a platform interface. The coding module may be a module that compiles or operates the source code of the virtual object to be created by the virtual space creation processor 320. The robot control module may be a module for controlling a real object synchronized with the virtual object. The platform interface may be an interface for communication between the device 300 and the user personal device 360.

[0096] Figure 4 is a diagram illustrating a connection between an object synchronization apparatus and a real object according to an embodiment of the present disclosure.

[0097] The apparatus 400 may include access to a memory (eg, Figure 1 memory 120) and executes instructions (e.g., Figure 1 For example, the device 400 may include a virtual space creation processor 420, a synchronization processor 430, a virtual object creation processor 440, and a virtual object verification processor 450.

[0098] For reference, the virtual space creation processor 420 may correspond to a virtual space creation processor (eg, Figure 3 The interface 470 may correspond to an interface (eg, Figure 3 The virtual object creation processor 440 may correspond to a virtual object creation processor (eg, Figure 3 The virtual object creation processor 340). The virtual object verification processor 450 may correspond to a virtual object verification processor (e.g., Figure 3 Virtual object verification processor 350).

[0099] The real object 480 may include a server and a component management device. The server of the real object 480 may refer to a server capable of performing communication with the device 400. The component management device of the real object 480 may refer to a module that controls a component (e.g., a robot arm) in the real object 480. The device 400 may access the server and the component management device of the real object 480 through the interface 470.

[0100] The synchronization processor 430 may perform robot monitoring, robot synchronization, and object tracking. For example, robot monitoring may refer to monitoring the state of the real object 480. Robot synchronization may refer to synchronization between the robot component of the virtual space created by the virtual space creation processor 420 and the real object 480 after monitoring the state of the real object 480. The device 400 may store the information generated by the synchronization processor 430 (e.g., the state of the real object 480) in the storage device 460. For reference, the following will be referred to Figure 6 Object tracking is described.

[0101] Figure 5 is a diagram illustrating a method for performing synchronization between a virtual object and a real object in an object synchronization apparatus according to an embodiment of the present disclosure.

[0102] The user personal device 510 may send a request for encoding and compiling the source code to the object synchronization device 500. In detail, the user personal device 510 may send a request for encoding and compiling the source code to the development environment providing processor 520. The development environment providing processor 520 may request the virtual object creation processor 530 to create a virtual object in response to the user's request. The virtual space creation processor 540 may create a virtual space in response to the request to create the virtual object, and may express the virtual object to be deployed in the created virtual space.

[0103] When a virtual object is created, the development environment providing processor 520 may request the virtual object verification processor 550 to verify the created object. The virtual object verification processor 550 may perform verification for the virtual object created in the virtual space in response to the request to verify the created object. Afterwards, the virtual object verification processor 550 may send information about the completion of compilation and the completion of virtual object creation to the user personal device 510.

[0104] The object synchronization device 500 may perform synchronization between the virtual object and the real object 570 in response to a virtual object control request of the user personal device 510. For example, the synchronization processor 560 may send a synchronization progress request to the real object 570. In detail, the synchronization processor 560 may change the virtual object to a control mode, may create a control command protocol capable of controlling the real object 570, and may apply the control command protocol to the real object 570.

[0105] When synchronization between the virtual object and the real object 570 is performed, the object synchronization device 500 may provide a synchronization notification and a control approval notification for a control request (eg, a synchronization request) to the user personal device 510 .

[0106] Figure 6 is a diagram illustrating a method for performing synchronization between a detected target object and a child virtual object in an object synchronization apparatus according to an embodiment of the present disclosure.

[0107] When real objects (e.g. Figure 5 When the object synchronization device 600 detects the target object 610 (real object 570), the object synchronization device 600 may transmit an object discovery notification to the synchronization processor 620. Thereafter, the synchronization processor 620 may store information about the target object 610 in the memory 630, and may request the virtual object creation processor 640 to create a virtual object (e.g., a child virtual object) related to the target object 620.

[0108] The virtual object creation processor 640 may create a child virtual object corresponding to the target object 610 based on the above request, and may apply the child virtual object to the virtual space. Afterwards, the object synchronization device 600 may request the virtual object verification processor 650 to verify the child virtual object 610 related to the target object based on the information about the target object 610 stored in the memory 630.

[0109] The virtual object verification processor 650 may perform verification on the created child virtual object in the virtual space created by the virtual space creation processor 660. In this article, the verification on the child virtual object may be the first verification, including software verification, movement verification, and action verification. When the verification of the child virtual object related to the target object 610 is completed, the object synchronization device 600 may send a target object application notification to the user (e.g., Figure 5 of the user's personal device 510).

[0110] The object synchronization device 600 can track the state of the detected target object 610. For example, the object synchronization device 600 can track at least one of the position state or the hardware state of the target object 610 or any combination thereof. In this article, when the state of the target object 610 changes, the object synchronization device 600 can use the tracking result to update the state of the target object 610. When the state of the detected target object 610 is updated, the object synchronization device 600 can perform synchronization between the updated target object and the child virtual object. In detail, the synchronization processor 620 can perform synchronization between the updated target object and the child virtual object.

[0111] Figure 7 is a diagram illustrating a connection between an object synchronization device and a user's personal device according to an embodiment of the present disclosure.

[0112] In S710, a device (eg, Figure 1 The object synchronization device 100 of the present invention can provide a virtual environment to the user's personal device, and can perform the compilation of the source code encoded in the provided virtual environment. For example, the device can provide the user with a development environment that is the same as the development environment included in the user's personal device.

[0113] In S720, the device may create a virtual object corresponding to the source code input by the user into the development environment. The device may perform verification for the created virtual object. In this article, the verification may include a first verification and a second verification, the first verification including software verification, movement verification and action verification, and the second verification including verification for compatibility of hardware components, which is associated with the power consumption of the real object, the allocation of the CPU and the memory, and the movement amount corresponding to the movement.

[0114] In S730, the device may perform synchronization between the verified virtual object and the real object in response to the control request for the verified virtual object. For example, the device may change the virtual object to a control mode in response to the control request of the user, may create a control command protocol capable of controlling the real object, and may apply the control command protocol to the real object, thereby performing synchronization. When performing synchronization, the device may provide the user with a control approval notification for the control request.

[0115] In S740, the device may apply the user's control command to the real object. For example, the user may transmit a control request for a command to move the real object to the device. The device may apply the user's control request to the virtual object, thereby providing the user with the virtual object to which the control request is applied. At the same time, the device may perform synchronization between the virtual object to which the control request is applied and the real object. When the device performs synchronization, the real object may execute the user's control request.

[0116] Figure 8 is a diagram illustrating connection between an object synchronization device and a plurality of user personal devices according to an embodiment of the present disclosure.

[0117] In S810, a device (eg, Figure 1 The object synchronization device 100 can be multiple based on the number of users, providing an independent development environment and an independent virtual space for each of the multiple users. For example, the device can provide different users (e.g., Figure 7 , hereinafter referred to as the "first user") (for example, Figure 8 A user (shown as “User 2” in FIG. 1 , hereinafter referred to as “second user”) is provided with a development environment different from that of the first user.

[0118] In S820, the device may create a virtual object corresponding to the source code of a first user among the multiple users in an independent virtual space, and may create a virtual object corresponding to the source code of a second user among the multiple users in an independent virtual space. Figure 8 A description is given, mainly describing an example of the second user.

[0119] In S830, when there is a control request for the virtual object of the second user while the virtual object of the first user is changed to the control mode, the device can set the control request of the second user to the waiting mode. For example, the device can set the control request of the second user to the waiting mode and provide a waiting round for the real object synchronization.

[0120] In S840, after changing the virtual object of the first user to the control mode, the device may change the virtual object of the second user to the control mode. For example, the device may apply the control command of the second user to the real object. The device may apply the control request of the second user to the virtual object corresponding to the source code of the second user, thereby providing the virtual object to which the control request is applied to the second user. At the same time, the device may perform synchronization between the virtual object to which the control request of the second user is applied and the real object. When synchronization is performed by the device, the real object may execute the control request of the second user.

[0121] Fig. 9 is a diagram illustrating a method for verifying a virtual object and applying the verified virtual object to a real object in an object synchronization apparatus according to an embodiment of the present disclosure.

[0122] In S910, a device (eg, Figure 1 The object synchronization device 100) can be encoded and compiled in a virtual environment by a user's personal device. In this article, the development environment included in the user's personal device can be the same as the development environment of the device.

[0123] In S920, the device may perform a first verification on the created virtual object. For example, the first verification may refer to verification of software intercommunication of the created virtual object. In detail, the first verification may include software verification of multiple call functions included in the source code and verification of movement and action of the virtual object in the virtual space.

[0124] In S930, when the first verification is performed on the virtual object, the device may perform a second verification. For example, the second verification may include verification of compatibility of a hardware component, the compatibility of the hardware component being associated with at least one of power consumption of the real object, allocation of CPU and memory, movement amount corresponding to movement, or the number of wheels of the virtual object and the number of wheels of the real object, or any combination thereof.

[0125] In S940, when the verification (e.g., the first verification and the second verification) for the virtual object is completed, the device may transmit, create and verify the virtual object completion information to the user. When the creation and verification completion information of the virtual object is received from the device, the user may control the virtual object. In detail, the user may transmit a control request for the virtual object to the device, thereby controlling the virtual object changed to the control mode.

[0126] Fig.10 A computing system related to an object synchronization method according to an embodiment of the present disclosure is shown.

[0127] Reference Fig.10The computing system 1000 for the object synchronization method may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage device 1600 , and a network interface 1700 interconnected via a bus 1200 .

[0128] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and / or the storage device 1600. The memory 1300 and the storage device 1600 may include various types of volatile storage media or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) 1310 and a RAM (Random Access Memory) 1320.

[0129] Therefore, the operations of the methods or algorithms described in conjunction with the embodiments disclosed in the specification may be directly implemented using hardware modules, software modules, or a combination thereof executed by the processor 1100. The software modules may reside on a storage medium (i.e., the memory 1300 and / or the storage device 1600) such as RAM, flash memory, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, and compact disk ROMs (CD-ROMs).

[0130] The storage medium may be coupled to the processor 1100. The processor 1100 may read information from the storage medium and may write information to the storage medium. Alternatively, the storage medium may be integrated with the processor 1100. The processor and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside in a user terminal. In another case, the processor and the storage medium may reside in a user terminal as separate components.

[0131] In the above, although the present disclosure is described with reference to the embodiments and the accompanying drawings, the present disclosure is not limited thereto. Various modifications and changes can be made to the embodiments of the present disclosure by ordinary technicians in the field to which the present disclosure belongs without departing from the spirit and scope of the present disclosure claimed in the claims.

[0132] The above-mentioned embodiments can be implemented with hardware components, software components and / or their combinations. For example, the devices, methods and components described in the disclosed embodiments can be implemented using general-purpose computers or special-purpose computers such as processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPGAs), programmable logic units (PLUs), microprocessors, or any device that can execute instructions and respond. The processing unit can execute an operating system (OS) or execute software applications running on the OS. In addition, the processing unit can access, store, operate, process and generate data in response to the operation of the software. It should be understood by those of ordinary skill in the art that, although a single processing unit can be illustrated for ease of understanding, the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or a processor and a controller. In addition, the processing unit may have different processing configurations, such as parallel processors.

[0133] Software may include computer programs, codes, instructions, or a combination of one or more of them, and may configure a processing unit to operate in a desired manner, or may issue instructions to a processing unit independently or collectively. Software and / or data may be implemented permanently or temporarily in any type of machine, component, physical device, virtual device, computer storage medium or unit, or in a transmitted signal wave so that instructions or data are interpreted by a processing unit or provided to a processing unit. Software may be distributed throughout a computer system connected by a network and may be stored or run in a decentralized manner. Software and data may be recorded in a computer-readable storage medium.

[0134] The method according to the embodiment can be implemented in the form of program instructions, which can be run by various computer devices and can be recorded in a computer-readable medium. The computer-readable medium can include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded on the medium can be specially designed and configured as an example, or known and available to technicians in the field of computer software. Examples of computer-readable media include: magnetic media such as hard disks, floppy disks, and tapes; optical media such as compact disk read-only memory (CD-ROM) disks and digital versatile disks (DVDs); magneto-optical media such as floppy disks; and hardware devices such as read-only memories (ROMs), random access memories (RAMs), flash memories, etc. that are specially configured to store and execute program instructions. Program instructions include both machine codes generated by compilers, for example, and high-level codes that can be run by computers using interpreters.

[0135] The aforementioned hardware devices may be configured to operate as one or more software modules for executing the disclosed embodiments, and vice versa.

[0136] Although the disclosed embodiments are described with reference to various drawings, it is obvious to those skilled in the art that the disclosed embodiments may be variously changed or modified based on the above description. For example, even if the above processes and methods are performed in a different order from that described above, and / or the above components (such as systems, structures, devices or circuits) are combined or connected in a different form and mode from that described above, or replaced or substituted with other components or equivalent components, sufficient effects can be achieved.

[0137] Effects of the object synchronization device and method thereof according to an embodiment of the present disclosure are described.

[0138] According to at least one embodiment of the present disclosure, the object synchronization apparatus can simplify the process of a user responding to a compilation request based on the user's source code, applying the source code to a real object using a cable and performing testing by verifying a virtual object corresponding to the source code.

[0139] In addition, according to at least one disclosed embodiment of the present disclosure, the object synchronization device can respond to a control request for a virtual object, and before performing synchronization between a verified virtual object and a real object, enable multiple users to participate in robot coding education using the metaverse space through a waiting mode and a control mode.

[0140] In addition, according to at least one disclosed embodiment of the present disclosure, the object synchronization device can update the state of a target object detected in a virtual space, and can enable a user to visualize the space where a real object exists through the virtual space through synchronization between the updated target object and a sub-virtual object without directly identifying the space.

[0141] In addition, various effects directly or indirectly determined by the present disclosure can be provided.

[0142] Accordingly, other implementations, other examples, and equivalents of the claims are within the scope of the claims.

[0143] Therefore, the embodiments of the present disclosure are not intended to limit the technical spirit of the present disclosure, but are provided for illustrative purposes only. The scope of the present disclosure should be interpreted based on the appended claims, and all technical ideas within the scope equivalent to the claims should be included in the scope of the present disclosure.

Claims

1. An object synchronization device, comprising: Memory, which stores instructions that can be executed by the computer; as well as at least one processor, accessing the memory and executing the instructions, Wherein, the at least one processor: In response to a compilation request based on a user's source code, creating a virtual object corresponding to the source code in a virtual space, Perform validation on the created virtual object, In response to a control request for the verified virtual object, performing synchronization between the verified virtual object and a real object, and When the synchronized real object detects the target object, a child virtual object corresponding to the detected target object is created in the virtual space.

2. The object synchronization device according to claim 1, wherein: The at least one processor provides the user with a development environment for a virtual object to be created in the virtual space, and creates a virtual object corresponding to a source code input into the development environment by the user.

3. The object synchronization device according to claim 2, wherein: The at least one processor: When there are multiple users, providing an independent development environment and an independent virtual space for each of the multiple users; creating a virtual object corresponding to a source code of a first user among the plurality of users in an independent virtual space of the first user; and A virtual object corresponding to a source code of a second user different from the first user among the plurality of users is created in an independent virtual space of the second user.

4. The object synchronization device according to claim 1, wherein: The at least one processor: Performing software verification on a plurality of calling functions included in the source code; performing a first verification on the movement and action of the virtual object in the virtual space; and A second verification is performed for compatibility of hardware components associated with movement and action of the real object to be synchronized with the virtual object.

5. The object synchronization device according to claim 4, wherein: The at least one processor performs verification on at least one of movement of the virtual object in the virtual space in a first direction, movement in a second direction different from the first direction, or an angle between the first direction and the second direction, or any combination thereof.

6. The object synchronization device according to claim 4, wherein: When the first verification is performed, the at least one processor performs the second verification different from the first verification, the first verification including the software verification, the verification for the movement, and the verification for the action; and The second verification includes verification of compatibility of hardware components, the compatibility being associated with at least one of power consumption of the real object, allocation of a central processing unit (CPU) and memory, or movement amount corresponding to the movement, or any combination thereof.

7. The object synchronization device according to claim 1, wherein: The at least one processor: In response to a control request from the user, changing the virtual object to a control mode; Creating a control command protocol capable of controlling the real object according to the user's control command for the virtual object; performing said synchronization by applying said control command protocol to said real object; and When the synchronization is performed, a control approval notification for the control request is provided to the user.

8. The object synchronization device according to claim 3, wherein: The at least one processor: when there is a control request for the virtual object of the second user while the virtual object of the first user is changed to a control mode in response to a control request for the virtual object of the first user, setting the control request of the second user to a waiting mode; and After the first user's virtual object is changed to the control mode, the second user's virtual object is changed to the control mode.

9. The object synchronization device according to claim 1, wherein: The at least one processor performs verification on the movement and action of the child virtual object based on the detected target object being a robot object different from the real object.

10. The object synchronization device according to claim 1, wherein: The at least one processor tracks the state of the detected target object and performs synchronization between the updated target object and the child virtual object based on the state of the detected target object being updated.

11. An object synchronization method, comprising: In response to a compilation request based on the source code of the user, creating a virtual object corresponding to the source code in the virtual space; Perform validation on the created virtual object; In response to a control request for the verified virtual object, performing synchronization between the verified virtual object and a real object; as well as When the synchronized real object detects the target object, a child virtual object corresponding to the detected target object is created in the virtual space.

12. The object synchronization method according to claim 11, wherein creating the virtual object comprises: providing the user with a development environment for virtual objects to be created in the virtual space; as well as A virtual object is created corresponding to source code input into the development environment by the user.

13. The object synchronization method according to claim 12, further comprising: When there are multiple users, providing an independent development environment and an independent virtual space for each of the multiple users; creating, in an independent virtual space of a first user among the plurality of users, a first virtual object corresponding to a source code of the first user; as well as A second virtual object corresponding to a source code of a second user different from the first user among the plurality of users is created in an independent virtual space of the second user.

14. The object synchronization method according to claim 11, wherein performing verification comprises: Performing software verification on a plurality of calling functions included in the source code; performing verification on the movement and action of the virtual object in the virtual space; as well as Verification is performed for compatibility of hardware components associated with movement and action of the real object to be synchronized with the virtual object.

15. The object synchronization method according to claim 14, wherein performing verification on the movement and the action comprises: Verification is performed on at least one of movement of the virtual object in the virtual space in a first direction, movement in a second direction different from the first direction, or an angle between the first direction and the second direction, or any combination thereof.

16. The object synchronization method according to claim 14, wherein performing verification on compatibility of the hardware components comprises: When the first verification is performed, a second verification different from the first verification is performed, the first verification including the software verification, the verification for the movement, and the verification for the action, and The second verification includes verification of compatibility of the hardware components, the compatibility being associated with at least one of power consumption of the real object, allocation of a central processing unit (CPU) and memory, or movement amount corresponding to the movement, or any combination thereof.

17. The object synchronization method according to claim 11, wherein performing the synchronization comprises: In response to a control request from the user, changing the virtual object to a control mode; Creating a control command protocol capable of controlling the real object according to the control command for the virtual object of the user; performing said synchronization by applying said control command protocol to said real object; as well as When the synchronization is performed, a control approval notification for the control request is provided to the user.

18. The object synchronization method according to claim 13, wherein: Changing the virtual object to a control mode comprises: When there is a control request for the virtual object of the second user while the virtual object of the first user is changed to the control mode in response to the control request for the virtual object of the first user, setting the control request of the second user to a waiting mode; and After the first user's virtual object is changed to the control mode, the second user's virtual object is changed to the control mode.

19. The object synchronization method according to claim 11, further comprising: Based on the detected target object being a robot object different from a real object, verification is performed on the movement and action of the child virtual object.

20. The object synchronization method according to claim 11, further comprising: Tracking the status of the detected target object; as well as Based on the detected state of the target object being updated, synchronization between the updated target object and the child virtual object is performed.

Images