OBJECT SYNCHRONIZATION DEVICE AND METHOD

Abstract

An object synchronization device comprises a memory storing computer-executable instructions and at least one processor accessing the memory and executing the instructions. The at least one processor: generates a virtual object corresponding to a user's source code in a virtual space in response to a compilation request based on the source code; performs verification for the generated virtual object; performs synchronization between the verified virtual object and a real object in response to a control request for the verified virtual object; and generates a subvirtual object corresponding to a recognized target object in the virtual space when the synchronized real object recognizes the target object.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an object synchronization apparatus and method therefor, and more particularly relates to technologies that enable a plurality of students to participate in robot coding training simultaneously using a metaverse space. BACKGROUND

[0002] Coding is the process of creating code, which is the language of a program to accomplish a specific purpose. Since coding promotes creativity, coding education has increased in elementary and secondary schools. Furthermore, since coding education has recently been identified as essential education in elementary and secondary schools, coding education for students has increased.

[0003] However, when applying a source code for controlling a robot to the robot, the number of students should be consistent with the number of robots. This still makes it difficult for students to obtain a source code for controlling a robot. It is necessary to develop a technology that solves this problem. SUMMARY

[0004] The present disclosure has been made to solve the above-mentioned problems encountered in the prior art while maintaining advantages achieved by the prior art.

[0005] One aspect of the present disclosure provides an object synchronization device for generating a virtual object corresponding to a user's source code in a virtual space in response to a compilation request based on the source code and performing verification for the generated virtual object. Another aspect of the present disclosure provides a method therefor.

[0006] Another aspect of the present disclosure provides an object synchronization device for performing synchronization between a verified virtual object and a real object in response to a control request for the verified virtual object. Another aspect of the present disclosure provides a method therefor.

[0007] Another aspect of the present disclosure provides an object synchronization device for tracking the state of a detected target object when the synchronized real object detects the target object, and updating the state of the detected target object to perform synchronization between the updated target object and a sub-virtual object. Another aspect of the present disclosure provides a method therefor.

[0008] The technical problems to be solved by the present disclosure are not limited to the aforementioned problems. Any other technical problems not mentioned herein should be more clearly understood by those skilled in the art to whom this disclosure pertains from the following description.

[0009] According to one aspect of the present disclosure, an object synchronization device may include a memory storing computer-executable instructions and at least one processor accessing the memory and executing the instructions. The at least one processor may: generate a virtual object corresponding to a user's source code in a virtual space in response to a compilation request based on the source code; perform verification for the generated virtual object; perform synchronization between the verified virtual object and a real object in response to a control request for the verified virtual object; and generate a sub-virtual object corresponding to a recognized target object in the virtual space when the synchronized real object recognizes the target object.

[0010] In one embodiment, the at least one processor may provide the user with a development environment for a virtual object to be created in the virtual space and may create the virtual object according to the source code entered by the user into the development environment.

[0011] In one embodiment, the at least one processor may: provide each of a plurality of users with an independent development environment and an independent virtual space when the user is a plurality of people; generate a virtual object corresponding to source code of a first user among the plurality of users in an independent virtual space of the first user; and generate a virtual object corresponding to source code of a second user different from the first user among the plurality of users in an independent virtual space of the second user.

[0012] In one embodiment, the at least one processor may perform software verification for a plurality of call functions included in the source code, may perform verification for movement and action of the virtual object in the virtual space, and may perform verification for the compatibility of hardware components. The compatibility may be related to a movement and action of the real object that is to be synchronized with the virtual object.

[0013] In one embodiment, the at least one processor may perform verification for at least one of movement in a first direction of the virtual object in the virtual space, movement in a second direction different from the first direction, or an angle between the first direction and the second direction; or any combination thereof.

[0014] In one embodiment, the at least one processor may perform a second verification distinct from a first verification, including software verification, movement verification, and action verification, when performing the first verification. The second verification may include verifying the compatibility of hardware components, wherein the compatibility is associated with at least one of a power consumption of the real object, allocated amounts of a central processing unit (CPU) and memory, or a movement amount corresponding to the movement, or any combination thereof.

[0015] In one embodiment, the at least one processor may: change the virtual object to a control mode in response to a user control request; generate a control command protocol capable of controlling the real object in response to a user control command for the virtual object; perform the synchronization by applying the control command protocol to the real object; and provide the user with a control approval notification for the control request when the synchronization is performed.

[0016] In one embodiment, the at least one processor may place a control request from the second user into a wait mode when a control request for a virtual object of the second user is present while a virtual object of the first user is being changed to a control mode in response to a control request for the virtual object of the first user. The at least one processor may change the virtual object of the second user to the control mode after the virtual object of the first user is changed to the control mode.

[0017] In one embodiment, the at least one processor may perform verification for a movement and an action of the sub-virtual object based on the detected target object being a robot object that is different from the real object.

[0018] In one embodiment, the at least one processor may track a state of the detected target object and perform synchronization between an updated target object and the sub-virtual object based on the state of the detected target object being updated. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present disclosure should become more apparent from the following detailed description taken in conjunction with the accompanying drawings: Fig. 1 is a drawing showing an object synchronization device according to an embodiment of the present disclosure; Fig. 2 is a flowchart for describing an object synchronization method according to an embodiment of the present disclosure; Fig. 3 is a drawing showing a connection between an object synchronization device and a personal user device according to an embodiment of the present disclosure; Fig. 4 is a drawing showing a connection between an object synchronization device and a real object according to an embodiment of the present disclosure; Fig. 5 is a drawing showing a method of performing synchronization between a virtual object and a real object in an object synchronization device according to an embodiment of the present disclosure; Fig. 6 is a drawing showing a method of performing synchronization between a detected target object and a sub-virtual object in an object synchronization device according to an embodiment of the present disclosure; Fig. 7 is a drawing showing a connection between an object synchronization device and a personal user device according to an embodiment of the present disclosure; Fig. 8 is a drawing showing a connection between an object synchronization device and a plurality of personal user devices according to an embodiment of the present disclosure; Fig. 9 is a drawing showing a method for verifying a virtual object and applying the verified virtual object to a real object in an object synchronization device according to an embodiment of the present disclosure; and Fig. 10 shows a computer system via an object synchronization method according to an embodiment of the present disclosure.

[0020] With regard to the description of the drawings, the same or similar designations may be used for the same or similar components. DETAILED DESCRIPTION

[0021] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the drawings. Throughout the drawings, the same reference numerals are used to designate the same or equivalent components. Furthermore, a detailed description of well-known features or functions has been omitted in order not to unnecessarily obscure the gist of the present disclosure. Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that this is not intended to limit the present disclosure to particular embodiments. The present disclosure includes various modifications, equivalents, and / or alternatives of embodiments of the present disclosure.

[0022] The terms used in the present disclosure, such as "first," "second," "1st," "2nd," A, B, (a), (b), or the like, may be used to refer to various components, regardless of order and / or priority, and to distinguish one component from another component, but do not limit the components. For example, a first user device and a second user device refer to different user devices, regardless of order and / or priority. For example, without departing from the scope of the present disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

[0023] In the present disclosure, the terms "have," "may have," "comprise," and "have," or "may include" and "may have" indicate the presence of corresponding features (e.g., components such as numerical values, functions, operations, or parts), but do not preclude the presence of additional features.

[0024] It should be understood that when a component (e.g., a component) is described as being "(operatively or communicatively) coupled to" or "connected to" another component (e.g., a second component), the component may be directly coupled or connected to the other component, or there may be an intervening component (e.g., a third component). Conversely, when a component (e.g., a first component) is described as being "directly coupled to" or "directly connected to" another component (e.g., a second component), the component should be understood as having no intervening component (e.g., a third component).

[0025] Depending on the situation, the term "configured to" used in the present disclosure may be used interchangeably, e.g., with the terms "suitable for," "capable of," "designed for," "adapted for," "made for," or "capable of." When a component, device, element, or the like of the present disclosure is described as having a purpose or performing an operation, function, or the like, the component, device, or element should be considered herein as "configured to" fulfill that purpose or perform that operation or function.

[0026] The term "configured to" does not just mean "specially designed for" when referring to hardware. For example, a "processor configured to perform A, B, and C" may mean a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) that can perform corresponding operations by executing one or more software programs, a special-purpose processor (e.g., an embedded processor) for performing a corresponding operation, or a memory device. The singular terms include the plural unless the context clearly indicates otherwise. All terms used herein, including technical or scientific terms, may have the same meaning generally understood by a person of ordinary skill in the art described in the present disclosure.It should be further understood that terms defined in a dictionary and commonly used should also be interpreted as is customary in the relevant art, rather than in an idealized or overly formal sense, unless expressly defined herein in various embodiments of the present disclosure. In some cases, terms, even if defined in the description, should not be interpreted to exclude those embodiments of the present disclosure.

[0027] In the present disclosure, the terms "A or B," "at least one of A and / or B," or "one or more of A and / or B," and the like may include any and all combinations of the associated listed elements. For example, the term "A or B," "at least one of A and B," or "at least one of A or B" may refer to all cases (1) where at least one A is included, to the case (2) where at least one B is included, or to the case (3) where both at least one A and at least one B are included.Furthermore, in describing an embodiment of the present disclosure, each of the terms such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," "at least one of A, B, or C," and "at least one of A, B, or C, or any combination thereof" may include any or all possible combinations of the elements enumerated together in a corresponding one of the terms. In particular, the term such as "at least one of A, B, or C, or any combination thereof" may include "A," "B," or "C," or "AB" or "ABC," which is a combination thereof.

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the Fig. 1-10 described in detail.

[0029] Fig. 1 is a drawing showing an object synchronization device according to an embodiment of the present disclosure.

[0030] An object synchronization device 100 according to one embodiment may include a processor 110, a memory 120 storing instructions 122, and an interface 130.

[0031] The object synchronization device 100 may provide a real-time object synchronization device that generates a virtual object corresponding to a user's source code and performs synchronization between the generated virtual object and a real object 140 to control the real object 140 through a virtual environment. For example, the object synchronization device 100 may generate the virtual object corresponding to the user's source code in a virtual space in response to a compilation request based on the source code to control the real object 140 by controlling the virtual object created in the virtual environment (e.g., the virtual space).

[0032] The object synchronization device 100 may perform compilation to convert a text file (e.g., source code) describing a computer program in a human-readable programming language into machine language that can be interpreted and executed by an interpreter of the object synchronization device 100. For example, the object synchronization device 100 may generate a virtual object to be expressed in virtual space based on machine language corresponding to the user's source code. The user's source code may include at least one function for performing a control function, such as a movement and an action, of the virtual object.

[0033] The object synchronization device 100 may provide the user with a scene in which a virtual object is located. The scene may, for example, consist of one or more images showing the virtual object and a virtual space in which the virtual object exists and is created. A plurality of individual images may represent consecutive images. The virtual space may refer to a two-dimensional (2D) or three-dimensional (3D) space in which a virtual object is located. For simplicity, the virtual space will be primarily described as a 3D space in this specification.

[0034] The virtual object may refer to a thing, a liquid, a gas, a solid, a rigid body, a soft body, a deformable object (e.g., an object including both the rigid body and the soft body), and the like, which are formed with a set of one or more particles. For example, a position, a shape, a size, a state, and the like of each of the particles can be changed according to an arbitrary motion model in a scene or a virtual space for each of the particles. In particular, as an example where the virtual object is the solid body in the specification, the virtual object is mainly described as being able to be created in the virtual space by means of functions specifying a function related to the control of a robot, which were written by the user.

[0035] The object synchronization device 100 may represent a target in a real world (e.g., the real object 140 such as the robot, hereinafter referred to as "real object 140") as a virtual object in a virtual space. For example, the object synchronization device 100 may generate a virtual object corresponding to source code in a virtual space, and may express the generated virtual object through the virtual space and / or provide it to the user. Although described below, the user may control the real object 140 (e.g., move a robot) using the virtual object.

[0036] The object synchronization device 100 may perform verification for the generated virtual object. For example, before controlling the real object 140, the object synchronization device 100 may perform verification as to whether the virtual object should be applied or synchronized with the real object 140. In detail, the verification may include verifying the interaction of the software and verifying hardware compatibility. The object synchronization device 100 may apply a function associated with the control of the robot (e.g., a function included in the source code) written by the user to the virtual object to first verify the virtual object and thus synchronize the virtual object with the real object 140.To illustrate, the object synchronization device 100 may first perform a verification of the interoperability of the software of the virtual object and then a verification of the hardware compatibility. A detailed description of the verification is provided below with reference to FIG. Fig. 9 given.

[0037] The object synchronization device 100 can perform synchronization between the verified virtual object and the real object 140 in response to a control request for the verified virtual object. For example, the object synchronization device 100 can perform synchronization based on the control request, a control command, and a control command protocol. However, the synchronization process is not limited to this. The synchronization between the virtual object and the real object 140 can be performed based on a protocol other than the above-mentioned synchronization process.

[0038] The object synchronization device 100 can generate a sub-virtual object corresponding to a detected target object in the virtual space when the synchronized real object 140 detects the target object. For example, in contrast to the process in which the object synchronization device 100 generates the virtual object corresponding to the user's source code, the object synchronization device 100 can generate a sub-virtual object corresponding to a target object (e.g., a robot object, a wall object, a stair object, a road object, a human object, or the like) that is different from the real object 140. In the description, the sub-virtual object is mainly described as an object detected by the virtual object. However, the sub-virtual object is not limited to this.For example, the subvirtual object may be a virtual object corresponding to source code of a user other than the aforementioned user. Therefore, even if the subvirtual object is not recognized by the virtual object, it can be created in virtual space.

[0039] The object synchronization device 100 may generate a sub-virtual object corresponding to the target object detected by the real object 140 to provide the user with interaction between virtual objects. Specifically, the object synchronization device 100 may generate the sub-virtual object and thus provide the user, through the expression of the sub-virtual object, with a target object encompassed in a space different from a space in which the user is actually present (e.g., a space between a current position of the user and another position separated from the current position by a predetermined distance).

[0040] Processor 110 may execute software and control at least one other component (e.g., a hardware or software component) connected to processor 110. Furthermore, processor 110 may perform a variety of data processing or computations. For example, processor 110 may store source code and a control request in memory 120.

[0041] For reference, the processor 110 can perform all the operations performed by the object synchronization device 100. Therefore, for simplicity, in the description, the operation performed by the object synchronization device 100 is mainly described as an operation performed by the processor 110.

[0042] The memory 120 may temporarily and / or permanently store various data and / or information necessary to perform object synchronization.

[0043] The interface 130 may refer to an interface capable of supporting communication between the object synchronization device 100 and the real object 140. In detail, the interface 130 may indicate a common boundary for the exchange of information or a signal between two different devices (e.g., the object synchronization device 100 and the real object 140).

[0044] The interface 130 can provide the object synchronization device 100 with a common boundary for performing synchronization between the virtual object and the real object 140. Furthermore, the real object 140 can detect a target object and can provide information about the target object to the object synchronization device 100 via the interface 130. Finally, the object synchronization device 100 can generate a sub-virtual object corresponding to the detected target object in the virtual space based on the information about the target object transmitted via the interface 130.

[0045] Fig. 2 is a flowchart for describing an object synchronization method according to an embodiment of the present disclosure.

[0046] In S210, a device (e.g., an object synchronization device 100 in Fig. 1) create a virtual object corresponding to a user's source code in a virtual space in response to a compilation request based on the source code.

[0047] The device may provide the user with a development environment. For example, the user may generate source code through a development environment of a personal user device (e.g., a user's personal computer (PC). However, the performance (e.g., the computing speed of a central processing unit (CPU), a network speed, or the like) of the personal user device may be different for each of multiple users. To address such a difference, the device may provide the user with the same development environment.

[0048] Specifically, the device may provide the user with a development environment for a virtual object, whereby the virtual object is generated in response to the compilation request based on the source code. The device may include a personal user device connected to an interface (e.g., an interface 130 in Fig. 1), provided with a development environment comprising the same performance and environmental settings.

[0049] The device may provide an independent development environment and an independent virtual space to each of a plurality of users when there are a plurality of users. For example, the independent virtual space may be a space paired with the independent development environment provided to each of the plurality of users, and in which a virtual object is created or expressed. Specifically, when the user who wishes to create the virtual object using the device is a plurality of users, the device may provide the independent development environment and the independent virtual space to each of the plurality of users, thereby creating virtual objects of the plurality of users in parallel in response to a compilation request for a plurality of source codes.

[0050] The device may generate a virtual object corresponding to the source code of a first user among the plurality of users in an independent virtual space of the first user, and may generate a virtual object corresponding to the source code of a second user among the plurality of users in an independent virtual space of the second user who is different from the first user. Here, the first user and the second user may refer to different users. However, the source code of the first user and the source code of the second user are mainly described as different source codes, but are not limited thereto.

[0051] Furthermore, the plurality of users is not limited to the number described above. For example, the device may provide each of the plurality of users with an independent development environment and a dependent virtual space based on the plurality of users. For example, the dependent virtual space may be a space bound to the independent development environment provided to each of the plurality of users, and in which the plurality of virtual objects are created or expressed. As a result, the device may provide the users with a virtual space in which interaction between the plurality of users is applied. For example, the device may generate a virtual object corresponding to the source code of the first user and a virtual object corresponding to the source code of the second user in one virtual space.

[0052] In S220, the device may perform verification for the generated virtual object.

[0053] The device may perform a first verification and a second verification for the generated virtual object. For example, the first verification may specify the verification of the software interoperability for the generated virtual object. Furthermore, the second verification may specify the verification of the hardware compatibility for the generated virtual object.

[0054] The first verification may include verification for a plurality of call functions included in the source code and verification for movement and action of the virtual object in the virtual space.

[0055] For example, if user-written source code is compiled through software integration, the software verification may be verification for software executed by a processor (e.g., a processor 110 in Fig. 1) of the device. Furthermore, the source code may include the plurality of calling functions, specifically, functions in which functions about the plurality of robot movements and actions are written in text according to a predetermined format. The device may execute software for executing the function about the aforementioned robot movement and action by compiling the source code. In doing so, the device may perform verification for a syntactic element and a semantic element of the text in which the function about the aforementioned robot movement and action is written, thus performing software verification.

[0056] The device can perform verification for a movement and an action of the virtual object in the virtual space. For example, the device can move (e.g., control) the generated virtual object in a first direction (e.g., a positive direction of the x-axis in a 3D space represented by the x-axis, y-axis, and z-axis) in a virtual space (e.g., a space on a 3D plane). Thereafter, the device can move the virtual object in a second direction (e.g., a positive direction of the y-axis in the 3D space represented by the x-axis, y-axis, and z-axis) that is different from the above-mentioned first direction.

[0057] Here, the device may measure an angle between the first direction and the second direction to perform verification for the movement and action of the virtual object. For example, if the angle between the first direction and the second direction deviates from a predetermined angle (e.g., 90 degrees), the device may provide the user with information that it cannot verify the movement and action of the virtual object.

[0058] However, the verification of the movement and action of the virtual object is not limited to this. For example, the device may perform at least a movement speed of the virtual object in a specific direction, a movement of the wheel, or movements of components included in the virtual object, or any combination thereof as verification of the movement and action of the virtual object.

[0059] The second verification may indicate the verification of the compatibility of hardware components associated with a movement and an action of a real object (e.g., a real object 140 in Fig. 1) that is to be synchronized with the virtual object. For example, the second verification may include verifying the compatibility of hardware components associated with at least one of the physical object's power consumption, allocated amounts of a central processing unit (CPU) and memory, or a movement amount corresponding to the movement, or any combination thereof.

[0060] In S230, the device may perform synchronization between the verified virtual object and the real object in response to a control request for the verified virtual object.

[0061] The device may change the virtual object to a control mode in response to the user's control request for synchronization between the virtual object and the real object. For example, the virtual object mode may include a wait mode, a control mode, and an error mode.

[0062] The waiting mode may include a waiting state in which the virtual object waits in the virtual space from a time when the virtual object is created until a time when the user's control request is received. Specifically, the waiting mode may include a mode in which the virtual object's waiting state is maintained for the user's control request. A description of the waiting state and the waiting mode of the virtual object for the user's control request is given below.

[0063] The control mode may include a state in which the virtual object is controlled in the virtual space from a time when the user's control request is received to a time when control of the virtual object is terminated.

[0064] The failure mode may include a state in which it is impossible to control the virtual object due to a failure of the virtual object from a time when the virtual object is created to a time when control of the virtual object is terminated.

[0065] The device may generate a control command protocol capable of controlling a real object in response to a control command for the user's virtual object. The control command protocol may include an instruction, via a specific format and protocol, to transmit the user's control command to the real object and execute the control command on the real object side.

[0066] The device can apply the control command protocol to the real object, thus performing synchronization between the virtual object and the real object. For example, the device can synchronize a state of the virtual object with a state of the real object through synchronization between the virtual object and the real object. Furthermore, the device can limit access to shared data between the virtual object and the real object through synchronization between the virtual object and the real object. Furthermore, through synchronization between the virtual object and the real object, the device can prevent deadlock between the virtual object and the real object.

[0067] The device may provide the user with a control approval notification for the control request when synchronization is performed. When the user receives the control approval notification from the device, the user may control the virtual object to control the real object synchronized with the virtual object.

[0068] The device may perform synchronization between the plurality of virtual objects and the real object in response to the control request for the plurality of virtual objects. For example, the device may change the first user's virtual object to control mode in response to the control request for the first user's virtual object. While the first user's virtual object is changing to control mode, the device may receive a control request for the second user's virtual object. If the device receives the second user's control request while changing the first user's virtual object to control mode, the device may put the second user's control request into standby mode.

[0069] The device may separate and synchronize the first user's control request and the second user's control request for a real object. For example, after changing the first user's virtual object to the control mode, the device may change the second user's virtual object to the control mode. However, the synchronization method for the control requests of the plurality of users is not limited to this. For example, after the control mode of the first user's virtual object is terminated, the device may change the wait mode of the second user's virtual object to the control mode. Here, the first user's virtual object and the second user's virtual object may be the same virtual objects.

[0070] In S240, the device may generate a sub-virtual object corresponding to a detected target object in the virtual space when the synchronized real object detects the target object.

[0071] If the detected target object is a robot object (hereinafter referred to as a "robot") that is different from the real object, the device may perform verification for a movement and an action of the sub-virtual object. For example, if the target object is a robot, the device may generate a sub-virtual object (e.g., a robot in virtual space) corresponding to the target object (e.g., the robot in real space). The device may express and generate a movement and an action of the target object in virtual space by means of the verification for the movement and action of the generated sub-virtual object.

[0072] However, verification of the subvirtual object is not limited to this. For example, verification of the subvirtual object may include verification of an interaction with a virtual space previously created in the virtual space, as well as verification of movement and action. Specifically, verification for the interaction may include verification of a distance between the subvirtual object and the virtual object and verification of a movement line between the subvirtual object and the virtual object.

[0073] Fig. 3 is a drawing showing a connection between an object synchronization device and a personal user device according to an embodiment of the present disclosure.

[0074] A device 300 may include at least one processor that accesses a memory (e.g., a memory 120 in Fig. 1) and executes instructions (e.g. instructions 122 in Fig. 1). For example, the device 300 may include a processor 320 for generating a virtual space, a processor 330 for providing a development environment, a processor 340 for generating virtual objects, and a processor 350 for verifying virtual objects.

[0075] For example, device 300 may be connected to personal user device 360 ​​via an interface 310. Device 300 may allow a user to access processor 320 for generating a virtual space and processor 330 for providing a development environment via interface 310.

[0076] The virtual space generation processor 320 may generate a robot component 322 and an environment component 324. For example, the robot component 322 may include a virtual object and a virtual object corresponding to user source code and a sub-virtual object corresponding to a detected target object.

[0077] The environment component 324 may include a human object, a wall object, a non-device object, and a device object. The human object, the wall object, the non-device object, and the device object may each correspond to a person, a wall, a non-device, and a device.

[0078] The device 300 can express and generate an interaction between the robot component 322 and the environment component 324 using the virtual space generation processor 320. In detail, the device 300 can express the interaction between the robot component 322 and the environment component 324 in a virtual space generated by the virtual space generation processor 320 using the personal user device 360 ​​via the interface 310.

[0079] The interaction may include a relationship between the robot component 322 and the environment component 324. For example, if the robot component 322 collides with a wall object included in the environment component 324 in the virtual space, the interaction may include an action in which the state of the robot component 322 is changed by the environment component 324.

[0080] The development environment provision processor 330 may provide a modular virtual environment (Env.) and a compiled source code repository. For example, the modular virtual environment (Env.) may specify a module that creates and manages a virtual environment, and the compiled source code repository may include source code that is not compiled and source code that is compiled by a user's compilation request.

[0081] The virtual object generation processor 340 may generate a virtual object based on the source code included in the compiled source code memory. For example, the virtual object generation processor 340 may generate a virtual object and apply the generated virtual object. In particular, the virtual object generation processor 340 may generate and apply an object included in the robot component 322 and an object included in the environment component 324 in the virtual space.

[0082] The virtual object verification processor 350 may perform verification of virtual objects created by the virtual object generation processor 340. For example, the virtual object verification processor 350 may perform software (SW) integration for a plurality of call functions included in the source code of the virtual object, verification for a movement and action of the virtual object in the virtual space, and verification for the hardware (HW) compatibility of hardware components associated with a movement and action of a real object to be synchronized with the virtual object.

[0083] The personal user device 360 ​​may include a coding module, a robot control module, and a platform interface. The coding module may be a module in which source code for a virtual object to be generated by the processor 320 for generating a virtual space is compiled or run. The robot control module may be a module for controlling the real object synchronized with the virtual object. The platform interface may be an interface for communicating the device 300 with the personal user device 360.

[0084] Fig. 4 is a drawing showing a connection between an object synchronization device and a real object according to an embodiment of the present disclosure.

[0085] A device 400 may include at least one processor that accesses a memory (e.g., a memory 120 in Fig. 1) and executes instructions (e.g. instructions 122 in Fig. 1). For example, the device 400 may include a virtual space creation processor 420, a synchronization processor 430, a virtual object creation processor 440, and a virtual object verification processor 450.

[0086] For illustration, the virtual space generation processor 420 may correspond to a virtual space generation processor (e.g., a virtual space generation processor 320 in Fig. 3). An interface 470 may correspond to an interface (e.g., an interface 310 in Fig. 3). A virtual object generation processor 440 may correspond to a virtual object generation processor (e.g., a virtual object generation processor 340 in Fig. 3). The virtual object verification processor 450 may correspond to a virtual object verification processor (e.g., a virtual object verification processor 350 in Fig. 3).

[0087] A real object 480 may include a server and a parts manager. The server of the real object 480 may refer to a server capable of communicating with the device 400. The parts manager of the real object 480 may refer to a module that controls a component (e.g., a robot arm) of the real object 480. The device 400 may access the server and the parts manager of the real object 480 via the interface 470.

[0088] The synchronization processor 430 can perform robot monitoring, robot synchronization, and object tracking. For example, robot monitoring can refer to monitoring a state of the real object 480. Robot synchronization can refer to synchronization between a robot component of the virtual space generated by the processor 420 to generate the virtual space and the real object 480 after monitoring the state of the real object 480. The device 400 can store information generated by the synchronization processor 430 (e.g., the state of the real object 480) in the memory 460. For illustrative purposes, object tracking is described below with reference to Fig. 6 described.

[0089] Fig. 5 is a drawing showing a method of performing synchronization between a virtual object and a real object in an object synchronization device according to an embodiment of the present disclosure.

[0090] A personal user device 510 may transmit a request for coding and compiling source code to an object synchronization device 500. Specifically, the personal user device 510 may transmit the request for coding and compiling the source code to a development environment provision processor 520. The development environment provision processor 520 may request a virtual object generation processor 530 to generate a virtual object in response to a user request. The virtual space generation processor 540 may generate a virtual space in response to the virtual object generation request and may express a virtual object to be applied in the generated virtual space.

[0091] Once the virtual object is created, the development environment provisioning processor 520 may request a virtual object verification processor 550 to verify the created object. In response to the request to verify the created object, the virtual object verification processor 550 may perform verification for the virtual object created in the virtual space. Thereafter, the virtual object verification processor 550 may transmit information about the completion of compilation and the completion of the virtual object creation to the personal user device 510.

[0092] The object synchronization device 500 may perform synchronization between the virtual object and a real object 570 in response to a request to control the virtual object from the personal user device 510. For example, the synchronization processor 560 may transmit a synchronization progress request to the real object 570. In detail, the synchronization processor 560 may place the virtual object into a control mode, generate a control command protocol capable of controlling the real object 570, and apply the control command protocol to the real object 570.

[0093] The object synchronization device 500 may provide the personal user device 510 with a synchronization notification for a control request (e.g., a synchronization request) and a control approval notification when the synchronization between the virtual object and the real object 570 is performed.

[0094] Fig. 6 is a drawing showing a method of performing synchronization between a detected target object and a sub-virtual object in an object synchronization device according to an embodiment of the present disclosure.

[0095] If a target object 610 is separated from a real object (e.g. a real object 570 from Fig. 5), an object synchronization device 600 may provide object detection and notification to a synchronization processor 620. Thereafter, the synchronization processor 620 may store information about the target object 610 in the memory 630 and request a virtual object creation processor 640 to create a virtual object (e.g., a subvirtual object) about the target object 610.

[0096] The virtual object creation processor 640 may generate a subvirtual object corresponding to the target object 610 based on the above-mentioned request and may apply the subvirtual object to a virtual space. Thereafter, the object synchronization device 600 may request a virtual object verification processor 650 to verify the subvirtual object via the target object 610 based on the information about the target object 610 stored in the memory 630.

[0097] The virtual object verification processor 650 may perform verification for the generated subvirtual object in a virtual space generated by a virtual space generation processor 660. The verification for the subvirtual object may be a first verification, including a software verification, a motion verification, and an action verification. When the verification of the subvirtual object via the target object 610 is completed, the object synchronization device 600 may send a target object application notification to a user (e.g., a personal user device 510 in Fig. 5) transferred.

[0098] The object synchronization device 600 may track a state of the detected target object 610. For example, the object synchronization device 600 may track at least one of a positional state of the target object 610 or a hardware state, or any combination thereof. At this time, the object synchronization device 600 may update the state of the target object 610 using the tracked result when the state of the target object 610 is changed. The object synchronization device 600 may perform synchronization between the updated target object and the sub-virtual object when the state of the detected target object 610 is updated. In detail, the synchronization processor 620 may perform synchronization between the updated target object and the sub-virtual object.

[0099] Fig. 7 is a drawing showing a connection between an object synchronization device and a personal user device according to an embodiment of the present disclosure.

[0100] In S710, a device (e.g., an object synchronization device 100 in Fig. 1) Provide the personal user device with a virtual environment and compile source code encoded in the provided virtual environment. For example, the device may provide a user with the same development environment as a development environment included in the personal user device.

[0101] In S720, the device may create a virtual object corresponding to source code entered by the user into the development environment. The device may perform verification of the created virtual object. The verification may include a first verification including software verification, movement verification, and action verification, and a second verification including hardware component compatibility verification related to the power consumption of a real object, the allocated amounts of CPU and memory, and the movement amount corresponding to a movement.

[0102] In S730, the device may perform synchronization between the verified virtual object and the real object in response to a control request for the verified virtual object. For example, the device may put the virtual object into a control mode in response to a user's control request, generate a control command protocol capable of controlling the real object, and apply the control command protocol to the real object, thereby performing synchronization. The device may provide the user with a control approval notification for the control request when the synchronization is performed.

[0103] In S740, the device may apply a user's control command to the real object. For example, the user may transmit a control request to the device via a command to move the real object. The device may apply the user's control request to the virtual object, thus providing the user with the virtual object to which the control request is applied. At the same time, the device may perform synchronization between the virtual object to which the control request is applied and the real object. The real object may execute the user's control request when the synchronization is performed by the device.

[0104] Fig. 8 is a drawing showing a connection between an object synchronization device and a plurality of personal user devices according to an embodiment of the present disclosure.

[0105] In S810, a device (e.g., an object synchronization device 100 in Fig. 1) provide each of a plurality of users with an independent environment and an independent virtual space based on the user being a plurality of users. For example, the device may provide a user (e.g., a user who is in Fig. 8 as “User 2” and is referred to below as a “second user”), which is separated from a user (e.g. a user in Fig. 7, hereinafter referred to as a “first user”), provide a development environment that is different from a development environment of the first user.

[0106] In S820, the device may generate a virtual object corresponding to a source code of the first user among the plurality of users in an independent virtual space of the first user, and may generate a virtual object corresponding to a source code of the second user among the plurality of users in an independent virtual space of the second user. Hereinafter, for the sake of simplicity of description, Fig. 8 mainly describes an example of the second user.

[0107] In S830, if a control request for a virtual object of the second user is present while a virtual object of the first user is changing to a control mode, the device may put the control request of the second user into a waiting mode. For example, the device may put the control request of the second user into the waiting mode and provide a waiting loop for synchronization of the real object.

[0108] In S840, after placing the first user's virtual object in control mode, the device may place the second user's virtual object in control mode. For example, the device may apply a control command of the second user to a real object. The device may apply the second user's control request to the virtual object corresponding to the second user's source code, thus providing the second user with the virtual object to which the control request is applied. At the same time, the device may perform synchronization between the virtual object to which the second user's control request is applied and the real object. The real object may execute the second user's control request when the synchronization is performed by the device.

[0109] Fig. 9 is a drawing showing a method of verifying a virtual object and applying the verified virtual object to a real object in an object synchronization device according to an embodiment of the present disclosure.

[0110] In S910, a device (e.g., an object synchronization device 100 in Fig. 1) Continue coding and compiling in a virtual environment using a personal user device. A development environment included in the personal user device may be the same as a development environment of the device.

[0111] In S920, the device may perform a first verification for a created virtual object. For example, the first verification may specify the verification of the software interaction for the created virtual object. In detail, the first verification may include a software verification for a plurality of call functions included in a source code and a verification for a movement and an action of the virtual object in a virtual space.

[0112] In S930, the device may perform a second verification when performing the first verification for the virtual object. For example, the second verification may include verifying the compatibility of hardware components associated with at least one of a real object's power consumption, allocated amounts of CPU and memory, a movement amount corresponding to a movement, and the number of wheels of the virtual object, or any combination thereof.

[0113] In S940, the device may provide complete information about the virtual object to the user, create, and verify it when the verification (e.g., the first verification and the second verification) for the virtual object is completed. When the user receives the complete creation and verification information of the virtual object from the device, the user may control the virtual object. Specifically, the user may provide a control request for the virtual object to the device, thereby changing the control of the virtual object to a control mode.

[0114] Fig. 10 shows a computer system via an object synchronization method according to an embodiment of the present disclosure.

[0115] Referring to Fig.10, a computer system 1000 for the object synchronization method may include at least a processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a memory 1600, and a network interface 1700 interconnected via a bus 1200.

[0116] Processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory 1300 and / or storage 1600. Memory 1300 and storage 1600 may include various types of volatile or non-volatile storage media. For example, memory 1300 may include a ROM (read-only memory) 1310 and a RAM (random-access memory) 1320.

[0117] Accordingly, the operations of the methods or algorithms described in connection with the embodiments disclosed in the specification may be implemented directly with a hardware module, a software module, or a combination thereof executed by processor 1100. The software module may be located on a storage medium (i.e., memory 1300 and / or storage 1600), such as RAM, flash memory, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), a register, a hard disk, a removable disk, and a compact disc-ROM (CD-ROM).

[0118] The storage medium may be connected to the processor 1100. The processor 1100 may read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may be integrated into the processor 1100. The processor and the storage medium may be located in an application-specific integrated circuit (ASIC). The ASIC may be located in a user terminal. Alternatively, the processor and the storage medium may be located as separate components in the user terminal.

[0119] Although the present disclosure has been described with reference to embodiments and the accompanying drawings, the present disclosure is not limited thereto. The embodiments of the present disclosure may be modified and changed in various ways by those skilled in the art to which this disclosure pertains without departing from the spirit and scope of the present disclosure as claimed in the following claims.

[0120] The embodiments described above may be implemented with hardware components, software components, and / or a combination thereof. For example, the devices, methods, and components described in the disclosed embodiments may be implemented with general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field-programmable array (FPGA), a programmable logic unit (PLU), a microprocessor, or any device capable of executing and responding to instructions. A processing unit may execute an operating system (OS) or a software application running on the OS. Furthermore, the processing unit may access, store, manipulate, process, and generate data in response to software execution.It should be understood by those skilled in the art that, although a single processing unit is illustrated for clarity, the processing unit may include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing unit may include a plurality of processors or a processor and a controller. The processing unit may also have another processing configuration, such as a parallel processor.

[0121] Software may comprise computer programs, codes, instructions, or one or more combinations thereof, and may configure a processing unit to operate in a desired manner or may instruct the processing unit independently or jointly. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical equipment, virtual equipment, computer storage media or units, or transmitted signal waves to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed across computer systems interconnected by networks and may be stored or executed in distributed form. Software and data may be recorded in a single computer-readable storage medium.

[0122] The methods according to embodiments may be implemented in the form of program instructions that can be executed by various computer means and recorded in computer-readable media. The computer-readable media may include program instructions, data files, data structures, and the like, individually or in combination. The program instructions recorded on the media may be specifically designed and configured for an example or may be known and applicable to those skilled in the art. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as compact disc read-only memory disks (CD-ROMs) and digital versatile discs (DVDs); magneto-optical media such as floppy disks; and hardware devices specifically configured to store and execute program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like.Program instructions include both machine code, as generated by a compiler, and higher-level code that can be executed by the computer using an interpreter.

[0123] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the disclosed embodiments, or vice versa.

[0124] Although the disclosed embodiments are described with reference to various drawings, it should be apparent to those skilled in the art that the disclosed embodiments can be changed or modified in various ways based on the above description. For example, adequate effects can be achieved even if the aforementioned processes and methods are performed in a different order than described above and / or the aforementioned components, such as systems, structures, devices, or circuits, are combined or coupled in forms and modes other than those described above, or are replaced or exchanged with other components or equivalents.

[0125] A description will be given of the effects of the object synchronization apparatus and method therefor according to an embodiment of the present disclosure.

[0126] According to at least one embodiment of the present disclosure, the object synchronization apparatus may simplify a process in which users apply and test source code to a real object using a cable by checking a virtual object corresponding to the user's source code in response to a compilation request based on the source code.

[0127] Furthermore, the object synchronization device according to at least one of the disclosed embodiments of the present disclosure may enable a plurality of users to participate in the robot coding training using a metaverse space through a wait mode and a control mode before performing the synchronization between the verified virtual object and the real object in response to the control request for the virtual object.

[0128] Furthermore, the object synchronization device according to at least one of the disclosed embodiments of the present disclosure may update a state of a detected target object in a virtual space and may enable the user to visualize a space in which a real object is located through a virtual space without directly identifying the space by means of synchronization between the updated target object and a sub-virtual object.

[0129] In addition, various effects directly or indirectly determined by the present disclosure can be provided.

[0130] Therefore, other implementations, other embodiments, and equivalents to the claims are intended to be within the scope of the following claims.

[0131] Therefore, the embodiments of the present disclosure are not intended to limit the technical spirit of the present disclosure, but are provided for illustrative purposes only. The scope of the present disclosure should be interpreted based on the appended claims, and all technical ideas within the scope corresponding to the claims should be included in the scope of the present disclosure.

Claims

[1] Object synchronization device, comprising: a memory that stores computer-executable instructions; and at least one processor configured to access the memory and execute the instructions, wherein the at least one processor creates a virtual object corresponding to a user's source code in a virtual space in response to a compilation request based on the source code, performs a verification for the created virtual object, in response to a control request for the verified virtual object, performs a synchronization between the verified virtual object and a real object, and creates a subvirtual object corresponding to a detected target object in the virtual space when the synchronized real object detects the target object. [2] The object synchronization device according to claim 1, wherein the at least one processor provides the user with a development environment for a virtual object to be created in the virtual space and creates the virtual object according to the source code input by the user into the development environment. [3] The object synchronization device of claim 2, wherein the at least one processor: provides each of a plurality of users with an independent development environment and an independent virtual space if the user is a plurality of persons; creates a virtual object corresponding to a source code of a first user among the plurality of users in an independent virtual space of the first user; and creates a virtual object corresponding to source code of a second user different from the first user among the plurality of users in an independent virtual space of the second user. [4] The object synchronization device of claim 1, wherein the at least one processor: performs software verification for a plurality of call functions included in the source code; performs a first verification for a movement and an action of the virtual object in the virtual space; and performs a second verification for compatibility of hardware components, wherein the compatibility is linked to a movement and an action of the real object that are to be synchronized with the virtual object. [5] The object synchronization device of claim 4, wherein the at least one processor performs verification for at least one of a movement in a first direction of the virtual object in the virtual space, a movement in a second direction different from the first direction, or an angle between the first direction and the second direction, or any combination thereof. [6] Object synchronization device according to claim 4, wherein: the at least one processor performs the second verification, which is different from the first verification, comprising the software verification, the movement verification, and the action verification when the first verification is performed; and the second verification comprises a verification for compatibility of hardware components, wherein the compatibility is associated with at least one of a power consumption of the real object, allocated amounts of a central processing unit (CPU) and memory, or a movement amount corresponding to the movement, or any combination thereof. [7] The object synchronization device of claim 1, wherein the at least one processor: changes the virtual object to a control mode in response to a control request from the user; generates a control command protocol capable of controlling the real object in response to a control command for the user's virtual object; to perform the synchronization by applying the control command protocol to the real object; and provides the user with a tax approval notification for the tax request when the synchronization is performed. [8] The object synchronization device of claim 3, wherein the at least one processor: puts a control request of the second user into a waiting mode if there is a control request for a virtual object of the second user, while a virtual object of the first user is changed into a control mode in response to a control request for the virtual object of the first user; and the second user's virtual object changes to control mode after the first user's virtual object has been changed to control mode. [9] The object synchronization device according to claim 1, wherein the at least one processor performs verification for a movement and an action of the sub-virtual object based on the detected target object being a robot object different from the real object. [10] The object synchronization device of claim 1, wherein the at least one processor tracks a state of the detected target object and performs synchronization between an updated target object and the sub-virtual object based on the state of the detected target object being updated. [11] Object synchronization method, comprising: Creating a virtual object corresponding to a user's source code in a virtual space in response to a compilation request based on the source code; Performing verification for the created virtual object; Performing synchronization between the verified virtual object and a real object in response to a control request for the verified virtual object; and Creating a subvirtual object corresponding to a detected target object in the virtual space when the synchronized real object detects the target object. [12] The object synchronization method of claim 11, wherein creating the virtual object comprises: Providing the user with a development environment for a virtual object to be created in virtual space; and Creating the virtual object according to the source code entered by the user in the development environment. [13] The object synchronization method according to claim 12, further comprising: Providing an independent development environment and an independent virtual space for each of a plurality of users when the user is a plurality of persons; Creating a first virtual object corresponding to a source code of a first user among the plurality of users in an independent virtual space of the first user; and Creating a second virtual object corresponding to source code of a second user different from the first user among the plurality of users in an independent virtual space of the second user. [14] The object synchronization method of claim 11, wherein performing the verification comprises: Performing software verification for a plurality of call functions included in the source code; Performing a verification for a movement and an action of the virtual object in the virtual space; and Performing a verification for the compatibility of hardware components, where the compatibility is associated with a movement and an action of the real object to be synchronized with the virtual object. [15] The object synchronization method of claim 14, wherein performing the verification for the movement and the action comprises: Performing the verification for at least one of a movement in a first direction of the virtual object in the virtual space, a movement in a second direction different from the first direction, or an angle between the first direction and the second direction, or any combination thereof. [16] The object synchronization method of claim 14, wherein performing the verification for the compatibility of the hardware components comprises: Performing a second verification different from a first verification, comprising the software verification, the movement verification, and the action verification when the first verification is performed, and wherein the second verification comprises a verification for the compatibility of hardware components, the compatibility being associated with at least one of power consumption of the real object, allocated amounts of a central processing unit (CPU) and memory, or a movement amount corresponding to the movement, or any combination thereof. [17] The object synchronization method of claim 11, wherein performing the synchronization comprises: Changing the virtual object to a control mode in response to a control request from the user; Generating a control command protocol capable of controlling the real object in response to a control command for the user's virtual object; Performing the synchronization by applying the control command protocol to the real object; and Provide the user with a tax approval notification for the tax request when the synchronization is performed. [18] The object synchronization method of claim 13, wherein changing the virtual object to the control mode comprises: Setting a control request of the second user to a wait mode when there is a control request for a virtual object of the second user while a virtual object of the first user is changed to a control mode in response to a control request for the virtual object of the first user; and Changing the second user's virtual object to control mode after the first user's virtual object has been changed to control mode. [19] The object synchronization method of claim 11, further comprising: Performing verification for a motion and an action of the subvirtual object based on the detected target object being a robot object different from the real object. [20] The object synchronization method according to claim 11, further comprising: Tracking a state of the detected target object; and Performing synchronization between an updated target object and the subvirtual object based on the state of the detected target object being updated.

Images