Device, method and computer program for projecting video to object

The apparatus and method address the limitations of conventional projection mapping on moving objects by segmenting and center-point derivation for real-time 3D dynamic projection mapping, enabling cost-effective mixed reality content creation with one projector and RGB camera.

KR103044453B1Active Publication Date: 2026-11-06KT CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
KR1020210061032
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-05-11
Publication Date
2026-11-06
Estimated Expiration
2041-05-11

AI Technical Summary

Technical Problem

Conventional projection methods cannot accurately apply projection mapping to moving objects due to fixed image size constraints, and existing deep learning-based object tracking methods are limited to objects of learned shapes, obstructive when used in art spaces, and unreliable with RGB-D cameras.

Method used

An apparatus and method that divides objects into segments, derives a center point using a deep learning prediction model, and projects a virtual object onto the object, allowing real-time tracking and 3D dynamic projection mapping with an RGB camera and one projector.

Benefits of technology

Enables real-time 3D dynamic projection mapping on moving objects without shape or size constraints, using one projector and one RGB camera, reducing installation and production costs, and applicable to mixed reality content creation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112021054611093-PAT00014_ABST
    Figure 112021054611093-PAT00014_ABST
Patent Text Reader

Abstract

A device for projecting an image onto an object includes a detection unit for detecting an object included in image data, a segmentation unit for segmenting the detected object into the size of at least one segmentation unit, a center point derivation unit for deriving an individual center point for each segmented object from information about the segmented object based on a deep learning prediction model and deriving an overall center point for the object based on the derived individual center point, a virtual object generation unit for generating a virtual object based on the derived overall center point for the object, and a projection unit for projecting the virtual object onto the object, wherein the size of the segmentation unit may include a first segmentation unit and a second segmentation unit of different sizes.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to an apparatus, a method, and a computer program for projecting an image onto an object. Background Technology

[0002] Projection methods using projectors onto walls or screens are widely utilized in various fields such as art, performing arts, and education.

[0003] With the recent widespread adoption of affordable projectors, such as mini projectors, their use is increasing not only in homes but also in exhibition and performing arts spaces. Consequently, new exhibition methods, such as Media Facade Art, have emerged. Media Facade Art is a compound term combining "media" and "facade" (meaning the exterior wall of a building), referring to the method of projecting various video content onto the exterior walls of a structure.

[0004] On the other hand, conventional screen projection methods using projectors have the problem that they cannot be applied to moving objects because the size of the content image projected onto the screen is fixed.

[0005] While existing screen projection methods do not pose a problem for projection mapping on stationary buildings or walls, they have a limitation in that they cannot accurately apply projection mapping when the target is a moving object.

[0006] In order to perform 3D projection mapping on a moving object, motion detection sensors must be attached to the moving object to track its position in real time, or vision recognition technology using an RGB-D camera must be used.

[0007] The method using RGB-D cameras has the disadvantage of low reliability when detecting the location of moving objects in real time, while the method using motion detection sensors has the disadvantage that if motion detection sensors are used on moving objects in art spaces where exhibitions or performances take place, they obstruct the audience's view, consequently reducing their concentration on the artwork.

[0008] Meanwhile, due to the recent advancement of AI technology, deep learning-based media facade art is being widely used. This deep learning-based object tracking method uses a single RGB-D camera to track objects through real-time deep learning and projects images toward moving objects. However, this deep learning-based object tracking method can only estimate objects identical to the learned object shape before estimating the object. For example, if only objects with a cube shape have been learned, only the learned cube objects can be tracked during the object estimation stage. Prior art literature

[0009] Korean Patent Publication No. 10-1392357 (Registered on April 29, 2014) The problem to be solved

[0010] The present invention aims to solve the problems of the aforementioned prior art by dividing an object included in image data into segments of the size of a segmentation unit, deriving a center point for the object from information about the segmented object using a deep learning prediction model, and projecting a virtual object generated based on the derived center point onto the object.

[0011] However, the technical problems that this embodiment aims to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0012] As a technical means for achieving the technical problem described above, an apparatus for projecting an image onto an object according to the first aspect of the present invention comprises: a detection unit for detecting an object included in image data; a division unit for dividing the detected object into at least one division unit; a center point derivation unit for deriving an individual center point for each of the divided objects from information about the divided objects based on a deep learning prediction model, and deriving an overall center point for the object based on the derived individual center points; a virtual object generation unit for generating a virtual object based on the derived overall center point for the object; and a projection unit for projecting the virtual object onto the object, wherein the size of the division unit may include a first division unit and a second division unit of different sizes.

[0013] A method for projecting an image onto an object, performed by an image projection device according to a second aspect of the present invention, comprises: a step of detecting an object included in image data; a step of dividing the detected object into at least one segmentation unit; a step of deriving an individual center point for each of the segmented objects from information about the segmented objects based on a deep learning prediction model, and deriving an overall center point for the object based on the derived individual center points; a step of creating a virtual object based on the derived overall center point for the object; and a step of projecting the virtual object onto the object, wherein the size of the segmentation unit may include a first segmentation unit and a second segmentation unit of different sizes.

[0014] A computer program stored in a computer-readable recording medium comprising a sequence of instructions for projecting an image onto an object according to a third aspect of the present invention, when executed by a computing device, may include a sequence of instructions for detecting an object included in image data, dividing the detected object into at least one dividing unit, deriving an individual center point for each of the divided objects from information about the divided objects based on a deep learning prediction model, deriving an overall center point for the object based on the derived individual center point, creating a virtual object based on the derived overall center point for the object, projecting the virtual object onto the object, and the size of the dividing unit may include a sequence of instructions including a first dividing unit and a second dividing unit of different sizes.

[0015] The above-described means for solving the problem are merely exemplary and should not be interpreted as intended to limit the invention. In addition to the exemplary embodiments described above, additional embodiments described in the drawings and the detailed description of the invention may exist. Effects of the invention

[0016] According to any one of the means for solving the problem of the present invention described above, the present invention can divide an object included in image data into the size of a division unit, derive a center point for the object from information about the divided object using a deep learning prediction model, and project a virtual object generated based on the derived center point onto the object.

[0017] Through this, the present invention can track moving objects in real time using an RGB camera regardless of the shape and size of the object, and can perform 3D dynamic projection mapping that can be perceived as three-dimensional by utilizing this.

[0018] In addition, the present invention can perform 3D dynamic projection mapping on an object that projects an image using only one projector and one RGB camera.

[0019] In addition, the present invention derives a center point for an object using at least one division unit and performs 3D dynamic projection mapping using the derived center point for the object, so it is not constrained by the shape and size of the object.

[0020] In addition, since the present invention uses only one projector and one RGB camera, installation and production costs for creating mixed reality content are low.

[0021] In addition, the present invention can be used as a base technology for creating mixed reality content using 3D dynamic projection mapping, and can be widely used in various fields such as performance, art, and education.

[0022] Furthermore, since the present invention can create virtual objects using any type of object, it is not dependent on hardware and is not limited to a specific projector. Therefore, it can be applied to any mixed reality content regardless of the projector's specifications. Brief explanation of the drawing

[0023] FIG. 1 is a block diagram of an image projection device according to one embodiment of the present invention. FIG. 2 is a diagram illustrating the process of deriving the overall center point of an object according to an embodiment of the present invention. FIGS. 3a and 3b are drawings for explaining a method of dividing an object according to an embodiment of the present invention. FIGS. 4a and 4b are drawings for explaining a method for calculating a score for a divided unit according to an embodiment of the present invention. FIGS. 5a to 5c are drawings for explaining a method for deriving the overall center point of an object according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a method for projecting an image onto an object according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating a method of projecting an image onto an object according to an embodiment of the present invention. Specific details for implementing the invention

[0024] Embodiments of the present invention are described below in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0025] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0026] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware.

[0027] Some of the operations or functions described in this specification as being performed by a terminal or device may instead be performed by a server connected to said terminal or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal or device connected to said server.

[0028] Hereinafter, specific details for implementing the present invention will be described with reference to the attached configuration diagram or process flowchart.

[0029] FIG. 1 is a block diagram of an image projection device (10) according to one embodiment of the present invention.

[0030] Referring to FIG. 1, the image projection device (10) may include a detection unit (100), a segmentation unit (110), a center point derivation unit (120), a virtual object creation unit (130), a projection unit (140), a calculation unit (150), and a learning unit (160). However, the image projection device (10) illustrated in FIG. 1 is merely one embodiment of the present invention, and various modifications are possible based on the components illustrated in FIG. 1.

[0031] Figure 2 is a diagram for generalizing the process of deriving the overall center point of an object.

[0032] Referring to FIG. 2, the detection unit (100) can detect an object included in the image data when receiving image data from an image capturing device that captures an object. For example, the image capturing device may include an RGB camera. Additionally, the detection unit (100) can detect an object from the image data based on image data (e.g., RGB data) for an object received from the image capturing device. For example, the detection unit (100) can detect an object from the image data by utilizing RGB data that includes environmental information in which the object was captured (e.g., a projection mapping environment used in a dark lighting and space).

[0033] Next, the dividing unit (110) can divide the detected object into at least one dividing unit. Here, the dividing unit is a basic unit for dividing the object detected from image data, and, for example, may be in the shape of a square with equal width and height. Additionally, for the purpose of dividing various shapes of objects, the size of the dividing unit may include a first dividing unit and a second dividing unit of different sizes. In this case, the size of the first dividing unit may be larger than the size of the second dividing unit.

[0034] For example, referring to FIGS. 3a and 3b together, the dividing unit (110) can first divide a detected object (e.g., a triangle) into a first dividing unit and then further divide it into a second dividing unit.

[0035] The calculation unit (150) can calculate a score for a division unit based on the degree of overlap between a bounding box corresponding to the size of the division unit and a bounding box corresponding to the surplus area when a surplus area occurs from the divided object. Here, the score for the degree of overlap between bounding boxes can be calculated, for example, through [Equation 1].

[0037] [Mathematical Formula 1]

[0038] Score =

[0039] Here, is a bounding box corresponding to the size of the partition unit, and represents the bounding box corresponding to the redundant area.

[0041] For example, referring together with FIG. 4a, when an object is first divided by the size of the first division unit by the division unit (110) and a first surplus area is generated, the calculation unit (150) can calculate a score for the first division unit based on the degree of overlap between the bounding box (B) corresponding to the size of the first division unit and the bounding box (A) corresponding to the first surplus area.

[0042] Next, when a second surplus area is generated after the object is secondarily divided by the size of the second division unit by the division unit (110), the calculation unit (150) can calculate a score for the second division unit based on the degree of overlap between the bounding box corresponding to the size of the second division unit and the bounding box corresponding to the second surplus area.

[0043] For example, referring to FIGS. 4a and 4b together, it is assumed that the green box is a bounding box (B) corresponding to the size of the first division unit, and the red box is a bounding box (A) corresponding to the first surplus area.

[0044] If the score for the first division unit (e.g., 0.44) is less than a preset threshold (e.g., 0.6), the division unit (110) may determine the first surplus area (the area remaining after dividing the object into the first division unit) as an area for dividing into the second division unit.

[0045] Alternatively, if the score for the first division unit (e.g., 0.69 or 0.95) exceeds a preset threshold (e.g., 0.6), the division unit (110) may determine that the object has been properly divided without estimating the first surplus area as an area to be divided into the second division unit.

[0046] Meanwhile, the learning unit (160) can train a deep learning prediction model (20) to derive individual center points from test objects divided by at least one division unit.

[0047] Specifically, the learning unit (160) can train a deep learning prediction model (20) to derive individual center points for each divided test object based on information about test objects divided by the size of the first division unit, and train a deep learning prediction model (20) to derive individual center points for each divided test object based on information about test objects divided by the size of the second division unit.

[0048] The center point derivation unit (120) can derive individual center points for each divided object from information about the object divided by at least one division unit based on a deep learning prediction model (20).

[0049] The center point derivation unit (120) can derive individual center points for each of the divided objects through the deep learning prediction model (20) by inputting information about the divided objects into the size of at least one division unit into the deep learning prediction model (20).

[0050] The center point derivation unit (120) can derive an overall center point for an object based on individual center points for each segmented object derived through a deep learning prediction model (20).

[0051] The center point derivation unit (120) can predict individual center points for each divided object through a deep learning prediction model (20) based on whether the score for at least one division unit exceeds a preset threshold.

[0052] For example, referring to FIG. 3a and FIG. 3b, after an object is first divided by the size of a first division unit by the division unit (110), the center point derivation unit (120) can select a first area (e.g., an area containing a large star) that can be covered by the first division unit among the first divided objects based on a score for the first division unit. Subsequently, after an object is secondarily divided by the size of a second division unit by the division unit (110), the center point derivation unit (120) can select a second area (e.g., an area containing a small star) that can be covered by the second division unit among the areas not selected as the first area in the second divided objects based on a score for the second division unit.

[0053] The center point derivation unit (120) can derive an overall center point for an object based on the individual center points of each of the first area that can be covered by the selected first division unit and the second area that can be covered by the selected second division unit.

[0054] Referring to FIG. 5a, the learning unit (160) can train a deep learning prediction model so that the difference between the 2D correct coordinate information of an object and the 2D coordinate information of an object estimated from image data is reduced. Here, the 2D correct coordinate information of an object may include 2D pixel coordinate information of an object derived from image data. The 2D coordinate information of an object may be information estimated from each image frame constituting the image data through a coordinate estimation model. Here, the coordinate estimation model may be composed of, for example, at least one neural network of the CNN (Convolutional Neural Network) type.

[0055] For example, if the object has a cuboid shape, the 2D ground truth coordinate information of the object may include 2D pixel coordinate information for each vertex of the object extracted from an image of the object and 2D pixel coordinate information for the center point of the object.

[0056] The difference between the object's 2D correct coordinate information and the object's 2D coordinate information is reduced based on the Mean Square Error method, and the object's 2D coordinate information with reduced difference from the 2D correct coordinate information is estimated to be close to the object's 2D correct coordinate information through back propagation.

[0057] For example, the 2D coordinate information of an object ( , ) and the object's 2D correct coordinate information ( , In the case of ), the learning unit (160) can reduce the difference between the object's 2D coordinate information and the object's 2D correct coordinate information based on the mean squared error method of [Equation 2].

[0059] [Mathematical Formula 2]

[0060]

[0062] Looking at [Mathematical Formula 2], the Euclidean distance formula is used to calculate the difference in distance between the object's 2D coordinate information and the object's 2D correct coordinate information, and this is reduced through the mean squared error method.

[0063] The center point derivation unit (120) can convert the 2D coordinate information of an object into 3D pose coordinate information of an object based on the 3D correct coordinate information of an object, the 2D correct coordinate information of an object, and the camera parameter information of a camera that captured the object, and can derive an individual center point for each of the divided objects based on the converted 3D pose coordinate information of the object.

[0064] The center point derivation unit (120) can derive the overall center point for the object using individual center points for each segmented object derived based on the 3D pose coordinate information of the transformed object.

[0065] For example, referring to FIG. 5b, in order to derive the overall center point for an object, the center point derivation unit (120) can derive the overall center point for an object based on a formula that generates a polygon for each divided object using the individual center points of each divided object, divides the polygon for each divided object into diagonal faces, and calculates the center points of the upper and lower triangles formed on each divided polygon. For example, the individual center point of the first divided object and, the individual center points of the second divided object and, the individual center points of the third divided object and, the individual center point of the fourth divided object When this happens, the overall center of the object can be derived through [Equation 3].

[0067] [Mathematical Formula 3]

[0068]

[0070] Meanwhile, since information regarding the Z-axis of the overall center point of an object cannot be estimated from image data, information regarding the Z-axis of the object must be estimated using the length information of the Z-axis of the object measured by the user. Referring to FIG. 5c, the center point derivation unit (120) can derive information regarding the Z-axis of the overall center point of the object in proportion to the information regarding the Z-axis of the division unit by comparing the information regarding the Z-axis of the division unit with the length information of the Z-axis of the object measured by the division unit.

[0071] The virtual object creation unit (130) can create a virtual object based on the overall center point of the derived object. For example, the virtual object creation unit (130) can create a virtual object having a shape different from the object and can create a virtual object having a size smaller than the object.

[0072] The projection unit (140) can project a virtual object onto an object. For example, referring to FIG. 6, the projection unit (140) of the image projection device (10) can render a virtual object (62) based on the overall center point of the derived object and project the rendered virtual object (62) toward the object (60). Additionally, when the rotation value of the object (60) is derived based on the overall center point of the derived object, the projection unit (140) of the image projection device (10) can perform real-time 3D rendering by reflecting the rotation value of the object (60) in real-time to the rotation value of the virtual object (62), and perform projection mapping by projecting the 3D rendered virtual object (62) onto the object (60) in real-time. Here, if the virtual object (62) is configured to be smaller than the object (60), it can provide a pseudo-holographic or similar holographic effect capable of producing a 3D effect.

[0073] Meanwhile, those skilled in the art will fully understand that the detection unit (100), the division unit (110), the center point derivation unit (120), the virtual object creation unit (130), the projection unit (140), the calculation unit (150), and the learning unit (160) may each be implemented separately, or that one or more of these may be implemented in an integrated manner.

[0074] FIG. 7 is a flowchart illustrating a method of projecting an image onto an object according to an embodiment of the present invention.

[0075] Referring to FIG. 7, in step S701, the image projection device (10) can detect an object included in the image data.

[0076] In step S703, the image projection device (10) can divide the detected object into at least one division unit.

[0077] In step S705, the image projection device (10) can derive individual center points for each of the segmented objects from information about the segmented objects based on a deep learning prediction model.

[0078] In step S707, the image projection device (10) can derive an overall center point for the object based on the derived individual center points.

[0079] In step S709, the image projection device (10) can generate a virtual object based on the overall center point of the derived object.

[0080] In step S711, the image projection device (10) can project a virtual object onto the object.

[0081] In the description above, steps S701 to S711 may be further divided into additional steps or combined into fewer steps, depending on an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order between steps may be changed.

[0082] One embodiment of the present invention may also be implemented in the form of a recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include all computer storage media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0083] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0084] The scope of the present invention is defined by the claims set forth below rather than by the detailed description, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0085] 10: Image projection device 100: Detector 110: Dividing section 120: Center point derivation part 130: Virtual Object Creation Section 140: Projection section 150: Calculation section 160: Learning Department

Claims

Claim 1 An image projection device comprising: a detection unit for detecting an object included in image data; a division unit for dividing the detected object into at least one division unit size; a center point derivation unit for deriving an individual center point for each of the divided objects from information about the divided objects based on a deep learning prediction model, and deriving an overall center point for the object based on the derived individual center points; a virtual object generation unit for generating a virtual object based on the derived overall center point for the object; and a projection unit for projecting the virtual object onto the object, wherein the size of the division unit includes a first division unit and a second division unit of different sizes, and the division unit determines whether a redundant area occurs from the divided object and determines whether to perform additional division of the redundant area based on the determination result. Claim 2 An image projection device according to claim 1, wherein the dividing unit divides the object from the image data based on RGB data for the object. Claim 3 An image projection device according to claim 1, further comprising a calculation unit that calculates a score for the division unit based on the degree of overlap between a bounding box corresponding to the size of the division unit and a bounding box corresponding to the surplus area when a surplus area occurs from the division object. Claim 4 In claim 3, the image projection device, wherein the center point derivation unit predicts an individual center point for each of the segmented objects through the deep learning prediction model based on whether the score for the segmentation unit exceeds a preset threshold, Claim 5 An image projection device according to claim 1, wherein the dividing unit performs a first division of the object to the size of the first dividing unit, and the center point derivation unit selects a first area among the first divided objects that the first dividing unit can cover based on a score for the first dividing unit. Claim 6 An image projection device according to claim 5, wherein the size of the first division unit is larger than the size of the second division unit, the division unit performs a second division of the object with respect to the size of the second division unit, and the center point derivation unit selects a second area that the second division unit can cover among the areas not selected as the first area in the secondly divided object based on the score for the second division unit. Claim 7 An image projection device according to claim 6, wherein the center point derivation unit derives an overall center point for the object based on individual center points of each of the selected first region and the selected second region. Claim 8 An image projection device according to claim 1, further comprising a learning unit that trains the deep learning prediction model so as to reduce the difference between the two-dimensional correct coordinate information of the object and the two-dimensional coordinate information of the object estimated from the image data, wherein the two-dimensional correct coordinate information of the object includes two-dimensional pixel coordinate information for the object derived from the image data. Claim 9 An image projection device according to claim 8, wherein the center point derivation unit converts the 2D coordinate information of the object into 3D pose coordinate information of the object based on 3D correct coordinate information of the object, 2D correct coordinate information of the object, and camera parameter information of the camera that captured the object, and derives an individual center point for each of the divided objects based on the converted 3D pose coordinate information of the object. Claim 10 A method for projecting an image onto an object performed by an image projection device, comprising: a step of detecting an object included in image data; a step of dividing the detected object into at least one segmentation unit; a step of deriving an individual center point for each of the segmented objects from information about the segmented objects based on a deep learning prediction model, and deriving an overall center point for the object based on the derived individual center points; a step of generating a virtual object based on the derived overall center point for the object; and a step of projecting the virtual object onto the object, wherein the size of the segmentation unit includes a first segmentation unit and a second segmentation unit of different sizes, and the segmentation step determines whether a redundant area occurs from the segmented object and determines whether to perform additional segmentation of the redundant area based on the result of the determination. Claim 11 An image projection method according to claim 10, further comprising the step of calculating a score for the segmented unit based on the degree of overlap between a bounding box corresponding to the size of the segmented unit and a bounding box corresponding to the surplus area when a surplus area occurs from the segmented object. Claim 12 An image projection method according to claim 11, wherein the deriving step includes the step of predicting individual center points for each of the segmented objects through the deep learning prediction model based on whether the score for the segmented unit exceeds a preset threshold. Claim 13 An image projection method according to claim 10, wherein the dividing step includes the step of performing a first division of the object to the size of the first division unit, and the deriving step includes the step of selecting a first area that the first division unit can cover among the first divided objects based on a score for the first division unit. Claim 14 An image projection method according to claim 13, wherein the size of the first division unit is larger than the size of the second division unit, the division step includes the step of performing a second division of the object to the size of the second division unit, and the derivation step includes the step of selecting a second area that the second division unit can cover among the areas not selected as the first area in the secondly divided object based on the score for the second division unit. Claim 15 An image projection method according to claim 14, wherein the deriving step comprises the step of deriving an overall center point for the object based on the individual center points of each of the selected first region and the selected second region. Claim 16 An image projection method according to claim 10, further comprising the step of training the deep learning prediction model so as to reduce the difference between the two-dimensional correct coordinate information of the object and the two-dimensional coordinate information of the object estimated from the image data, wherein the two-dimensional correct coordinate information of the object includes two-dimensional pixel coordinate information for the object derived from the image data. Claim 17 An image projection method according to claim 16, wherein the deriving step comprises converting the 2D coordinate information of the object into 3D pose coordinate information of the object based on 3D correct coordinate information of the object, 2D correct coordinate information of the object, and camera parameter information of the camera that captured the object, and deriving an individual center point for each of the divided objects based on the converted 3D pose coordinate information of the object. Claim 18 A computer program stored on a computer-readable recording medium comprising a sequence of instructions for projecting an image onto an object, wherein, when the computer program is executed by a computing device, the computer program comprises a sequence of instructions for detecting an object included in image data, dividing the detected object into at least one dividing unit, deriving an individual center point for each of the divided objects from information about the divided objects based on a deep learning prediction model, deriving an overall center point for the object based on the derived individual center point, creating a virtual object based on the derived overall center point for the object, projecting the virtual object onto the object, wherein the size of the dividing unit includes a first dividing unit and a second dividing unit of different sizes, determining whether a redundant area occurs from the divided object, and determining whether to further divide the redundant area based on the determination result.

Citation Information

Patent Citations

  • Method for Real-time Object Detection Based on Lidar Sensor and Camera Using CNN

    KR1020210026412A

  • Autogrid analysis

    US6714675B1

  • Control method, learning device, discrimination device, and program

    WO2020178881A1