Interactive educational content providing device using lidar sensor and camera sensor

Abstract

An aspect of the present invention may provide an interactive educational content providing device using a LiDAR sensor and a camera sensor, the device comprising: a housing unit; a touch screen formed on the front surface of the housing unit and configured to execute a platform launcher for driving content for a user and to selectively control a corresponding UI; multiple image projection units provided in the housing unit to simultaneously or selectively project an image for interactive educational content on a floor and a wall surface; a projector for providing an image signal to the image projection units; a projector installation unit installed on the rear surface of the housing unit to allow the projector to be installed thereon; a LiDAR sensor unit deployed to both sides of the bottom of the housing unit to be installed toward the projection direction of the image projection units, the LiDAR sensor unit recognizing a user position according to interactive educational content to track a user motion; a sensor connection arm for mechanically connecting the LiDAR sensor unit and the housing unit and protecting a cable therein; a camera sensor unit which is provided on the front surface of the housing unit, is associated with a signal from the LiDAR sensor unit, detects the user's motion to generate and track the user's body skeleton, tracks a hand and a foot of the user's body skeleton to identify whether the hand and the foot is the left or right hand and foot, and provides linking information to the interactive educational content; a speaker unit provided in the housing unit to transmit the sound of the interactive educational content to the user; an exhaust fan installed in the housing unit to discharge heat inside the housing unit; and a control unit for controlling the projector, the image projection units, the LiDAR sensor unit, and the camera sensor unit.

Description

Interactive educational content provider using LiDAR and camera sensors

[0001] The present invention relates to an interactive educational content providing device using a LiDAR sensor and a camera sensor.

[0002] Generally, with the recent advancement of IT technology, e-learning services that implement online education alongside offline education are gaining prominence. Here, offline education refers to education conducted using textbooks in institutions such as schools and academies, while online education refers to education conducted using network access technologies, such as the Internet and intranets, and various multimedia content.

[0003] Currently, various effects utilizing multimedia content such as text, graphics, and images, as well as audio and video, are being applied to such interactive educational content. In addition, e-learning services are being implemented that allow users to directly participate in interactive educational content on the client side.

[0004] Among various technologies for directly producing interactive educational content composed of various objects using a combination of text, images, video, and sound, and for combining 3D effects therewith, Korean Patent Publication No. 10-2017-0057736 discloses the “Educational VR Content Production System and Control Method Thereof.” The aforementioned prior art comprises a communication unit that receives a control signal from a creator terminal; a template generation unit that selects a 3D experience environment and a 3D object constituting a VR experience environment according to the control signal, and generates at least one template information by combining the selected 3D experience environment and the 3D object; a memory that stores the 3D experience environment, the 3D object, and the generated template information; an educational VR experience environment generation unit that generates a data file according to one of the at least one template information generated by the template generation unit and the selected template information; and a display unit that displays at least one of the generated data file, the at least one template information, the stored 3D experience environment, and the 3D object on a web screen, thereby enabling the production of educational VR content and its sharing on the web. A creator terminal, a client terminal, and a service server of a production system, a method for producing VR content using these, and a method for operating VR content are disclosed.

[0005] However, interactive educational content utilizing such conventional technology is limited to visually providing users with virtual experience information. Consequently, there is a lack of proposals for experiential activity-type interactive educational content that allows users to actively participate in the content, as well as interactive educational content experience devices that provide such content; thus, development of such content is required.

[0006] The present invention has been devised to solve the aforementioned problems, and its purpose is to provide an interactive educational content providing device using LiDAR sensors and camera sensors, which is equipped with a plurality of video projection units in an enclosure that project images for interactive educational content onto the floor and walls simultaneously or selectively. This allows not only the indoor floor space but also the walls to be used as an interactive educational content experience space, thereby enabling diversification and enlargement of the experience space, which improves the realism of the interactive educational content, enhances immersion by giving the user the feeling of actually participating in the interactive educational content, and allows a larger number of people to participate together due to the expandability of the experience space.

[0007] Another objective of the present invention is to provide an interactive educational content providing device using a LiDAR sensor and a camera sensor that enables the projector to be mounted on a projector mounting unit installed on the rear of an enclosure and project the video signal of the projector through a video projection unit installed on the front of the enclosure, thereby making it possible to increase the size and performance of the projector, and consequently provide large-screen interactive educational content with clear images, enhance the three-dimensionality of interactive educational content by installing multiple projectors or multiple video projection units, and enable the implementation of more diverse content.

[0008] According to one aspect of the present invention for achieving the above-mentioned purpose, the invention comprises: an enclosure; a touch screen provided on the front of the enclosure, for executing a platform launcher for driving content for a user and for selectively controlling the corresponding UI; a plurality of image projection units provided on the enclosure, for simultaneously or selectively projecting images for interactive educational content onto a floor and a wall; a projector providing image signals to the image projection units; a projector installation unit installed on the rear of the enclosure to allow the projector to be installed; a LiDAR sensor unit deployed on both sides of the floor of the enclosure and installed facing the projection direction of the image projection units, for recognizing the user's position according to the interactive educational content and tracking user motion; a sensor connection arm mechanically connecting the LiDAR sensor unit and the enclosure and protecting a cable inside; and a camera sensor unit provided on the front of the enclosure, linked with the signal from the LiDAR sensor unit, detecting user movements to generate and track the user's body skeleton, tracking the hands and feet among the user's body skeleton to identify whether the hands and feet are left or right, and providing linkage information to the interactive educational content. An interactive educational content providing device using a LiDAR sensor and a camera sensor may be provided, characterized by comprising: a speaker unit provided in the above-mentioned housing unit and transmitting the sound of interactive educational content to a user; an exhaust fan installed in the above-mentioned housing unit and discharging heat inside the housing unit; and a control unit that controls the projector, image projection unit, LiDAR sensor unit, and camera sensor unit to provide interactive educational content.

[0009] In addition, a fixing means may be further formed such that one end of the sensor connecting arm is pivotably hinged to the housing, and the other end of the sensor connecting arm is detachably connected to the upper side of the housing by magnetic force.

[0010] According to another aspect of the present invention, the apparatus comprises: an enclosure; a touch screen provided on the front of the enclosure, for executing a platform launcher for driving content for a user and for selectively controlling the corresponding UI; a plurality of image projection units provided on the enclosure, for simultaneously or selectively projecting images for interactive educational content onto a floor and a wall; a projector providing image signals to the image projection units; a projector installation unit installed on the rear of the enclosure to allow the projector to be installed; a LiDAR sensor unit deployed on both sides of the floor of the enclosure and installed toward the projection direction of the image projection units, for recognizing the user's position according to the interactive educational content and tracking user motion; a sensor connection arm mechanically connecting the LiDAR sensor unit and the enclosure and protecting a cable inside; and a camera sensor unit provided on the front of the enclosure, linked with the signal from the LiDAR sensor unit, for detecting user movements to generate and track the user's body skeleton, and for tracking the hands and feet of the user's body skeleton to identify whether the hands and feet are left or right and providing linkage information to the interactive educational content. An interactive educational content providing device using a LiDAR sensor and a camera sensor may be provided, comprising: a speaker unit provided in the above-mentioned housing unit for delivering sound of interactive educational content to a user; an exhaust fan installed in the above-mentioned housing unit for discharging heat inside the housing unit; and a control unit that controls the projector, image projection unit, LiDAR sensor unit, and camera sensor unit to provide interactive educational content; wherein a projector lifting device is provided in the projector installation unit to cause the projector to be automatically raised and lowered by an electric device.

[0011] In addition, an arm receiving portion that slides in the transverse direction of the sensor connecting arm may be further formed inside the above-mentioned housing portion, and a rail portion may be formed so that the sensor connecting arm is cross-received.

[0012] In addition, a sensor insertion section is formed in the entrance of the arm receiving section to which a lidar sensor section is inserted, and the front of the sensor insertion section is open so that motion tracking can be performed with the lidar sensor inserted into the arm receiving section.

[0013] As described above, the present invention provides a plurality of video projection units that simultaneously or selectively project images for interactive educational content onto the floor and walls of an enclosure, thereby allowing not only the indoor floor space but also the walls to be used as an experience space for interactive educational content. This enables diversification of the experience space and allows for the enlargement of the experience space, thereby enhancing the realism of the interactive educational content. Furthermore, it increases immersion by giving users the feeling of actually participating in the interactive educational content, and allows a larger number of people to participate together due to the expandability of the experience space.

[0014] Furthermore, the present invention provides a dual structure in which a projector is mounted on a projector mounting unit installed on the rear of a housing and the video signal of the projector is projected through a video projection unit installed on the front of the housing. This enables the projector to be enlarged and perform well, thereby providing the effect of providing large-screen interactive educational content as a clear image. Additionally, by installing multiple projectors or multiple video projection units, the three-dimensional effect of the interactive educational content can be enhanced, and a wider variety of content can be implemented.

[0015] FIG. 1 is a perspective view of an interactive educational content providing device using a LiDAR sensor and a camera sensor according to an embodiment of the present invention.

[0016] FIG. 2 is a perspective view of an interactive educational content providing device using a LiDAR sensor and a camera sensor according to another embodiment of the present invention.

[0017] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.

[0018] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0019] FIG. 1 is a perspective view of an interactive educational content providing device using a lidar sensor and a camera sensor according to one embodiment of the present invention, and FIG. 2 is a perspective view of an interactive educational content providing device using a lidar sensor and a camera sensor according to another embodiment of the present invention.

[0020] FIG. 2 is a perspective view of an interactive educational content providing device using a LiDAR sensor and a camera sensor according to another embodiment of the present invention.

[0021] Referring to FIG. 1 and FIG. 2, the interactive educational content providing device according to the present invention may largely include an enclosure (110); a touch screen (120); a projection unit (130); a projector (140); a projector installation unit (150); a LiDAR sensor unit (170); a sensor connecting arm (180); a camera sensor unit (190); a speaker unit (200); and a control unit.

[0022] An interactive educational content providing device according to an embodiment of the present invention may include an enclosure (110).

[0023] The above-mentioned housing (110) can be manufactured in the form of a cabinet, and components necessary for operating the device can be provided and protected inside.

[0024] An anchor hole may be provided at the bottom of the above-mentioned housing part (110) to allow it to be fixed to the floor of the installation location. Additionally, wheels may be provided on the bottom surface of the above-mentioned housing part (110) to facilitate movement.

[0025] According to an embodiment of the present invention, a touch screen (120) may be provided on the front of the housing portion (110). The touch screen (120) may be installed on the upper part of the housing portion (110), and a content screen may be provided in which a platform launcher for running content for a user is executed and a corresponding UI can be selected and controlled.

[0026] According to an embodiment of the present invention, an image projection unit (130) may be provided in the housing unit (110). The image projection unit (130) is optically connected to a projector (140) installed on the rear of the housing unit (110), and may project interactive educational content images transmitted from the projector (140) onto the floor and walls. Multiple such image projection units may be installed on the front, side, top surface, etc., of the housing unit (110), and each image projection unit (130) may project images for interactive educational content simultaneously or selectively.

[0027] According to an embodiment of the present invention, a projector (140) may be provided. The projector (140) provides an image signal to the image projection unit (130), and a large-screen dedicated projector (140) for projecting a large screen onto a wall inside the room may be used.

[0028] One or more of the above projectors (140) may be installed, and each projector (140) may be used independently or interconnected to display a single interactive educational content video.

[0029] According to an embodiment of the present invention, a projector installation part (150) may be provided.

[0030] The projector installation part (150) may be a shelf-shaped structure installed on the rear of the housing part (110). The projector installation part (150) may be equipped with devices that ensure stable installation of the projector (140) and prevent shaking during movement.

[0031] Additionally, the projector installation section (150) may be provided with a partitioned mounting bracket for installing a plurality of projectors (140).

[0032] In addition, the projector installation part (150) may be equipped with a projector lifting device (160) that enables the projector (140) to be automatically raised and lowered by an electric device.

[0033] The above projector lifting device (160) is a device that enables the mounting bracket on which the projector (140) is mounted to be raised and lowered in a vertical direction, and may be composed of a driving part consisting of a ball screw that rotates by a motor and a screw block that is axially coupled to the ball screw, and a guide part that prevents shaking when the mounting bracket is raised and lowered around the driving part.

[0034] The above projector lifting device (160) can be controlled by operating the touch screen (120) or by an operator operating the motor.

[0035] The projection position (height) of the projector (140) can be varied by the projector lifting device (160), and through this, precise setting work can be performed on the target wall and floor surfaces where the image is projected.

[0036] According to an embodiment of the present invention, a LiDAR sensor unit (170) may be provided that is deployed on both sides of the bottom of the housing unit (110) and installed toward the projection direction of the image projection unit (130), and tracks user motion by recognizing the user position according to interactive educational content.

[0037] The above-mentioned LiDAR sensor unit (170) is located at both lower ends of the device and can perform tracking operations using LiDAR (Light Detection and Ranging) technology. The LiDAR sensor is used for distance measurement and environment scanning. It can determine distance values ​​by emitting a laser from a designated location and measuring the time it takes for the laser to reflect off an object and return to the sensor. Through this method, the LiDAR sensor can precisely measure information about the distance and shape of the surrounding environment.

[0038] LiDAR sensors are highly efficient at mapping environments or detecting obstacles by performing numerous or millions of distance measurements at very high speeds, and by placing them stereo on both sides of the device, the precision of the sensors can be maximized by minimizing blind spots for the users.

[0039] According to an embodiment of the present invention, a sensor connecting arm (180) may be provided.

[0040] The sensor connecting arm (180) mechanically connects the lidar sensor part (170) and the housing part (110) and can serve to protect the cable inside.

[0041] The sensor connecting arm (180) can be provided as a pipe structure.

[0042] Referring to FIG. 1, one end of the sensor connecting arm (180) is pivotally hinged to the housing part (110), and a fixing means (220) may be further formed so that the other end of the sensor connecting arm (180) is detachably connected to the upper side of the housing part (110) by magnetic force.

[0043] At this time, the fixing means (220) may use a neodymium magnet. The neodymium magnet may serve to fix the sensor connecting arm (180) so that it does not move during movement.

[0044] Referring to FIG. 2, the sensor connecting arm (180) can be stored by being received within the housing (110) rather than being stored upright. To this end, an arm receiving portion (220) that slides in the lateral direction of the sensor connecting arm (180) can be further formed inside the housing (110). At this time, the arm receiving portion (220) can be provided in a tubular form in which the pipe-shaped sensor connecting arm (180) is double-coupled.

[0045] In addition, the sensor connecting arms (180) installed on both sides of the housing (110) can be cross-accepted so that they do not interfere with each other, and a rail section (221) can be formed to guide the movement path of the sensor connecting arms (180).

[0046] At this time, the sensor connecting arm (180) can be positioned at the front and back on a horizontal plane to avoid mutual interference, or positioned at the top and bottom on a vertical plane to avoid mutual interference.

[0047] At this time, a sensor insertion section (223) into which a lidar sensor section (170) is inserted can be formed at the entrance of the arm receiving section (220). Such a sensor insertion section (223) can prevent the lidar sensor section (170) from protruding outward from the housing section (110) and being damaged during the movement of the device.

[0048] In addition, by making the front of the sensor insertion part (223) open, motion tracking can be performed with the LiDAR sensor part (170) inserted into the arm receiving part (220). The present invention, which is equipped with such an arm receiving part (220), has the advantage of being able to easily expand or contract the position of the LiDAR sensor part (170), making it convenient for application and placement according to the shape of the experience space.

[0049] According to an embodiment of the present invention, a camera sensor unit (190) may be provided. The camera sensor unit (190) may be provided to be located at the front of the housing unit (110), particularly below the touch screen (120).

[0050] The camera sensor unit (190) is linked with the signal of the lidar sensor unit (170), detects the user's movements to generate and track the user's body skeleton, and tracks the hands and feet of the user's body skeleton to identify whether the hands and feet are left or right, thereby providing linked information to interactive educational content.

[0051] According to an embodiment of the present invention, a speaker unit (200) may be provided. A plurality of the speaker units (200) may be installed on the front or side of the housing unit (110) to deliver sound of interactive educational content to a user.

[0052] In addition, an exhaust fan (210) installed in the above-mentioned housing (110) to discharge heat inside the housing may be installed.

[0053] The exhaust fan (210) lowers the temperature inside the enclosure where a lot of heat is generated and discharges the heat, and regulates the internal temperature by circulating air inside the enclosure (110). This prevents the equipment from overheating or becoming too hot, thereby providing a stable operating environment, and optimizes the performance and improves the durability of the equipment through heat discharge. By maintaining the temperature inside the enclosure, it helps the equipment operate stably and extends the lifespan of the equipment.

[0054] According to an embodiment of the present invention, a control unit (not shown) that controls the projector (140), image projection unit (130), lidar sensor unit (170), and camera sensor unit (190) to provide interactive educational content may be provided.

[0055] As described above, the present invention provides a plurality of video projection units in an enclosure that simultaneously or selectively project images for interactive educational content onto the floor and walls. This allows not only the indoor floor space but also the walls to be used as an experience space for interactive educational content, thereby enabling diversification and enlargement of the experience space, which can enhance the realism of the interactive educational content, increase immersion by giving users the feeling of actually participating in the interactive educational content, and allow a larger number of people to participate together due to the expandability of the experience space.

[0056] In addition, the present invention provides a dual structure in which a projector is mounted on a projector mounting unit installed on the rear of a housing and the video signal of the projector is projected through a video projection unit installed on the front of the housing. This enables the projector to be enlarged and high-performance, thereby providing the effect of providing large-screen interactive educational content as a clear image. Furthermore, by installing multiple projectors or multiple video projection units, the three-dimensional effect of the interactive educational content can be enhanced, and there is an advantage in being able to implement a wider variety of content.

[0057] As described above, the present invention is not limited to the specific preferred embodiments described above, and any person skilled in the art to which the invention pertains can make various modifications without departing from the essence of the invention as claimed in the claims, and such modifications will be within the scope of the claims.

Claims

1. Hull section; A touch screen provided on the front of the above-mentioned housing, for executing a platform launcher for driving content for a user and for selecting and controlling the corresponding UI; A plurality of image projection units provided in the above-mentioned housing, which simultaneously or selectively project images for interactive educational content onto the floor and walls; A projector that provides an image signal to the above-mentioned image projection unit; A projector installation part installed on the rear of the above-mentioned housing part to allow the projector to be installed; A LiDAR sensor unit deployed on both sides of the bottom of the above-mentioned housing and installed facing the projection direction of the above-mentioned image projection unit, which recognizes the user's position according to interactive educational content and tracks user motion; A sensor connecting arm that mechanically connects the above-mentioned lidar sensor unit and housing unit and protects the cable inside; A camera sensor unit provided on the front of the above-mentioned housing unit and linked with the signal of the above-mentioned lidar sensor unit, detects user movements to generate and track the user's body skeleton, tracks the hands and feet within the user's body skeleton to identify whether the hands and feet are left or right, and provides linkage information to interactive educational content; A speaker unit provided in the above-mentioned housing and delivering the sound of interactive educational content to the user; An exhaust fan installed in the above-mentioned enclosure to discharge heat from inside the enclosure; and A control unit that controls the above-mentioned projector, image projection unit, LiDAR sensor unit, and camera sensor unit to provide interactive educational content; An interactive educational content providing device using a LiDAR sensor and a camera sensor, characterized by including 2. In Paragraph 1, An interactive educational content providing device using a LiDAR sensor and a camera sensor, characterized in that one end of the sensor connecting arm is pivotally hinge-coupled to the housing, and a fixing means is further formed to allow the other end of the sensor connecting arm to be detachably coupled by magnetic force to the upper side of the housing.

3. Hull section; A touch screen provided on the front of the above-mentioned housing, for executing a platform launcher for driving content for a user and for selecting and controlling the corresponding UI; A plurality of image projection units provided in the above-mentioned housing, which simultaneously or selectively project images for interactive educational content onto the floor and walls; A projector that provides an image signal to the above-mentioned image projection unit; A projector installation part installed on the rear of the above-mentioned housing part to allow the projector to be installed; A LiDAR sensor unit deployed on both sides of the bottom of the above-mentioned housing and installed facing the projection direction of the above-mentioned image projection unit, which recognizes the user's position according to interactive educational content and tracks user motion; A sensor connecting arm that mechanically connects the above-mentioned lidar sensor unit and housing unit and protects the cable inside; A camera sensor unit provided on the front of the above-mentioned housing unit and linked with the signal of the above-mentioned lidar sensor unit, detects user movements to generate and track the user's body skeleton, tracks the hands and feet within the user's body skeleton to identify whether the hands and feet are left or right, and provides linkage information to interactive educational content; A speaker unit provided in the above-mentioned housing and delivering the sound of interactive educational content to the user; An exhaust fan installed in the above-mentioned enclosure to discharge heat from inside the enclosure; and A control unit that controls the projector, image projection unit, LiDAR sensor unit, and camera sensor unit to provide interactive educational content; and An interactive educational content providing device using a LiDAR sensor and a camera sensor, characterized in that a projector lifting device is provided in the projector installation part to automatically raise and lower the projector by means of an electric motor.

4. In Paragraph 3, An interactive educational content providing device using a LiDAR sensor and a camera sensor, characterized by further forming an arm receiving portion that slides in the transverse direction of a sensor connecting arm inside the above-mentioned housing portion, and forming a rail portion so that the sensor connecting arm is cross-received.

5. In Paragraph 4, An interactive educational content providing device using a LiDAR sensor and a camera sensor, characterized by forming a sensor insertion section in which a LiDAR sensor section is inserted at the entrance of the arm receiving section, wherein the front of the sensor insertion section is open, so as to perform motion tracking work with the LiDAR sensor inserted into the arm receiving section.

Images