An interactive holographic blackboard system

Abstract

The embodiment of the present invention provides an interactive holographic blackboard system, including an L-shaped projector, a positioning pointer, a motion capture device and a server, wherein the server is connected to the L-shaped projector and the motion capture device respectively. The L-shaped projector is used to receive the image signal and the switching signal input by the server, and use the image signal to project the image to the preset vertical plane and horizontal plane vector respectively, and is also used to switch the projected content according to the switching signal; the positioning pointer is used to move the position according to the user's operation and transmit the pointer positioning signal; the motion capture device is used to capture the pointer positioning signal and output the pointer positioning signal to the server; the server is used to output the image signal, and determine the real-time position of the positioning pointer according to the pointer positioning signal, and output the switching signal according to the real-time position. Thus, the interaction between the projected image and the user's action is realized, thereby improving the user experience during teaching.

Description

Technical Field

[0001] The invention relates to the technical field of educational supplies, in particular to an interactive holographic blackboard system. Background Art

[0002] Blackboard is an important teaching tool. In fact, blackboard does not refer to a black board. In the traditional sense, it is a hard surface that can be written on repeatedly. Blackboards are generally made of wood or frosted glass. You can write on them with chalk or other specific writing materials. They are mostly black, dark green, white or beige. With the advancement of science and technology, projectors or other electronic display devices have become popular in schools and various training institutions to display teaching information.

[0003] Compared with the interactive features of the blackboard, which is easy to write and erase, when users use a projector for teaching, it is more troublesome to operate the transmitted information, and the projector cannot provide users with interactive features, resulting in a poor user experience during teaching. Summary of the invention

[0004] In view of this, the present invention provides an interactive holographic blackboard system to solve the problem of poor interactivity when using a projector for teaching, thereby causing poor user experience.

[0005] In order to solve the above problems, the present invention discloses an interactive holographic blackboard system, comprising an L-shaped projector, a positioning pointer, a motion capture device and a server, wherein the server is connected to the L-shaped projector and the motion capture device respectively, wherein:

[0006] The L-shaped projector is used to receive the image signal and the switching signal input by the server, and use the image signal to project the image to the preset vertical plane and horizontal plane vector respectively, and is also used to switch the projection content according to the switching signal, and the vertical plane is adjacent to the horizontal plane;

[0007] The positioning pointer is used to be held by a user, moves with the user's operation, and emits a pointer positioning signal;

[0008] The motion capture device is used to capture the pointer positioning signal and output the pointer positioning signal to the server;

[0009] The server is used to output the image signal, determine the real-time position of the positioning pointer according to the pointer positioning signal, and output the switching signal according to the real-time position.

[0010] Optionally, the L-shaped projector includes a first projector and a second projector, wherein:

[0011] The first projector is used to project an image onto the vertical plane;

[0012] The second projector is used for projecting an image onto the horizontal plane.

[0013] Optionally, the positioning pointer includes a pointer body and a first signal transmitting unit, wherein:

[0014] The pointer body is used for being held by a user;

[0015] The first signal transmitting unit is arranged on the pointer body and is used for transmitting a pointer positioning signal.

[0016] Optionally, the first signal transmitting unit is a wireless positioning signal transmitting unit or an optical signal transmitting unit.

[0017] Optionally, it is characterized by further comprising 3D glasses, wherein:

[0018] The stereo glasses are used for allowing users to observe the stereoscopic images projected by the L-shaped projector.

[0019] Optionally, the 3D glasses are shutter-type 3D glasses, wherein:

[0020] The shutter-type stereo glasses are used to switch the light transmittance of the lenses according to the synchronous signal output by the server.

[0021] Optionally, the stereo glasses are provided with a second signal transmitting unit, wherein:

[0022] The second signal transmitting unit is used to transmit a glasses positioning signal, the server is used to determine the user's position according to the glasses positioning signal obtained by the motion capture unit, and output the switching signal according to the user's position and the real-time position of the pointer.

[0023] Optionally, the second signal transmitting unit is a wireless positioning signal transmitting unit or an optical signal transmitting unit.

[0024] It can be seen from the above technical scheme that the present invention provides an interactive holographic blackboard system, including an L-shaped projector, a positioning pointer, a motion capture device and a server, and the server is connected to the L-shaped projector and the motion capture device respectively. The L-shaped projector is used to receive the image signal and the switching signal input by the server, and use the image signal to project the image to the preset vertical plane and horizontal plane vector respectively, and is also used to switch the projection content according to the switching signal, and the vertical plane is adjacent to the horizontal plane; the positioning pointer is used for the user to hold, move the position with the user's operation, and transmit the pointer positioning signal; the motion capture device is used to capture the pointer positioning signal and output the pointer positioning signal to the server; the server is used to output the image signal, and determine the real-time position of the positioning pointer according to the pointer positioning signal, and output the switching signal according to the real-time position. The server outputs the corresponding switching signal to the L-shaped projector for the real-time position of the positioning pointer, and the L-shaped projector switches the projected image under the control of the switching signal, realizing the interaction between the projected image and the user's action, and the user can conveniently switch the image by moving the positioning pointer in the hand, thereby improving the user's experience in teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram of the structure of an interactive holographic blackboard system embodiment provided by the present invention;

[0027] Figure 2 A schematic structural diagram of another interactive holographic blackboard system embodiment provided by the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Embodiment 1

[0030] Figure 1 A schematic structural diagram of an interactive holographic blackboard system embodiment provided by the present invention.

[0031] like Figure 1 As shown, the interactive holographic blackboard system provided in this embodiment is used to provide image support for users to conduct teaching or training, and specifically includes an L-shaped projector 10, a positioning pointer 20 with a positioning element, a motion capture device 30 and a server 40, wherein the server is respectively connected to the L-shaped projector and the motion capture device.

[0032] The server stores corresponding image data, which is connected to the L-shaped projector via a data cable or a wireless connection. The image data can be output to the L-shaped projector in the form of an image signal through the above connection method, so that the projector projects the image onto a predetermined plane.

[0033] The L-shaped projector for projecting images is used to project images onto two planes, one of which is a vertical plane 100 perpendicular to the ground, and the other is a horizontal plane 200 parallel to the ground. The two planes are at an angle of ninety degrees and are arranged adjacent to each other to present the image projected by the L-shaped projector to the user.

[0034] The L-shaped projector includes two independently working projectors, namely a first projector 11 and a second projector 12. The first projector is used to project images onto a vertical plane, and the second projector is used to project images onto a horizontal plane, thereby creating an image space with a panoramic effect for users.

[0035] The positioning pointer is used for the user to hold and move under the user's guidance, and the positioning pointer is used to emit positioning signals for the motion capture device to perform motion capture. Specifically, the positioning pointer includes a pointer body for the user to hold and a first signal emitting unit 21 arranged on the pointer body.

[0036] The first signal transmitting unit is used to transmit a pointer positioning signal for the motion capture device to position the pointer. The first signal transmitting unit can be a wireless positioning signal transmitting unit for transmitting a wireless positioning signal, or a light transmitting unit for transmitting a light positioning signal. As a light transmitting unit, it is used to transmit a flashing signal according to a certain rule and period, and the flashing signal constitutes a certain code.

[0037] The motion capture device is used to obtain the pointer positioning signal emitted by the positioning pointer and send the pointer positioning signal to the server.

[0038] Motion capture, referred to as Mocap, involves data that can be directly understood and processed by computers, such as size measurement, positioning and orientation of objects in physical space.

[0039] With the rapid development of computer software and hardware technology and the increasing requirements for animation production, motion capture has entered the practical stage in developed countries. Many manufacturers have launched a variety of commercial motion capture equipment, such as MotionAnalysis, Polhemus, Sega Interactive, MAC, X-Ist, FilmBox, etc., which have been successfully used in virtual reality, games, ergonomics research, simulation training, biomechanics research and many other aspects.

[0040] From a technical perspective, the essence of motion capture is to measure, track, and record the motion trajectory of an object in three-dimensional space. A typical motion capture device generally consists of the following parts:

[0041] Sensor, the so-called sensor is a tracking device fixed to a specific part of a moving object. It will provide the MotionCapture system with the position information of the moving object. Generally, the number of trackers will be determined by the level of capture detail.

[0042] Signal capture device. This device varies depending on the type of motion capture system. It is responsible for capturing position signals. For mechanical systems, it is a circuit board that captures electrical signals. For optical motion capture systems, it is a high-resolution infrared camera.

[0043] Data transmission equipment. Motion capture systems, especially those that require real-time effects, need to quickly and accurately transmit large amounts of motion data from the signal capture device to the computer system for processing, and data transmission equipment is used to complete this task.

[0044] Data processing equipment. The data captured by the Motion capture system needs to be corrected and processed, and then combined with the 3D model to complete the work of computer animation production. This requires us to apply data processing software or hardware to complete this work. Whether it is software or hardware, they all rely on the computer's high-speed data computing ability to complete data processing, so that the 3D model can move truly and naturally. There are several types of motion capture, including mechanical motion capture, acoustic motion capture, electromagnetic motion capture, optical motion capture, and inertial navigation motion capture.

[0045] Mechanical motion capture relies on mechanical devices to track and measure motion trajectories. A typical system consists of multiple joints and rigid links. An angle sensor is installed in the rotatable joint to measure the change in the rotation angle of the joint. When the device moves, the position and motion trajectory of the end point of the rod in space can be obtained based on the angle change measured by the angle sensor and the length of the link. In fact, the motion trajectory of any point on the device can be calculated, and the rigid link can also be replaced with a telescopic rod with variable length, and its length change can be measured with a displacement sensor.

[0046] An early mechanical motion capture device uses joints and connecting rods with angle sensors to form a "digital model with adjustable posture", whose shape can simulate the human body, or other animals or objects. The user can adjust the posture of the model according to the needs of the plot, and then lock it. The angle sensor measures and records the rotation angle of the joint. Based on these angles and the mechanical dimensions of the model, the posture of the model can be calculated, and these posture data can be transmitted to the animation software so that the character model in it can also make the same posture. This is an early motion capture device, but it still has a certain market until now. Foreign countries have given this device a very vivid name: "monkey".

[0047] One application form of mechanical motion capture is to connect the moving object to be captured to a mechanical structure. The movement of the object drives the mechanical device, which is then recorded in real time by the sensor.

[0048] The advantages of this method are low cost, high accuracy, real-time measurement, and the ability to allow multiple characters to perform simultaneously. However, its disadvantages are also very obvious. The main disadvantage is that it is very inconvenient to use, and the mechanical structure greatly hinders and restricts the performer's movements. The "monkey" is difficult to use for real-time capture of continuous movements, and the operator needs to constantly adjust the "monkey"'s posture according to the plot requirements, which is very troublesome. It is mainly used for static modeling capture and key frame determination.

[0049] Commonly used acoustic motion capture devices consist of a transmitter, a receiver, and a processing unit. The transmitter is a fixed ultrasonic generator, and the receiver is generally composed of three ultrasonic probes arranged in a triangle. By measuring the time or phase difference of the sound wave from the transmitter to the receiver, the system can calculate and determine the position and direction of the receiver.

[0050] This type of device has a low cost, but has a large delay and lag in capturing motion, poor real-time performance, and generally low accuracy. There cannot be large obstructions between the sound source and the receiver, and it is more susceptible to interference from noise and multiple reflections. Since the speed of sound waves in the air is related to air pressure, humidity, and temperature, corresponding compensation must also be made in the algorithm.

[0051] Electromagnetic motion capture system is a commonly used motion capture device. It is generally composed of a transmitter, a receiving sensor and a data processing unit. The transmitter generates an electromagnetic field distributed in space according to a certain time and space law; the receiving sensors (usually 10 to 20) are placed at key positions on the performer's body, move in the electromagnetic field with the performer's movements, and are connected to the data processing unit through cables or wirelessly.

[0052] When the performer performs in the electromagnetic field, the receiving sensor transmits the received signal to the processing unit through the cable, and the spatial position and direction of each sensor can be solved based on these signals. Polhemus and Ascension are both famous for producing electromagnetic motion capture equipment. The sampling rate of such systems is generally 15 to 120 times per second (depending on the number of models and sensors). In order to eliminate jitter and interference, the sampling rate is generally below 15Hz. For some high-speed sports, such as boxing and basketball games, this sampling rate cannot meet the requirements. The advantage of electromagnetic motion capture is that it records six-dimensional information, that is, it can not only obtain spatial position, but also direction information, which is very valuable for some special applications. Secondly, it is fast and real-time. When the performer performs, the character model in the animation system can react at the same time, which is convenient for rehearsal, adjustment and modification. The calibration of the device is relatively simple, the technology is relatively mature, the robustness is good, and the cost is relatively low.

[0053] Its disadvantage is that it has strict requirements on the environment. There must be no metal objects near the performance venue, otherwise it will cause electromagnetic field distortion and affect the accuracy. The system's permissible performance range is smaller than that of the optical type, especially the cable has a greater restriction on the performer's movement, and it is not suitable for more intense sports and performances.

[0054] Optical motion capture completes the task of motion capture by monitoring and tracking specific light spots on the target. Most common optical motion capture is based on computer vision principles. Theoretically, for a point in space, as long as it can be seen by two cameras at the same time, the position of the point in space at this moment can be determined based on the images and camera parameters taken by the two cameras at the same time. When the camera continuously shoots at a high enough rate, the motion trajectory of the point can be obtained from the image sequence.

[0055] A typical optical motion capture system usually uses 6 to 8 cameras arranged around the performance venue. The overlapping area of ​​the field of view of these cameras is the range of the performer's movements. For ease of processing, performers are usually required to wear monochrome clothing and attach some special signs or luminous points, called "Markers", to key parts of the body, such as joints, hips, elbows, wrists, etc. The visual system will recognize and process these signs. After the system is calibrated, the camera continuously captures the performer's movements and saves the image sequence, which is then analyzed and processed to identify the marker points and calculate their spatial position at each moment, thereby obtaining their motion trajectory. In order to obtain an accurate motion trajectory, the camera should have a high shooting rate, generally reaching more than 60 frames per second.

[0056] If markers are attached to the key points of the performer's facial expressions, expression capture can be achieved. Most expression capture methods use optical methods.

[0057] Some optical motion capture systems do not rely on markers as identification marks, for example, they extract motion information based on the silhouette of the target, or use a gridded background to simplify the processing process. Researchers are studying the use of image recognition and analysis technology without relying on markers, using the visual system to directly identify the key parts of the performer's body and measure its motion trajectory. It is estimated that this technology will soon be put into practical use.

[0058] The advantages of optical motion capture are that the performer has a large range of activities, there is no restriction of cables or mechanical devices, the performer can perform freely, and it is very convenient to use. Its sampling rate is relatively high, which can meet the needs of most high-speed motion measurements. The number of markers can be purchased and added according to the actual application, which is convenient for system expansion.

[0059] The disadvantage of this method is that the system is expensive. It can capture real-time motion, but the post-processing (including marker identification, tracking, and calculation of spatial coordinates) is relatively heavy and is therefore suitable for scientific research applications.

[0060] The inertial navigation sensor AHRS (Attitude Reference System) and IMU (Inertial Measurement Unit) are used to measure the acceleration, orientation, tilt angle and other characteristics of the performer's movement. It is not affected by environmental interference and is not afraid of occlusion. It has high capture accuracy and high sampling speed, reaching 1,000 times per second or more. Due to the use of highly integrated chips and modules, it is small in size, light in weight and cost-effective. The inertial navigation sensor is worn on the performer's head, or a data suit is worn through 17 sensors, which is connected to the host through USB cable, Bluetooth, 2.4GzhDSSS wireless, etc., and can track head and whole body movements respectively, and display complete movements in real time.

[0061] After receiving the pointer positioning signal, the server can calculate the real-time position of the pointer based on the preset spatial coordinate system and the position of the motion capture device in the spatial coordinate system. After obtaining the real-time position, it determines whether the spatial coordinates corresponding to the real-time position reflect the user's need to switch the image. When it is determined that switching is required, a switching signal is output to the L-shaped projector.

[0062] When the L-shaped projector is performing image projection according to the image signal output by the server, upon receiving a switching signal, the projected image is switched, thereby realizing action interaction between the image and the user.

[0063] It can be seen from the above technical scheme that this embodiment provides an interactive holographic blackboard system, including an L-shaped projector, a positioning pointer, a motion capture device and a server, and the server is connected to the L-shaped projector and the motion capture device respectively. The L-shaped projector is used to receive the image signal and the switching signal input by the server, and use the image signal to project the image to the preset vertical plane and horizontal plane vector respectively, and is also used to switch the projection content according to the switching signal, and the vertical plane is adjacent to the horizontal plane; the positioning pointer is used for the user to hold, move the position with the user's operation, and transmit the pointer positioning signal; the motion capture device is used to capture the pointer positioning signal and output the pointer positioning signal to the server; the server is used to output the image signal, and determine the real-time position of the positioning pointer according to the pointer positioning signal, and output the switching signal according to the real-time position. The server outputs the corresponding switching signal to the L-shaped projector for the real-time position of the positioning pointer, and the L-shaped projector switches the projected image under the control of the switching signal, realizing the interaction between the projected image and the user's action, and the user can conveniently switch the image by moving the positioning pointer in the hand, thereby improving the user's experience in teaching.

[0064] Embodiment 2

[0065] Figure 2 A schematic structural diagram of another interactive holographic blackboard system embodiment provided by the present invention.

[0066] The interactive blackboard system provided in this embodiment is based on the previous embodiment and is equipped with stereo glasses 50 for users to wear.

[0067] After the user wears the stereoscopic glasses, he can watch the stereoscopic images projected by the L-shaped projector in a stereoscopic manner, thereby further improving the user's experience.

[0068] The 3D glasses are preferably shutter-type 3D glasses, which enable the user's left and right eyes to view different image frames by quickly switching the light transmittance of the lenses. When there is a slight parallax between adjacent image frames, the time-difference images obtained by the two eyes allow the user to experience a 3D image. The switching of the shutter-type 3D glasses is synchronized with the image transmitted by the L-shaped projector.

[0069] The shutter-type stereo glasses are also provided with a second signal transmitting unit 51, which is used to transmit a glasses positioning signal for the motion capture device to capture the user's head motion. The second signal transmitting unit is also a wireless positioning signal transmitting unit for transmitting a wireless positioning signal, and can also be a light transmitting unit for transmitting an optical positioning signal. As a light transmitting unit, it is used to transmit a flashing signal according to a certain rule and period, and the flashing signal constitutes a certain code.

[0070] The motion capture device is used to obtain the glasses positioning signal emitted by the second signal transmitting unit and output it to the server. After receiving the glasses positioning signal, the server calculates the position of the glasses, i.e. the user's head, according to the positioning principle of the positioning pointer, and then outputs a corresponding switching signal according to the position of the glasses and the real-time position of the positioning pointer. In this way, the user can switch the image by combining his own position with the position of the positioning pointer, thus enriching the route.

[0071] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0072] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0073] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0074] The technical solution provided by the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An interactive holographic blackboard system, characterized in that: The system comprises an L-shaped projector, a positioning pointer, a motion capture device and a server, wherein the positioning pointer comprises a first signal transmitting unit, and the server is connected to the L-shaped projector and the motion capture device respectively, wherein: The L-shaped projector is used to receive the image signal and the switching signal input by the server, and use the image signal to project the image to the preset vertical plane and horizontal plane vector respectively, and is also used to switch the projection content according to the switching signal, and the vertical plane is adjacent to the horizontal plane; The positioning pointer is used for being held by a user, and moves with the user's operation, and transmits a pointer positioning signal through the first signal transmitting unit; the pointer positioning signal is a wireless positioning signal or an optical positioning signal; The motion capture device is used to capture the pointer positioning signal and output the pointer positioning signal to the server; The server is used to output the image signal, determine the real-time position of the positioning pointer according to the pointer positioning signal, a preset spatial coordinate system and the position of the motion capture device in the spatial coordinate system, and output the switching signal according to the real-time position.

2. The interactive holographic blackboard system according to claim 1, characterized in that: The L-shaped projector comprises a first projector and a second projector, wherein: The first projector is used to project an image onto the vertical plane; The second projector is used for projecting an image onto the horizontal plane.

3. The interactive holographic blackboard system according to claim 1, characterized in that: The positioning pointer comprises a pointer body, wherein: The pointer body is used for being held by a user; The first signal transmitting unit is arranged on the pointer body.

4. The interactive holographic blackboard system according to claim 3, characterized in that: The first signal transmitting unit is a wireless positioning signal transmitting unit or an optical signal transmitting unit.

5. The interactive holographic blackboard system according to any one of claims 1 to 4, characterized in that: Also included are stereo glasses, wherein: The stereo glasses are used for allowing users to observe the stereoscopic images projected by the L-shaped projector.

6. The interactive holographic blackboard system according to claim 5, characterized in that: The 3D glasses are shutter-type 3D glasses, wherein: The shutter-type 3D glasses are used to switch the light transmittance of the lenses according to the synchronization signal output by the server.

7. The interactive holographic blackboard system according to claim 5, characterized in that: The stereo glasses are provided with a second signal transmitting unit, wherein: The second signal transmitting unit is used to transmit a glasses positioning signal, the server is used to determine the user's position according to the glasses positioning signal obtained by the motion capture device, and output the switching signal according to the user's position and the real-time position of the pointer.

8. The interactive holographic blackboard system according to claim 7, characterized in that: The second signal transmitting unit is a wireless positioning signal transmitting unit or an optical signal transmitting unit.

Images