Linkage interaction system based on special-shaped projection

Abstract

The invention provides a linkage interaction system based on special-shaped projection, and relates to the technical field of projection. The linkage interaction system based on special-shaped projection comprises a touch all-in-one machine, a plurality of projectors, a three-dimensional automobile model and a multimedia server, a fusion program is arranged in the multimedia server, and the fusion program is sent to a specified projector to perform Mapping projection display and establish a three-dimensional model. The graffiti image is projected to the surface of the automobile, so that unique visual experience is provided. The system is combined with an interaction technology, so that the audience interactively experiences graffiti creation, and immersive personalized experience is created. The Mapping technology is utilized, the system creates a super-reality artistic effect on the surface of the automobile, shocking visual impact is brought, combination of the projection fusion technology and the automobile model is achieved, and audiences can easily enjoy the design pleasure.

Description

Technical Field

[0001] The present invention relates to the field of projection technology, in particular to a linkage interaction system based on special-shaped projection. Background Art

[0002] In recent years, with the continuous progress and innovation of science and technology, lights have emerged in various places, attracting the attention and concern of countless people. In these light shows, 3D Mapping technology has gradually become the focus of heated discussion. 3D Mapping, also known as 3D projection technology, Facade Mapping, three-dimensional building projection, three-dimensional building art, and three-dimensional vocal image projection, is a magical technology that can break through the limitations of traditional physical space. Through advanced projection methods, it can turn the surface of any object into a vivid animation, realizing the seamless integration of objects and animations, thereby creating a shocking effect with great visual impact.

[0003] However, although 3D Mapping technology has been widely used in the field of outdoor projection, it is currently mainly concentrated on a certain plane of the building for display. This display method is mostly based on content mapping, and the audience can only passively accept it during the viewing process, and cannot participate in the creation process. Therefore, although the visual effect is amazing, its interactivity is relatively low, and the audience's sense of participation and experience is also limited to a certain extent. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the deficiencies in the prior art, the present invention provides a linkage interactive system based on special-shaped projections, which solves the problem that the interactivity is relatively low and the audience's sense of participation and experience are also subject to certain limitations.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a linkage interactive system based on special-shaped projection, including a touch all-in-one machine, multiple projectors, a three-dimensional car model and a multimedia server, wherein a fusion program is provided in the multimedia server, and the fusion program is sent to a designated projector for mapping projection display and establishment of a three-dimensional model;

[0008] The Mapping projection fusion principle in the fusion program includes three-dimensional model and UV expansion, image content generation, real-time inverse calculation technology, edge fusion technology, geometric correction technology, color balance correction and distortion correction;

[0009] The three-dimensional model building includes a model coordinate system, a world coordinate system, an observation coordinate system, a projection plane and a device coordinate system.

[0010] Preferably, the model coordinate system is connected to a model transformation, the world coordinate system is connected to an observation transformation, the observation coordinate system is connected to three-dimensional clipping with respect to a viewing frustum, the model transformation is connected to an observation transformation, the observation transformation is connected to the three-dimensional clipping with respect to a viewing frustum, the three-dimensional clipping with respect to a viewing frustum is connected to a projection, the projection plane is connected to a window-to-viewport transformation, the device coordinate system is connected to a display, the projection is connected to the window-to-viewport transformation, and the window-to-viewport transformation is connected to the display.

[0011] Preferably, the three-dimensional model and UV unfolding include stereo Mapping projection fusion software, the core of which is to generate image content based on the UV unfolding or perspective projection principle of the three-dimensional model, which means that the software processes the three-dimensional model and unfolds the two-dimensional image or animation on the surface of the model so that they can be accurately mapped to the actual physical surface.

[0012] Preferably, the generation of the image content includes computationally generating image content of a suitable three-dimensional shape, involving simulating the position, orientation and angle of the projector in three-dimensional space, and processing the image to fit the geometry of the projected object.

[0013] Preferably, the real-time back-calculation technology includes mapping the content onto a real model using back-calculation technology, which involves dynamically adjusting the image content according to the actual position and angle of the projector to ensure that the image can accurately fit the surface of the projected object, even if the object is an irregular shape.

[0014] Preferably, the edge fusion technology includes an edge fusion technology for eliminating uneven brightness and matching effects in overlapping areas of multiple projectors, and the processing method is to multiply each pixel of the single-sided projection in the fusion zone area by a certain value so that when it is superimposed on the corresponding pixel in another image, a corresponding expected pixel value can be obtained;

[0015]

[0016] Among them, xx takes the value of 0 and 1, representing the left edge and right edge of the fusion band respectively, and ww and hh are the pixel width of the fusion band and the single-side image respectively.

[0017] Preferably, in the geometric correction technology, the geometric correction is to eliminate the distortion effect of the projection system, and the image needs to be registered, including translation, rigidity, affine, projection and other transformations. The geometric correction needs to find the transformation relationship between the projector and the projection screen, which can be achieved by using the camera as an intermediate variable.

[0018] P′=A·P+B

[0019] Among them, PP is the pixel coordinate of the point in the camera space, P′P′ is the pixel coordinate of the point in the projection screen space, AA and BB are the transformation matrix and displacement vector, which need to be determined by designing a fusion solution based on on-site conditions and requirements.

[0020] Preferably, the color balance is used to correct the color cast, oversaturation, undersaturation, etc. of the image, and the gamma correction is a nonlinear transfer function used to compensate for the nonlinear characteristics of the display device;

[0021] L∝V γ

[0022] Wherein, L, V and γ represent the power exponent of brightness, input signal voltage and Gamma value respectively.

[0023] Preferably, in the distortion correction, since image deformation and distortion may occur when projecting onto a non-planar object, the software must have a distortion correction function to ensure that the correct image display effect is obtained on the non-planar screen. The distortion correction is used to correct lens distortion, including radial distortion and tangential distortion.

[0024] x dr =x(1+k1r 2 +k2r 4 +k3r 6 )

[0025] y dr =y(1+k1r 2 +k2r 4 +k3r 6 )

[0026] Among them, (x, y) is the ideal undistorted coordinate, (x dr ,y dr ) is the coordinate of the pixel point of the distorted image, r is the distance from the center of the lens, and k1, k2, k3 are the distortion coefficients.

[0027] Working principle: The audience selects or customizes the painting on the car model on the all-in-one machine, transmits it to the fusion program in the multimedia server, and then sends it to the designated projector for mapping projection display. Through the parallel projection and perspective projection methods in computer graphics, the process includes framing and observing the projected object to establish the corresponding three-dimensional model, and establish coordinates for factors such as the position, direction and angle of the projector, and then finally realize it through projection transformation. The exhibit uses a variety of technologies including projection technology and high-precision fusion technology to project the picture onto the car model to create a surreal scene, which gives people a strong visual impact.

[0028] (III) Beneficial effects

[0029] The present invention provides a linkage interaction system based on special-shaped projection, which has the following beneficial effects:

[0030] The present invention introduces an innovative special-shaped projection linkage interactive system, which gives the car an unprecedented visual experience by projecting the designed graffiti image onto the surface of the car. The system integrates advanced interactive technology, allowing the audience to directly interact with the car model and experience the graffiti creation process firsthand, thereby creating a more immersive and personalized experience. In addition, with the help of Mapping technology, the system can create a surreal phantom art effect on the surface of the car, bringing a shocking visual impact to the audience. It provides a unique three-dimensional dynamic experience, subverts traditional audio-visual products, presents art and design in a brand-new way, and effectively attracts children and young people to participate in creation. Through the combination of projection fusion technology and car models, the audience can enjoy the fun of designing car shapes without any professional skills. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is an overall schematic diagram of the exhibit of the present invention;

[0032] Figure 2 A flow chart for the audience operation of the present invention;

[0033] Figure 3 A three-dimensional graphical display flow chart of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example:

[0036] like Figure 1 As shown, an embodiment of the present invention provides a linkage interactive system based on special-shaped projection, including a touch integrated machine, multiple projectors, a three-dimensional car model and a multimedia server. A fusion program is set in the multimedia server, and the fusion program is sent to a designated projector for Mapping projection display and establishment of a three-dimensional model. The Mapping projection fusion principle in the fusion program includes three-dimensional model and UV expansion, image content generation, real-time back calculation technology, edge fusion technology, geometric correction technology, color balance correction and distortion correction;

[0037] Specifically, in the above specific embodiment, car graffiti mapping creates an impressive visual experience by projecting specific animations and visual effects onto the surface of the car. The following are some technical effects achieved by the present invention:

[0038] Three-dimensional display: Using 3D mapping technology, complex three-dimensional models and animations can be displayed on the car surface, providing richer visual content.

[0039] Dynamic effects: Dynamic effects such as car color change and movement can be achieved, and the interactivity and fun of the display can be enhanced through animation and real-time rendering technology.

[0040] Interactive experience: Combined with the touch screen, users can interact with the 3D mapped car model and select different car colors or configurations, which is highly interactive.

[0041] Enhanced reality: 3D mapping technology can enhance the realism of car models, and improve lighting effects and color reproduction through mirror reflections, HDR (high dynamic range) technology, etc.

[0042] Visual impact: 3D mapping technology is often used in architectural projection, which brings strong visual impact to users by integrating animation with the car surface.

[0043] Data integration: 3D mapping technology can be integrated with automotive test and measurement data to achieve conversion from real scenes to virtual reality, providing support for automotive development.

[0044] like Figure 2 and 3 As shown, establishing a three-dimensional model includes a model coordinate system, a world coordinate system, an observation coordinate system, a projection plane and a device coordinate system, the model coordinate system is connected to a model transformation, the world coordinate system is connected to an observation transformation, the observation coordinate system is connected to a three-dimensional clipping related to a viewing frustum, the model transformation is connected to an observation transformation, the observation transformation is connected to a three-dimensional clipping related to a viewing frustum, the three-dimensional clipping related to a viewing frustum is connected to a projection, the projection plane is connected to a window-to-viewport transformation, the device coordinate system is connected to a display, the projection is connected to a window-to-viewport transformation, and the window-to-viewport transformation is connected to a display;

[0045] Specifically, in the above specific embodiment, the audience can touch the screen of the all-in-one machine, select a preset pattern or draw on the screen by themselves, and transmit the image information to the fusion program in the multimedia server. After receiving this information, the fusion program will send it to a specific projector for mapping projection display. In this process, the parallel projection and perspective projection techniques in computer graphics are used. First, the projected object is framed and observed, and then the corresponding three-dimensional model is established. Then, according to factors such as the position, direction and angle of the projector projection, the corresponding coordinate system is established. Finally, through projection transformation, the image is finally projected onto the car model to create a surreal scene.

[0046] To achieve this process, the exhibit uses a variety of advanced technologies, including high-precision projection technology and high-precision fusion technology. The combination of these technologies enables the image to be accurately projected onto the car model, creating amazing visual effects. When watching, the audience will feel a strong visual impact, as if they are in a surreal world.

[0047] Specifically, the present invention relates to a 3D model and UV unfolding technology, specifically including stereo mapping projection fusion software. The core function of the software is based on the UV unfolding or perspective projection principle of the 3D model, which is used to generate image content. This means that the software can process the 3D model and unfold the 2D image or animation on the model surface to ensure that they can be accurately mapped to the actual physical surface.

[0048] The image content generation process involves calculations to produce image content that fits the 3D shape. This includes simulating the position, orientation and angle of the projector in 3D space and adjusting the image to fit the geometry of the object being projected.

[0049] Real-time inverse calculation technology uses inverse calculation technology to map content onto a real model. It dynamically adjusts the image content according to the actual position and angle of the projector to ensure that the image can accurately fit the irregularly shaped surface of the projection object.

[0050] Edge blending technology is used to eliminate uneven brightness and matching effects in overlapping areas of multiple projectors. The processing method is to multiply each pixel of the single-sided projection in the fusion zone area by a specific value so that when it is superimposed on the corresponding pixel of another image, the expected pixel value can be obtained. The specific calculation formula is as follows:

[0051]

[0052] Among them, x takes the value of 0 and 1 to represent the left edge and right edge of the fusion band, respectively, and w and h are the pixel width of the fusion band and the single-side image, respectively.

[0053] Geometric correction technology aims to eliminate the distortion effect of the projection system. This requires the registration of the image, including translation, rigidity, affine, projection and other transformations. Geometric correction requires determining the transformation relationship between the projector and the projection screen, which can be achieved by using the camera as an intermediate variable. The specific transformation formula is:

[0054] P′=A·P+B

[0055] Among them, P is the pixel coordinate of the point in the camera space, P′ is the pixel coordinate of the point in the projection screen space, A and B are the transformation matrix and displacement vector, which need to be determined by designing a fusion solution based on the on-site conditions and needs.

[0056] Color balance technology is used to correct image color cast, oversaturation, undersaturation and other problems. Gamma correction is a nonlinear transfer function used to compensate for the nonlinear characteristics of display devices. Its formula is:

[0057] L∝V γ

[0058] Wherein, L, V and γ represent the power exponent of brightness, input signal voltage and Gamma value respectively.

[0059] In terms of distortion correction, since projection onto non-planar objects may cause image deformation and distortion, the software must have a distortion correction function to ensure the correct image display effect on non-planar screens. Distortion correction is used to correct lens distortion, including radial distortion and tangential distortion. The formula is:

[0060] x dr =x(1+k1r 2 +k2r 4 +k3r 6 )

[0061] y dr =y(1+k1r 2 +k2r 4 +k3r 6 )

[0062] Among them, (x, y) is the ideal undistorted coordinate, (x dr ,y dr ) is the coordinate of the pixel point of the distorted image, r is the distance from the center of the lens, and k1, k2, k3 are the distortion coefficients.

[0063] Finally, in the present invention, a multi-projector synchronization control technology was successfully developed, which ensures the synchronous operation of all projectors, effectively avoids screen tearing or delay, and is of decisive significance for building a seamless stereoscopic visual experience. Through the self-developed advanced edge fusion algorithm, pixel-level fine adjustment of the edges of the output images of different projectors is achieved, thereby smoothly connecting the images, eliminating the hard edge effect, and ensuring a smooth transition of the image. In terms of model surface mapping technology, through detailed 3D scanning and analysis, the model surface mapping is accurately created, ensuring that the projection content can be accurately mapped to every curved surface of the model.

[0064] The present invention also involves a dynamic generation and rendering technology of projection content, which is based on the 3D data of the model and can generate and render image content suitable for projection in real time. In addition, the color and brightness uniformity technology adjusts the brightness through a software algorithm to ensure that all projectors output consistent color and brightness on the model surface, thereby avoiding visual inconsistency caused by differences in device performance.

[0065] The present invention further integrates a user interaction feedback system and combines it with a touch all-in-one machine to develop a system that can respond to user input in real time and adjust the projected content to achieve a truly interactive experience. In terms of software and hardware integration technology, the present invention integrates all the technologies used into a unified system, including a projector, a computer, a touch all-in-one machine, and related software to achieve a seamless three-dimensional projection interactive experience. Finally, the projection design technology simulates the effect and designs the projection position in combination with the characteristics of the device to ensure that the projector is stably and accurately positioned at a suitable position above the model to achieve the best projection effect.

[0066] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A linkage interactive system based on special-shaped projection, comprising a touch all-in-one machine, multiple projectors, a three-dimensional car model and a multimedia server, characterized in that: The multimedia server is provided with a fusion program, and the fusion program is sent to a designated projector for mapping projection display and building a three-dimensional model; The Mapping projection fusion principle in the fusion program includes three-dimensional model and UV expansion, image content generation, real-time inverse calculation technology, edge fusion technology, geometric correction technology, color balance correction and distortion correction; The three-dimensional model building includes a model coordinate system, a world coordinate system, an observation coordinate system, a projection plane and a device coordinate system.

2. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: The model coordinate system is connected to the model transformation, the world coordinate system is connected to the observation transformation, the observation coordinate system is connected to the three-dimensional clipping with respect to the viewing frustum, the model transformation is connected to the observation transformation, the observation transformation is connected to the three-dimensional clipping with respect to the viewing frustum, the three-dimensional clipping with respect to the viewing frustum is connected to the projection, the projection plane is connected to the window-to-viewport transformation, the device coordinate system is connected to the display, the projection is connected to the window-to-viewport transformation, and the window-to-viewport transformation is connected to the display.

3. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: The three-dimensional model and UV unfolding include stereoscopic Mapping projection fusion software, the core of which is to generate image content based on the UV unfolding or perspective projection principle of the three-dimensional model, which means that the software processes the three-dimensional model and unfolds the two-dimensional image or animation on the surface of the model so that they can be accurately mapped to the actual physical surface.

4. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: The generation of the image content includes generating image content of a suitable three-dimensional shape by calculation, which involves simulating the position, direction and angle of the projector in three-dimensional space, and processing the image to adapt to the geometric shape of the projected object.

5. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: The real-time inverse calculation technology includes mapping the content onto a real model using inverse calculation technology, which involves dynamically adjusting the image content according to the actual position and angle of the projector to ensure that the image can accurately fit the surface of the projected object, even if the object is an irregular shape.

6. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: The edge fusion technology includes edge fusion technology used to eliminate uneven brightness and matching effects in overlapping areas of multiple projectors, and the processing method is to multiply each pixel of the single-sided projection in the fusion zone area by a certain value so that when it is superimposed on the corresponding pixel in another image, the corresponding expected pixel value can be obtained; Among them, xx takes the value of 0 and 1, representing the left edge and right edge of the fusion band respectively, and ww and hh are the pixel width of the fusion band and the single-side image respectively.

7. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: In the geometric correction technology, the geometric correction is to eliminate the distortion effect of the projection system, and the image needs to be registered, including translation, rigidity, affine, projection and other transformations. The geometric correction needs to find the transformation relationship between the projector and the projection screen, which can be achieved by using the camera as an intermediate variable. P′=A·P+B Among them, PP is the pixel coordinate of the point in the camera space, P′P′ is the pixel coordinate of the point in the projection screen space, AA and BB are the transformation matrix and displacement vector, which need to be determined by designing a fusion solution based on on-site conditions and requirements.

8. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: The color balance is used to correct the color cast, oversaturation, undersaturation, etc. of the image. Gamma correction is a nonlinear transfer function used to compensate for the nonlinear characteristics of the display device. L∝V γ Wherein, L, V and γ represent the power exponent of brightness, input signal voltage and Gamma value respectively.

9. The linkage interactive system based on special-shaped projection according to claim 1, characterized in that: In the distortion correction, since the image may be deformed when projected onto a non-planar object, the software must have a distortion correction function to ensure that the correct image display effect is obtained on the non-planar screen. Distortion correction is used to correct lens distortion, including radial distortion and tangential distortion; x dr =x(1+k1r 2 +k2r 4 +k3r 6 ) and dr =y(1+k1r 2 +k2r 4 +k3r 6 ) Among them, (x, y) is the ideal undistorted coordinate, (x dr ,y dr ) is the coordinate of the pixel point of the distorted image, r is the distance from the center of the lens, and k1, k2, k3 are the distortion coefficients.

Images