Immersive interactive deduction space construction system
Through the immersive interactive deduction space construction system in an open public space, the deduction content is dynamically adjusted using lidar and collaborative control modules, the problem of insufficient fixedness and interactivity of visual space construction in the existing technology is solved, and the problem of insufficient fixedness and interaction of visual space construction in the existing technology is realized. In-depth participation and rich experience of the audience.
Patent Information
- Application Number
- CN202411121688.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-12-03
AI Technical Summary
The lighting projection mode of the existing stage visual space construction system is fixed, and it cannot adapt to the interpretation requirements of different spatial scenes, resulting in limited visual range of the audience, lack of interactivity, and in-depth sense of participation and experience.
Using an immersive interactive deduction space construction system including sensing modules, control modules and deduction modules, collect environmental information through lidar devices, signal processing and environmental analysis generate real-time position and attitude information, and coordinate the control module to adjust the projection and lighting device to realize Dynamic interpretation content updates and changes allow viewers to move freely in open public spaces.
It realizes dynamic adjustment and interactivity of the deductive space, enhances the audience's sense of immersion and participation, adapts to the needs of different spatial scenarios, provides a richer experience, and reduces physical occupation, saving materials and energy.
Smart Images

Figure CN119065272A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of visual space construction, and more specifically, relates to an immersive interactive interpretation space construction system. Background Art
[0002] In the existing stage visual space construction, lighting, shadows and projections are used to allow the audience to gain a real sense of virtual influence. However, in the existing space construction system, interpretation methods such as lighting and projection can often only be operated according to preset modes. The operation mode is relatively fixed and limited by the size of the stage. Audiences in different directions can get different viewing experiences. There are always audiences whose visual range is limited and who cannot get a good viewing experience. It is difficult to adapt to the interpretation requirements of different space scenes. There is a lack of interaction between the audience and the system, which makes it impossible for the audience to get a deeper sense of participation and experience. Summary of the invention
[0003] The technical problem to be solved by the present invention is to provide an immersive interactive interpretation space construction system, which can create an immersive interpretation space scene for the audience. The interactive interpretation content can be updated and changed at any time, making the interpretation space fresh; the interpretation space can be adjusted according to the actual space and has strong applicability.
[0004] The invention discloses an immersive interactive interpretation space construction system, wherein the interpretation space is an open public space, the audience can move freely in the interpretation space, and the construction system is placed in the interpretation space. The construction system includes a sensor module, a control module and an interpretation module.
[0005] The sensor module includes a laser radar device. The laser radar device is distributed in the interpretation space. After receiving the main program monitoring command, it collects environmental information in the interpretation space by rotating and / or scanning, forms a data signal and transmits it to the control module.
[0006] The control module includes a signal processing device, an environmental analysis device and a collaborative control device. The signal processing device is used to receive data signals and convert them into point cloud data. The environmental analysis device performs preprocessing, feature extraction and matching positioning operations based on the original point cloud data, calculates the position and posture information of the target audience, and compares the real-time position and posture information with the preset scene positioning in the virtual environment to generate an action mapping. The collaborative control device is used to adjust the deduction module according to the mapping rules.
[0007] The performance module includes audio device, projection device and lighting device, and works together according to the instructions issued by the control module.
[0008] As a further improvement of the present invention, the data signal includes distance and angle information in the environment. The point cloud data includes reflection information of the audience and the surface of the object. Communication is performed by means of the tuio protocol.
[0009] As a further improvement of the present invention, the preprocessing operation includes removing noise, filtering and smoothing, and scaling the point cloud data.
[0010] As a further improvement of the present invention, effective information for feature extraction in the point cloud data after the preprocessing operation includes edge contours and foot points of the audience.
[0011] As a further improvement of the present invention, in the positioning and matching operation, feature matching is performed between the point cloud data of the current frame and the reference frame of the known position. The calculation is performed using an iterative nearest point projection algorithm or a variant algorithm thereof through the NDI protocol.
[0012] As a further improvement of the present invention, according to the actions of the audience and the mapping rules of the system, the projection content of the projection device is rendered in real time by the cooperative control device. The projection content includes the shape, color, window size, position and motion trajectory of the projection image.
[0013] As a further improvement of the present invention, according to the audience's actions and the mapping rules of the system, the lighting device is controlled by the cooperative control device to adjust the lighting effect changes and mechanical start and stop. The sound device is controlled by the cooperative control device to adjust the loudness, scale and speed of the sound effect. According to the scale and layout of the performance space, the sensor module, the control module and the performance module are installed in the performance space.
[0014] As a further improvement of the present invention, a cloud service module is also included. The cloud service module is used for point cloud data synchronization and remote control of the construction system, and integrates maintenance operations and data statistics of the construction system.
[0015] As a further improvement of the present invention, the projection device includes a main projection area and a secondary projection area; the main projection area is arranged in the central area of the performance space, and the secondary projection areas are dispersedly arranged around the main projection area; the position of the main projection area is fixed and the projection content in the main projection area remains unchanged, and the number of main projection areas is single; the number of secondary projection areas is multiple, and multiple different secondary projection areas are output and displayed by different projection devices respectively, and the position of the secondary projection area changes according to the movement of the audience, and its projection content changes according to the audience behavior.
[0016] As a further improvement of the present invention, it also includes a search system, the projection device is controlled by the search system, and the search system is used to control the movement of the secondary projection area in the interpretation space, and the movement is performed according to the state of the audience staying or entering and exiting the interpretation space; The audience includes the audience P1 who has just entered the performance space, the audience P2 staying in the performance space, the duration of P2 staying in the performance space is T, and the audience P3 leaving the performance space. Among them, the number of secondary projection areas is P, and the number of audiences in the coverage area of a single secondary projection area does not exceed the maximum number of audiences Pmax that it can cover. Then, the product of the number P of secondary projection areas and the number of audiences in the coverage area of a single secondary projection area not exceeding the maximum number of audiences Pmax that it can cover is the maximum number of people PPmax in the secondary projection areas in the performance space. The secondary projection areas include a working state and an idle state, while the main projection area always remains in the working state. When P2 < PPmax, the working steps of the search system include steps Z1-Z2. Step Z1: The search system searches for the audience P1 who has just entered the performance space. If there are unused secondary projection areas, it calls the remaining secondary projection areas to cover P1. If there are no unused secondary projection areas, it guides P1 into a single secondary projection area that does not exceed Pmax. Step Z2: When all the audiences P3 in a single secondary projection area have left, turn off the projection area, mark the projection area as the idle state, and return to Step Z1 to end. When P2 ≥ PPmax, at this time all secondary projection areas are in the working state, and the working steps of the search system include steps Y1-Y3. Step Y1: The search system searches for the audience P1 who has just entered the performance space and directly guides the audience into the main projection area. Step Y2: When there are audiences P3 leaving a single secondary projection area, guide the audiences in the main projection area into the secondary projection area with available empty positions. Step Y3: When the number of people in the main projection area reaches the maximum value, no guiding indication is given to the newly entered audience P1 and no projection interaction is carried out, and it ends. According to the duration T of P2 staying in the performance space in a single secondary projection area, calculate the average duration T' of the audiences in this secondary projection area. Move the position of the secondary projection area according to the average duration. The shorter the average duration T' in a single secondary projection area, the closer it is to the center of the performance space. The longer the average duration T' in a single secondary projection area, the closer it is to the edge of the performance space.
[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: By setting up a performance space in an open public space and placing it in a construction system, the audience can move freely in the performance space; the sensor module collects environmental information in the performance space through a lidar device, forms a data signal and transmits it to the control module, and the control module issues instructions to make the control modules work together, so as to create an immersive performance space scene for the audience. The interactive performance content can be updated and changed at any time, making the performance space fresh; the performance space can be adjusted according to the actual space and has strong applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 is a flowchart of the process of the present invention; Figure 2 It is the work flow chart of the present invention. DETAILED DESCRIPTION
[0019] Specific embodiment 1: Please refer to Figure 1-Figure 2 The invention relates to an immersive interactive interpretation space construction system, wherein the interpretation space is an open public space, the audience can move freely in the interpretation space, and the construction system is placed in the interpretation space. The construction system includes a sensor module, a control module and an interpretation module. The open public space can be a small art activity venue to a large commercial atrium space, such as a commercial center, a museum, an art gallery and a tourist attraction. When the audience moves freely in the interpretation space, the system interacts with the audience, rather than the system unilaterally outputting content to the audience. The audience's behavior can get feedback from the system to create an immersive interpretation space, so that the audience can get a better sense of reality and immersion.
[0020] According to the scale and layout of the performance space, the sensor module, control module and performance module are installed in the performance space. The performance space is an open scene, but it is still a limited space area. The range of data collected by the sensor module at least includes the entire performance space. It should be noted that the installation of the construction system is completed in advance. Compared with the traditional physical construction method, this construction system reduces the physical occupation of public space through virtual reality, which can not only save materials and energy, but also provide more play space for the audience.
[0021] The sensor module includes a laser radar device. The laser radar device is distributed in the performance space. After receiving the monitoring command of the main program, it collects environmental information in the performance space by rotating and / or scanning, forms a data signal and transmits it to the control module. Compared with the traditional stage where the audience's position is restricted to a fixed position, except for the swing of the limbs, the range of movement is limited. In the open scene of this embodiment, the audience's movement mode is uncertain, and the audience can enter and exit the performance space at any time, and the freedom of movement in the performance space is higher. The laser radar device can collect information in the performance space more accurately and quickly.
[0022] The control module includes a signal processing device, an environmental analysis device and a collaborative control device. The signal processing device is used to receive data signals and convert them into point cloud data. The environmental analysis device performs preprocessing, feature extraction and matching positioning operations based on the original point cloud data, calculates the position and posture information of the target audience, and compares the real-time position and posture information with the preset scene positioning in the virtual environment to generate an action mapping. The collaborative control device is used to adjust the deduction module according to the mapping rules.
[0023] The data signal includes the distance and angle information in the environment. The point cloud data includes the reflection information of the audience and the surface of the object. Communication is carried out through the tuio protocol.
[0024] The performance module includes audio device, projection device and lighting device, and works together according to the instructions issued by the control module.
[0025] The preprocessing operation includes removing noise, filtering and smoothing, and scaling the point cloud data. The effective information extracted from the point cloud data after the preprocessing operation includes the edge contour and foot points of the audience. In the positioning and matching operation, the point cloud data of the current frame is matched with the reference frame of the known position. The calculation is performed using the iterative nearest point projection algorithm or its variant algorithm through the NDI protocol.
[0026] According to the audience's actions and the mapping rules of the system, the projection content of the projection device is rendered in real time by the collaborative control device. The projection content includes the shape, color, window size, position and movement trajectory of the projected image. The lighting device is regulated by the collaborative control device to adjust the lighting effect changes and mechanical start and stop. The sound device is regulated by the collaborative control device to adjust the loudness, scale and speed of the sound effect.
[0027] Among them, the projection device includes a holographic projection device. The holographic projection is an image with a sense of depth and reality. The image includes virtual scenes and characters. Holographic projection belongs to the existing conventional technical means and will not be repeated here. The color tone and brightness of the light, the melody and loudness of the sound match the projection content, and the style is consistent. The lighting device and the sound device cooperate with the projection device to make it more realistic. The projection content is pre-made, and the projection can be controlled by humans and / or machines.
[0028] Among them, the preset scenes in the construction system include preset scenes of multiple different themes. It can flexibly display multiple contents in the same performance space. The switching of multiple themes provides more choices and creative space for the exhibitors. The interactive performance content can be updated and changed at any time, making the performance space have a continuous sense of freshness. It can select appropriate theme content according to the scale requirements of the performance space, which has strong adaptability and practicality.
[0029] It also includes a cloud service module, which is used for point cloud data synchronization and remote control of the construction system, and integrates the maintenance operations and data statistics of the construction system.
[0030] Working principle: Step S1: First, in the performance space, a construction system is pre-built according to the scale and layout of the performance space; Step S2: The system is in the initial state and is in the main program monitoring judgment. After the sensor module is initialized and activated, it makes a judgment. If it detects that the environmental information is collected, it proceeds to the next step and transmits the data signal to the control module for processing. Otherwise, it returns to the initial state; Step S3: The control module converts the data signal into point cloud data, and performs data preprocessing and feature extraction on the point cloud data; Step S4: judging the extracted features, if valid, adopting them, otherwise returning to the initial state; performing environmental analysis at the same time, matching and locating the point cloud data, if valid, adopting and mapping the content for real-time rendering, otherwise returning to the initial state; Step S5: The control module sends instructions to the deduction module, adjusts the deduction module according to the mapping rules, ends and returns to the initial state.
[0031] Specific embodiment 2: The same as specific embodiment 1 is not repeated here, and the difference is that the projection device includes a main projection area and a secondary projection area. The main projection area is set in the central area of the interpretation space, and the secondary projection areas are dispersed around the main projection area.
[0032] The position of the main projection area is fixed and the projection content in the main projection area remains unchanged. The number of main projection areas is single. As the basic overall performance area and a landmark area, the main projection area shows the overall style and characteristics to the audience, continuously attracts the audience to approach the performance area, and plays a role in gathering people. The main projection area is output and displayed by a separate projection device.
[0033] There are multiple secondary projection areas, and different secondary projection areas are output and displayed by different projection devices. The position of the secondary projection area changes according to the movement of the audience, and its projection content changes according to the audience's behavior. The projection content in the secondary projection area is consistent with the theme in the main projection area. As a discrete area, the secondary projection area is an extension and expansion of the visual space of the main projection area. It displays point-to-point to the audience to refine the interpretation of the projection. The main projection area and the secondary projection area complement each other, so that the interpretation space has a sense of hierarchy and provides the audience with a better immersive interactive interpretation experience.
[0034] Determine the number of secondary projection areas according to the number of projection devices. The more viewers there are in the performance area, the more viewers each secondary projection area covers; the fewer viewers there are in the performance area, the fewer viewers each secondary projection area covers. However, ensure that the secondary projection areas can cover all the viewers in the entire performance space so that all viewers can access the projection content.
[0035] Specific Embodiment 3: The same parts as Specific Embodiment 2 will not be elaborated here. The differences are as follows: It further includes a search system. The projection device is controlled by the search system, and the search system is used to control the movement of the secondary projection areas within the performance space, and the movement is based on the states of the viewers staying or entering and leaving within the performance space.
[0036] The viewers include viewer P1 who has just entered the performance space, viewer P2 who stays in the performance space, the duration of P2 staying in the performance space is T, and viewer P3 who leaves the performance space. Among them, the number of secondary projection areas is P, and the number of viewers in the coverage area of each secondary projection area does not exceed the maximum number of viewers Pmax that it can cover. Then, the product of the number P of secondary projection areas and the number of viewers in the coverage area of each secondary projection area not exceeding the maximum number of viewers Pmax that it can cover is the maximum number of people PPmax in the secondary projection areas within the performance space. The secondary projection areas include a working state and an idle state, while the main projection area always remains in the working state.
[0037] When P2 < PPmax, the working steps of the search system include steps Z1 - Z2.
[0038] Step Z1: The search system searches for the viewer P1 who has recently entered the performance space. If there are unused secondary projection areas, call the remaining secondary projection areas to cover P1; if there are no unused secondary projection areas, then guide P1 into a single secondary projection area that does not exceed Pmax. Step Z2: When all the viewers P3 in a single secondary projection area have left, turn off this projection area, mark this projection area as the idle state, and return to Step Z1 to end.
[0039] When P2 ≥ PPmax, at this time all the secondary projection areas are in the working state, and the working steps of the search system include steps Y1 - Y3.
[0040] Step Y1: The search system searches for the viewer P1 who has recently entered the performance space and directly guides the viewer into the main projection area. Step Y2: When there is a viewer P3 leaving a single secondary projection area, then guide the viewers in the main projection area into the secondary projection area with available empty positions. Step Y3: When the number of people in the main projection area reaches the maximum value, no guidance instructions are given to the audience P1 who has just entered the performance space, and no projection interaction is performed, and the process ends.
[0041] The entire system can serve the largest number of audiences to the greatest extent, and at the same time, the audiences in different projection areas can be managed more accurately to avoid chaos and crowds in the performance space during operation.
[0042] At the same time, according to the time T that P2 in a single sub-projection area stays in the interpretation space, the average time T' of the audience in the sub-projection area is calculated, and the position of the sub-projection area is moved according to the average time. The shorter the average time T' in a single sub-projection area, the closer it is to the center of the interpretation space; the longer the average time T' in a single sub-projection area, the closer it is to the edge of the interpretation space; this allows new audiences to fully immerse themselves in the experience, and at the same time allows audiences in the interpretation space to quickly leave the interpretation space, and new audiences can quickly join the interpretation space, reducing the movement of the projection device, which is conducive to speeding up the flow of audiences in the interpretation space.
Claims
1. An immersive interactive interpretation space construction system, characterized by: The performance space is an open public space where the audience can move freely. The construction system is placed in the performance space. The construction system includes a sensor module, a control module and a performance module. The sensor module includes a laser radar device; the laser radar device is distributed in the interpretation space, and after receiving the monitoring instruction of the main program, it collects environmental information in the interpretation space by rotating and / or scanning, forms a data signal and transmits it to the control module; The control module includes a signal processing device, an environment analysis device and a collaborative control device; the signal processing device is used to receive data signals and convert them into point cloud data; the environment analysis device performs preprocessing, feature extraction and matching positioning operations based on the original point cloud data, calculates the position and posture information of the target audience, and compares the real-time position and posture information with the preset scene positioning in the virtual environment to generate action mapping; the collaborative control device is used to adjust the deduction module according to the mapping rules; The interpretation module includes an audio device, a projection device and a lighting device, and they work together according to the instructions issued by the control module.
2. The immersive interactive performance space construction system according to claim 1, characterized in that: The data signal includes the distance and angle information in the environment; the point cloud data includes the reflection information of the audience and the surface of the object; and the communication is carried out through the TUIO protocol.
3. The immersive interactive performance space construction system according to claim 1, characterized in that: The preprocessing operations include noise removal, filtering and smoothing, and data scaling of the point cloud data.
4. The immersive interactive performance space construction system according to claim 1, characterized in that: The effective information for feature extraction in the point cloud data after preprocessing operations includes the edge contours and foot points of the audience.
5. The immersive interactive performance space construction system according to claim 1, characterized in that: In the positioning and matching operation, the point cloud data of the current frame is feature matched with the reference frame of the known position; the calculation is performed using the iterative nearest point projection algorithm or its variant algorithm through the NDI protocol.
6. The immersive interactive performance space construction system according to claim 1, characterized in that: According to the audience's actions and the system's mapping rules, the projection content of the projection device is rendered in real time by the collaborative control device; the projection content includes the shape, color, window size, position and motion trajectory of the projected image.
7. The immersive interactive performance space construction system according to claim 1, characterized in that: According to the audience's movements and the system's mapping rules, the lighting device is controlled by a collaborative control device to adjust the lighting effects and mechanical start and stop; the audio device is controlled by a collaborative control device to adjust the loudness, scale and speed of the sound effects; according to the scale and layout of the performance space, the sensor module, control module and performance module are installed in the performance space.
8. The immersive interactive performance space construction system according to claim 1, characterized in that: It also includes a cloud service module; the cloud service module is used for point cloud data synchronization and remote control of the construction system, and integrates maintenance operations and data statistics of the construction system.
9. The immersive interactive interpretation space construction system according to claim 1, characterized in that: The projection device includes a main projection area and a secondary projection area; the main projection area is arranged in the central area of the performance space, and the secondary projection areas are dispersedly arranged around the main projection area; the position of the main projection area is fixed and the projection content in the main projection area remains unchanged, and the number of the main projection area is single; the number of the secondary projection areas is multiple, and multiple different secondary projection areas are output and displayed by different projection devices respectively, and the position of the secondary projection area changes according to the movement of the audience, and its projection content changes according to the behavior of the audience.
10. The immersive interactive performance space construction system according to claim 9, characterized in that: It also includes a search system, the projection device is controlled by the search system, and the search system is used to control the movement of the secondary projection area in the performance space, and the movement is performed according to the state of the audience staying or entering and exiting the performance space; The audience includes the audience P1 who has just entered the performance space, the audience P2 staying in the performance space, the duration of P2 staying in the performance space is T, and the audience P3 who has left the performance space; where the number of secondary projection areas is P, and the number of audiences covered by a single secondary projection area does not exceed its maximum coverage number Pmax, then the product of the number P of secondary projection areas and the number of audiences covered by a single secondary projection area not exceeding its maximum coverage number Pmax is the maximum number PPmax of the secondary projection areas in the performance space; the secondary projection areas include a working state and an idle state, while the main projection area always remains in the working state; When P2 < PPmax, the working steps of the search system include steps Z1-Z2; Step Z1: The search system searches for the audience P1 who has recently entered the performance space. If there are unused secondary projection areas, it calls the remaining secondary projection areas to cover P1; if there are no unused secondary projection areas, it guides P1 into a single secondary projection area not exceeding Pmax; Step Z2: When all the audiences P3 in a single secondary projection area have left, turn off this projection area, mark this projection area as the idle state, and return to Step Z1 to end; When P2 ≥ PPmax, at this time all secondary projection areas are in the working state, and the working steps of the search system include steps Y1-Y3; Step Y1: The search system searches for the audience P1 who has recently entered the performance space and directly guides the audience into the main projection area; Step Y2: When there is an audience P3 leaving in a single secondary projection area, guide the audience in the main projection area into the secondary projection area with available empty positions; Step Y3: When the number of people in the main projection area reaches the maximum value, no guiding indication is given to the newly entered audience P1 in the performance space and no projection interaction is carried out, and it ends; According to the duration T of P2 staying in the performance space in a single secondary projection area, calculate the average duration T' of the audiences in this secondary projection area, and move the position of the secondary projection area according to the average duration. The shorter the average duration T' in a single secondary projection area, the closer it is to the center of the performance space; the longer the average duration T' in a single secondary projection area, the closer it is to the edge of the performance space.
