Wall interactive projection method and system based on CIM digital twinning

Abstract

The invention provides a wall interactive projection method and system based on CIM digital twinning, and is applied to the field of projection data processing. The environment change data of the wall projection area is collected in real time, the projection content can be dynamically adjusted in combination with the CIM model, the stability of the projection content in the physical space is ensured, meanwhile, the interaction content and the projection content input by the user are precisely synchronized by dividing the projection subareas and compensating the interaction influence in real time, and the user experience is improved. Projection distortion or jumping is automatically corrected under the environment change, accurate positioning and dynamic calibration are performed by using three-dimensional space coordinate data, and a projection path and angle are optimized, so that the distortion problem caused by angle change or shielding is avoided, and the stability of the projection effect and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the field of projection data processing, and in particular to a wall interactive projection method and system based on CIM digital twins. Background Art

[0002] With the continuous development of the times, various types of entertainment games are emerging, such as projection games, including desktop projection games, floor projection games and wall projection games, and all of them are through the real-time combination of interactive sensing and image display. All special programs will make the image generation and interactive sensing change synchronously.

[0003] Since the projection system needs to present clear and stable images on the wall, and factors such as the wall's material, color, and lighting changes will affect the projection effect, it is difficult to ensure high-quality projection output on different types of walls. Summary of the invention

[0004] The present invention aims to solve the problem of how to ensure the stability of the projection image in the physical space, avoid the problem of distortion or jumping of the projection content due to angle changes or occlusion by moving objects, and provide a wall interactive projection method and system based on CIM digital twins.

[0005] The present invention adopts the following technical means to solve the technical problem:

[0006] The present invention provides a wall interactive projection method based on CIM digital twin, comprising:

[0007] Based on a projector preset in the wall projection area, real-time collection of environmental change data of the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data and person movement data;

[0008] Determining whether the environmental change data detects a preset projection distortion jump;

[0009] If so, dividing the wall projection area into at least two projection partitions according to the preset projection parameters of the wall projection area, identifying the interactive influence information of the projection partitions, and dynamically compensating the corresponding projection content by applying the preset CIM model according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure and wall texture;

[0010] Determining whether the projected content can be displayed synchronously with the interactive content input by the user;

[0011] If not, based on the spatial coordinate data preset for the wall projection area by the CIM model, the coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area, wherein the spatial coordinate data are specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0012] Furthermore, the step of applying a preset CIM model to dynamically compensate the corresponding projection content according to the interactive impact information further includes:

[0013] Based on the preset content requirements of the projection content, generating the current projection area of ​​the wall projection area, wherein the content requirements specifically include content ratio, resolution, brightness and contrast;

[0014] Determining whether the current projection area matches a preset optimal projection area;

[0015] If not, then obtain the reflection parameters of the light from the wall projection area through the light emitted by the projector, collect the parameters to be corrected of the wall projection area according to the reflection parameters, and control the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

[0016] Furthermore, before the step of applying a preset CIM model to dynamically compensate the corresponding projection content according to the interactive impact information, the method further includes:

[0017] Based on the coordinate origin predefined by the CIM model for the wall projection area, construct a virtual three-dimensional coordinate system applicable to the wall projection area, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0018] Determining whether the virtual three-dimensional coordinate system can match the projection field of view of the projector;

[0019] If so, the wall parameters of the wall projection area are measured through the virtual three-dimensional coordinate system to obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0020] Furthermore, the step of collecting coordinate information corresponding to the interactive content and dynamically adjusting the interactive response of the projection content according to the coordinate information further includes:

[0021] Collecting user coordinate information of the user in the wall projection area, and obtaining an interaction area between the user coordinate information and the projection content;

[0022] Determining whether the interactive area can accommodate the projection content and the interactive content at the same time;

[0023] If not, a preset layer is deployed in the same space of the wall projection area, and based on the preset layer, the projection content and the interactive content are synchronously displayed using a preset display order, wherein the preset layer specifically includes a front-end layer and a background layer.

[0024] Furthermore, the step of determining whether the environmental change data detects a preset projection distortion jump further includes:

[0025] Based on the projection information preset by the projector, detecting the jumping amplitude of the projection information;

[0026] Determining whether the jitter amplitude reaches a preset distortion standard;

[0027] If so, apply the preset optical flow method to collect continuous frame data of the projection message before and after the change, and calculate the motion vector of each pixel point corresponding to the projection message by comparing the continuous frame data, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

[0028] Furthermore, the step of determining whether the projection content can be displayed synchronously with the interactive content input by the user further includes:

[0029] Based on the preset sensors in the wall projection area, the movement trajectory corresponding to the interactive content is captured in real time, wherein the sensors specifically include infrared sensors, touch sensors and pressure sensors;

[0030] Determining whether the movement trajectory matches the mapping rule preset in the CIM model;

[0031] If so, respond to the corresponding dynamic display content on the wall projection area according to the movement trajectory, detect the position change information of the user in the wall projection area, and dynamically adjust the display position of the projection content according to the position change information.

[0032] Furthermore, the step of collecting the environmental change data of the wall projection area in real time based on the projector preset in the wall projection area also includes:

[0033] Based on the projection effect preset by the projector, the color deviation information of the wall projection area is detected, wherein the projection effect specifically includes brightness, focus and color;

[0034] Determining whether the color deviation information meets a preset projection requirement;

[0035] If not, then according to the projection requirements, the influencing parameters of the color deviation information on the projection message are identified, and the influencing parameters are generated in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

[0036] The present invention also provides a wall interactive projection system based on CIM digital twin, comprising:

[0037] A collection module, which is used to collect environmental change data of the wall projection area in real time based on a projector preset in the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data and person movement data;

[0038] A judging module, used for judging whether the environmental change data detects a preset projection distortion jump;

[0039] an execution module, configured to, if yes, divide the wall projection area into at least two projection partitions according to the preset projection parameters of the wall projection area, identify the interactive influence information of the projection partitions, and dynamically compensate the corresponding projection content by applying the preset CIM model according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure and wall texture;

[0040] A second judgment module is used to judge whether the projection content can be displayed synchronously with the interactive content input by the user;

[0041] The second execution module is used for, if not possible, collecting coordinate information corresponding to the interactive content based on the spatial coordinate data preset for the wall projection area by the CIM model, dynamically adjusting the interactive response of the projection content according to the coordinate information, and synchronously connecting the time nodes of the interactive content and the projection content on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in the three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0042] Furthermore, the execution module also includes:

[0043] A generating unit, configured to generate a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness and contrast;

[0044] A judging unit, used to judge whether the current projection area matches a preset optimal projection area;

[0045] an execution unit, for, if not, obtaining reflection parameters of the light from the wall projection area through the light emitted by the projector, collecting parameters to be corrected of the wall projection area according to the reflection parameters, and controlling the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, the reflection angle and the material properties of the reflection surface, and the parameters to be corrected specifically include the wall curvature and the wall inclination.

[0046] Furthermore, it also includes:

[0047] A construction module, configured to construct a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin predefined by the CIM model for the wall projection area, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0048] A third judgment module is used to judge whether the virtual three-dimensional coordinate system can match the projection field of view of the projector;

[0049] The third execution module is used to measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, and obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0050] The present invention provides a wall interactive projection method and system based on CIM digital twin, which has the following beneficial effects:

[0051] The present invention collects environmental change data of the wall projection area in real time and can dynamically adjust the projection content in combination with the CIM model to ensure its stability in the physical space. At the same time, by dividing the projection into partitions and compensating for interactive effects in real time, the interactive content input by the user and the projection content are accurately synchronized, and projection distortion or jitter is automatically corrected under environmental changes. The three-dimensional space coordinate data is used for precise positioning and dynamic calibration to optimize the projection path and angle, thereby avoiding distortion problems caused by angle changes or occlusions, and improving the stability of the projection effect and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of an embodiment of the wall interactive projection method based on CIM digital twin of the present invention;

[0053] Figure 2 This is a structural block diagram of an embodiment of the wall interactive projection system based on CIM digital twin of the present invention. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The implementation of the objectives, functional features and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Reference Figure 1 , is a wall interactive projection method based on CIM digital twin in an embodiment of the present invention, comprising:

[0057] S1: Based on a projector preset in a wall projection area, real-time collection of environmental change data of the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data, and person movement data;

[0058] S2: Determine whether the environmental change data detects a preset projection distortion jump;

[0059] S3: If yes, then according to the preset projection parameters of the wall projection area, at least two projection partitions are divided from the wall projection area, the interactive influence information of the projection partitions is identified, and the preset CIM model is applied to dynamically compensate the corresponding projection content according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure and wall texture;

[0060] S4: Determine whether the projection content can be displayed synchronously with the interactive content input by the user;

[0061] S5: If not, based on the spatial coordinate data preset for the wall projection area by the CIM model, the coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area, wherein the spatial coordinate data are specifically the positioning coordinates of the wall in the three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0062] In this embodiment, the system collects environmental change data of the wall projection area in real time based on the projector pre-installed in the wall projection area. The environmental change data specifically includes object occlusion data, angle change data and person movement data. Then the system determines whether these environmental change data detect the preset projection distortion jump to execute the corresponding steps; for example, when the system determines that the environmental change data of the wall projection area does not detect the preset projection distortion jump, the system will consider that the projection information remains stable in the current environment and is not affected by object occlusion, angle change or person movement. The system will continue to output the current projection content to ensure the stability and accuracy of the picture, and continue to collect environmental change data of the wall projection area in real time to ensure timely response to any changes in the future, and record relevant data of the current environment and projection status (such as wall structure, projection angle, ambient light, etc.) to provide data support for subsequent tuning and analysis. ; For example, when the system determines that the environmental change data of the wall projection area detects a preset projection distortion jump, the system will consider that the projection information is affected. The system will divide the wall projection area into at least two projection partitions based on the preset projection parameters of the wall projection area, including the wall size, wall structure and wall texture, identify the interactive impact information of these projection partitions, and dynamically compensate the corresponding projection content based on the preset CIM model according to different interactive impact information; the system divides the wall projection area into at least two projection partitions according to preset projection parameters such as wall size, structure and texture, so that the affected area can be accurately located, and adjustments can be made to the specific projection area to avoid unnecessary waste of resources or affecting the projection effects of other parts due to global adjustments. At the same time, according to different interactive impact information, the system dynamically compensates the corresponding projection content by applying the CIM model. This compensation can adjust the projection content in real time according to the changes in the wall projection area (such as occlusion, angle change or person movement) to ensure the stability of the picture quality, and identify the specific interactive influences on different projection partitions (such as user behavior, object movement, etc.), and make adjustments based on these influence information. In this way, the projection content can be better synchronized with the user's interaction to achieve a highly accurate interactive effect; then the system determines whether these projection contents can be synchronously displayed to the wall projection area with the interactive content input by the user in real time, so as to execute the corresponding steps;For example, when the system determines that the dynamically compensated projection content can be synchronously displayed with the interactive content input by the user in real time to the wall projection area, the system will consider that the projection content and the interactive content have been successfully connected and can be kept synchronized, thereby achieving a seamless connection between the projection content and the user's interactive behavior. The system will continue to monitor the user's input changes to ensure that the projection content can remain synchronized under any new interactive behavior or environmental changes in the future, maintain the continuous stability of the interactive experience, prevent synchronization problems caused by user behavior or environmental changes, and ensure the stability of the interaction. At the same time, if the environmental conditions (such as lighting, object occlusion, etc.) change, the system should be able to automatically adjust the projection content to ensure that the effect of the synchronous display is not affected, and ensure the projection The projection screen always maintains the best display effect under different environmental conditions, improves the adaptability and stability of the system, and records the user's interactive behavior and the synchronous display of the projection content, generates corresponding logs, so as to facilitate the later analysis and optimization of system performance. By recording and analyzing user behavior data, the system can find potential room for improvement in the subsequent optimization process, and further improve the stability of the system and user experience; for example, when the system determines that the dynamically compensated projection content cannot be synchronously displayed to the wall projection area with the interactive content input by the user in real time, the system will think that the projection content and the interactive content cannot be synchronously connected, and the system will pre-set the spatial coordinate data of the wall projection area based on the CIM model. The spatial coordinate data is specifically the wall The coordinates of the positioning in three-dimensional space are collected to collect the coordinate information corresponding to the interactive content. The coordinate information specifically includes the azimuth, tilt angle and normal vector of the wall. The interactive response of the projected content is dynamically adjusted according to these coordinate information, and the time nodes of the interactive content and the projected content are synchronized in the wall projection area. By obtaining the coordinate information such as the azimuth, tilt angle and normal vector of the wall, the system can adjust the projection angle, position and content in real time to confirm that the projection content matches the user's interactive behavior, ensure the precise synchronization between the interactive content and the projected image, avoid image distortion, jumping or delay problems caused by changes in the wall angle or asynchronous interactive content, and use the precise coordinate data of the wall and interactive content to quickly respond to users. Input changes, and adjust the display position, direction and content of the projected content. This real-time dynamic compensation makes the interaction between the user and the system more sensitive and natural, enhances the system's response speed to the environment and user interaction, reduces latency, improves the system's interactivity and flexibility, and enhances user experience. Adjustment based on the wall space coordinate data can not only improve the accuracy of projection, but also optimize the use of the projection area. For example, when the wall angle changes, the projector can adjust the direction of the projected content in real time, so that the picture always adapts to the wall structure, avoids picture distortion or edge blur, improves space utilization, and ensures that even on irregular or complex walls, the projected content can still be displayed clearly and completely, improving the adaptability and quality of the projection effect. ;

[0063] It should be noted that the interactive influence information of the projection partition is identified, and the corresponding projection content is dynamically compensated by applying a preset CIM model according to the interactive influence information. The specific examples are as follows:

[0064] The scene is described as,

[0065] Assume that there is currently a wall projection in a virtual fitting mirror. When a user stands in front of a wall, virtual clothes are projected on the wall. The user can view the styles and combinations of different clothes by moving, turning around, raising hands, and other actions. The wall projection content will be adjusted in real time according to the user's actions to ensure that the clothes appear natural and realistic on the user.

[0066] The interactive impact information is:

[0067] For example, when the user turns around or raises his arm, the clothing in the virtual fitting mirror needs to be adjusted according to the user's movements to ensure that the virtual clothing matches the user's posture; for example, when the user turns around, part of the body may block part of the projection area, resulting in some clothing details (such as the back or cuffs) being blocked; if the angle of the projector changes, or the tilt angle of the wall projection area changes when the user moves, the projection content may be distorted;

[0068] Apply CIM model for dynamic compensation,

[0069] Real-time acquisition of user position and posture data: Through motion capture devices (such as depth cameras or infrared sensors), the system acquires the user's body position, posture and motion data in real time; these data include the user's turning angle, the height and direction of the arm raised, etc.; when the user turns around or moves, the system determines whether the projected content is blocked based on the data fed back by the sensor; for example, when the user turns around, the back details or cuff design in the projection may be blocked; the system determines the part that needs to be compensated by calculating these blocked areas and determining which projected content will be blocked;

[0070] Apply CIM model to adjust projection content,

[0071] If the user turns around or raises his hand, the CIM model will adjust the display position of the virtual clothing in real time to keep it consistent with the user's posture. For example, after the user turns around, the virtual clothing on the back will automatically rotate and display instead of staying at the original front-view angle. If a user's action (such as bending or stretching his arms) blocks a part of the projection area, the CIM model will calculate a new display area and adjust the blocked clothing part to another angle or position to ensure that the display of the clothing is not blocked. For example, the system may move the virtual cuff from the back to the side of the user to avoid being blocked by the arm. If the angle of the projector changes (for example, when the user walks, the projector tilts slightly), the CIM model will adjust the position and scale of the projected content in real time to avoid distortion of the projected content due to angle offset. For example, the system will calculate the new projection coordinates on the wall and recalibrate the display of the virtual clothing according to the new projection angle. If the user walks in the projection area, or the ambient light changes (such as the curtains are opened), the system will also adjust the brightness and contrast of the projected content according to the environmental changes to ensure that the virtual clothing is clearly visible under different light conditions.

[0072] That is, when the user turns around, raises his hand or makes other movements, the virtual clothing projected on the wall will respond instantly and always be consistent with the user's body posture, ensuring that the user can see different angles of the clothing; even if the user's movements cause some parts of the clothing to be blocked, the system will automatically adjust the projection content and re-display the blocked parts in other locations to ensure that the details of the clothing are not lost; the display effect of the projected content on the wall always remains clear, and no matter how the user moves, the virtual clothing can accurately synchronize with the user's movements to avoid distortion or dislocation;

[0073] In summary, the above examples describe how the system uses the CIM model to dynamically compensate for projection content in real time, ensuring that the wall projection can be synchronized with the user's interaction and actions, thereby improving the experience of the virtual fitting mirror.

[0074] It should be supplemented that, based on the spatial coordinate data preset for the wall projection area by the CIM model, the coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area. The specific examples are as follows:

[0075] Suppose in a virtual home design application, users can control virtual furniture (such as sofas, tables, decorations, etc.) displayed on the wall through gestures or touch; the wall projection area will display a virtual room layout, and users can change the position, size, rotation angle, etc. of the furniture by interacting with the wall projection; the entire system is adjusted in real time based on the CIM model to ensure that the time nodes of the projection content and the interactive content are synchronized;

[0076] First, when the system is initialized, the CIM model will establish a three-dimensional coordinate system based on the size, structure, texture and other data of the wall space. For example, assuming that the wall is 4 meters long and 3 meters wide, and the distance between the projector and the wall is 2 meters; the normal vector of the wall (that is, the direction perpendicular to the ground) will be preset to (0, 1, 0), so that the system knows the spatial location of the wall; the CIM model will also consider parameters such as the angle, inclination, and structural details of the wall; if there are decorations or special textures on the wall, the system will also preset them in the model; for example, if there is a special protrusion on the wall, the CIM model will adjust the projection content according to the protrusion so that the furniture is not blocked or distorted;

[0077] Then, the coordinate information of the interactive content is collected, and the user can operate the virtual furniture on the wall through gestures or touch. For example, when the user points to the sofa in the wall projection, or clicks the virtual table on the wall with his finger, the system will collect the specific coordinates of the user's gesture through sensors (such as infrared sensors, depth cameras, etc.). Assuming that the user's right fingertip is located at a certain position of the wall projection (for example, 3 meters away from the center of the wall projection area and 1 meter to the right), the system will transmit this coordinate point to the CIM model to indicate the user's gesture or touch position. Assuming that the user clicks on the virtual sofa in the projection area, the system will identify the user's finger coordinate information, including the fingertip position on the wall, the distance from the wall surface, the finger direction, and other data.

[0078] Then, the interactive response of the projected content is dynamically adjusted. According to the coordinates of the user's gesture (for example, the position where the user points to the sofa), the system will calculate the optimal display position of the sofa on the wall in real time through the CIM model; the system will dynamically adjust the position of the sofa according to the three-dimensional coordinates of the wall projection area so that it is displayed synchronously with the user's pointing; if the user points to the left side of the sofa, the system will slightly move the display position of the sofa so that the place where the user points appears exactly on the left side of the sofa; if the user's gesture indicates dragging the virtual sofa, the system will dynamically adjust the size and angle of the furniture according to the changes in the position and direction of the user's gesture; for example, if the user drags the sofa from the right to the left, the system will calculate the size and rotation angle of the sofa in the new position in real time according to the size change of the wall space, and adjust it through the CIM model to ensure that the projected content is not distorted on the wall; if the user's action causes some furniture to be blocked, the CIM model will automatically adjust the projected content to ensure that the furniture is not blocked by the body or arm; for example, when the user raises his arm, if the arm blocks part of the sofa, the system will move the sofa to the other side or adjust the blocked part to the visible area;

[0079] Then adjust the delay compensation of the interactive response. In the wall projection area, the system needs to ensure that the projected content is synchronized with the user's interactive content at the same time. For example, when the user clicks on the virtual sofa, the sofa's movement, rotation, scaling and other actions must be consistent with the moment of the finger click. Therefore, the system will compensate for the interactive delay in real time to ensure that each user's action can respond to the changes in the projected content immediately. If the user clicks on the sofa, the system will calculate the exact time point of the user's finger click and connect it with the time node of the sofa movement. By dynamically adjusting the projection frame rate, ensure that the projection content is synchronized with the user's gesture changes. For example, if the user quickly clicks on the sofa, the system will quickly adjust the position of the sofa on the wall to avoid projection distortion caused by delay. Assuming that the projector is slightly distorted due to some reasons (such as user movement, object occlusion or projection angle adjustment), the CIM model will detect these changes in real time and automatically adjust the display position of the projected content. For example, when the angle of the projector is offset, the system will correct the projection angle of the virtual furniture according to the three-dimensional coordinate data of the wall to ensure that the display of the furniture is not affected.

[0080] That is, no matter how the user moves, points or clicks, the system can respond to the user's actions in real time and dynamically adjust the furniture in the projection according to the user's input; the display of virtual furniture is always consistent with the user's gestures, position and angle; for example, when the user points to the sofa on the wall, the sofa will immediately appear at the position where the user's finger is pointing, and its angle, size, etc. will change synchronously with the user's actions; even if the user moves or interacts quickly, the projected content on the wall will always remain clear and distortion-free, and the furniture will not be blocked; every action of the user (such as moving the arm, turning around, approaching the wall, etc.) will be instantly reflected in the changes in the position, angle and size of the virtual furniture in the projection; when the user places virtual furniture on the wall projection, the whole experience is very smooth and natural; the system can accurately respond to the user's gestures or touches to ensure that there is no time difference or spatial dislocation between the projected content and the interactive content;

[0081] To sum up, in the above-mentioned application scenarios of virtual home placement, through the application of the CIM model, the wall projection system can accurately capture and respond in real time to the user's gestures, touch and other interactive information; the system dynamically adjusts the position, size, angle, etc. of the projection content according to the preset spatial coordinate data to ensure that the time nodes of the projection and user interaction are completely synchronized; in this way, the interaction between the user and the projection becomes more intuitive, smooth and real, enhancing the overall interactive experience.

[0082] In this embodiment, the step S3 of dynamically compensating the corresponding projection content using a preset CIM model according to the interactive influence information further includes:

[0083] S31: generating a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness and contrast;

[0084] S32: Determine whether the current projection area matches a preset optimal projection area;

[0085] S33: If not, obtaining reflection parameters of the light from the wall projection area through the light emitted by the projector, collecting parameters to be corrected of the wall projection area according to the reflection parameters, and controlling the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflection surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

[0086] In this embodiment, the system generates a current projection area of ​​the wall projection area based on the pre-set content requirements of the projection content, which specifically include content ratio, resolution, brightness and contrast, and then the system determines whether the current projection area matches the pre-set optimal projection area to execute the corresponding steps; for example, when the system determines that the current projection area of ​​the wall projection area can match the pre-set optimal projection area, the system will consider that the current projection effect has met the preset quality requirements, and the system will continue to project under the current settings to ensure the smoothness of the user experience and the stability of the visual effect without any additional correction or adjustment. At the same time, after ensuring that the current projection area matches the optimal projection area, the system will continue to interact with the user. If the user changes the angle or spatial position of the wall projection area, the system will continue Use existing parameters to adjust the projection to ensure that the new interactive area still meets the optimal projection standards, and ensure that the projection content is synchronized with the interactive content input by the user, so that the user can get instant and clear feedback every time they interact, providing a smooth interactive experience; for example, when the system determines that the current projection area of ​​the wall projection area cannot match the preset optimal projection area, the system will consider that the current projection effect cannot meet the preset quality requirements. The system will use the light emitted by the projector to obtain the reflection parameters of these lights from the wall projection area. The reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflection surface. According to these reflection parameters, the parameters to be corrected of the wall projection area are collected. The parameters to be corrected specifically include wall curvature and wall inclination. According to these parameters to be corrected, the projection content is controlled to perform geometric distortion correction;The system can accurately measure the reflection of the projection area and identify the shape and characteristics of the wall through the light emitted by the projector and its reflection path, angle, and surface material characteristics. According to the parameters to be corrected collected by the reflection parameters, the system can dynamically perform geometric correction on the projection content. By controlling the geometric shape and angle of the projection content, the image distortion caused by the irregular wall surface is avoided, and the clarity and accuracy of the projection content is maintained. At the same time, the reflection parameters can provide the projection system with real-time data on the physical characteristics of the wall surface, ensuring that the projected image will not be distorted or blurred due to the irregular wall surface, and preventing the irregular surface characteristics of the wall (such as uneven and tilted wall surface) from causing distortion or blur. The system can accurately reflect the impact of wall changes by collecting and adjusting the reflection parameters of the wall projection area in real time, ensuring that the projection content can be adjusted in time and presented synchronously with the user's operation, avoiding interaction delays or distortion caused by inconsistent wall shapes. That is, the system can adapt to different wall environments. Even if the wall is irregular or deformed during the projection process, the system can adjust the projection effect in real time to ensure the stability of the quality of the wall projection area and avoid adverse effects caused by wall changes. ;

[0087] It should be noted that, through the light emitted by the projector, the reflection parameters of the light from the wall projection area are obtained, the parameters to be corrected of the wall projection area are collected according to the reflection parameters, and the projection content is controlled to perform geometric distortion correction according to the parameters to be corrected. The specific examples are as follows:

[0088] Suppose there is an interactive projection system. The projection content is the interface of a virtual game, which contains a round virtual object, a "virtual sphere". The wall is used as the projection area, and the wall has a certain curvature and inclination. At this time, the system hopes to ensure that the projected virtual sphere can always remain round, regardless of how the distortion on the wall changes.

[0089] First, the projector is located in a corner of the room, and it projects an image of a virtual sphere onto the wall. The virtual sphere should appear as a complete circle on the wall. During this process, the light from the projector shines onto the wall, forming a light spot, which is the display area of ​​the virtual sphere.

[0090] Then, the reflection parameters are collected. When the light hits the wall, it will be reflected according to the shape of the wall. At this time, the system will analyze the condition of the wall through the reflected light. The sensor detects the propagation path of each light and records the propagation angle from the projector to each point on the wall.

[0091] For example, light from a projector reflects from a certain point onto a wall, but the paths of reflections may be different in different areas of the wall. If the wall is tilted or curved, these paths may change slightly because after hitting the wall, the light will reflect according to the normal direction of the wall surface. The system calculates the angle of each reflected light ray, especially in areas where the wall is curved and tilted.

[0092] For example, in areas where the wall is tilted, the reflection angle will be different, causing the projected image to be stretched or compressed to different degrees in these areas; if the wall is rough, it will cause the reflected light to scatter; if it is a smooth surface, the reflected light will be reflected more concentratedly; the system can infer the wall material through this difference and evaluate the reflection effect in different areas;

[0093] Then the parameters to be corrected are identified. By collecting the path and angle of the reflected light, the system finds that part of the wall is slightly curved. This curvature causes the projected image to appear stretched or compressed on the wall, especially when close to the curved area.

[0094] For example, in an area where the wall is curved, a virtual sphere that should be displayed as a circle may become an ellipse; the system will recognize this and mark the area as a target area that needs geometric distortion correction; the system also finds that the wall is slightly tilted, and the angle of the reflected light shows that the projected content in the tilted area has been stretched or deformed; for example, in an area where the wall is tilted, the originally vertical virtual sphere will appear irregular, or even appear to be an ellipse or an elongated shape;

[0095] Then, geometric distortion correction is performed. The system uses perspective transformation to correct the curved areas of the wall. Due to the curvature of the wall, the image is stretched or compressed in some areas. The perspective transformation will scale and distort these areas to restore the originally elongated sphere to a true circle. For example, in the area where the wall is curved, the system will adjust the display ratio of the sphere in this area so that it looks like a circle displayed on a flat surface without being affected by the curvature of the wall.

[0096] Finally, the tilted area is corrected. For the tilted part of the wall, the system uses the affine transformation algorithm to correct it. The affine transformation can adjust the content of the tilted area to be perpendicular to the observer while maintaining the shape ratio. Through this transformation, the originally tilted and distorted virtual sphere will be restored to the correct circle. For example, if the wall is tilted to the left, the system will perform reverse tilt correction on the image of the virtual sphere, making it look like it is suspended in a position perpendicular to the ground, avoiding deformation or stretching of the virtual object.

[0097] When the user interacts with the virtual sphere, the system will adjust the projection content in real time to ensure that the virtual sphere always maintains the correct shape; no matter where the user stands in front of the wall, the system will adjust the projection based on real-time feedback data; for example, the user may walk to the side of the wall, causing the viewing angle of the projected image to change; the system recalculates the reflection path of the light based on the user's interactive behavior to ensure that the projection content is updated in real time and always conforms to the shape of the wall;

[0098] In summary, through the above steps, the system successfully uses reflected light to collect data on the wall shape and perform geometric distortion correction; no matter whether the wall is curved or tilted, the system can make real-time corrections to the projection content so that the virtual sphere always remains round; no matter at what angle or position the user interacts with the virtual object, the projection effect can meet expectations, ensuring the quality and accuracy of the picture.

[0099] In this embodiment, before step S3 of dynamically compensating the corresponding projection content using a preset CIM model according to the interactive influence information, the method further includes:

[0100] S301: constructing a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin predefined by the CIM model for the wall projection area, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0101] S302: Determine whether the virtual three-dimensional coordinate system matches the projection field of view of the projector;

[0102] S303: If so, measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, and obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0103] In this embodiment, the system constructs a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin pre-defined for the wall projection area by the CIM model. The virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis. The system then determines whether these virtual three-dimensional coordinate systems can match the projection field of view of the projector to execute corresponding steps. For example, when the system determines that the virtual three-dimensional coordinate system applicable to the wall projection area cannot match the projection field of view of the projector, the system will believe that there is a difference between the coordinate system of the wall projection area and the visible range of the projector, resulting in the projection content may not be fully covered or distorted, or even unable to be accurately aligned or stably displayed. The system will change the position of the projector to make it cover the wall projection area. The entire coordinate range of the virtual three-dimensional coordinate system is ensured to be connected with the projection field of view of the projector, including moving the projector forward and backward, or adjusting the projection angle of the projector as needed, and appropriately reducing the range of the virtual three-dimensional coordinate system to match the projection field of view of the projector. The maximum values ​​of the X, Y, and Z axes of the virtual coordinate system can be reset to avoid conflicts with the physical limitations of the projector. If the wall projection area is too large or irregular, the system can dynamically adjust the size and angle of the projection content according to the shape of the wall and the projection field of view of the projector to adapt to the field of view of the projector, so as to ensure that the display effect of the projection content on the wall is more accurate and stable; for example, when the system determines that the virtual three-dimensional coordinate system is suitable for the wall projection area The system can match the projection field of view of the projector. At this time, the system will consider that the coordinate system of the wall projection area is the same as that of the projector. The system will measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system. The wall parameters specifically include the wall size and wall shape, and obtain the path information from the projector to the wall projection area. The path information specifically includes the distance between the projector's position and the wall, the projector's focal length and field of view. By matching the virtual three-dimensional coordinate system with the projector's projection field of view, the system can ensure that the size, shape, angle and other characteristics of the wall projection area are consistent with the content output by the projector, avoid image distortion, blur or dislocation problems, obtain the path information from the projector to the wall, ensure that the projection content can be accurately projected to a specific position on the wall, and optimize After measuring the clarity and display effect of the picture and the wall parameters (such as wall size and shape), the system can dynamically adjust the size, proportion, direction, etc. of the projected content to adapt it to the actual characteristics of the wall, helping the system to adjust the projection parameters according to the actual projection environment, such as projection angle, focus setting and content resolution, in order to obtain the best projection effect. By accurately measuring the wall size and shape, the system can automatically adjust the match between the projection content and the wall under different environmental conditions, not only to adapt to static walls, but also to cope with dynamic environmental changes. When changes occur in the wall projection area (such as object movement, wall tilt, etc.), the system can quickly adjust based on real-time feedback of path information and wall parameters to ensure the synchronization of projection and interactive content.

[0104] In this embodiment, the step S5 of collecting coordinate information corresponding to the interactive content and dynamically adjusting the interactive response of the projection content according to the coordinate information further includes:

[0105] S51: collecting user coordinate information of the user in the wall projection area, and obtaining an interaction area between the user coordinate information and the projection content;

[0106] S52: Determine whether the interactive area can accommodate the projection content and the interactive content at the same time;

[0107] S53: If not, then deploy preset layers in the same space of the wall projection area, and based on the preset layers, apply a preset display order to synchronously display the projection content and the interactive content, wherein the preset layers specifically include a front-end layer and a background layer.

[0108] In this embodiment, the system acquires the user coordinate information of the user on the wall projection area to obtain the interactive area of ​​the user coordinate information and the projection content, and then the system determines whether the interactive area can accommodate the projection content and the interactive content at the same time to execute the corresponding steps; for example, when the system determines that the interactive area of ​​the user coordinate information and the projection content can accommodate the projection content and the interactive content at the same time, the system will consider that there is no conflict between the current projection area and the user's interactive needs in the physical space, and efficient interactive display can be achieved. The system will monitor the user's coordinate information in real time, and the system can adjust the projection content according to the user's actions to ensure that the projection and the user's interaction are in the same spatial coordinate range, and the interactive area can accommodate the projection. In the case of content and interactive content, the system can automatically adjust the projection brightness, resolution, contrast and other parameters to adapt to different lighting environments or changes in user position to ensure the best quality of the display effect. At the same time, the system can also adjust the response of the projected content based on the user's specific interactive behavior (for example, finger touch, body movement, etc.), so that the user's action can directly affect the content or form of projection, and the system can effectively use the entire wall projection area for content display and user interaction, avoiding the problem of overlapping or occlusion of projection content and interactive content due to insufficient interactive space, achieving a seamless transition between the user and the projected content, and ensuring that the interaction between the user and the projection is not limited by physical space, thereby optimizing the space. for improving the efficiency of use; for example, when the system determines that the interactive area between the user coordinate information and the projected content cannot accommodate both the projected content and the interactive content, the system will consider that there is a conflict between the current projection area and the user's interactive needs in physical space, and the system will deploy pre-set layers in the same space of the wall projection area. The preset layers specifically include the front-end layer and the background layer, and the projection content and the interactive content are displayed synchronously based on the pre-set display order of the layer application; the system can effectively solve the spatial conflict problem between the user's interactive needs and the projection content by introducing the layer management of the front-end layer and the background layer. When the interactive area cannot accommodate the projection and interactive content, the system will allocate the interactive content and the projection content to Different layers ensure that the two will not block or overlap each other. At the same time, the relationship between the user's operation feedback and the projected content can be accurately controlled through the allocation and display order of layers. Through the front-end layer, the user's operation is not affected by the background content. The system can display the real-time response of the user's interaction on the front-end layer, while the background layer continues to display the projected content, ensuring the accuracy of the operation and the immediacy of the feedback. In addition, through the preset layers and display sorting rules, the system can intelligently schedule the display order of various types of content to meet different application scenarios and user needs. For example, when displaying product advertisements, the advertising content of the background layer can be covered by the interactive elements of the front-end layer, and when no interaction is required, the advertising content will be restored to full-screen display.

[0109] In this embodiment, the step S2 of determining whether the environmental change data detects a preset projection distortion jump further includes:

[0110] S21: Based on the projection information preset by the projector, detecting the jumping amplitude of the projection information;

[0111] S22: Determine whether the jitter amplitude reaches a preset distortion standard;

[0112] S23: If yes, apply the preset optical flow method to collect continuous frame data of the projection message before and after the change, and calculate the motion vector of each pixel point corresponding to the projection message by comparing the continuous frame data, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

[0113] In this embodiment, the system detects the amplitude of the projection information based on the projection information preset by the projector, and then the system determines whether the amplitude of the projection information reaches the preset distortion standard to execute the corresponding steps; for example, when the system determines that the amplitude of the projection information does not reach the preset distortion standard, the system will consider that the current projection effect is stable, and there is no distortion or picture jump phenomenon. The system will continue to maintain the current projection settings, including brightness, contrast, color and other parameters, and continuously monitor the environmental changes in the wall projection area, such as object occlusion, angle change and person movement, to ensure that the projection effect will not be affected by these factors. Sudden distortion and jumps occur, and the synchronous response of the projected content and the interactive content is further adjusted to enhance the user's interactive experience. For example, the clarity of the content is adjusted or the response delay is reduced to make the feedback of the projected content and the user's operation smoother, and the display of the interactive content can be continuously optimized to make its synchronous response with the projected content more accurate. By real-time tracking of the user's input, the system can adjust the interactive response time, the position of the interactive interface, etc., to make the temporal and spatial relationship between the projected content and the user's operation smoother. For example, when the system determines that the jump amplitude of the projection message has reached the preset distortion standard, the system will consider that the current projection effect is not good. Stable, prone to distortion or screen jump, the system will apply the pre-set optical flow method to collect continuous frame data of the projection information before and after the change, and calculate the motion vector of each pixel point corresponding to the projection information by comparing these continuous frame data. The motion vector specifically includes the movement speed and movement direction of the pixel point; the system can adjust the projection content based on the motion vector by obtaining the motion information of each pixel point, and correct the distortion or jump in the picture in real time. For example, the system can adjust the edge, focus area or geometric deformation of the projection according to the direction of movement to avoid the projection content from changing due to the environment. Through precise motion compensation, it can avoid image jump. The optical flow method can reduce the visual interference caused by users and improve the stability and smoothness of the viewing experience. At the same time, the real-time data acquisition and processing of the optical flow method can enable the system to quickly identify the distortion or jump of the projection image, and adjust the projection content in a very short time to achieve rapid correction and reduce the intervention of users and operators on the projector. That is, the system can automatically correct the projection content according to the calculated motion vector without manual adjustment of the projection equipment. In interactive projection applications, the optical flow method can ensure that the projection content and the user's actions (such as gestures, touch, etc.) are always synchronized, avoiding interaction delays or incoherence caused by picture jumps or distortions, and improving the naturalness and accuracy of the interactive experience.

[0114] It should be noted that, by applying the preset optical flow method, continuous frame data of the projection information before and after the change is collected, and by comparing the continuous frame data, the motion vector of each pixel point corresponding to the projection information is calculated. The specific example is as follows:

[0115] Suppose that in an exhibition hall, there is a wall used as a projection display wall. A projector projects a dynamic virtual artwork on this wall. Users can interact with the projected content through gestures or movements to change certain elements in the picture, such as making the virtual character walk, rotate, or change color.

[0116] The projected content changes dynamically with the user's interaction. However, the wall may be tilted, blocked by objects, or the projection may be distorted or jump due to the user's rapid movement. In order to ensure the stability and accuracy of the projected content in the physical space, the system uses the optical flow method to correct the projected image in real time.

[0117] First, the projection content is that a virtual character walks on the screen and interacts with the background scene; the user interacts by touching the wall or waving his arms to affect the character's movements; it is assumed that the system captures a frame of image every 50 milliseconds; the first frame shows that the virtual character is standing on the left side of the scene, and the background is a static city scene; the second frame shows that the character starts to walk, and the moving distance is about 10 pixels. The background city scene also changes dynamically with the movement of the character; by comparing the continuous frame data, the system uses the optical flow method to compare the changes between the first frame and the second frame; the optical flow method extracts the motion information of each pixel by analyzing the pixel differences in continuous image frames, including the motion direction (horizontal / vertical) and motion speed;

[0118] For example, in the first frame, the head of the person is located in the upper left corner of the image, and in the second frame, the head of the person moves 10 pixels to the lower right. The system uses the optical flow method to calculate the motion vector of the head area, and the movement direction of the head of the person is "lower right" and the movement speed is 10 pixels / frame. Through the algorithm of the optical flow method, the system will obtain the motion vector of each pixel. The following are some possible calculation results:

[0119] The character's head moves from left to right, and the movement direction of each pixel is "rightward";

[0120] The character's arm moves from top to bottom, and the movement direction of each pixel is "downward";

[0121] The buildings in the background will also change as the characters move, so the pixels in the background will also have a certain motion vector, and the motion direction is "horizontal right shift";

[0122] For example, suppose the character's arm is located at the upper left of the screen in the first frame and at the lower right in the second frame. The system compares the images and finds that the motion vector of the area is "from the upper left to the lower right", and calculates the moving speed of each pixel to be 5 pixels / frame; that is, the optical flow compensation calculation is performed. Once the system obtains the motion vector of each pixel, the next task is to dynamically adjust the projection content to ensure that the projection image is not distorted or jumpy.

[0123] For example, if in the projection area, the person's arm is located in the occlusion area of ​​the object, causing the projection content in that area to be distorted or blurred, the system can predict the real content of that area based on the motion vector (i.e., the movement direction and speed of the arm) calculated by the optical flow method, and restore it to a normal state;

[0124] Then, real-time geometric distortion correction is performed. If the wall is tilted, the position and angle of the projected content in the physical space will change, resulting in geometric distortion of the image. The system dynamically adjusts the geometric shape of the projected content based on the motion vector and wall angle information to ensure the correct display of the content. For example, if the wall is tilted by 10 degrees, the system will correct the display position of each pixel according to the tilt angle to avoid deformation or stretching of the projected image.

[0125] Then, the projected content is corrected. The system will make precise corrections to the projected content based on the dynamically calculated motion vector. For example, if the head of a person moves to the lower right in the image, the system will adjust the display position of the projected content based on the motion vector calculated by the optical flow method to align the head of the person correctly in the image. If the position of the arm changes due to the waving of the user's arm, the system will smooth the area of ​​the arm to eliminate the distortion effect. The buildings and scenery in the background scene change dynamically as the person walks. The system predicts the changes in the background content by calculating the motion vectors of the pixels in the background and dynamically updates the projected image.

[0126] Finally, the corrected projection content is output. According to the above correction steps, the system projects the optimized projection content onto the wall in real time. Regardless of the movement of the characters, the change of the background, or the tilt of the wall, all dynamic compensations ensure the stability of the projection content in the physical space, avoiding distortion or picture jumps.

[0127] Suppose the user is interacting and waving his arms to make the character dance; due to the rapid movement of the arms, the position of the arms in the projected image may be distorted, and the edges of the arms may become blurred or jump; through the optical flow method, the system calculates the movement direction and speed of each pixel in the arm area (for example, the arm moves from the upper left to the lower right, moving 3 pixels per second), and then smoothes the area based on this information; the system will reposition each pixel in the arm area according to the motion vector, correct its position, and restore the normal display effect;

[0128] In summary, in the above example, through the optical flow method, the system can calculate the motion vector of each pixel in real time, and then accurately correct the distortion or jump of the projected content; whether it is occlusion by wall objects, wall tilt, or rapid interactive actions by users, the optical flow method can effectively help the system understand the dynamics of image changes and optimize the projection effect, thereby ensuring the consistency and stability of the picture.

[0129] In this embodiment, the step S4 of determining whether the projection content can be displayed synchronously with the interactive content input by the user further includes:

[0130] S41: Based on the sensors preset in the wall projection area, capturing the movement trajectory corresponding to the interactive content in real time, wherein the sensors specifically include infrared sensors, touch sensors and pressure sensors;

[0131] S42: Determine whether the movement trajectory matches the mapping rule preset in the CIM model;

[0132] S43: If yes, respond to the corresponding dynamic display content on the wall projection area according to the movement trajectory, detect the position change information of the user in the wall projection area, and dynamically adjust the display position of the projection content according to the position change information.

[0133] In this embodiment, the system is based on the sensors pre-set in the wall projection area, which specifically include infrared sensors, touch sensors and pressure sensors, to capture the movement trajectory corresponding to the interactive content in real time, and then the system determines whether these movement trajectories match the mapping rules pre-set in the CIM model to execute the corresponding steps; for example, when the system determines that the movement trajectory corresponding to the interactive content cannot match the mapping rules pre-set in the CIM model, the system will consider that the user's interactive behavior deviates from the interactive model preset by the system, and the system will dynamically update the mapping rules by real-time monitoring and learning the user's interactive behavior, and at the same time, through artificial intelligence or machine learning algorithms, the system can gradually adjust CIM model is used to adapt to the behavior patterns of different users. For example, the system collects the user's trajectory data (such as gestures, touch points, pressure points, etc.) and reconstructs the mapping rules according to the user's interaction habits to make it more flexible and accurate. The sensor is calibrated regularly to ensure that it can accurately capture the interaction trajectory under different environmental conditions. For example, the sensitivity of the infrared sensor is enhanced, or the response curve of the pressure sensor is optimized to ensure stable operation in various environments. For example, when the system determines that the movement trajectory corresponding to the interactive content can match the mapping rules preset by the CIM model, the system will consider that the user's interactive behavior has successfully interacted with the system's preset interaction model, and the system will According to these movement trajectories, the corresponding dynamic display content is responded to on the wall projection area, the user's position change information in the wall projection area is detected, and the display position of the projection content is dynamically adjusted according to different position change information; the system can provide accurate projection content by responding to the user's actions and changes in real time, ensuring that the user's interactive operations receive instant feedback. This smooth experience can enhance the user's sense of immersion and make it easier for the user to interact naturally with the system. The projection content is adjusted according to the user's actions and position changes, and personalized visual effects can be provided for different users. This dynamic response capability allows each user to enjoy customized interactive content, and at the same time can be based on the user's position on the wall projection. The system can monitor and adjust the projection content in real time based on the position change information in the projection area, and flexibly adjust the display position of the projection content. That is, no matter how the user moves, the projection content can always be accurately aligned to the preset position, avoiding distortion or jumping, ensuring that the visual effect is always stable and clear, and through precise movement trajectory and position change information, the system can monitor and adjust the projection content in real time to reduce the projection distortion or unstable picture caused by irregular wall or too fast user movement. According to the user's position change in the wall projection area, the system dynamically adjusts the position of the projection content to ensure that the projection content and the interactive content are always consistent. This synchronization can effectively avoid projection misalignment or incoordination caused by user displacement.

[0134] In this embodiment, the step S1 of collecting the environmental change data of the wall projection area in real time based on the projector preset in the wall projection area also includes:

[0135] S11: Based on the projection effect preset by the projector, detecting the color deviation information of the wall projection area, wherein the projection effect specifically includes brightness, focus and color;

[0136] S12: Determine whether the color deviation information meets the preset projection requirement;

[0137] S13: If not, then according to the projection requirement, identify the influencing parameters of the color deviation information on the projection message, and generate the influencing parameters in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

[0138] In this embodiment, the system detects the color deviation information of the wall projection area based on the projection effect preset by the projector, which specifically includes brightness, focus and color, and then the system determines whether the color deviation information meets the preset projection requirements to execute the corresponding steps; for example, when the system determines that the color deviation information of the wall projection area can meet the preset projection requirements, the system will consider that the color performance of the wall projection effect meets expectations and the visual quality of the projected content has reached the required standard, and the system will maintain the current projection settings to ensure that the picture does not have color distortion or inconsistency during continuous use, which helps to maintain the picture. The system can regularly or as needed fine-tune the projection effect to keep the color and brightness at the best state to cope with possible environmental changes. After the color deviation reaches the preset standard, the system can enter a stable operation state to ensure that there will be no color deviation problem when the projected content is displayed for a long time. This is very important for application scenarios that require long-term content display (such as conferences, performances, digital signage, etc.); for example, when the system determines that the color deviation information of the wall projection area cannot meet the preset projection requirements, the system will automatically adjust the projection effect to ensure that the color deviation information of the wall projection area can be adjusted to the optimal state. At this time, the system will think that the color of the wall projection effect does not meet expectations. The system will identify the influencing parameters of the color deviation information on the projection information according to the pre-set projection requirements. The influencing parameters specifically include color temperature, color gamut and color difference, and generate these influencing parameters on the panel pre-set by the projector; the system can maintain the color accuracy of the projected content under different environmental conditions (such as light changes, wall material changes, etc.) by real-time identification and adjustment of the parameters affecting the color (such as color temperature, color gamut and color difference). This can effectively avoid the color deviation problem caused by environmental changes or equipment differences, and at the same time, by adjusting the color parameters, the projection content can be adaptively adjusted. The ability of the system enables the projection to maintain ideal color effects under different wall surfaces and environmental conditions, reducing the need for manual intervention and increasing the intelligence level of the system. By identifying color deviations and adjusting the projector panel, the system can achieve automatic color correction without manual intervention. This not only saves adjustment time, but also reduces human errors and improves overall efficiency and accuracy. Color correction can respond to environmental changes in real time, such as changes in light intensity and different wall materials. The system automatically adjusts the color parameters of the projected content by detecting changes in the environment and equipment status to ensure that the color always meets the preset requirements and improves the stability of the projection effect.

[0139] Reference Figure 2 , is a wall interactive projection system based on CIM digital twin in one embodiment of the present invention, comprising:

[0140] The acquisition module 10 is used to acquire the environmental change data of the wall projection area in real time based on the projector preset in the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data and person movement data;

[0141] A judging module 20, used to judge whether the environmental change data detects a preset projection distortion jump;

[0142] An execution module 30 is configured to, if yes, divide the wall projection area into at least two projection partitions according to the preset projection parameters of the wall projection area, identify the interactive influence information of the projection partitions, and dynamically compensate the corresponding projection content by applying a preset CIM model according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure and wall texture;

[0143] A second determination module 40 is used to determine whether the projection content can be displayed synchronously with the interactive content input by the user;

[0144] The second execution module 50 is used for, if not possible, collecting coordinate information corresponding to the interactive content based on the spatial coordinate data preset for the wall projection area by the CIM model, dynamically adjusting the interactive response of the projection content according to the coordinate information, and synchronously connecting the time nodes of the interactive content and the projection content on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in the three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0145] In this embodiment, the acquisition module 10 acquires the environmental change data of the wall projection area in real time based on the projector pre-installed in the wall projection area. The environmental change data specifically includes object occlusion data, angle change data and person movement data. Then the judgment module 20 determines whether these environmental change data detect the preset projection distortion jump to execute the corresponding steps; for example, when the system determines that the environmental change data of the wall projection area does not detect the preset projection distortion jump, the system will consider that the projection information remains stable in the current environment and is not affected by object occlusion, angle change or person movement. The system will continue to output the current projection content to ensure the stability and accuracy of the picture, and continue to acquire the environmental change data of the wall projection area in real time to ensure that it can respond in time to any changes in the future, and record the relevant data of the current environment and projection status (such as wall structure, projection angle, ambient light, etc.) to provide data for subsequent tuning and analysis. Support; for example, when the system determines that the environmental change data of the wall projection area detects a preset projection distortion jump, the execution module 30 will consider that the projection information is affected, and the system will divide at least two projection partitions from the wall projection area according to the projection parameters preset in the wall projection area, the projection parameters specifically include wall size, wall structure and wall texture, identify the interactive impact information of these projection partitions, and dynamically compensate the corresponding projection content according to different interactive impact information using a preset CIM model; the system divides the wall projection area into at least two projection partitions according to preset projection parameters such as wall size, structure and texture, so that the affected area can be accurately located, and adjustments can be made to the specific projection area to avoid unnecessary waste of resources or affecting the projection effects of other parts due to global adjustments. At the same time, according to different interactive impact information, the system dynamically compensates the corresponding projection content by applying the CIM model. This compensation can adjust the projection content in real time according to the changes in the wall projection area (such as occlusion, angle change or character movement), ensure the stability of the picture quality, and identify the specific interactive influences (such as user behavior, object movement, etc.) on different projection partitions, and make adjustments based on these influence information. In this way, the projection content can be better synchronized with the user's interaction to achieve a highly accurate interactive effect; then the second judgment module 40 determines whether these projection contents can be synchronously displayed to the wall projection area with the interactive content input by the user in real time, so as to execute the corresponding steps;For example, when the system determines that the dynamically compensated projection content can be synchronously displayed with the interactive content input by the user in real time to the wall projection area, the system will consider that the projection content and the interactive content have been successfully connected and can be kept synchronized, thereby achieving a seamless connection between the projection content and the user's interactive behavior. The system will continue to monitor the user's input changes to ensure that the projection content can still be kept synchronized under any new interactive behavior or environmental changes in the future, maintain the continuous stability of the interactive experience, prevent synchronization problems caused by user behavior or environmental changes, and ensure the stability of the interaction. At the same time, if the environmental conditions (such as lighting, object occlusion, etc.) change, the system should be able to automatically adjust the projection content to ensure that the effect of the synchronous display is not affected, and ensure that the projection picture The surface always maintains the best display effect under different environmental conditions, improves the adaptability and stability of the system, and records the user's interactive behavior and the synchronous display of the projection content, generates corresponding logs, so as to facilitate the later analysis and optimization of system performance. By recording and analyzing user behavior data, the system can find potential room for improvement in the subsequent optimization process, and further improve the stability of the system and user experience; for example, when the system determines that the dynamically compensated projection content cannot be synchronously displayed to the wall projection area with the interactive content input by the user in real time, the second execution module 50 will think that the projection content and the interactive content cannot be synchronously connected, and the system will pre-set the spatial coordinate data of the wall projection area based on the CIM model. The specific spatial coordinate data The coordinate information corresponding to the interactive content is collected for the positioning coordinates of the wall in three-dimensional space. The coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall. The interactive response of the projected content is dynamically adjusted according to these coordinate information, and the time nodes of the interactive content and the projected content are synchronously connected in the wall projection area. By obtaining the coordinate information such as the azimuth, inclination angle and normal vector of the wall, the system can adjust the projection angle, position and content in real time to confirm that the projection content matches the user's interactive behavior, ensure the precise synchronization between the interactive content and the projected image, avoid image distortion, jumping or delay problems caused by changes in the wall angle or asynchronous interactive content, and use the precise coordinate data of the wall and interactive content to quickly respond to user requests. The real-time dynamic compensation can adjust the display position, direction and content of the projected content according to the changes in user input. This real-time dynamic compensation makes the interaction between the user and the system more sensitive and natural, enhances the system's response speed to the environment and user interaction, reduces delays, improves the interactivity and flexibility of the system, and enhances the user experience. In addition, the adjustment based on the wall space coordinate data can not only improve the accuracy of the projection, but also optimize the use of the projection area. For example, when the wall angle changes, the projector can adjust the direction of the projected content in real time, so that the picture always adapts to the wall structure, avoids picture distortion or edge blur, improves space utilization, and ensures that even on irregular or complex walls, the projected content can still be displayed clearly and completely, improving the adaptability and quality of the projection effect. ;

[0146] In this embodiment, the execution module further includes:

[0147] A generating unit, configured to generate a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness and contrast;

[0148] A judging unit, used to judge whether the current projection area matches a preset optimal projection area;

[0149] an execution unit, for, if not, obtaining reflection parameters of the light from the wall projection area through the light emitted by the projector, collecting parameters to be corrected of the wall projection area according to the reflection parameters, and controlling the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, the reflection angle and the material properties of the reflection surface, and the parameters to be corrected specifically include the wall curvature and the wall inclination.

[0150] In this embodiment, the system generates a current projection area of ​​the wall projection area based on the pre-set content requirements of the projection content, which specifically include content ratio, resolution, brightness and contrast, and then the system determines whether the current projection area matches the pre-set optimal projection area to execute the corresponding steps; for example, when the system determines that the current projection area of ​​the wall projection area can match the pre-set optimal projection area, the system will consider that the current projection effect has met the preset quality requirements, and the system will continue to project under the current settings to ensure the smoothness of the user experience and the stability of the visual effect without any additional correction or adjustment. At the same time, after ensuring that the current projection area matches the optimal projection area, the system will continue to interact with the user. If the user changes the angle or spatial position of the wall projection area, the system will continue Use existing parameters to adjust the projection to ensure that the new interactive area still meets the optimal projection standards, and ensure that the projection content is synchronized with the interactive content input by the user, so that the user can get instant and clear feedback every time they interact, providing a smooth interactive experience; for example, when the system determines that the current projection area of ​​the wall projection area cannot match the preset optimal projection area, the system will consider that the current projection effect cannot meet the preset quality requirements. The system will use the light emitted by the projector to obtain the reflection parameters of these lights from the wall projection area. The reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflection surface. According to these reflection parameters, the parameters to be corrected of the wall projection area are collected. The parameters to be corrected specifically include wall curvature and wall inclination. According to these parameters to be corrected, the projection content is controlled to perform geometric distortion correction;The system can accurately measure the reflection of the projection area and identify the shape and characteristics of the wall through the light emitted by the projector and its reflection path, angle, and surface material characteristics. According to the parameters to be corrected collected by the reflection parameters, the system can dynamically perform geometric correction on the projection content. By controlling the geometric shape and angle of the projection content, the image distortion caused by the irregular wall surface is avoided, and the clarity and accuracy of the projection content is maintained. At the same time, the reflection parameters can provide the projection system with real-time data on the physical characteristics of the wall surface, ensuring that the projected image will not be distorted or blurred due to the irregular wall surface, and preventing the irregular surface characteristics of the wall (such as uneven and tilted wall surface) from causing distortion or blur. The system can accurately reflect the impact of wall changes by collecting and adjusting the reflection parameters of the wall projection area in real time, ensuring that the projection content can be adjusted in time and presented synchronously with the user's operation, avoiding interaction delays or distortion caused by inconsistent wall shapes. That is, the system can adapt to different wall environments. Even if the wall is irregular or deformed during the projection process, the system can adjust the projection effect in real time to ensure the stability of the quality of the wall projection area and avoid adverse effects caused by wall changes. ;

[0151] In this embodiment, it also includes:

[0152] A construction module, configured to construct a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin predefined by the CIM model for the wall projection area, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0153] A third judgment module is used to judge whether the virtual three-dimensional coordinate system can match the projection field of view of the projector;

[0154] The third execution module is used to measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, and obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0155] In this embodiment, the system constructs a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin pre-defined for the wall projection area by the CIM model. The virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis. The system then determines whether these virtual three-dimensional coordinate systems can match the projection field of view of the projector to execute corresponding steps. For example, when the system determines that the virtual three-dimensional coordinate system applicable to the wall projection area cannot match the projection field of view of the projector, the system will believe that there is a difference between the coordinate system of the wall projection area and the visible range of the projector, resulting in the projection content may not be fully covered or distorted, or even unable to be accurately aligned or stably displayed. The system will change the position of the projector to make it cover the wall projection area. The entire coordinate range of the virtual three-dimensional coordinate system is ensured to be connected with the projection field of view of the projector, including moving the projector forward and backward, or adjusting the projection angle of the projector as needed, and appropriately reducing the range of the virtual three-dimensional coordinate system to match the projection field of view of the projector. The maximum values ​​of the X, Y, and Z axes of the virtual coordinate system can be reset to avoid conflicts with the physical limitations of the projector. If the wall projection area is too large or irregular, the system can dynamically adjust the size and angle of the projection content according to the shape of the wall and the projection field of view of the projector to adapt to the field of view of the projector, so as to ensure that the display effect of the projection content on the wall is more accurate and stable; for example, when the system determines that the virtual three-dimensional coordinate system is suitable for the wall projection area The system can match the projection field of view of the projector. At this time, the system will consider that the coordinate system of the wall projection area is the same as that of the projector. The system will measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system. The wall parameters specifically include the wall size and wall shape, and obtain the path information from the projector to the wall projection area. The path information specifically includes the distance between the projector's position and the wall, the projector's focal length and field of view. By matching the virtual three-dimensional coordinate system with the projector's projection field of view, the system can ensure that the size, shape, angle and other characteristics of the wall projection area are consistent with the content output by the projector, avoid image distortion, blur or dislocation problems, obtain the path information from the projector to the wall, ensure that the projection content can be accurately projected to a specific position on the wall, and optimize After measuring the clarity and display effect of the picture and the wall parameters (such as wall size and shape), the system can dynamically adjust the size, proportion, direction, etc. of the projected content to adapt it to the actual characteristics of the wall, helping the system to adjust the projection parameters according to the actual projection environment, such as projection angle, focus setting and content resolution, in order to obtain the best projection effect. By accurately measuring the wall size and shape, the system can automatically adjust the match between the projection content and the wall under different environmental conditions, not only to adapt to static walls, but also to cope with dynamic environmental changes. When changes occur in the wall projection area (such as object movement, wall tilt, etc.), the system can quickly adjust based on real-time feedback of path information and wall parameters to ensure the synchronization of projection and interactive content.

[0156] In this embodiment, the second execution module further includes:

[0157] An acquisition unit, configured to collect user coordinate information of the user in the wall projection area, and acquire an interaction area between the user coordinate information and the projection content;

[0158] A second determination unit, configured to determine whether the interactive area can accommodate both the projection content and the interactive content;

[0159] The second execution unit is used to deploy preset layers in the same space of the wall projection area if not, and based on the preset layers, apply a preset display order to synchronously display the projection content and the interactive content, wherein the preset layers specifically include a front-end layer and a background layer.

[0160] In this embodiment, the system acquires the user coordinate information of the user on the wall projection area to obtain the interactive area of ​​the user coordinate information and the projection content, and then the system determines whether the interactive area can accommodate the projection content and the interactive content at the same time to execute the corresponding steps; for example, when the system determines that the interactive area of ​​the user coordinate information and the projection content can accommodate the projection content and the interactive content at the same time, the system will consider that there is no conflict between the current projection area and the user's interactive needs in the physical space, and efficient interactive display can be achieved. The system will monitor the user's coordinate information in real time, and the system can adjust the projection content according to the user's actions to ensure that the projection and the user's interaction are in the same spatial coordinate range, and the interactive area can accommodate the projection. In the case of content and interactive content, the system can automatically adjust the projection brightness, resolution, contrast and other parameters to adapt to different lighting environments or changes in user position to ensure the best quality of the display effect. At the same time, the system can also adjust the response of the projected content based on the user's specific interactive behavior (for example, finger touch, body movement, etc.), so that the user's action can directly affect the content or form of projection, and the system can effectively use the entire wall projection area for content display and user interaction, avoiding the problem of overlapping or occlusion of projection content and interactive content due to insufficient interactive space, achieving a seamless transition between the user and the projected content, and ensuring that the interaction between the user and the projection is not limited by physical space, thereby optimizing the space. for improving the efficiency of use; for example, when the system determines that the interactive area between the user coordinate information and the projected content cannot accommodate both the projected content and the interactive content, the system will consider that there is a conflict between the current projection area and the user's interactive needs in physical space, and the system will deploy pre-set layers in the same space of the wall projection area. The preset layers specifically include the front-end layer and the background layer, and the projection content and the interactive content are displayed synchronously based on the pre-set display order of the layer application; the system can effectively solve the spatial conflict problem between the user's interactive needs and the projection content by introducing the layer management of the front-end layer and the background layer. When the interactive area cannot accommodate the projection and interactive content, the system will allocate the interactive content and the projection content to Different layers ensure that the two will not block or overlap each other. At the same time, the relationship between the user's operation feedback and the projected content can be accurately controlled through the allocation and display order of layers. Through the front-end layer, the user's operation is not affected by the background content. The system can display the real-time response of the user's interaction on the front-end layer, while the background layer continues to display the projected content, ensuring the accuracy of the operation and the immediacy of the feedback. In addition, through the preset layers and display sorting rules, the system can intelligently schedule the display order of various types of content to meet different application scenarios and user needs. For example, when displaying product advertisements, the advertising content of the background layer can be covered by the interactive elements of the front-end layer, and when no interaction is required, the advertising content will be restored to full-screen display.

[0161] In this embodiment, the judgment module further includes:

[0162] A detection unit, configured to detect a fluctuation amplitude of the projection information based on a projection information preset by the projector;

[0163] A third judging unit, used to judge whether the jitter amplitude reaches a preset distortion standard;

[0164] The third execution unit is used to apply a preset optical flow method to collect continuous frame data of the projection message before and after the change, and calculate the motion vector of each pixel point corresponding to the projection message by comparing the continuous frame data, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

[0165] In this embodiment, the system detects the amplitude of the projection information based on the projection information preset by the projector, and then the system determines whether the amplitude of the projection information reaches the preset distortion standard to execute the corresponding steps; for example, when the system determines that the amplitude of the projection information does not reach the preset distortion standard, the system will consider that the current projection effect is stable, and there is no distortion or picture jump phenomenon. The system will continue to maintain the current projection settings, including brightness, contrast, color and other parameters, and continuously monitor the environmental changes in the wall projection area, such as object occlusion, angle change and person movement, to ensure that the projection effect will not be affected by these factors. Sudden distortion and jumps occur, and the synchronous response of the projected content and the interactive content is further adjusted to enhance the user's interactive experience. For example, the clarity of the content is adjusted or the response delay is reduced to make the feedback of the projected content and the user's operation smoother, and the display of the interactive content can be continuously optimized to make its synchronous response with the projected content more accurate. By real-time tracking of the user's input, the system can adjust the interactive response time, the position of the interactive interface, etc., to make the temporal and spatial relationship between the projected content and the user's operation smoother. For example, when the system determines that the jump amplitude of the projection message has reached the preset distortion standard, the system will consider that the current projection effect is not good. Stable, prone to distortion or screen jump, the system will apply the pre-set optical flow method to collect continuous frame data of the projection information before and after the change, and calculate the motion vector of each pixel point corresponding to the projection information by comparing these continuous frame data. The motion vector specifically includes the movement speed and movement direction of the pixel point; the system can adjust the projection content based on the motion vector by obtaining the motion information of each pixel point, and correct the distortion or jump in the picture in real time. For example, the system can adjust the edge, focus area or geometric deformation of the projection according to the direction of movement to avoid the projection content from changing due to the environment. Through precise motion compensation, it can avoid image jump. The optical flow method can reduce the visual interference caused by users and improve the stability and smoothness of the viewing experience. At the same time, the real-time data acquisition and processing of the optical flow method can enable the system to quickly identify the distortion or jump of the projection image, and adjust the projection content in a very short time to achieve rapid correction and reduce the intervention of users and operators on the projector. That is, the system can automatically correct the projection content according to the calculated motion vector without manual adjustment of the projection equipment. In interactive projection applications, the optical flow method can ensure that the projection content and the user's actions (such as gestures, touch, etc.) are always synchronized, avoiding interaction delays or incoherence caused by picture jumps or distortions, and improving the naturalness and accuracy of the interactive experience.

[0166] In this embodiment, the second determination module further includes:

[0167] A capturing unit, configured to capture the movement trajectory corresponding to the interactive content in real time based on a sensor preset in the wall projection area, wherein the sensor specifically includes an infrared sensor, a touch sensor, and a pressure sensor;

[0168] A fourth judgment unit, used to judge whether the movement trajectory matches the mapping rule preset in the CIM model;

[0169] The fourth execution unit is used to respond to the corresponding dynamic display content on the wall projection area according to the movement trajectory, detect the position change information of the user in the wall projection area, and dynamically adjust the display position of the projection content according to the position change information.

[0170] In this embodiment, the system is based on the sensors pre-set in the wall projection area, which specifically include infrared sensors, touch sensors and pressure sensors, to capture the movement trajectory corresponding to the interactive content in real time, and then the system determines whether these movement trajectories match the mapping rules pre-set in the CIM model to execute the corresponding steps; for example, when the system determines that the movement trajectory corresponding to the interactive content cannot match the mapping rules pre-set in the CIM model, the system will consider that the user's interactive behavior deviates from the interactive model preset by the system, and the system will dynamically update the mapping rules by real-time monitoring and learning the user's interactive behavior, and at the same time, through artificial intelligence or machine learning algorithms, the system can gradually adjust CIM model is used to adapt to the behavior patterns of different users. For example, the system collects the user's trajectory data (such as gestures, touch points, pressure points, etc.) and reconstructs the mapping rules according to the user's interaction habits to make it more flexible and accurate. The sensor is calibrated regularly to ensure that it can accurately capture the interaction trajectory under different environmental conditions. For example, the sensitivity of the infrared sensor is enhanced, or the response curve of the pressure sensor is optimized to ensure stable operation in various environments. For example, when the system determines that the movement trajectory corresponding to the interactive content can match the mapping rules preset by the CIM model, the system will consider that the user's interactive behavior has successfully interacted with the system's preset interaction model, and the system will According to these movement trajectories, the corresponding dynamic display content is responded to on the wall projection area, the user's position change information in the wall projection area is detected, and the display position of the projection content is dynamically adjusted according to different position change information; the system can provide accurate projection content by responding to the user's actions and changes in real time, ensuring that the user's interactive operations receive instant feedback. This smooth experience can enhance the user's sense of immersion and make it easier for the user to interact naturally with the system. The projection content is adjusted according to the user's actions and position changes, and personalized visual effects can be provided for different users. This dynamic response capability allows each user to enjoy customized interactive content, and at the same time can be based on the user's position on the wall projection. The system can monitor and adjust the projection content in real time based on the position change information in the projection area, and flexibly adjust the display position of the projection content. That is, no matter how the user moves, the projection content can always be accurately aligned to the preset position, avoiding distortion or jumping, ensuring that the visual effect is always stable and clear, and through precise movement trajectory and position change information, the system can monitor and adjust the projection content in real time to reduce the projection distortion or unstable picture caused by irregular wall or too fast user movement. According to the user's position change in the wall projection area, the system dynamically adjusts the position of the projection content to ensure that the projection content and the interactive content are always consistent. This synchronization can effectively avoid projection misalignment or incoordination caused by user displacement.

[0171] In this embodiment, the acquisition module further includes:

[0172] A second detection unit is used to detect color deviation information of the wall projection area based on a projection effect preset by the projector, wherein the projection effect specifically includes brightness, focus and color;

[0173] A fifth judgment unit, used to judge whether the color deviation information meets a preset projection requirement;

[0174] The fifth execution unit is used to, if not, identify the influencing parameters of the color deviation information on the projection message according to the projection requirement, and generate the influencing parameters in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

[0175] In this embodiment, the system detects the color deviation information of the wall projection area based on the projection effect preset by the projector, which specifically includes brightness, focus and color, and then the system determines whether the color deviation information meets the preset projection requirements to execute the corresponding steps; for example, when the system determines that the color deviation information of the wall projection area can meet the preset projection requirements, the system will consider that the color performance of the wall projection effect meets expectations and the visual quality of the projected content has reached the required standard, and the system will maintain the current projection settings to ensure that the picture does not have color distortion or inconsistency during continuous use, which helps to maintain the picture. The system can regularly or as needed fine-tune the projection effect to keep the color and brightness at the best state to cope with possible environmental changes. After the color deviation reaches the preset standard, the system can enter a stable operation state to ensure that there will be no color deviation problem when the projected content is displayed for a long time. This is very important for application scenarios that require long-term content display (such as conferences, performances, digital signage, etc.); for example, when the system determines that the color deviation information of the wall projection area cannot meet the preset projection requirements, the system will automatically adjust the projection effect to ensure that the color deviation information of the wall projection area can be adjusted to the optimal state. At this time, the system will think that the color of the wall projection effect does not meet expectations. The system will identify the influencing parameters of the color deviation information on the projection information according to the pre-set projection requirements. The influencing parameters specifically include color temperature, color gamut and color difference, and generate these influencing parameters on the panel pre-set by the projector; the system can maintain the color accuracy of the projected content under different environmental conditions (such as light changes, wall material changes, etc.) by real-time identification and adjustment of the parameters affecting the color (such as color temperature, color gamut and color difference). This can effectively avoid the color deviation problem caused by environmental changes or equipment differences, and at the same time, by adjusting the color parameters, the projection content can be adaptively adjusted. The ability of the system enables the projection to maintain ideal color effects under different wall surfaces and environmental conditions, reducing the need for manual intervention and increasing the intelligence level of the system. By identifying color deviations and adjusting the projector panel, the system can achieve automatic color correction without manual intervention. This not only saves adjustment time, but also reduces human errors and improves overall efficiency and accuracy. Color correction can respond to environmental changes in real time, such as changes in light intensity and different wall materials. The system automatically adjusts the color parameters of the projected content by detecting changes in the environment and equipment status to ensure that the color always meets the preset requirements and improves the stability of the projection effect.

[0176] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The wall interactive projection method based on CIM digital twin is characterized by: The following steps are involved: Based on a projector preset in the wall projection area, real-time collection of environmental change data of the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data and person movement data; Determining whether the environmental change data detects a preset projection distortion jump; If so, at least two projection partitions are divided from the wall projection area according to the preset projection parameters of the wall projection area, the interactive influence information of the projection partitions is identified, and the preset CIM model is applied to dynamically compensate the corresponding projection content according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure and wall texture: Determining whether the projected content can be displayed synchronously with the interactive content input by the user; If not, based on the spatial coordinate data preset for the wall projection area by the CIM model, the coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area, wherein the spatial coordinate data are specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

2. The wall interactive projection method based on CIM digital twin according to claim 1 is characterized in that: The step of applying a preset CIM model to dynamically compensate the corresponding projection content according to the interactive impact information further includes: Based on the preset content requirements of the projection content, generating the current projection area of ​​the wall projection area, wherein the content requirements specifically include content ratio, resolution, brightness and contrast; Determining whether the current projection area matches a preset optimal projection area; If not, then obtain the reflection parameters of the light from the wall projection area through the light emitted by the projector, collect the parameters to be corrected of the wall projection area according to the reflection parameters, and control the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

3. The wall interactive projection method based on CIM digital twin according to claim 1 is characterized in that: Before the step of applying a preset CIM model to dynamically compensate the corresponding projection content according to the interactive impact information, the method further includes: Based on the coordinate origin predefined by the CIM model for the wall projection area, construct a virtual three-dimensional coordinate system applicable to the wall projection area, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis; Determining whether the virtual three-dimensional coordinate system can match the projection field of view of the projector; If so, the wall parameters of the wall projection area are measured through the virtual three-dimensional coordinate system to obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

4. The wall interactive projection method based on CIM digital twin according to claim 1 is characterized in that: The step of collecting coordinate information corresponding to the interactive content and dynamically adjusting the interactive response of the projection content according to the coordinate information further includes: Collecting user coordinate information of the user in the wall projection area, and obtaining an interaction area between the user coordinate information and the projection content; Determining whether the interactive area can accommodate the projection content and the interactive content at the same time; If not, a preset layer is deployed in the same space of the wall projection area, and based on the preset layer, the projection content and the interactive content are synchronously displayed using a preset display order, wherein the preset layer specifically includes a front-end layer and a background layer.

5. The wall interactive projection method based on CIM digital twin according to claim 1 is characterized in that: The step of determining whether the environmental change data detects a preset projection distortion jump also includes: Based on the projection information preset by the projector, detecting the jumping amplitude of the projection information; Determining whether the jitter amplitude reaches a preset distortion standard; If so, apply the preset optical flow method to collect continuous frame data of the projection message before and after the change, and calculate the motion vector of each pixel point corresponding to the projection message by comparing the continuous frame data, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

6. The wall interactive projection method based on CIM digital twin according to claim 1 is characterized in that: The step of determining whether the projected content can be displayed synchronously with the interactive content input by the user further includes: Based on the preset sensors in the wall projection area, the movement trajectory corresponding to the interactive content is captured in real time, wherein the sensors specifically include infrared sensors, touch sensors and pressure sensors; Determining whether the movement trajectory matches the mapping rule preset in the CIM model; If so, respond to the corresponding dynamic display content on the wall projection area according to the movement trajectory, detect the position change information of the user in the wall projection area, and dynamically adjust the display position of the projection content according to the position change information.

7. The wall interactive projection method based on CIM digital twin according to claim 1 is characterized in that: The step of collecting the environmental change data of the wall projection area in real time by the projector preset based on the wall projection area also includes: Based on the projection effect preset by the projector, the color deviation information of the wall projection area is detected, wherein the projection effect specifically includes brightness, focus and color; Determining whether the color deviation information meets a preset projection requirement; If not, then according to the projection requirements, the influencing parameters of the color deviation information on the projection message are identified, and the influencing parameters are generated in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

8. The wall interactive projection system based on CIM digital twin is characterized by: include: A collection module, which is used to collect environmental change data of the wall projection area in real time based on a projector preset in the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data and person movement data; A judging module, used for judging whether the environmental change data detects a preset projection distortion jump; an execution module, configured to, if yes, divide the wall projection area into at least two projection partitions according to the preset projection parameters of the wall projection area, identify the interactive influence information of the projection partitions, and dynamically compensate the corresponding projection content by applying the preset CIM model according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure and wall texture; A second judgment module is used to judge whether the projection content can be displayed synchronously with the interactive content input by the user; The second execution module is used for, if not possible, collecting coordinate information corresponding to the interactive content based on the spatial coordinate data preset for the wall projection area by the CIM model, dynamically adjusting the interactive response of the projection content according to the coordinate information, and synchronously connecting the time nodes of the interactive content and the projection content on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in the three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

9. The CIM digital twin-based wall interactive projection system according to claim 8, characterized in that: The execution module also includes: A generating unit, configured to generate a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness and contrast; A judging unit, used to judge whether the current projection area matches a preset optimal projection area; an execution unit, for, if not, obtaining reflection parameters of the light from the wall projection area through the light emitted by the projector, collecting parameters to be corrected of the wall projection area according to the reflection parameters, and controlling the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, the reflection angle and the material properties of the reflection surface, and the parameters to be corrected specifically include the wall curvature and the wall inclination.

10. The wall interactive projection system based on CIM digital twin according to claim 8, characterized in that: Also includes: A construction module, configured to construct a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin predefined by the CIM model for the wall projection area, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis; A third judgment module is used to judge whether the virtual three-dimensional coordinate system can match the projection field of view of the projector; The third execution module is used to measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, and obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

Images