Wall surface interactive projection method and system based on CIM digital twinning

Abstract

The application provides a wall surface interactive projection method and system based on CIM digital twinning, and is applied to the field of projection data processing; through real-time collection of environmental change data of a wall surface projection area, the application can dynamically adjust projection content in combination with a CIM model, ensures stability in a physical space, synchronizes interactive content input by a user with projection content with high precision through division of projection partitions and real-time compensation of interactive influence, automatically corrects projection distortion or jumping under environmental changes, and uses three-dimensional space coordinate data for accurate positioning and dynamic calibration, optimizes a projection path and angle, thereby avoiding distortion problems caused by angle changes or shielding, and improving stability of projection effect and user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of projection data processing, in particular to a wall interactive projection method and system based on CIM digital twinning. BACKGROUND

[0002] With the continuous development of the times, various different entertainment games are emerging, such as projection games, including desktop projection games, ground projection games and wall projection games, and all through the real-time combination of interactive sensing and image display, all special programs will make the image generation and interactive sensing change synchronously.

[0003] Since the projection system needs to present clear and stable images on the wall, factors such as the material, color and light change of the wall will affect the projection effect, and it is difficult to persistently guarantee high-quality projection output on different types of walls. SUMMARY

[0004] The present application aims to solve the problem of how to ensure the stability of the projection picture in the physical space and avoid the distortion or jumping of the projection content caused by the angle change or the shielding of the moving object, and provides a wall interactive projection method and system based on CIM digital twinning.

[0005] The present application adopts the following technical means to solve the technical problems:

[0006] The present application provides a wall interactive projection method based on CIM digital twinning, comprising:

[0007] A projector preset based on a wall projection area, which collects environmental change data of the wall projection area in real time, wherein the environmental change data specifically includes object shielding data, angle change data and character movement data;

[0008] Judging whether the environmental change data detects a preset projection distortion and jumping;

[0009] If yes, at least two projection subareas are divided from the wall projection area according to the projection parameters preset for the wall projection area, the interactive influence information received by the projection subareas is identified, and the corresponding projection content is dynamically compensated according to the interactive influence information by applying a preset CIM model, wherein the projection parameters specifically include wall size, wall structure and wall texture;

[0010] Judging whether the projection content can be synchronously displayed with the interactive content input by the user;

[0011] If not, based on the spatial coordinate data preset for the wall projection area by the CIM model, the coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0012] Furthermore, the step of applying a preset CIM model to dynamically compensate corresponding projection content according to the interactive impact information further includes:

[0013] Based on the preset content requirements of the projection content, generating a current projection area of ​​the wall projection area, wherein the content requirements specifically include content ratio, resolution, brightness and contrast;

[0014] Determining whether the current projection area matches a preset optimal projection area;

[0015] If not, the reflection parameters of the light from the wall projection area are obtained through the light emitted by the projector, the parameters to be corrected of the wall projection area are collected according to the reflection parameters, and the projection content is controlled to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

[0016] Furthermore, before the step of applying a preset CIM model to dynamically compensate the corresponding projection content according to the interactive impact information, the method further includes:

[0017] Based on the coordinate origin predefined for the wall projection area by the CIM model, a virtual three-dimensional coordinate system applicable to the wall projection area is constructed, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0018] Determining whether the virtual three-dimensional coordinate system matches the projection field of view of the projector;

[0019] If so, the wall parameters of the wall projection area are measured through the virtual three-dimensional coordinate system to obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0020] Furthermore, the step of collecting coordinate information corresponding to the interactive content and dynamically adjusting the interactive response of the projected content according to the coordinate information further includes:

[0021] Collecting user coordinate information of the user in the wall projection area, and obtaining an interaction area between the user coordinate information and the projection content;

[0022] Determining whether the interactive area can accommodate both the projection content and the interactive content;

[0023] If not, a preset layer is deployed in the same space of the wall projection area, and based on the preset layer, the projection content and the interactive content are synchronously displayed using a preset display order, wherein the preset layer specifically includes a front-end layer and a background layer.

[0024] Furthermore, the step of determining whether the environmental change data detects a preset projection distortion jump further includes:

[0025] Based on the projection information preset by the projector, detecting the beating amplitude of the projection information;

[0026] Determining whether the jitter amplitude reaches a preset distortion standard;

[0027] If so, the preset optical flow method is applied to collect continuous frame data of the projection message before and after the change, and by comparing the continuous frame data, the motion vector of each pixel point corresponding to the projection message is calculated, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

[0028] Furthermore, the step of determining whether the projected content can be displayed synchronously with the interactive content input by the user further includes:

[0029] Based on the sensors preset in the wall projection area, the movement trajectory corresponding to the interactive content is captured in real time, wherein the sensors specifically include infrared sensors, touch sensors and pressure sensors;

[0030] Determining whether the movement trajectory matches a mapping rule preset in the CIM model;

[0031] If so, the corresponding dynamic display content is displayed on the wall projection area according to the movement trajectory, the position change information of the user in the wall projection area is detected, and the display position of the projection content is dynamically adjusted according to the position change information.

[0032] Furthermore, the step of collecting the environmental change data of the wall projection area in real time based on the projector preset in the wall projection area further includes:

[0033] Based on the projection effect preset by the projector, detecting the color deviation information of the wall projection area, wherein the projection effect specifically includes brightness, focus and color;

[0034] Determining whether the color deviation information meets a preset projection requirement;

[0035] If not, then according to the projection requirements, the influencing parameters of the color deviation information on the projection message are identified, and the influencing parameters are generated in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

[0036] The present invention also provides a wall interactive projection system based on CIM digital twins, comprising:

[0037] A collection module is used to collect environmental change data of the wall projection area in real time based on a projector preset in the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data, and person movement data;

[0038] A judging module, configured to judge whether a preset projection distortion jump is detected in the environmental change data;

[0039] an execution module, configured to, if yes, divide the wall projection area into at least two projection partitions based on preset projection parameters of the wall projection area, identify interactive influence information of the projection partitions, and dynamically compensate corresponding projection content using a preset CIM model based on the interactive influence information, wherein the projection parameters specifically include wall size, wall structure, and wall texture;

[0040] A second judgment module is used to judge whether the projected content can be displayed synchronously with the interactive content input by the user;

[0041] The second execution module is used to, if not possible, collect coordinate information corresponding to the interactive content based on the spatial coordinate data preset for the wall projection area by the CIM model, dynamically adjust the interactive response of the projection content according to the coordinate information, and synchronously connect the time nodes of the interactive content and the projection content on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0042] Furthermore, the execution module further includes:

[0043] A generating unit, configured to generate a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness, and contrast;

[0044] A judging unit, configured to judge whether the current projection area matches a preset optimal projection area;

[0045] an execution unit, configured to, if not, obtain reflection parameters of the light from the wall projection area through the light emitted by the projector, collect parameters to be corrected of the wall projection area based on the reflection parameters, and control the projection content to perform geometric distortion correction based on the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, the reflection angle, and the material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

[0046] Furthermore, it also includes:

[0047] A construction module, configured to construct a virtual three-dimensional coordinate system applicable to the wall projection area based on a coordinate origin predefined for the wall projection area by the CIM model, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0048] A third judgment module is used to judge whether the virtual three-dimensional coordinate system can match the projection field of view of the projector;

[0049] The third execution module is used to measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, and obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0050] The present invention provides a wall interactive projection method and system based on CIM digital twin, which has the following beneficial effects:

[0051] The present invention collects environmental change data of the wall projection area in real time and combines it with the CIM model to dynamically adjust the projection content to ensure its stability in the physical space. At the same time, by dividing the projection into zones and compensating for interactive effects in real time, the interactive content input by the user and the projection content are accurately synchronized, and projection distortion or jitter is automatically corrected under environmental changes. In addition, the three-dimensional spatial coordinate data is used for precise positioning and dynamic calibration to optimize the projection path and angle, thereby avoiding distortion problems caused by angle changes or occlusions, and improving the stability of the projection effect and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of an embodiment of the wall interactive projection method based on CIM digital twins of the present invention;

[0053] Figure 2 This is a structural block diagram of an embodiment of the wall interactive projection system based on CIM digital twin of the present invention. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The implementation, functional features and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] Reference Attachment Figure 1 , is a wall interactive projection method based on CIM digital twin in one embodiment of the present invention, comprising:

[0057] S1: Based on a projector preset in a wall projection area, collect environmental change data of the wall projection area in real time, wherein the environmental change data specifically includes object occlusion data, angle change data, and person movement data;

[0058] S2: Determine whether the environmental change data detects a preset projection distortion jump;

[0059] S3: If yes, dividing the wall projection area into at least two projection partitions according to preset projection parameters of the wall projection area, identifying interactive influence information of the projection partitions, and dynamically compensating corresponding projection content using a preset CIM model according to the interactive influence information, wherein the projection parameters specifically include wall size, wall structure, and wall texture;

[0060] S4: Determine whether the projected content can be displayed synchronously with the interactive content input by the user;

[0061] S5: If not, based on the spatial coordinate data preset for the wall projection area by the CIM model, the coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0062] In this embodiment, the system collects environmental change data of the wall projection area in real time based on a projector pre-installed in the wall projection area. The environmental change data specifically includes object occlusion data, angle change data and person movement data. The system then determines whether these environmental change data detect pre-set projection distortion jumps to execute corresponding steps; for example, when the system determines that the environmental change data of the wall projection area does not detect the pre-set projection distortion jump, the system will consider that the projection information remains stable in the current environment and is not affected by object occlusion, angle change or person movement. The system will continue to output the current projection content to ensure the stability and accuracy of the picture, and at the same time continue to collect environmental change data of the wall projection area in real time to ensure that it can respond in time to any changes in the future, and record relevant data of the current environment and projection status (such as wall structure, projection angle, ambient light, etc.) to provide data support for subsequent tuning and analysis. For example, when the system determines that the environmental change data of the wall projection area detects a preset projection distortion and jump, the system will consider that the projection information is affected. The system will divide the wall projection area into at least two projection partitions based on the projection parameters preset in the wall projection area. The projection parameters specifically include wall size, wall structure and wall texture, identify the interactive impact information of these projection partitions, and dynamically compensate the corresponding projection content based on different interactive impact information using a preset CIM model; the system divides the wall projection area into at least two projection partitions based on preset projection parameters such as wall size, structure and texture, so that the affected area can be accurately located, and adjustments can be made to specific projection areas to avoid unnecessary resource waste or affecting the projection effects of other parts caused by global adjustments. At the same time, based on different interactive impact information, the system dynamically compensates the corresponding projection content by applying the CIM model. This compensation can adjust the projected content in real time based on changes in the wall projection area (such as occlusion, angle changes, or person movement) to ensure stable image quality. It also identifies the specific interactive influences (such as user behavior, object movement, etc.) on different projection areas and makes adjustments based on this influence information. In this way, the projected content can be better synchronized with the user's interaction, achieving highly accurate interactive effects. The system then determines whether these projected contents can be displayed synchronously with the interactive content input by the user in real time to the wall projection area, and then executes the corresponding steps.For example, when the system determines that the dynamically compensated projection content can be displayed synchronously with the interactive content input by the user in real time to the wall projection area, the system will consider that the projection content and the interactive content have been successfully connected and can be kept synchronized, thereby achieving a seamless connection between the projection content and the user's interactive behavior. The system will continue to monitor the user's input changes to ensure that the projection content can still be kept synchronized under any new interactive behavior or environmental changes in the future, maintain the continuous stability of the interactive experience, prevent synchronization problems caused by user behavior or environmental changes, and ensure the stability of the interaction. At the same time, if the environmental conditions (such as lighting, object occlusion, etc.) change, the system should be able to automatically adjust the projection content to ensure that the effect of the synchronous display is not affected, and ensure the projection The projection screen always maintains the best display effect under different environmental conditions, improves the adaptability and stability of the system, and records the user's interactive behavior and the synchronous display of the projection content, generates corresponding logs, so as to facilitate the subsequent analysis and optimization of system performance. By recording and analyzing user behavior data, the system can find potential improvement space in the subsequent optimization process, and further improve the stability of the system and user experience; for example, when the system determines that the dynamically compensated projection content cannot be synchronously displayed with the interactive content input by the user in real time to the wall projection area, the system will think that the projection content and the interactive content cannot be synchronized, and the system will pre-set the spatial coordinate data of the wall projection area based on the CIM model. The spatial coordinate data is specifically the wall The positioning coordinates in three-dimensional space are collected to collect the coordinate information corresponding to the interactive content. The coordinate information specifically includes the azimuth, tilt angle and normal vector of the wall. The interactive response of the projected content is dynamically adjusted according to these coordinate information, and the time nodes of the interactive content and the projected content are synchronized on the wall projection area. By obtaining the coordinate information such as the azimuth, tilt angle and normal vector of the wall, the system can adjust the projection angle, position and content in real time to confirm that the projection content matches the user's interactive behavior, ensure the precise synchronization between the interactive content and the projected image, avoid the image distortion, jump or delay caused by the change of the wall angle or the asynchronous interactive content, and use the precise coordinate data of the wall and interactive content to quickly respond to the user. This real-time dynamic compensation adapts to changes in input and adjusts the display position, direction, and content of the projected content. This makes user interaction with the system more sensitive and natural, enhances the system's responsiveness to the environment and user interactions, reduces latency, improves system interactivity and flexibility, and enhances the user experience. Furthermore, adjustments based on wall spatial coordinate data not only improve projection accuracy but also optimize the use of the projection area. For example, when the wall angle changes, the projector can adjust the direction of the projected content in real time, ensuring that the image always adapts to the wall structure, avoiding image distortion or blurred edges, improving space utilization, and ensuring that even on irregular or complex wall surfaces, the projected content remains clear and complete, enhancing the adaptability and quality of the projection effect.

[0063] It should be noted that the interactive impact information of the projection partition is identified, and the corresponding projection content is dynamically compensated by applying a preset CIM model according to the interactive impact information. A specific example is as follows:

[0064] The scene is described as,

[0065] Imagine a virtual dressing mirror with wall projection. A user stands in front of a wall, where virtual clothing is projected onto it. The user can move, turn, raise their hands, and other actions to view different clothing styles and combinations. The projection adjusts in real time based on the user's movements, ensuring the clothing appears natural and realistic on the user.

[0066] The interactive impact information is,

[0067] For example, when a user turns around or raises their arm, the clothing in the virtual fitting mirror needs to adjust according to the user's movements to ensure that the virtual clothing matches the user's posture. For example, when a user turns around, part of their body may block part of the projection area, resulting in certain clothing details (such as the back or cuffs) being blocked. If the angle of the projector changes, or the tilt angle of the wall projection area changes when the user moves, the projected content may be distorted.

[0068] Apply CIM model for dynamic compensation,

[0069] Real-time acquisition of user position and posture data: Through motion capture devices (such as depth cameras or infrared sensors), the system acquires the user's body position, posture, and motion data in real time. This data includes the user's turning angle, the height and direction of arm raises, etc. When the user turns or moves, the system determines whether the projected content is blocked based on the data fed back by the sensor. For example, when the user turns around, the details of the back or the design of the cuffs in the projection may be blocked. The system calculates these blocked areas and determines which projected content will be blocked to determine the part that needs to be compensated.

[0070] Apply CIM model to adjust projection content,

[0071] If the user turns around or raises their hand, the CIM model adjusts the display position of the virtual garment in real time to align with the user's posture. For example, when the user turns around, the virtual garment on their back automatically rotates instead of remaining at the original front-facing angle. If a user's movement (such as bending or extending their arm) blocks part of the projection area, the CIM model calculates a new display area and adjusts the obscured garment portion to a different angle or position to ensure that the garment is displayed unobstructed. For example, the system might move the virtual cuff from the back to the side of the user to avoid being blocked by the arm. If the projector angle changes (for example, when the projector tilts slightly as the user walks), the CIM model adjusts the position and scale of the projected content in real time to avoid distortion caused by the angle shift. For example, the system calculates the new projection coordinates on the wall and recalibrates the virtual garment display based on the new projection angle. If the user moves within the projection area or the ambient light changes (such as when a curtain is drawn), the system also adjusts the brightness and contrast of the projected content to ensure that the virtual garment is clearly visible under different lighting conditions.

[0072] That is, when the user turns around, raises their hand, or makes other movements, the virtual clothing projected on the wall will respond instantly, always aligning with the user's body posture, ensuring that the user can see different angles of the clothing; even if the user's movements cause certain parts of the clothing to be blocked, the system will automatically adjust the projection content and re-display the blocked parts in other locations to ensure that the clothing details are not lost; the display effect of the projected content on the wall always remains clear, and no matter how the user moves, the virtual clothing can be accurately synchronized with the user's movements to avoid distortion or misalignment;

[0073] In summary, the above examples describe how the system uses the CIM model to dynamically compensate for projection content in real time, ensuring that the wall projection can be synchronized with the user's interactions and actions, thereby improving the experience of the virtual fitting mirror.

[0074] It should be supplemented that, based on the spatial coordinate data preset for the wall projection area by the CIM model, coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area. A specific example is as follows:

[0075] Imagine a virtual home design app where users can control virtual furniture (such as sofas, tables, and decorations) displayed on the wall through gestures or touch. A virtual room layout is projected onto the wall, and users can interact with the projection to change the position, size, and rotation of the furniture. The entire system makes real-time adjustments based on the CIM model, ensuring synchronization between the projected and interactive content.

[0076] During system initialization, the CIM model establishes a three-dimensional coordinate system based on the dimensions, structure, and texture of the wall space. For example, suppose the wall is 4 meters long and 3 meters wide, and the distance between the projector and the wall is 2 meters. The wall's normal vector (the direction perpendicular to the ground) is preset to (0, 1, 0), allowing the system to determine the spatial location of the wall. The CIM model also considers parameters such as the wall's angle, inclination, and structural details. If there are any decorative objects or special textures on the wall, the system will also preset them in the model. For example, if the wall has a special protrusion, the CIM model will adjust the projected content accordingly to ensure that the furniture is not obscured or distorted.

[0077] The coordinate information of the interactive content is then collected, and the user can operate the virtual furniture on the wall through gestures or touch. For example, the user points to the sofa projected on the wall, or clicks the virtual table on the wall with a finger. The system will collect the specific coordinates of the user's gesture through sensors (such as infrared sensors, depth cameras, etc.). Assuming that the user's right fingertip is located at a certain position on the wall projection (for example, 3 meters away from the center of the wall projection area and 1 meter to the right), the system will transmit this coordinate point to the CIM model to indicate the user's gesture or touch position. Assuming that the user clicks on the virtual sofa in the projection area, the system will identify the user's finger coordinate information, including the fingertip position on the wall, the distance from the wall surface, the direction of the finger, and other data.

[0078] The interactive response of the projected content is then dynamically adjusted. Based on the coordinates of the user's gesture (for example, the position where the user points to the sofa), the system will use the CIM model to calculate the optimal display position of the sofa on the wall in real time. The system will dynamically adjust the position of the sofa based on the three-dimensional coordinates of the wall projection area so that it is displayed synchronously with the user's pointing. If the user points to the left side of the sofa, the system will slightly move the display position of the sofa so that the point pointed by the user appears exactly to the left of the sofa. If the user's gesture indicates dragging the virtual sofa, the system will dynamically adjust the size and angle of the furniture based on the changes in the position and direction of the user's gesture. For example, if the user drags the sofa from the right to the left, the system will calculate the size and rotation angle of the sofa in the new position in real time based on the changes in the size of the wall space, and adjust it through the CIM model to ensure that the projected content is not distorted on the wall. If the user's action causes some furniture to be blocked, the CIM model will automatically adjust the projected content to ensure that the furniture is not blocked by the body or arm. For example, if the user raises his arm and the arm blocks part of the sofa, the system will move the sofa to the other side or adjust the blocked part to the visible area.

[0079] Then, the delay compensation of the interactive response is adjusted. In the wall projection area, the system needs to ensure that the projected content and the user's interactive content are synchronized at the same time. For example, when the user clicks on the virtual sofa, the sofa's movement, rotation, scaling, and other actions must coincide with the moment of the finger click. Therefore, the system will compensate for the interactive delay in real time to ensure that each user's action can instantly respond to changes in the projected content. If the user clicks on the sofa, the system will calculate the exact time of the user's finger click and align it with the time node of the sofa's movement. By dynamically adjusting the projection frame rate, the projection content is ensured to be synchronized with the user's gesture changes. For example, if the user clicks on the sofa quickly, the system will quickly adjust the position of the sofa on the wall to avoid projection distortion caused by delay. If the projector is slightly distorted due to some reasons (such as user walking, object occlusion, or projection angle adjustment), the CIM model will detect these changes in real time and automatically adjust the display position of the projected content. For example, if the projector angle is offset, the system will correct the projection angle of the virtual furniture based on the three-dimensional coordinate data of the wall to ensure that the display of the furniture is not affected.

[0080] That is, no matter how the user moves, points or clicks, the system can respond to the user's actions in real time and dynamically adjust the furniture in the projection according to the user's input; the display of virtual furniture is always consistent with the user's gestures, position and angle; for example, when the user points to a sofa on the wall, the sofa will immediately appear at the position where the user's finger is pointing, and its angle, size, etc. will change synchronously with the user's movements; even if the user moves or interacts quickly, the projected content on the wall will always remain clear and undistorted, and the furniture will not be obscured; every user's action (such as moving the arm, turning around, approaching the wall, etc.) will be instantly reflected in the changes in the position, angle and size of the virtual furniture in the projection; when the user places virtual furniture on the wall projection, the whole experience is very smooth and natural; the system can accurately respond to the user's gestures or touches, ensuring that there is no time difference or spatial dislocation between the projected content and the interactive content;

[0081] To summarize, in the aforementioned virtual home placement application scenario, through the application of the CIM model, the wall projection system can accurately capture and respond in real time to user gestures, touches and other interactive information; the system dynamically adjusts the position, size, angle, etc. of the projected content based on preset spatial coordinate data to ensure that the projection and the time nodes of user interaction are completely synchronized; in this way, the interaction between the user and the projection becomes more intuitive, smooth and realistic, enhancing the overall interactive experience.

[0082] In this embodiment, the step S3 of dynamically compensating the corresponding projection content using a preset CIM model according to the interactive impact information further includes:

[0083] S31: generating a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness, and contrast;

[0084] S32: Determine whether the current projection area matches a preset optimal projection area;

[0085] S33: If not, obtain the reflection parameters of the light from the wall projection area through the light emitted by the projector, collect the parameters to be corrected of the wall projection area according to the reflection parameters, and control the projection content to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

[0086] In this embodiment, the system generates a current projection area of ​​the wall projection area based on the pre-set content requirements of the projection content, which specifically include content ratio, resolution, brightness and contrast, and then the system determines whether the current projection area matches the pre-set optimal projection area to execute the corresponding steps; for example, when the system determines that the current projection area of ​​the wall projection area can match the pre-set optimal projection area, the system will consider that the current projection effect has met the preset quality requirements, and the system will continue to project under the current settings to ensure the smoothness of the user experience and the stability of the visual effect without any additional correction or adjustment. At the same time, after ensuring that the current projection area matches the optimal projection area, the system will continue to interact with the user. If the user changes the angle or spatial position of the wall projection area, the system will continue Use existing parameters to adjust the projection to ensure that the new interactive area still meets the optimal projection standards, and ensure that the projected content is synchronized with the interactive content input by the user, so that the user can get instant and clear feedback on each interaction, providing a smooth interactive experience. For example, when the system determines that the current projection area of ​​the wall projection area cannot match the pre-set optimal projection area, the system will consider that the current projection effect cannot meet the preset quality requirements. The system will use the light emitted by the projector to obtain the reflection parameters of these lights from the wall projection area. The reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflecting surface. Based on these reflection parameters, the parameters to be corrected of the wall projection area are collected. The parameters to be corrected specifically include wall curvature and wall tilt. Based on these parameters to be corrected, the projection content is controlled to perform geometric distortion correction.The system uses the light emitted by the projector and its reflection path, angle, and surface material characteristics to accurately measure the reflection situation of the projection area and identify the shape and characteristics of the wall. Based on the parameters to be corrected collected by the reflection parameters, the system can dynamically perform geometric correction on the projected content. By controlling the geometric shape and angle of the projected content, it avoids image distortion caused by irregular wall surfaces and maintains the clarity and accuracy of the projected content. At the same time, the reflection parameters can provide the projection system with real-time data on the physical characteristics of the wall, ensuring that the projected image will not be distorted or blurred due to irregular wall surfaces, and preventing irregularities caused by irregular wall surface characteristics (such as uneven walls, tilted walls, etc.). This improves image clarity and visual quality by preventing blur or distortion in the image. Furthermore, in dynamic interactive environments, it is crucial that the projected content remains highly consistent with the user's interactive actions. By collecting and adjusting the reflection parameters of the wall projection area in real time, the system can accurately reflect the impact of wall changes, ensuring that the projected content can be adjusted in a timely manner and presented synchronously with the user's actions, avoiding interaction delays or distortion caused by inconsistent wall shapes. The system is adaptive to different wall environments. Even if the wall is irregular or deforms during projection, the system can adjust the projection effect in real time, ensuring stable quality in the wall projection area and avoiding adverse effects caused by wall changes.

[0087] It should be noted that, by using the light emitted by the projector, reflection parameters of the light from the wall projection area are obtained, parameters to be corrected of the wall projection area are collected based on the reflection parameters, and the projection content is controlled to perform geometric distortion correction based on the parameters to be corrected. Specific examples are as follows:

[0088] Consider an interactive projection system. The projected content is a virtual game interface, which includes a circular virtual object called a "virtual sphere." The projection area is a wall that is curved and tilted. The system needs to ensure that the projected virtual sphere remains circular regardless of any distortion on the wall.

[0089] First, the projector is placed in a corner of the room, projecting an image of a virtual sphere onto the wall. This virtual sphere should appear as a complete circle on the wall. During this process, light from the projector shines onto the wall, forming a light spot. This light spot is the display area of ​​the virtual sphere.

[0090] Then, reflection parameters are collected. When light hits the wall, it will reflect according to the shape of the wall. At this time, the system will analyze the condition of the wall through the reflected light. The sensor detects the propagation path of each light and records the propagation angle from the projector to each point on the wall.

[0091] For example, light from a projector bounces off a wall from a single point, but the paths it takes may be different in different areas of the wall. If the wall is tilted or curved, these paths may change slightly because the light, after hitting the wall, reflects along the normal to the wall's surface. The system calculates the angle of each reflected ray, especially in areas of curvature and tilt.

[0092] For example, in areas where the wall is tilted, the reflection angle will be different, causing the projected image to appear stretched or compressed to varying degrees in these areas; if the wall is rough, it will cause the reflected light to scatter; if it is a smooth surface, the reflected light will be reflected more concentratedly; the system can infer the wall material based on this difference and evaluate the reflection effect in different areas;

[0093] The system then identifies the parameters to be corrected. By collecting the path and angle of the reflected light, it discovers that a portion of the wall is slightly curved. This curvature causes the projected image to appear stretched or compressed on the wall, especially near the curved area.

[0094] For example, in areas with curved walls, a virtual sphere that should appear circular may become elliptical. The system recognizes this and marks the area as a target for geometric distortion correction. The system also detects that the wall is slightly tilted, and the angle of the reflected light causes the projected content in the tilted area to be stretched or deformed. For example, in areas with tilted walls, a virtual sphere that was originally upright may appear irregular, or even elliptical or elongated.

[0095] Then, geometric distortion correction is performed. The system uses perspective transformation to correct for curved areas of the wall. Due to the curvature of the wall, the image is stretched or compressed in some areas. The perspective transformation scales and distorts these areas, restoring the originally elongated sphere to a true circle. For example, in an area with a curved wall, the system adjusts the display ratio of the sphere in that area so that it looks like a circle displayed on a flat surface, unaffected by the curvature of the wall.

[0096] Finally, the system corrects tilted areas. For tilted parts of the wall, the system uses an affine transformation algorithm to correct them. Affine transformation can adjust the content of the tilted area to be perpendicular to the observer while maintaining the shape proportions. Through this transformation, the originally tilted and distorted virtual sphere will be restored to its correct circular shape. For example, if the wall is tilted to the left, the system will perform a reverse tilt correction on the image of the virtual sphere, making it appear to be suspended in a position perpendicular to the ground, avoiding deformation or stretching of the virtual object.

[0097] As the user interacts with the virtual sphere, the system adjusts the projection in real time to ensure it always maintains the correct shape. Regardless of where the user stands in front of the wall, the system adjusts the projection based on real-time feedback. For example, a user might walk to the side of the wall, causing the viewing angle of the projected image to change. Based on the user's interactive behavior, the system recalculates the light reflection path to ensure the projection is updated in real time and always conforms to the shape of the wall.

[0098] In summary, through the above steps, the system successfully uses reflected light to collect data on the wall shape and perform geometric distortion correction. Regardless of whether the wall is curved or tilted, the system can make real-time corrections to the projected content so that the virtual sphere always remains round. Regardless of the angle or position at which the user interacts with the virtual object, the projection effect can meet expectations, ensuring the quality and accuracy of the image.

[0099] In this embodiment, before step S3 of dynamically compensating the corresponding projection content using a preset CIM model according to the interactive impact information, the method further includes:

[0100] S301: Constructing a virtual three-dimensional coordinate system applicable to the wall projection area based on a coordinate origin predefined for the wall projection area by the CIM model, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis;

[0101] S302: Determine whether the virtual three-dimensional coordinate system matches the projection field of view of the projector;

[0102] S303: If so, measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system to obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

[0103] In this embodiment, the system constructs a virtual three-dimensional coordinate system applicable to the wall projection area based on the coordinate origin pre-defined for the wall projection area by the CIM model. The virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis. The system then determines whether these virtual three-dimensional coordinate systems can match the projection field of view of the projector to execute the corresponding steps. For example, when the system determines that the virtual three-dimensional coordinate system applicable to the wall projection area cannot match the projection field of view of the projector, the system will consider that there is a difference between the coordinate system of the wall projection area and the visual range of the projector, resulting in the projection content may not be fully covered or distorted, or even unable to be accurately aligned or stably displayed. The system will change the position of the projector to make it cover the wall projection area. The entire coordinate range of the virtual three-dimensional coordinate system is used to ensure that the virtual three-dimensional coordinate system can be docked with the projection field of view of the projector, including moving the projector forward and backward, or adjusting the projection angle of the projector as needed, and appropriately narrowing the range of the virtual three-dimensional coordinate system to match the projection field of view of the projector. The maximum values ​​of the X, Y, and Z axes of the virtual coordinate system can be reset to avoid conflicts with the physical limitations of the projector. If the wall projection area is too large or irregular, the system can dynamically adjust the size and angle of the projection content according to the shape of the wall and the projection field of view of the projector to adapt to the field of view of the projector. This can ensure that the display effect of the projection content on the wall is more accurate and stable; for example, when the system determines that the virtual three-dimensional coordinate system is suitable for the wall projection area The system can match the projection field of view of the projector. At this time, the system will think that the coordinate system of the wall projection area is the same as that of the projector. The system will measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system. The wall parameters specifically include the wall size and wall shape, and obtain the path information from the projector to the wall projection area. The path information specifically includes the distance between the projector's position and the wall, the projector's focal length and field of view. By matching the virtual three-dimensional coordinate system with the projector's projection field of view, the system can ensure that the size, shape, angle and other characteristics of the wall projection area are consistent with the content output by the projector, avoiding image distortion, blur or dislocation problems, obtaining the path information from the projector to the wall, and ensuring that the projection content can be accurately projected to a specific position on the wall, optimizing The system can dynamically adjust the size, scale, direction, etc. of the projected content to adapt to the actual characteristics of the wall after measuring the wall parameters (such as wall size and shape) to improve the clarity and display effect of the picture. It can also help the system adjust the projection parameters such as projection angle, focus setting and content resolution according to the actual projection environment to obtain the best projection effect. By accurately measuring the wall size and shape, the system can automatically adjust the match between the projected content and the wall under different environmental conditions. It can not only adapt to static walls, but also cope with dynamic environmental changes. When there are changes in the wall projection area (such as object movement, wall tilt, etc.), the system can quickly adjust based on the real-time feedback of path information and wall parameters to ensure the synchronization of projection and interactive content.

[0104] In this embodiment, the step S5 of collecting coordinate information corresponding to the interactive content and dynamically adjusting the interactive response of the projected content according to the coordinate information further includes:

[0105] S51: Collecting user coordinate information of the user in the wall projection area, and obtaining an interaction area between the user coordinate information and the projection content;

[0106] S52: Determine whether the interactive area can accommodate the projection content and the interactive content at the same time;

[0107] S53: If not, deploying preset layers in the same space of the wall projection area, and applying a preset display order based on the preset layers to synchronously display the projection content and the interactive content, wherein the preset layers specifically include a front-end layer and a background layer.

[0108] In this embodiment, the system acquires the user coordinate information of the user on the wall projection area to obtain the interaction area between the user coordinate information and the projection content, and then the system determines whether the interaction area can accommodate the projection content and the interactive content at the same time, so as to execute the corresponding steps; for example, when the system determines that the interaction area between the user coordinate information and the projection content can accommodate the projection content and the interactive content at the same time, the system will consider that there is no conflict between the current projection area and the user's interaction needs in the physical space, and efficient interactive display can be achieved. The system will monitor the user's coordinate information in real time, and the system can adjust the projection content according to the user's actions to ensure that the projection and the user's interaction are in the same spatial coordinate range, and the projection can be accommodated in the interaction area. In the case of content and interactive content, the system can automatically adjust the projection brightness, resolution, contrast and other parameters to adapt to different lighting environments or changes in user position to ensure the best quality of the display effect. At the same time, the system can also adjust the response of the projected content based on the user's specific interactive behavior (for example, finger touch, body movement, etc.), so that the user's action can directly affect the content or form of the projection, and the system can effectively utilize the entire wall projection area for content display and user interaction, avoiding the problem of overlapping or occlusion of projection content and interactive content due to insufficient interactive space, and achieving a seamless transition between the user and the projected content, ensuring that the interaction between the user and the projection is not restricted by physical space, thereby optimizing the space. For example, when the system determines that the interactive area between the user coordinate information and the projected content cannot accommodate both the projected content and the interactive content at the same time, the system will consider that there is a conflict between the current projection area and the user's interactive needs in physical space. The system will deploy pre-set layers in the same space of the wall projection area. The preset layers specifically include the front-end layer and the background layer. The projected content and the interactive content are displayed synchronously based on the pre-set display order of the layer application. The system can effectively solve the spatial conflict problem between the user's interactive needs and the projected content by introducing layer management of the front-end layer and the background layer. When the interactive area cannot accommodate the projection and interactive content, the system will allocate the interactive content and the projected content to Different layers ensure that the two will not block or overlap each other. At the same time, the relationship between the user's operation feedback and the projected content can be accurately controlled through the allocation and display order of layers. Through the front-end layer, the user's operation is not affected by the background content. The system can display the real-time response of the user's interaction on the front-end layer, while the background layer continues to display the projected content, ensuring the accuracy of the operation and the immediacy of the feedback. Moreover, through the preset layers and display sorting rules, the system can intelligently schedule the display order of various types of content to meet different application scenarios and user needs. For example, when displaying product advertisements, the advertising content of the background layer can be covered by the interactive elements of the front-end layer. When no interaction is required, the advertising content will be restored to full-screen display.

[0109] In this embodiment, the step S2 of determining whether the environmental change data detects a preset projection distortion jump further includes:

[0110] S21: Based on the projection information preset by the projector, detecting the amplitude of the projection information;

[0111] S22: Determine whether the jitter amplitude reaches a preset distortion standard;

[0112] S23: If yes, apply the preset optical flow method to collect continuous frame data of the projection information before and after the change, and calculate the motion vector of each pixel point corresponding to the projection information by comparing the continuous frame data, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

[0113] In this embodiment, the system detects the amplitude of the projection information based on the projection information preset by the projector, and then determines whether the amplitude of the projection information reaches the preset distortion standard to execute the corresponding steps; for example, when the system determines that the amplitude of the projection information does not reach the preset distortion standard, the system will consider that the current projection effect is stable and there is no distortion or picture jump phenomenon. The system will continue to maintain the current projection settings, including parameters such as brightness, contrast, and color, and continuously monitor environmental changes in the wall projection area, such as object occlusion, angle changes, and person movement, to ensure that the projection effect will not be affected by these factors. Sudden distortion and jumps occur, and at the same time, the synchronous response of the projected content and the interactive content is further adjusted to enhance the user's interactive experience. For example, the clarity of the content is adjusted or the response delay is reduced to make the feedback of the projected content and the user's operation smoother, and the display of the interactive content can be continuously optimized to make its synchronous response with the projected content more accurate. By tracking the user's input in real time, the system can adjust the interactive response time, the position of the interactive interface, etc., to make the temporal and spatial relationship between the projected content and the user's operation smoother. For example, when the system determines that the jump amplitude of the projection information has reached the preset distortion standard, the system will consider that the current projection effect is not good. Stable, prone to distortion or picture jumping, the system will apply the pre-set optical flow method to collect continuous frame data of the projection information before and after the change, and calculate the motion vector of each pixel point corresponding to the projection information by comparing these continuous frame data. The motion vector specifically includes the movement speed and movement direction of the pixel point; by obtaining the motion information of each pixel point, the system can adjust the projection content based on the motion vector and correct the distortion or jumping in the picture in real time. For example, the system can adjust the edge, focus area or geometric deformation of the projection according to the direction of movement to avoid the projection content from changing due to the environment. Through precise motion compensation, it can avoid image jumping. The visual interference caused by users can be improved, and the stability and smoothness of the viewing experience can be improved. At the same time, the real-time data collection and processing of the optical flow method can enable the system to quickly identify the distortion or jump of the projected image, and adjust the projected content in a very short time to achieve rapid correction, reducing the intervention of users and operators on the projector. That is, the system can automatically correct the projected content according to the calculated motion vector without manual adjustment of the projection equipment. In interactive projection applications, the optical flow method can ensure that the projected content and the user's actions (such as gestures, touch, etc.) are always synchronized, avoiding interaction delays or discontinuities caused by image jumps or distortion, and improving the naturalness and accuracy of the interactive experience.

[0114] It should be noted that the preset optical flow method is applied to collect continuous frame data of the projection information before and after the change, and by comparing the continuous frame data, the motion vector of each pixel point corresponding to the projection information is calculated. The specific example is as follows:

[0115] Suppose in an exhibition hall, there is a wall used as a projection display wall; a projector projects a dynamic virtual artwork on this wall; users can interact with the projection content through gestures or movements, changing certain elements in the picture, such as making virtual characters walk, rotate, or change colors;

[0116] The projection content dynamically changes with user interaction, but the wall may be tilted, obstructed by objects, or distorted or jittered due to rapid user movement; to ensure the stability and accuracy of the projection content in the physical space, the system uses optical flow to correct the projection picture in real time;

[0117] First, the projection content is a virtual character walking on the screen and interacting with the background scene; users interact by touching the wall or waving their arms to affect the character's movements; suppose the system collects a frame of image every 50 milliseconds; the first frame shows the virtual character standing on the left side of the scene, with a static cityscape in the background; the second frame shows the character starting to walk, moving about 10 pixels, and the cityscape in the background also dynamically changes with the character's movement; by comparing consecutive frame data, the system uses optical flow to compare the changes between the first and second frames; optical flow analyzes the pixel differences in consecutive image frames to extract the motion information of each pixel, including the direction (horizontal / vertical) and speed of movement;

[0118] For example, in the first frame, the character's head is in the upper left corner of the image, while in the second frame, the head moves 10 pixels to the right and down; the system uses optical flow to calculate the motion vector of this head region, resulting in a motion direction of "right down" and a motion speed of 10 pixels / frame; through the algorithm of optical flow, the system will get the motion vector of each pixel; here are some possible calculation results:

[0119] The character's head moves from left to right, and the motion direction of each pixel is "right";

[0120] The character's arm moves from top to bottom, and the motion direction of each pixel is "down";

[0121] The buildings in the background also change with the character's movement, so the pixels in the background also have certain motion vectors, with a motion direction of "horizontal right";

[0122] For example, suppose the character's arm is in the upper left corner of the screen in the first frame and in the lower right corner in the second frame; the system compares the images and finds that the motion vector of this region is "from the upper left to the lower right", and calculates the movement speed of each pixel point as 5 pixels / frame; that is, optical flow compensation calculation; once the system gets the motion vector of each pixel, the next task is to dynamically adjust the projection content to ensure that the projection image is not distorted or jittered;

[0123] For example, if in the projection area, the person's arm is located in the occlusion area of the object, causing the projected content in this area to be distorted or blurred, the system can predict the true content of this area according to the motion vector calculated by the optical flow method (i.e. the direction and speed of the arm movement), and restore it to normal state;

[0124] Then real-time geometric distortion correction is performed. Assuming that the wall surface is tilted, the position and angle of the projection content in the physical space will change, causing geometric distortion of the picture. The system dynamically adjusts the geometric shape of the projection content according to the motion vector and the wall surface angle information, to ensure correct display of the content. For example, if the wall surface is tilted by 10 degrees, the system will correct the display position of each pixel point according to the tilt angle, to avoid distortion or stretching of the projected image;

[0125] Subsequently, the projection content is corrected. The system will accurately correct the projection content according to the dynamically calculated motion vector. For example: the person's head moves to the right and down in the image, and the system adjusts the display position of the projection content according to the motion vector calculated by the optical flow method, so that the person's head is correctly aligned in the image. Due to the user's arm movement, the position of the arm changes, and the system will perform smoothing processing on the arm area to eliminate distortion effects. The buildings and scenery in the background scene change dynamically as the person moves, and the system predicts the changes in the background content by calculating the motion vector of the pixels in the background, and dynamically updates the projection image;

[0126] Finally, the corrected projection content is output. The system projects the optimized projection content to the wall surface in real time according to the above correction steps. Whether it is the movement of the person, the change of the background, or the tilt of the wall, all dynamic compensation ensures the stability of the projection content in the physical space, avoiding distortion or picture jumping;

[0127] Suppose the user is interacting and waving his arm to make the person dance. Due to the rapid movement of the arm, the position of the arm in the projection picture may be distorted, with the edges of the arm becoming blurred or jumping. Through the optical flow method, the system calculates the motion direction and speed of each pixel point in the arm area (for example, the arm moves from the top left to the bottom right at a speed of 3 pixels per second), and then performs smoothing processing on the area according to this information. The system will reposition each pixel point in the arm area according to the motion vector, correct its position, and restore the normal display effect;

[0128] In summary, the above examples show that the system can calculate the motion vector of each pixel in real time through the optical flow method, and then accurately correct the distortion or jumping of the projection content. Whether it is the occlusion of the wall surface object, the tilt of the wall surface, or the rapid interactive action of the user, the optical flow method can effectively help the system understand the dynamics of image changes and optimize the projection effect, thereby ensuring the consistency and stability of the picture.

[0129] In the embodiment, the step S4 of judging whether the projection content can be synchronously displayed with the interactive content input by the user further comprises:

[0130] S41: Real-time capture the moving track corresponding to the interactive content based on the preset sensor of the wall projection area, wherein the sensor specifically comprises an infrared sensor, a touch sensor and a pressure sensor;

[0131] S42: Judge whether the moving track matches the preset mapping rule of the CIM model;

[0132] S43: If yes, responsively display the corresponding dynamic display content on the wall projection area according to the moving track, detect the position change information of the user in the wall projection area, and dynamically adjust the display position of the projection content according to the position change information.

[0133] In this embodiment, the system captures the movement trajectory corresponding to the interactive content in real time based on the sensors pre-set in the wall projection area, which specifically include infrared sensors, touch sensors, and pressure sensors. Then the system determines whether these movement trajectories match the mapping rules pre-set in the CIM model to execute corresponding steps. For example, when the system determines that the movement trajectory corresponding to the interactive content cannot match the mapping rules pre-set in the CIM model, the system considers that the user's interactive behavior deviates from the pre-set interactive model. The system dynamically updates the mapping rules by real-time monitoring and learning the user's interactive behavior, and gradually adjusts the CIM model through artificial intelligence or machine learning algorithms to adapt to the behavior patterns of different users. For example, the system collects the user's trajectory data (such as gestures, touch points, pressure points, etc.), reconstructs the mapping rules according to the user's interactive habits to make them more flexible and accurate, and periodically calibrates the sensors to ensure that they can accurately capture the interactive trajectory in different environmental conditions, such as enhancing the sensitivity of infrared sensors or optimizing the response curve of pressure sensors to ensure stable operation in various environments. For example, when the system determines that the movement trajectory corresponding to the interactive content can match the mapping rules pre-set in the CIM model, the system considers that the user's interactive behavior successfully interacts with the pre-set interactive model. The system responds to the corresponding dynamic display content on the wall projection area according to these movement trajectories, detects the position change information of the user in the wall projection area, and dynamically adjusts the display position of the projection content according to different position change information. The system can provide accurate projection content by responding to the user's actions and changes in real time, ensuring that the user's interactive operation receives immediate feedback. This smooth experience enhances the user's sense of immersion, making it easier for users to interact naturally with the system. Adjusting the projection content according to the user's actions and position changes can provide personalized visual effects for different users. This dynamic response capability allows each user to enjoy customized interactive content, and the system can flexibly adjust the display position of the projection content according to the user's position change information in the wall projection area. No matter how the user moves, the projection content can always accurately align to the pre-set position, avoiding distortion or jumping phenomenon, ensuring that the visual effect is always stable and clear. Through accurate movement trajectory and position change information, the system can monitor and adjust the projection content in real time, reducing the problem of projection distortion or unstable picture caused by irregular wall surface or user's fast action. According to the user's position change in the wall projection area, the system dynamically adjusts the position of the projection content to ensure that the projection content always matches the interactive content. This synchronization can effectively avoid the projection misalignment or incoordination caused by user displacement.

[0134] In this embodiment, the projector pre-set in the wall projection area also includes the following steps in step S1 of real-time collection of environmental change data of the wall projection area:

[0135] S11: Detecting color deviation information of the wall projection area based on a preset projection effect of the projector, wherein the projection effect specifically includes brightness, focus, and color;

[0136] S12: Determine whether the color deviation information meets the preset projection requirements;

[0137] S13: If not, identifying the influencing parameters of the color deviation information on the projection message according to the projection requirement, and generating the influencing parameters in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

[0138] In this embodiment, the system detects the color deviation information of the wall projection area based on the projection effect preset by the projector, which specifically includes brightness, focus and color. The system then determines whether the color deviation information meets the preset projection requirements to execute the corresponding steps. For example, when the system determines that the color deviation information of the wall projection area can meet the preset projection requirements, the system will consider that the color performance of the wall projection effect meets expectations and the visual quality of the projected content has reached the required standard. The system will maintain the current projection settings to ensure that the picture does not have color distortion or inconsistency during continuous use, which helps to maintain the picture. The quality is stable to avoid visual interference in the subsequent interaction process of users. At the same time, the system continues to monitor the environmental changes in the wall projection area (such as lighting changes, wall material changes, etc.). The system can regularly or as needed fine-tune the projection effect to keep the color and brightness in the best state to cope with possible environmental changes. After the color deviation reaches the preset standard, the system can enter a stable operation state to ensure that there will be no color shift problem when the projected content is displayed for a long time. This is very important for application scenarios that require long-term content display (such as conferences, performances, digital signage, etc.); for example, when the system determines that the color deviation information of the wall projection area cannot meet the preset projection requirements, At this time, the system will think that the color of the wall projection effect does not meet expectations. The system will identify the influencing parameters of the color deviation information on the projection information according to the pre-set projection requirements. The influencing parameters specifically include color temperature, color gamut and color difference, and generate these influencing parameters on the panel pre-set by the projector; the system can maintain the color accuracy of the projected content under different environmental conditions (such as light changes, wall material changes, etc.) by real-time identification and adjustment of parameters affecting color (such as color temperature, color gamut and color difference). This can effectively avoid color deviation problems caused by environmental changes or equipment differences, and at the same time, by adjusting the color parameters, adaptive adjustment of the projection content can be achieved. The ability of , enables the projection to maintain ideal color effects under different wall surfaces and environmental conditions, reducing the need for manual intervention and increasing the intelligence level of the system. By identifying color deviations and adjusting the projector panel, the system can realize automatic color correction without manual intervention. This not only saves adjustment time, but also reduces human errors and improves overall efficiency and accuracy. The color correction can respond to environmental changes in real time, such as changes in light intensity and different wall materials. The system automatically adjusts the color parameters of the projected content by detecting changes in the environment and equipment status to ensure that the color always meets the preset requirements and improve the stability of the projection effect.

[0139] Reference Attachment Figure 2 , is a wall interactive projection system based on CIM digital twin in one embodiment of the present invention, comprising:

[0140] The acquisition module 10 is used to collect environmental change data of the wall projection area in real time based on a projector preset in the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data, and person movement data;

[0141] A judging module 20 is configured to judge whether a preset projection distortion jump is detected in the environmental change data;

[0142] an execution module 30 configured to, if yes, divide the wall projection area into at least two projection partitions based on preset projection parameters of the wall projection area, identify interactive influence information of the projection partitions, and dynamically compensate corresponding projection content using a preset CIM model based on the interactive influence information, wherein the projection parameters specifically include wall size, wall structure, and wall texture;

[0143] A second judgment module 40 is used to judge whether the projected content can be displayed synchronously with the interactive content input by the user;

[0144] The second execution module 50 is used to, if not possible, collect coordinate information corresponding to the interactive content based on the spatial coordinate data preset for the wall projection area by the CIM model, dynamically adjust the interactive response of the projection content according to the coordinate information, and synchronously connect the time nodes of the interactive content and the projection content on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth, inclination angle and normal vector of the wall.

[0145] In this embodiment, the acquisition module 10 collects environmental change data of the wall projection area in real time based on a projector pre-installed in the wall projection area. The environmental change data specifically includes object occlusion data, angle change data and person movement data. Then the judgment module 20 judges whether these environmental change data detect the preset projection distortion jump to execute the corresponding steps; for example, when the system determines that the environmental change data of the wall projection area does not detect the preset projection distortion jump, the system will consider that the projection information remains stable in the current environment and is not affected by object occlusion, angle change or person movement. The system will continue to output the current projection content to ensure the stability and accuracy of the picture, and at the same time continue to collect environmental change data of the wall projection area in real time to ensure that it can respond in time to any changes in the future, and record relevant data of the current environment and projection status (such as wall structure, projection angle, ambient light, etc.) to provide data for subsequent tuning and analysis. Support; for example, when the system determines that the environmental change data of the wall projection area detects a preset projection distortion jump, the execution module 30 will consider that the projection information is affected, and the system will divide the wall projection area into at least two projection partitions according to the projection parameters preset in the wall projection area. The projection parameters specifically include wall size, wall structure and wall texture, identify the interactive impact information of these projection partitions, and dynamically compensate the corresponding projection content based on different interactive impact information using a preset CIM model; the system divides the wall projection area into at least two projection partitions based on preset projection parameters such as wall size, structure and texture, so that the affected area can be accurately located, and adjustments can be made to the specific projection area to avoid unnecessary resource waste or impact on the projection effects of other parts caused by global adjustments. At the same time, based on different interactive impact information, the system dynamically compensates the corresponding projection content by applying the CIM model. This compensation can adjust the projected content in real time based on changes in the wall projection area (such as occlusion, angle changes, or person movement), ensuring the stability of image quality. It also identifies the specific interactive influences (such as user behavior, object movement, etc.) on different projection partitions and makes adjustments based on this influence information. In this way, the projected content can be better synchronized with the user's interaction, achieving a highly accurate interactive effect. The second judgment module 40 then determines whether these projected contents can be synchronously displayed on the wall projection area with the interactive content input by the user in real time, and then executes the corresponding steps.For example, when the system determines that the dynamically compensated projection content can be displayed synchronously with the interactive content input by the user in real time to the wall projection area, the system will consider that the projection content and the interactive content have been successfully connected and can be kept synchronized, thereby achieving a seamless connection between the projection content and the user's interactive behavior. The system will continue to monitor the user's input changes to ensure that the projection content can still be kept synchronized under any new interactive behavior or environmental changes in the future, maintain the continuous stability of the interactive experience, prevent synchronization problems caused by user behavior or environmental changes, and ensure the stability of the interaction. At the same time, if the environmental conditions (such as lighting, object occlusion, etc.) change, the system should be able to automatically adjust the projection content to ensure that the effect of the synchronous display is not affected, and ensure that the projection picture The surface always maintains the best display effect under different environmental conditions, improves the adaptability and stability of the system, and records the user's interactive behavior and the synchronous display of the projection content, generates corresponding logs, so as to facilitate the subsequent analysis and optimization of system performance. By recording and analyzing user behavior data, the system can find potential improvement space in the subsequent optimization process, and further improve the stability of the system and user experience; for example, when the system determines that the dynamically compensated projection content cannot be synchronously displayed with the interactive content input by the user in real time to the wall projection area, the second execution module 50 will think that the projection content and the interactive content cannot be synchronously connected, and the system will pre-set the spatial coordinate data of the wall projection area based on the CIM model. The specific spatial coordinate data To locate the coordinates of the wall in three-dimensional space, the coordinate information corresponding to the interactive content is collected. The coordinate information specifically includes the azimuth, tilt angle and normal vector of the wall. The interactive response of the projected content is dynamically adjusted according to these coordinate information, and the time nodes of the interactive content and the projected content are synchronized on the wall projection area. By obtaining the coordinate information such as the azimuth, tilt angle and normal vector of the wall, the system can adjust the projection angle, position and content in real time to confirm that the projection content matches the user's interactive behavior, ensure the precise synchronization between the interactive content and the projected image, avoid the image distortion, jump or delay caused by the change of the wall angle or the asynchronous interactive content, and use the precise coordinate data of the wall and interactive content to quickly respond to the user's interaction. This real-time dynamic compensation adapts to changes in user input and adjusts the display position, direction, and content of the projected content. This makes user interaction with the system more sensitive and natural, enhances the system's responsiveness to the environment and user interactions, reduces latency, improves system interactivity and flexibility, and enhances the user experience. Furthermore, adjustments based on wall spatial coordinate data not only improve projection accuracy but also optimize the use of the projection area. For example, when the wall angle changes, the projector can adjust the direction of the projected content in real time, ensuring that the image always adapts to the wall structure, avoiding image distortion or blurred edges, improving space utilization, and ensuring that even on irregular or complex walls, the projected content remains clear and complete, enhancing the adaptability and quality of the projection effect.

[0146] In this embodiment, the execution module further includes:

[0147] A generating unit, configured to generate a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness, and contrast;

[0148] A judging unit, configured to judge whether the current projection area matches a preset optimal projection area;

[0149] an execution unit, configured to, if not, obtain reflection parameters of the light from the wall projection area through the light emitted by the projector, collect parameters to be corrected of the wall projection area based on the reflection parameters, and control the projection content to perform geometric distortion correction based on the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, the reflection angle, and the material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

[0150] In the embodiment, the system generates the current projection area of the wall projection area based on the pre-set content requirements of the projection content, and the content requirements specifically include content proportion, resolution, brightness and contrast, and then the system judges whether the current projection area matches the pre-set optimal projection area to execute corresponding steps; for example, when the system determines that the current projection area of the wall projection area can match the pre-set optimal projection area, the system considers that the current projection effect has met the pre-set quality requirements, the system will continue to project under the current setting to ensure the smoothness of user experience and the stability of visual effect, without any additional correction or adjustment, and at the same time, after ensuring that the current projection area matches the optimal projection area, the system will continue to interact with the user, if the user changes the angle or spatial position of the wall projection area, the system will continue to use the existing parameters to adjust the projection, to ensure that the new interaction area can still meet the optimal projection standard, and to ensure that the projection content and the interactive content input by the user are kept in synchronization, so that the user can get instant and clear feedback every time of interaction, and a smooth interactive experience is provided; for example, when the system determines that the current projection area of the wall projection area cannot match the pre-set optimal projection area, the system considers that the current projection effect cannot meet the pre-set quality requirements, the system will obtain the reflection parameters of the light from the wall projection area through the light emitted by the projector, the reflection parameters specifically include the propagation path of the light, the reflection angle and the material characteristics of the reflection surface, according to these reflection parameters, the system collects the to-be-corrected parameters of the wall projection area, the to-be-corrected parameters specifically include wall bending and wall tilting, and the system controls the geometric distortion correction of the projection content according to these to-be-corrected parameters;The system can accurately measure the reflection of the projection area, identify the shape and characteristics of the wall surface, and dynamically correct the projection content based on the collected parameters, avoid image distortion caused by irregular wall surfaces, and maintain the clarity and accuracy of the projection content. The reflection parameters can provide real-time data on the physical characteristics of the wall surface to ensure that the projected image is not distorted or blurred due to irregular wall surfaces. This improves the clarity and viewing experience of the image. In a dynamic interactive environment, it is crucial for the projection content to be consistent with the user's actions. By collecting and adjusting the reflection parameters of the wall projection area in real time, the system can accurately reflect the impact of changes in the wall surface and ensure that the projection content adjusts in real time to match the user's actions. This avoids interaction delays or distortions caused by inconsistent wall shapes. The system can adapt to different wall environments, even if the wall is irregular or deforms during projection. It can adjust the projection effect in real time to ensure stable quality of the wall projection area and avoid adverse effects caused by changes in the wall surface.

[0151] In this embodiment, the system further includes:

[0152] A construction module is configured to construct a virtual three-dimensional coordinate system suitable for the wall projection area based on a predefined coordinate origin of the wall projection area in the CIM model. The virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis.

[0153] A third judgment module is configured to judge whether the virtual three-dimensional coordinate system can match the projection field of view of the projector.

[0154] A third execution module is configured to measure the wall parameters of the wall projection area and obtain the path information from the projector to the wall projection area through the virtual three-dimensional coordinate system if the answer is yes. The wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the projector and the wall, the focal length of the projector, and the field of view range.

[0155] In this embodiment, the system constructs a virtual three-dimensional coordinate system suitable for the wall projection area based on the coordinate origin of the wall projection area predefined by the CIM model, which specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis, and then the system determines whether these virtual three-dimensional coordinate systems can match the projection field of view of the projector to execute corresponding steps; for example, when the system determines that the virtual three-dimensional coordinate system suitable for the wall projection area cannot match the projection field of view of the projector, the system considers that there is a difference between the coordinate system of the wall projection area and the visual range of the projector, which may result in that the projection content cannot completely cover or be distorted, or even cannot be accurately aligned or stably displayed, so the system changes the position of the projector to cover the entire coordinate range of the wall projection area, ensures that the virtual three-dimensional coordinate system can be connected with the projection field of view of the projector, including moving the projector forward and backward, or adjusting the projection angle of the projector as needed, while appropriately reducing the range of the virtual three-dimensional coordinate system to match the projection field of view of the projector, the maximum value of the X, Y, and Z axes of the virtual coordinate system can be reset to avoid conflicts with the physical limitations of the projector, and if the wall projection area is too large or irregular, the system can dynamically adjust the size and angle of the projection content according to the shape of the wall and the projection field of view of the projector to adapt to the field of view of the projector, which can ensure that the display effect of the projection content on the wall is more accurate and stable; for example, when the system determines that the virtual three-dimensional coordinate system suitable for the wall projection area can match the projection field of view of the projector, the system considers that there is no difference between the coordinate system of the wall projection area and the projector, and the system measures the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, which specifically include the wall size and the wall shape, obtains the path information of the projector to the wall projection area, which specifically includes the distance between the position of the projector and the wall, the focal length and the field of view range of the projector; through the matching of the virtual three-dimensional coordinate system and the projection field of view of the projector, the system can ensure that the size, shape, angle, and other characteristics of the wall projection area are consistent with the content output by the projector, avoid picture distortion, blur, or misplacement problems, obtain the path information of the projector to the wall, ensure that the projection content can be accurately projected to a specific position on the wall, optimize the clarity and display effect of the picture, and after measuring the wall parameters (such as the wall size and shape), the system can dynamically adjust the size, proportion, direction, etc. of the projection content to adapt to the actual characteristics of the wall, help the system adjust the projection parameters such as the projection angle, focal point setting, and content resolution according to the actual projection environment, so as to obtain the best projection effect, and through the accurate measurement of the wall size and shape, the system can automatically adjust the matching between the projection content and the wall under different environmental conditions, not only adapt to static walls, but also cope with dynamic environmental changes, and when the wall projection area changes (such as object movement, wall inclination, etc.), the system can quickly adjust based on the real-time feedback of the path information and the wall parameters to ensure the synchronization of the projection and interactive content.

[0156] In the embodiment, the second execution module further comprises:

[0157] An acquisition unit is configured to collect user coordinate information of the user on the wall projection area, and acquire an interaction area of the user coordinate information and the projection content.

[0158] A second judgment unit is configured to judge whether the interaction area can simultaneously accommodate the projection content and the interactive content.

[0159] A second execution unit is configured to, if not, deploy a preset layer in the same space of the wall projection area, and based on the preset layer, apply a preset display sorting to synchronously display the projection content and the interactive content. The preset layer specifically comprises a front end layer and a background layer.

[0160] In the embodiment, the system acquires the user coordinate information and the interactive region of the projection content by collecting the user coordinate information of the user on the wall projection area, and then the system judges whether the interactive region can simultaneously accommodate the projection content and the interactive content to execute the corresponding steps; for example, when the system determines that the user coordinate information and the interactive region of the projection content can simultaneously accommodate the projection content and the interactive content, the system considers that there is no conflict between the current projection region and the user's interactive demand in the physical space, and efficient interactive display can be realized, the system can adjust the projection content according to the user's action by monitoring the user's coordinate information in real time, and ensure that the projection and the user's interaction are in the same space coordinate range, in the case that the interactive region can accommodate the projection content and the interactive content, the system can automatically adjust the parameters such as brightness, resolution and contrast of the projection to adapt to different light environments or user position changes, and ensure the best quality of display effect, at the same time, the system can also adjust the response of the projection content based on the specific interactive behavior of the user (for example, finger touch, body movement, etc.), so that the user's action can directly affect the content or form of the projection, and the system can effectively utilize the whole wall projection area for content display and user interaction, avoiding the problem of overlapping or shielding of the projection content and the interactive content due to insufficient interactive space, realizing seamless transition between the user and the projection content, ensuring that the interaction between the user and the projection is not limited by the physical space, thereby optimizing the use efficiency of the space; for example, when the system determines that the user coordinate information and the interactive region of the projection content cannot simultaneously accommodate the projection content and the interactive content, the system considers that there is a conflict between the current projection region and the user's interactive demand in the physical space, the system deploys a pre-set layer in the same space of the wall projection area, the pre-set layer specifically includes a front-end layer and a background layer, and synchronously displays the projection content and the interactive content based on the layer application of the pre-set display order; the system can effectively solve the conflict problem between the user's interactive demand and the projection content in space by introducing the layer management of the front-end layer and the background layer, when the interactive region cannot accommodate the projection and the interactive content, the system separates the interactive content and the projection content into different layers through the layer separation method, to ensure that they will not be mutually shielded or overlapped, at the same time, through the allocation and display order of the layers, the relationship between the user's operation feedback and the projection content is accurately controlled, through the front-end layer, the user's operation is not affected by the background content, the system can display the real-time response of the user's interaction in the front-end layer, while the background layer continues to display the projection content, to ensure the accuracy of the operation and the immediacy of the feedback, and through the pre-set layer and the display order rule, the system can intelligently schedule the display order of various contents to meet different application scenarios and user demands, for example, when displaying product advertisements, the advertisement content of the background layer can be covered by the interactive elements of the front-end layer, and when there is no need for interaction, the advertisement content will be restored to full-screen display.

[0161] In the embodiment, the judging module further comprises:

[0162] a detecting unit, configured to detect a jumping amplitude of the projection information based on a preset projection information of the projector;

[0163] a third judging unit, configured to judge whether the jumping amplitude reaches a preset distortion standard;

[0164] a third executing unit, configured to, if yes, collect continuous frame data of the projection information before and after the change by applying a preset optical flow method, and calculate a motion vector of each pixel point corresponding to the projection information by comparing the continuous frame data, wherein the motion vector specifically comprises a motion speed of the pixel point and a motion direction of the pixel point.

[0165] In the embodiment, the system detects the jumping amplitude of the projection information based on the projection information preset by the projector, and then judges whether the jumping amplitude reaches the distortion standard preset in advance to execute corresponding steps. For example, when the system judges that the jumping amplitude of the projection information does not reach the distortion standard preset in advance, the system considers that the current projection effect is stable and no distortion or picture jumping phenomenon occurs, and the system continues to maintain the current projection settings, including brightness, contrast, color and other parameters, to continuously monitor the environmental changes of the wall projection area, such as object blocking, angle change and character movement, to ensure that the projection effect will not suddenly distort and jump due to these factors, and further adjust the synchronization response of the projection content and the interactive content to improve the user's interactive experience, for example, adjusting the clarity of the content or reducing the response delay to make the feedback of the projection content and the user operation more smooth, and still continuously optimizing the display of the interactive content to make the synchronization response of the projection content and the interactive content more accurate. By tracking the user's input in real time, the system can adjust the interactive response time, the position of the interactive interface and the like to make the space-time relationship between the projection content and the user operation more smooth. For example, when the system judges that the jumping amplitude of the projection information reaches the distortion standard preset in advance, the system considers that the current projection effect is unstable and is prone to distortion or picture jumping. The system applies the optical flow method preset in advance to collect continuous frame data of the projection information before and after the change, calculates the motion vector of each pixel point corresponding to the projection information by comparing the continuous frame data, and the motion vector specifically includes the motion speed of the pixel point and the motion direction of the pixel point. The system can adjust the projection content based on the motion vector by obtaining the motion information of each pixel point to correct the distortion or jumping in the picture in real time, for example, the system can adjust the edge, focusing area or geometric deformation of the projection according to the motion direction to avoid the visual interference of image jumping to the user and improve the stability and smoothness of the viewing experience through accurate motion compensation. At the same time, the real-time data collection and processing of the optical flow method can enable the system to quickly identify the distortion or jumping of the projection picture and adjust the projection content in a very short time to realize rapid correction and reduce the intervention of the user and the operator on the projector. That is, the system can automatically correct the projection content according to the calculated motion vector without manually adjusting the projection device. In interactive projection applications, the optical flow method can ensure that the projection content and the user's actions (such as gestures, touches, etc.) are always synchronized to avoid interactive delay or incoherence caused by picture jumping or distortion, improve the naturalness and accuracy of the interactive experience.

[0166] In the embodiment, the second judging module further includes:

[0167] The capturing unit is configured to capture a moving track corresponding to the interactive content in real time based on a sensor preset for the wall projection area, and the sensor specifically includes an infrared sensor, a touch sensor and a pressure sensor.

[0168] a fourth determining unit, configured to determine whether the movement trajectory matches a mapping rule preset in the CIM model;

[0169] The fourth execution unit is used to respond to the corresponding dynamic display content on the wall projection area according to the movement trajectory, detect the position change information of the user in the wall projection area, and dynamically adjust the display position of the projection content according to the position change information.

[0170] In this embodiment, the system is based on sensors pre-installed in the wall projection area. The sensors specifically include infrared sensors, touch sensors and pressure sensors, which capture the movement trajectory corresponding to the interactive content in real time. The system then determines whether these movement trajectories match the mapping rules pre-installed in the CIM model to execute the corresponding steps; for example, when the system determines that the movement trajectory corresponding to the interactive content cannot match the mapping rules pre-installed in the CIM model, the system will consider that the user's interactive behavior deviates from the interactive model preset by the system. The system will dynamically update the mapping rules by real-time monitoring and learning the user's interactive behavior. At the same time, through artificial intelligence or machine learning algorithms, the system can gradually adjust CIM model is used to adapt to the behavior patterns of different users. For example, the system collects user trajectory data (such as gestures, touch points, pressure points, etc.) and reconstructs mapping rules according to the user's interaction habits to make it more flexible and accurate. The system also calibrates the sensor regularly to ensure that it can accurately capture the interaction trajectory under different environmental conditions. For example, the sensitivity of the infrared sensor is enhanced, or the response curve of the pressure sensor is optimized to ensure stable operation in various environments. For example, when the system determines that the movement trajectory corresponding to the interactive content can match the mapping rules preset by the CIM model, the system will consider that the user's interactive behavior has successfully interacted with the system's preset interaction model, and the system will According to these movement trajectories, the corresponding dynamic display content is responded to on the wall projection area, the user's position change information in the wall projection area is detected, and the display position of the projection content is dynamically adjusted according to different position change information; the system can provide accurate projection content by responding to the user's actions and changes in real time, ensuring that the user's interactive operations receive instant feedback. This smooth experience can enhance the user's sense of immersion and make it easier for users to interact naturally with the system. Adjusting the projection content according to the user's actions and position changes can provide personalized visual effects for different users. This dynamic response capability allows each user to enjoy customized interactive content, and at the same time can provide personalized visual effects according to the user's position on the wall projection. The system can flexibly adjust the display position of the projected content based on the position change information in the projection area, that is, no matter how the user moves, the projected content can always be accurately aligned to the preset position, avoiding distortion or jumping, ensuring that the visual effect is always stable and clear, and through precise movement trajectory and position change information, the system can monitor and adjust the projection content in real time, reducing the projection distortion or unstable picture problems caused by irregular wall surfaces or rapid user movements. According to the user's position changes in the wall projection area, the system dynamically adjusts the position of the projection content to ensure that the projection content and the interactive content are always consistent. This synchronization can effectively avoid projection dislocation or incoordination caused by user displacement.

[0171] In this embodiment, the acquisition module further includes:

[0172] A second detection unit is configured to detect color deviation information of the wall projection area based on a preset projection effect of the projector, wherein the projection effect specifically includes brightness, focus and color;

[0173] A fifth judgment unit is configured to judge whether the color deviation information meets a preset projection requirement.

[0174] A fifth execution unit is configured to, if not, identify an influence parameter of the color deviation information on projection information according to the projection requirement, and generate the influence parameter to a preset panel of the projector in real time, wherein the influence parameter specifically includes color temperature, color gamut and color difference.

[0175] In this embodiment, the system is based on the projection effect preset by the projector, and the projection effect specifically includes brightness, focusing, and color detection wall projection area color deviation information. Then the system determines whether these color deviation information meets the pre-set projection requirements to execute corresponding steps; for example, when the system determines that the color deviation information of the wall projection area can meet the pre-set projection requirements, the system considers that the color performance of the wall projection effect meets the expectation, and the visual quality of the projection content has reached the required standard. The system will maintain the current projection settings to ensure that the picture does not appear color distortion or inconsistency in continuous use, which helps to keep the picture quality stable, avoids visual interference in the subsequent interaction process of the user, and continues to monitor the environmental changes (such as light changes, wall material changes, etc.) of the wall projection area. The system can periodically or as needed fine-tune the projection effect to keep the color and brightness in the best state to cope with possible environmental changes, and the color deviation can enter a stable running state after reaching the pre-set standard to ensure that there is no color shift problem when projecting content for a long time. This is very important for application scenarios that need to display content for a long time (such as conferences, performances, digital signs, etc.); for example, when the system determines that the color deviation information of the wall projection area cannot meet the pre-set projection requirements, the system considers that the color of the wall projection effect does not meet the expectation. The system will identify the influence parameters of the color deviation information on the projection information according to the pre-set projection requirements, and the influence parameters specifically include color temperature, color gamut, and color difference. These influence parameters are generated to the panel preset by the projector; the system can maintain the color accuracy of the projection content under different environmental conditions (such as light changes, wall material changes, etc.) by real-time identification and adjustment of the parameters affecting the color (such as color temperature, color gamut, and color difference), which can effectively avoid the color deviation problem caused by environmental changes or equipment differences. At the same time, by adjusting the color parameters, the system realizes the ability to adaptively adjust the projection content, so that the projection can maintain the ideal color effect under different walls and environmental conditions, reducing the need for manual intervention and increasing the intelligence level of the system. By identifying color deviation and adjusting the panel of the projector, the system can realize automatic color correction without human intervention, which not only saves the calibration time, but also reduces human errors, improves the overall efficiency and accuracy, and the color correction can respond to environmental changes in real time, such as changes in light intensity and different wall materials. The system automatically adjusts the color parameters of the projection content by detecting changes in environmental and equipment states to ensure that the color always meets the pre-set requirements and improves the stability of the projection effect.

[0176] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. The interactive wall projection method based on CIM digital twin is characterized by: The following steps are involved: Based on a projector preset in the wall projection area, real-time collection of environmental change data of the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data, and person movement data; Determining whether a preset projection distortion jump is detected in the environmental change data; If so, dividing the wall projection area into at least two projection partitions according to preset projection parameters of the wall projection area, identifying interactive influence information of the projection partitions, and dynamically compensating corresponding projection content using a preset CIM model based on the interactive influence information, wherein the projection parameters specifically include wall size, wall structure, and wall texture; Determining whether the projected content can be displayed synchronously with the interactive content input by the user; If not, based on the spatial coordinate data preset for the wall projection area by the CIM model, coordinate information corresponding to the interactive content is collected, the interactive response of the projection content is dynamically adjusted according to the coordinate information, and the time nodes of the interactive content and the projection content are synchronously connected on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth angle, tilt angle and normal vector of the wall; The step of collecting coordinate information corresponding to the interactive content and dynamically adjusting the interactive response of the projected content according to the coordinate information further includes: Collecting user coordinate information of the user in the wall projection area, and obtaining an interaction area between the user coordinate information and the projection content; Determining whether the interactive area can accommodate both the projection content and the interactive content; If not, a preset layer is deployed in the same space of the wall projection area, and based on the preset layer, the projection content and the interactive content are synchronously displayed using a preset display order, wherein the preset layer specifically includes a front-end layer and a background layer.

2. The interactive wall projection method based on CIM digital twin according to claim 1 is characterized in that: The step of applying a preset CIM model to dynamically compensate corresponding projection content according to the interactive impact information further includes: Based on the preset content requirements of the projection content, generating a current projection area of ​​the wall projection area, wherein the content requirements specifically include content ratio, resolution, brightness and contrast; Determining whether the current projection area matches a preset optimal projection area; If not, the reflection parameters of the light from the wall projection area are obtained through the light emitted by the projector, the parameters to be corrected of the wall projection area are collected according to the reflection parameters, and the projection content is controlled to perform geometric distortion correction according to the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, reflection angle and material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

3. The interactive wall projection method based on CIM digital twin according to claim 1 is characterized in that: Before the step of applying a preset CIM model to dynamically compensate the corresponding projection content according to the interactive impact information, the method further includes: Based on the coordinate origin predefined for the wall projection area by the CIM model, a virtual three-dimensional coordinate system applicable to the wall projection area is constructed, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis; Determining whether the virtual three-dimensional coordinate system matches the projection field of view of the projector; If so, the wall parameters of the wall projection area are measured through the virtual three-dimensional coordinate system to obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

4. The CIM digital twin-based interactive wall projection method according to claim 1 is characterized in that: The step of determining whether the environmental change data detects a preset projection distortion jump further includes: Based on the projection information preset by the projector, detecting the beating amplitude of the projection information; Determining whether the jitter amplitude reaches a preset distortion standard; If so, the preset optical flow method is applied to collect continuous frame data of the projection message before and after the change, and by comparing the continuous frame data, the motion vector of each pixel point corresponding to the projection message is calculated, wherein the motion vector specifically includes the movement speed and movement direction of the pixel point.

5. The interactive wall projection method based on CIM digital twin according to claim 1 is characterized in that: The step of determining whether the projected content can be displayed synchronously with the interactive content input by the user further includes: Based on the sensors preset in the wall projection area, the movement trajectory corresponding to the interactive content is captured in real time, wherein the sensors specifically include infrared sensors, touch sensors and pressure sensors; Determining whether the movement trajectory matches a mapping rule preset in the CIM model; If so, the corresponding dynamic display content is displayed on the wall projection area according to the movement trajectory, the position change information of the user in the wall projection area is detected, and the display position of the projection content is dynamically adjusted according to the position change information.

6. The CIM digital twin-based interactive wall projection method according to claim 1 is characterized in that: The step of collecting the environmental change data of the wall projection area in real time by the projector preset based on the wall projection area also includes: Based on the projection effect preset by the projector, detecting the color deviation information of the wall projection area, wherein the projection effect specifically includes brightness, focus and color; Determining whether the color deviation information meets a preset projection requirement; If not, then according to the projection requirements, the influencing parameters of the color deviation information on the projection message are identified, and the influencing parameters are generated in real time on the preset panel of the projector, wherein the influencing parameters specifically include color temperature, color gamut and color difference.

7. The wall interactive projection system based on CIM digital twin is characterized by: include: A collection module is used to collect environmental change data of the wall projection area in real time based on a projector preset in the wall projection area, wherein the environmental change data specifically includes object occlusion data, angle change data, and person movement data; A judging module, configured to judge whether a preset projection distortion jump is detected in the environmental change data; an execution module, configured to, if yes, divide the wall projection area into at least two projection partitions based on preset projection parameters of the wall projection area, identify interactive influence information of the projection partitions, and dynamically compensate corresponding projection content using a preset CIM model based on the interactive influence information, wherein the projection parameters specifically include wall size, wall structure, and wall texture; A second judgment module is used to judge whether the projected content can be displayed synchronously with the interactive content input by the user; a second execution module, configured to, if not, collect coordinate information corresponding to the interactive content based on spatial coordinate data preset for the wall projection area by the CIM model, dynamically adjust the interactive response of the projection content according to the coordinate information, and synchronize the time nodes of the interactive content and the projection content on the wall projection area, wherein the spatial coordinate data is specifically the positioning coordinates of the wall in three-dimensional space, and the coordinate information specifically includes the azimuth angle, tilt angle, and normal vector of the wall; The second execution module further includes: an acquiring unit, configured to acquire user coordinate information of the user in the wall projection area, and acquire an interaction area between the user coordinate information and the projection content; a second determining unit, configured to determine whether the interactive area can accommodate both the projection content and the interactive content; The second execution unit is configured to, if not, deploy preset layers in the same space of the wall projection area, and based on the preset layers, apply a preset display order to synchronously display the projection content and the interactive content, wherein the preset layers specifically include a front-end layer and a background layer.

8. The CIM digital twin-based wall interactive projection system according to claim 7 is characterized in that: The execution module also includes: A generating unit, configured to generate a current projection area of ​​the wall projection area based on preset content requirements of the projection content, wherein the content requirements specifically include content ratio, resolution, brightness, and contrast; A judging unit, configured to judge whether the current projection area matches a preset optimal projection area; an execution unit, configured to, if not, obtain reflection parameters of the light from the wall projection area through the light emitted by the projector, collect parameters to be corrected of the wall projection area based on the reflection parameters, and control the projection content to perform geometric distortion correction based on the parameters to be corrected, wherein the reflection parameters specifically include the light propagation path, the reflection angle, and the material properties of the reflecting surface, and the parameters to be corrected specifically include wall curvature and wall inclination.

9. The CIM digital twin-based wall interactive projection system according to claim 7, characterized in that: Also includes: A construction module, configured to construct a virtual three-dimensional coordinate system applicable to the wall projection area based on a coordinate origin predefined for the wall projection area by the CIM model, wherein the virtual three-dimensional coordinate system specifically includes a horizontal X-axis, a vertical Y-axis, and a depth Z-axis; A third judgment module is used to judge whether the virtual three-dimensional coordinate system can match the projection field of view of the projector; The third execution module is used to measure the wall parameters of the wall projection area through the virtual three-dimensional coordinate system, and obtain the path information from the projector to the wall projection area, wherein the wall parameters specifically include the wall size and wall shape, and the path information specifically includes the distance between the position of the projector and the wall, the focal length of the projector, and the field of view.

Images