A complex interactive device

Abstract

This invention provides a digital-real fusion interactive device. The device determines a list of collected light intensities based on the light intensity acquired by each light intensity sensor during projection by the first and second projection mechanisms. It then determines a target light intensity list based on the light intensity of the projected image output by the first and second projection mechanisms in each sub-region at that projection moment. Next, it determines the first and second sub-regions based on the difference in light intensity between the collected and target light intensity lists for each sub-region. The device analyzes the light intensity data corresponding to the first and second sub-regions to determine the user's corresponding interaction position at that projection moment. Finally, it performs corresponding interactive feedback operations based on the movement trajectory formed by several interaction positions corresponding to several consecutive preset projection moments, making the interactive feedback operations more aligned with the user's interaction intentions.

Description

Technical Field

[0001] This invention relates to the field of data-real fusion technology, and more specifically to a data-real fusion interactive device. Background Technology

[0002] Digital-real fusion interactive devices are interactive devices that combine digital technology with physical scenes, such as physical sand table demonstration models. These devices are typically used in scene displays, using high-lumen laser projectors to project images onto the sand table. The projected images are often special effects films, such as simulated ecological scenes like ships sailing on a lake or seabirds soaring. To enhance the user experience, human-computer interaction functions are also added. For example, users can place their fingers on a certain area of ​​the sand table to interact with the projected images, increasing the interaction between the user and the digital-real fusion interactive device, allowing the user to more realistically experience the demonstrated content.

[0003] Current digital-real fusion interactive devices often fail to accurately determine the user's finger position during human-computer interaction, leading to erroneous operation or display of interactive content and a decline in user experience. Summary of the Invention

[0004] To achieve the objectives of this invention, the technical solution adopted is as follows: a digital-real fusion interactive device, comprising a first projection mechanism, a second projection mechanism, a projection area, and an interactive feedback system; the first and second projection mechanisms are used to map the output projected image onto the projection area; the projection angle between the first projection mechanism and the projection area is different from the projection angle between the second projection mechanism and the projection area; the projected images output by the first and second projection mechanisms are identical in real time; and at the same time, the light intensity of the projected image output by the first projection mechanism is greater than the light intensity of the projected image output by the second projection mechanism; a plurality of light intensity sensors are evenly arranged on the projection area; each light intensity sensor corresponds to a sub-area of ​​the projection area.

[0005] The interactive feedback system is used to execute the following methods: Step S100: Based on the light intensity obtained by each light intensity sensor at the current projection moment when the first projection mechanism and the second projection mechanism are performing projection work, determine the list of collected light intensities corresponding to the projection moment. Step S200: Determine the target light intensity list corresponding to the projection moment based on the light intensity of the projected image output by the first projection mechanism and the second projection mechanism in each sub-region at the projection moment. Step S300: Based on the difference in illumination intensity between each sub-region in the light intensity list and the target light intensity list corresponding to the projection time, determine the first sub-region and the second sub-region from several sub-regions; the first sub-region is characterized as the sub-region where the projected image output by the first projection mechanism is blocked; the second sub-region is characterized as the sub-region where the projected image output by the second projection mechanism is blocked. Step S400: Perform data analysis on the illumination intensity corresponding to the first sub-region and the second sub-region to determine the interaction position of the user's interactive terminal on the projection area at the projection moment. Step S500: Based on the movement trajectory formed by several interactive positions corresponding to several projection times of the user's interactive terminal at several consecutive preset position trajectory judgment times, perform the corresponding interactive feedback operation.

[0006] In one exemplary embodiment of this application, step S100 includes: Step S110: Obtain the light intensity collected by each light intensity sensor at the current projection moment when the first and second projection mechanisms are performing projection operations, so as to obtain the light intensity list A=(A1,A2,...,A1) corresponding to the projection moment. m ,...,A n ); where m = 1, 2, ..., n; n is the number of light intensity sensors; A m This refers to the light intensity collected by the m-th light intensity sensor at the current projection moment when the first and second projection mechanisms are performing projection operations.

[0007] In one exemplary embodiment of this application, step S200 includes: Step S210: Obtain the illumination intensity of the projected image output by the first projection mechanism at the projection moment in each sub-region, so as to obtain the first light intensity list B=(B1,B2,...,B m ,...,B n ); where B m Let be the light intensity of the projected image output by the first projection mechanism at the projection moment in the m-th sub-region; the m-th sub-region is the sub-region corresponding to the m-th light intensity sensor; Step S220: Obtain the illumination intensity of the projected image output by the second projection mechanism at the projection moment in each sub-region, so as to obtain the second light intensity list C=(C1,C2,...,C m ,...,C n ); where C m Let be the illumination intensity of the projected image output by the second projection mechanism at the projection moment on the m-th sub-region; Step S230: Based on the first light intensity list B and the second light intensity list C, determine the target light intensity list D=(D1,D2,...,D...) corresponding to the projection time. m ,...,D n ); where D m D represents the superimposed light intensity of the projected image output by the first and second projection mechanisms at the projection moment on the m-th sub-region; m =B m +C m .

[0008] In one exemplary embodiment of this application, step S300 includes: Step S310: Based on the target light intensity list D and the acquired light intensity list A, determine the light intensity difference list E = (E1, E2, ..., E...). m ,...,E n ); where E m =D m -A m ; Step S320: Traverse the light intensity difference list E. If E0 < E m ≤C m If C m <E m ≤B m If , then the m-th sub-region is determined as the first sub-region; where E0 is the preset light intensity difference threshold.

[0009] In one exemplary embodiment of this application, step S320 further includes: Step S321, if E m >B m Then the m-th sub-region is determined as the user's interactive position on the projection area at that projection moment.

[0010] In one exemplary embodiment of this application, step S400 includes: Step S410: Traverse the region type corresponding to each sub-region to obtain the type identifier list F=(F1,F2,...,F...). m ,...,F n ); where F m This is the type identifier for the region type corresponding to the m-th sub-region; If the m-th subregion is the first subregion, then F m As the first identifier; If the m-th sub-region is the second sub-region, then F m For the second identifier; If the m-th subregion is neither the first subregion nor the second subregion, then Fm For third identification; Step S420: Input the type identifier list F into the preset position determination model to obtain the user's interactive position on the projection area at the projection time, as output by the position determination model. The location determination model is trained based on several types of identifiers corresponding to the projection area at historical projection times.

[0011] In one exemplary embodiment of this application, the location determination model is determined according to the following steps: Step S421: Obtain the type identifier of each sub-region of the projection area at several historical projection times to obtain several historical identifier lists G1, G2, ..., G i ,...,G j Where i = 1, 2, ..., j; j is the number of historical projection moments; G i This is the list of historical identifiers corresponding to the i-th historical projection moment; G i =(G i1 G i2 ,...,G im ,...,G in );G im This is the type identifier of the region type corresponding to the m-th sub-region of the projection region at the i-th historical projection time. If the m-th sub-region is the first sub-region at the i-th historical projection time, then G im As the first identifier; If the m-th sub-region is the second sub-region at the i-th historical projection time, then G im For the second identifier; If the m-th sub-region is neither the first sub-region nor the second sub-region at the i-th historical projection time, then G im For third identification; Step S422: Obtain the location identifier M corresponding to the user's interaction position on the projection area at the i-th historical projection time. i ; Step S423, G i As input samples, M i As output labels, supervised training is performed on a pre-defined neural network model to obtain a location determination model.

[0012] In one exemplary embodiment of this application, step S500 includes: Step S510: Obtain the interaction positions of the user's interactive terminal for the y consecutive projection times before the current projection time, so as to obtain the interaction position list H=(H0,H1,...,H...). d ,...,H y ); where d=1,2,...,y; y is the preset number of position trajectory judgments; H0 is the coordinates of the user's interactive position at the current projection time; H d The coordinates of the user's interaction position at the d-th historical projection moment before the current projection moment; Step S520: Based on the preset coordinate trajectory determination rules, determine the coordinate trajectory from H0, H1, ..., H d ,...,H y The corresponding coordinate point forms the movement trajectory Z; Step S530: If the vertical distance between the coordinate point corresponding to H0 and Z is less than a preset distance threshold, and there is a preset interaction feedback rule at the interaction position corresponding to H0, then execute the interaction feedback operation corresponding to the interaction feedback rule.

[0013] Compared with the prior art, the beneficial effects of the present invention are: When the digital-real fusion interactive device of the present invention performs projection, it first determines a list of collected light intensities corresponding to the projection moment based on the light intensity acquired by each light intensity sensor of the first and second projection mechanisms at the current projection moment, to characterize the light intensity of the projected image received in each sub-region. Then, based on the light intensity of the projected image output by the first and second projection mechanisms in each sub-region at the projection moment, it determines a list of target light intensities corresponding to the projection moment, to characterize the light intensity of the projected image actually output by the first and second projection mechanisms. Finally, based on the difference between the light intensity of each sub-region in the list of collected light intensities and the list of target light intensities corresponding to the projection moment, it determines the projected image represented by the output of the first projection mechanism from several sub-regions. The system identifies a first sub-region where the surface is obscured and a second sub-region where the projected image output by the second projection mechanism is obscured. Data analysis is performed on the light intensity corresponding to the first and second sub-regions to determine the user's interactive position on the projection area at that projection moment. Finally, based on the movement trajectory formed by the user's interactive terminal at several interactive positions corresponding to several consecutive preset projection moments, corresponding interactive feedback operations are performed. By analyzing the difference between the actual light intensity of the projected image output by the first and second projection mechanisms and the received light intensity of the projected image received by the projection area, the user's corresponding interactive position is accurately determined, making subsequent interactive feedback operations more aligned with the user's interactive intent, thereby improving the user's experience with the digital-real fusion interactive device. Attached Figure Description

[0014] Figure 1 A flowchart of the method executed by the interactive feedback system of the data-real fusion interactive device provided in the embodiments of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example: The present invention proposes a digital-real fusion interactive device, which includes a first projection mechanism, a second projection mechanism, a projection area, and an interactive feedback system.

[0017] The first and second projection mechanisms are used to map the output projected image onto the projection area. The first and second projection mechanisms can be laser projectors. The projection angle between the first and second projection mechanisms and the projection area is different. The projected images output by the first and second projection mechanisms are the same in real time, that is, at the same moment, the projected images output by the first and second projection mechanisms are the same, so as to ensure that the projected images output by the first and second projection mechanisms can be displayed on the projection area at the same time. At the same moment, the light intensity of the projected image output by the first projection mechanism is greater than the light intensity of the projected image output by the second projection mechanism, so as to ensure the accurate determination of the user's interaction position.

[0018] The projection area is the area where the projected image is displayed. The projection area can be the display area of ​​the display sand table. Several light intensity sensors, i.e. matrix light intensity sensors, are set on the projection area on average. Each light intensity sensor corresponds to a sub-area of ​​the projection area. Several sub-areas make up the projection area. Each light intensity sensor is used to collect the light intensity of the projected image on the sub-area where the light intensity sensor is located.

[0019] The interactive feedback system determines the user's interaction position based on the light intensity of the projected image on the projection area, and performs corresponding interactive feedback operations based on multiple interaction positions of the user at consecutive times. Specifically, for example... Figure 1 As shown, the interactive feedback system is used to perform the following methods: Step S100: Based on the light intensity obtained by each light intensity sensor at the current projection moment when the first projection mechanism and the second projection mechanism are performing projection work, determine the list of collected light intensities corresponding to the projection moment. Furthermore, step S100 includes step S110: Step S110: Obtain the light intensity collected by each light intensity sensor at the current projection moment when the first and second projection mechanisms are performing projection operations, so as to obtain the light intensity list A=(A1,A2,...,A1) corresponding to the projection moment. m ,...,A n ); where m = 1, 2, ..., n; n is the number of light intensity sensors; A m This refers to the light intensity collected by the m-th light intensity sensor at the current projection moment when the first and second projection mechanisms are performing projection operations.

[0020] The light intensity in the light intensity list is the actual light intensity collected by each light intensity sensor, that is, the light intensity of the projected image in each sub-region.

[0021] Step S200: Determine the target light intensity list corresponding to the projection moment based on the light intensity of the projected image output by the first projection mechanism and the second projection mechanism in each sub-region at the projection moment. Furthermore, step S200 includes steps S210-S230: Step S210: Obtain the illumination intensity of the projected image output by the first projection mechanism at the projection moment in each sub-region, so as to obtain the first light intensity list B=(B1,B2,...,B m ,...,B n ); where B m Let be the light intensity of the projected image output by the first projection mechanism at the projection moment in the m-th sub-region; the m-th sub-region is the sub-region corresponding to the m-th light intensity sensor; Step S220: Obtain the illumination intensity of the projected image output by the second projection mechanism at the projection moment in each sub-region, so as to obtain the second light intensity list C=(C1,C2,...,C m ,...,C n ); where C m Let be the illumination intensity of the projected image output by the second projection mechanism at the projection moment on the m-th sub-region; Step S230: Based on the first light intensity list B and the second light intensity list C, determine the target light intensity list D=(D1,D2,...,D...) corresponding to the projection time. m ,...,D n ); where D m D represents the superimposed light intensity of the projected image output by the first and second projection mechanisms at the projection moment on the m-th sub-region; m =B m +C m .

[0022] The target light intensity list is the superposition value of the actual light intensity of the projected image output by the first projection mechanism and the second projection mechanism, that is, the light intensity of the projected image that should be displayed on the projection area.

[0023] In addition, before determining the light intensity, the light intensity can be preprocessed. This involves setting up several ambient light intensity sensors next to the projection area to detect the light intensity of the surrounding environment. The average value of several light intensities collected by the ambient light intensity sensors at the projection time is determined as the ambient light intensity. After determining the list of collected light intensities, the ambient light intensity is subtracted from each collected light intensity in the list, and the difference is used to re-determine the light intensity in the list. This process removes the influence of ambient light on the collected light intensity, making the subsequent determination of the user's interaction position more accurate.

[0024] Step S300: Based on the difference in illumination intensity between each sub-region in the acquisition light intensity list and the target light intensity list corresponding to the projection time, determine the first sub-region and the second sub-region from several sub-regions. The first sub-region is characterized as the sub-region where the projected image output by the first projection mechanism is obscured; The second sub-region is characterized as the sub-region where the projected image output by the second projection mechanism is obscured; Furthermore, step S300 includes steps S310-S321: Step S310: Based on the target light intensity list D and the acquired light intensity list A, determine the light intensity difference list E = (E1, E2, ..., E...). m ,...,E n ); where E m =D m -A m ; The light intensity difference is represented as the difference between the actual illumination of the projected image output by the first and second projection mechanisms and the illumination of the projected image received by the projection area.

[0025] Step S320: Traverse the light intensity difference list E. If E0 < E m ≤C m If C m <E m ≤B m If , then the m-th sub-region is determined as the first sub-region; where E0 is the preset light intensity difference threshold; Step S321, if E m >B m Then the m-th sub-region is determined as the user's interactive position on the projection area at that projection moment.

[0026] If E m If ≤E0, it means that the difference between the actual illumination of the projected image output by the first projection mechanism and the second projection mechanism in the m-th sub-region and the illumination of the projected image received in the m-th sub-region is small. It is assumed that there are no obstacles or the obstacles are small in size in the m-th sub-region, so there is not much illumination loss. Therefore, there is no user interaction position in this sub-region.

[0027] If E0 < E m ≤C m If the difference between the actual illumination of the projected image output by the first projection mechanism and the second projection mechanism in the m-th sub-region and the illumination of the projected image received in the m-th sub-region is close to the light intensity of the projected image output by the second projection mechanism, then it can be assumed that there is an obstacle in this sub-region that blocks the projected image output by the second projection mechanism, but does not block the projected image output by the first projection mechanism (because the first projection mechanism and the second projection mechanism are intersected). Therefore, this sub-region is defined as the second sub-region.

[0028] If C m <E m ≤B m If the difference between the actual illumination of the projected image output by the first projection mechanism and the second projection mechanism in the m-th sub-region and the illumination of the projected image received in the m-th sub-region is close to the light intensity of the projected image output by the first projection mechanism, then it can be assumed that there is an obstacle in this sub-region that blocks the projected image output by the first projection mechanism, but does not block the projected image output by the second projection mechanism. Therefore, this sub-region is defined as the first sub-region.

[0029] If E m >B m If the difference between the actual illumination of the projected image output by the first and second projection mechanisms in the m-th sub-region and the illumination of the projected image received in the m-th sub-region is large, it can be assumed that there is an obstacle in the sub-region that blocks the projected images output by the first and second projection mechanisms. In this case, the sub-region can be directly determined as the interaction position of the user's interaction terminal in the projection area at the projection time.

[0030] Step S400: Perform data analysis on the illumination intensity corresponding to the first sub-region and the second sub-region to determine the interaction position of the user's interactive terminal on the projection area at the projection moment. The user's interaction point can be the user's hand or a handheld interaction device, and the interaction position is the projection position of the user's interaction point on the projection area or the position of direct contact on the projection area.

[0031] Furthermore, step S400 includes steps S410-S420: Step S410: Traverse the region type corresponding to each sub-region to obtain the type identifier list F=(F1,F2,...,F...). m ,...,F n ); where F m This is the type identifier for the region type corresponding to the m-th sub-region; If the m-th subregion is the first subregion, then F m As the first identifier; If the m-th sub-region is the second sub-region, then F m For the second identifier; If the m-th subregion is neither the first subregion nor the second subregion, then F m For third identification; Step S420: Input the type identifier list F into the preset position determination model to obtain the user's interactive position on the projection area at the projection time, as output by the position determination model. The location determination model is trained based on several types of identifiers corresponding to the projection area at historical projection times. Specifically, the location determination model is determined according to steps S421-S423: Step S421: Obtain the type identifier of each sub-region of the projection area at several historical projection times to obtain several historical identifier lists G1, G2, ..., G i ,...,G j Where i = 1, 2, ..., j; j is the number of historical projection moments; G i This is the list of historical identifiers corresponding to the i-th historical projection moment; G i =(G i1 G i2 ,...,G im ,...,G in );G im This is the type identifier of the region type corresponding to the m-th sub-region of the projection region at the i-th historical projection time. If the m-th sub-region is the first sub-region at the i-th historical projection time, then G im As the first identifier; If the m-th sub-region is the second sub-region at the i-th historical projection time, then G im For the second identifier; If the m-th sub-region is neither the first sub-region nor the second sub-region at the i-th historical projection time, then G im For third identification; Step S422: Obtain the location identifier M corresponding to the user's interaction position on the projection area at the i-th historical projection time. i ; Step S423, G i As input samples, M i As output labels, supervised training is performed on a pre-defined neural network model to obtain a location determination model.

[0032] Supervised training of neural network models can be performed using existing sample training methods for neural network models.

[0033] Step S500: Based on the movement trajectory formed by several interactive positions corresponding to several projection times of the user's interactive terminal at several consecutive preset position trajectory judgment times, perform the corresponding interactive feedback operation. Furthermore, step S500 includes steps S510-S530: Step S510: Obtain the interaction positions of the user's interactive terminal for the y consecutive projection times before the current projection time, so as to obtain the interaction position list H=(H0,H1,...,H...). d ,...,H y ); where d=1,2,...,y; y is the preset number of position trajectory judgments; H0 is the coordinates of the user's interactive position at the current projection time; H d The coordinates of the user's interaction position at the d-th historical projection moment before the current projection moment; Step S520: Based on the preset coordinate trajectory determination rules, determine the coordinate trajectory from H0, H1, ..., H d ,...,H y The corresponding coordinate point forms the movement trajectory Z; Step S530: If the vertical distance between the coordinate point corresponding to H0 and Z is less than a preset distance threshold, and there is a preset interaction feedback rule at the interaction position corresponding to H0, then execute the interaction feedback operation corresponding to the interaction feedback rule.

[0034] Because a user's interactive device might be located in a sub-region at a certain projection moment, but this sub-region is merely a sub-region the user's interactive device has passed through, and not the actual interaction location of the user's intended interaction, in order to make the interactive feedback operation more closely match the user's interaction intent, the system determines whether to execute an interactive feedback operation (such as clicking a button, switching pages, etc.) based on the movement trajectory formed by several interactive positions corresponding to several consecutive preset projection moments. If the interactive position at the current projection moment is a discrete point on the movement trajectory, it indicates that the interactive position is a sudden operation by the user, not a gradual movement process. Therefore, the interactive position is considered to be a position the user has passed through, and no interactive feedback operation is performed. Conversely, if the interactive position at the current projection moment is located on the movement trajectory, or the shortest straight distance to the movement trajectory is small, then the interactive position is considered to be the location corresponding to the user's interaction intent, and the interactive feedback operation corresponding to that interactive position (such as the data analysis and processing logic corresponding to a button click operation) is executed.

[0035] When the digital-real fusion interactive device of the present invention performs projection, it first determines a list of collected light intensities corresponding to the projection moment based on the light intensity acquired by each light intensity sensor of the first and second projection mechanisms at the current projection moment, to characterize the light intensity of the projected image received in each sub-region. Then, based on the light intensity of the projected image output by the first and second projection mechanisms in each sub-region at the projection moment, it determines a list of target light intensities corresponding to the projection moment, to characterize the light intensity of the projected image actually output by the first and second projection mechanisms. Finally, based on the difference between the light intensity of each sub-region in the list of collected light intensities and the list of target light intensities corresponding to the projection moment, it determines the projected image represented by the output of the first projection mechanism from several sub-regions. The system identifies a first sub-region where the surface is obscured and a second sub-region where the projected image output by the second projection mechanism is obscured. Data analysis is performed on the light intensity corresponding to the first and second sub-regions to determine the user's interactive position on the projection area at that projection moment. Finally, based on the movement trajectory formed by the user's interactive terminal at several interactive positions corresponding to several consecutive preset projection moments, corresponding interactive feedback operations are performed. By analyzing the difference between the actual light intensity of the projected image output by the first and second projection mechanisms and the received light intensity of the projected image received by the projection area, the user's corresponding interactive position is accurately determined, making subsequent interactive feedback operations more aligned with the user's interactive intent, thereby improving the user's experience with the digital-real fusion interactive device.

[0036] The embodiments disclosed herein are preferred embodiments, but are not limited thereto. Those skilled in the art can readily grasp the spirit of the present invention based on the above embodiments and make different extensions and variations, but as long as they do not depart from the spirit of the present invention, they are all within the protection scope of the present invention.

Claims

1. A digital-real fusion interactive device, characterized in that, The system includes a first projection mechanism, a second projection mechanism, a projection area, and an interactive feedback system. The first and second projection mechanisms are used to map the output projected image onto the projection area. The projection angle between the first and second projection mechanisms and the projection area are different. The projected images output by the first and second projection mechanisms are identical in real time. At the same time, the light intensity of the projected image output by the first projection mechanism is greater than the light intensity of the projected image output by the second projection mechanism. A plurality of light intensity sensors are evenly distributed on the projection area. Each light intensity sensor corresponds to a sub-area of ​​the projection area. The interactive feedback system is used to perform the following methods: Step S100: Based on the light intensity obtained by each light intensity sensor at the current projection moment when the first projection mechanism and the second projection mechanism are performing projection work, determine the list of collected light intensities corresponding to the projection moment. Step S200: Determine the target light intensity list corresponding to the projection time based on the light intensity of the projected image output by the first projection mechanism and the second projection mechanism in each sub-region at the projection time. Step S300: Based on the difference in illumination intensity between the acquired light intensity list and the target light intensity list corresponding to the projection time, determine a first sub-region and a second sub-region from the plurality of sub-regions; the first sub-region is characterized as the sub-region where the projected image output by the first projection mechanism is blocked; the second sub-region is characterized as the sub-region where the projected image output by the second projection mechanism is blocked. Step S400: Perform data analysis on the light intensity corresponding to the first sub-region and the second sub-region to determine the interaction position of the user's interactive terminal on the projection area at the projection time; Step S500: Based on the movement trajectory formed by several interactive positions corresponding to several projection times of the user's interactive terminal at several consecutive preset position trajectory judgment times, perform the corresponding interactive feedback operation. Wherein, step S400 includes steps S410-S420: Step S410: Traverse the region type corresponding to each sub-region to obtain a type identifier list F=(F1,F2,...,F...). m ,...,F n ); where F m The type identifier for the region type corresponding to the m-th sub-region; If the m-th sub-region is the first sub-region, then F m As the first identifier; If the m-th sub-region is the second sub-region, then F m For the second identifier; If the m-th sub-region is neither the first sub-region nor the second sub-region, then F m For third identification; Step S420: Input the type identifier list F into the preset position determination model to obtain the user's interactive terminal's position on the projection area at the projection time, as output by the position determination model; The location determination model is obtained by training on several types of identifiers corresponding to the projection area at historical projection times.

2. The data-real fusion interactive device according to claim 1, characterized in that, Step S100 includes: Step S110: Obtain the light intensity collected by each light intensity sensor at the current projection moment when the first projection mechanism and the second projection mechanism are performing projection work, so as to obtain the light intensity collection list A=(A1,A2,...,A1) at the projection moment. m ,...,A n ); where m=1,2,...,n; n is the number of light intensity sensors; A m The light intensity is the light intensity collected by the m-th light intensity sensor at the current projection moment when the first projection mechanism and the second projection mechanism are performing projection operations.

3. The data-real fusion interactive device according to claim 2, characterized in that, Step S200 includes: Step S210: Obtain the illumination intensity of the projected image output by the first projection mechanism at the projection time on each of the sub-regions, so as to obtain a first light intensity list B=(B1,B2,...,B m ,...,B n ); where B m The light intensity of the projected image output by the first projection mechanism at the projection moment is located in the m-th sub-region; the m-th sub-region is the sub-region corresponding to the m-th light intensity sensor. Step S220: Obtain the illumination intensity of the projected image output by the second projection mechanism at the projection time on each of the sub-regions, to obtain a second light intensity list C=(C1,C2,...,C m ,...,C n ); where C m The light intensity of the projected image output by the second projection mechanism at the projection moment on the m-th sub-region; Step S230: Based on the first light intensity list B and the second light intensity list C, determine the target light intensity list D=(D1,D2,...,D...) corresponding to the projection time. m ,...,D n ); where D m D is the superimposed light intensity of the projected images output by the first projection mechanism and the second projection mechanism at the projection moment on the m-th sub-region; m =B m +C m .

4. The data-real fusion interactive device according to claim 3, characterized in that, Step S300 includes: Step S310: Based on the target light intensity list D and the acquired light intensity list A, determine the light intensity difference list E = (E1, E2, ..., E...). m ,...,E n ); where E m =D m -A m ; Step S320: Traverse the light intensity difference list E. If E0 < E m ≤C m If C m <E m ≤B m Then the m-th sub-region is determined as the first sub-region; where E0 is a preset light intensity difference threshold.

5. The data-real fusion interactive device according to claim 4, characterized in that, Step S320 further includes: Step S321, if E m >B m Then the m-th sub-region is determined as the user's interactive position on the projection region at that projection time.

6. The data-real fusion interactive device according to claim 5, characterized in that, The location determination model is determined according to the following steps: Step S421: Obtain the type identifier of each sub-region of the projection area at several historical projection times to obtain several historical identifier lists G1, G2, ..., G i ,...,G j Where i = 1, 2, ..., j; j is the number of the historical projection times; G i This is a list of historical identifiers corresponding to the i-th historical projection moment; G i =(G i1 G i2 ,...,G im ,...,G in );G im The type identifier of the region type corresponding to the m-th sub-region of the projection region at the i-th historical projection time; If the m-th sub-region is the first sub-region at the i-th historical projection time, then G im As the first identifier; If the m-th sub-region is the second sub-region at the i-th historical projection time, then G im For the second identifier; If the m-th sub-region is neither the first sub-region nor the second sub-region at the i-th historical projection time, then G im For third identification; Step S422: Obtain the location identifier M corresponding to the interaction position of the user's interactive terminal on the projection area at the i-th historical projection time. i ; Step S423, G i As input samples, M i As output labels, a pre-defined neural network model is subjected to supervised training to obtain the location determination model.

7. The data-real fusion interactive device according to claim 6, characterized in that, Step S500 includes: Step S510: Obtain the interaction positions of the user's interactive terminal for the y consecutive projection times before the current projection time, so as to obtain the interaction position list H=(H0,H1,...,H...). d ,...,H y ); where d=1,2,...,y; y is the preset number of position trajectory judgments; H0 is the coordinates of the user's interactive position at the current projection time; H d The coordinates of the user's interaction position at the d-th historical projection moment before the current projection moment; Step S520: Based on the preset coordinate trajectory determination rules, determine the coordinate trajectory from H0, H1, ..., H d ,...,H y The corresponding coordinate point forms the movement trajectory Z; Step S530: If the vertical distance between the coordinate point corresponding to H0 and Z is less than a preset distance threshold, and there is a preset interaction feedback rule at the interaction position corresponding to H0, then execute the interaction feedback operation corresponding to the interaction feedback rule.

Images