Projection interaction method, projection device and computer-readable storage medium

Abstract

The present application provides a projection interaction method, a projection device, and a computer-readable storage medium; the method in the present application includes: obtaining the homography relationship between the original projection image and the projection image after trapezoidal correction when the projection device is projected sideways; obtaining the projection distance between the projection device and the projection surface; obtaining the target model surface corresponding to the projection distance from a preset front projection space calibration model, the front projection space calibration model including the depth information of the projection image at different projection distances; receiving an interaction operation, and determining the target interaction position of the interaction operation based on the homography relationship and the target model surface. In the embodiment of the present application, the homography relationship obtained by the trapezoidal correction of the projection image is applied to the target model surface identified by the depth sensor during side projection to determine the target interaction position of the interaction operation, thereby improving the accuracy of the interaction operation during side projection.

Description

Technical Field

[0001] The present application relates to the field of computer interaction technology, and in particular to a projection interaction method, a projection device, and a computer-readable storage medium. Background Art

[0002] Telephoto projectors typically use depth sensors to implement touch control on the projection surface. This requires calibration of the projected image and the depth sensor scanning dot matrix to identify and map the touch position of the image and operate interactive controls such as buttons in the image.

[0003] In fact, projectors generally have side projection application scenarios. In the side projection scenario, the projected image will become a rectangular image from the perspective of the projection surface after trapezoidal correction. In this way, there may be deviations between the user interaction coordinates detected by the depth sensor and the corresponding coordinates of the corrected projection image, resulting in the user's actual interaction position and the interaction target position in the corrected projection image being misaligned, so that the projection interaction in the side projection scenario is inaccurate. Summary of the Invention

[0004] The present application provides a projection interaction method, a projection device and a computer-readable storage medium, which aim to solve the technical problem of inaccurate projection interaction in side projection scenarios due to misalignment between the actual interaction position of the user and the interaction target position in the corrected projection image.

[0005] In one aspect, the present application provides a projection interaction method, the projection interaction method comprising the following steps:

[0006] Obtaining a homography relationship between an original projected image and a projected image after trapezoidal correction when the projection device is projected sideways;

[0007] Obtaining the projection distance between the projection device and the projection surface;

[0008] Acquire a target model surface corresponding to the projection distance from a preset front projection space calibration model, wherein the front projection space calibration model includes depth information of the projection image at different projection distances;

[0009] An interaction operation is received, and a target interaction position of the interaction operation is determined according to the homography relationship and the target model surface.

[0010] In some embodiments of the present application,

[0011] The front projection space calibration model is obtained by the following method:

[0012] orthographically projecting the projection image onto a first target object at a first projection distance within a maximum projection range, aligning the first target object with the projected projection image, and collecting first depth information of the first target object;

[0013] orthographically projecting the projection image onto a second target object at a second projection distance within a maximum projection range, aligning the second target object with the projected projection image, and acquiring second depth information of the second target object, wherein the first projection distance is different from the second projection distance;

[0014] Constructing a front projection space calibration model based on the first depth information and the second depth information, wherein the front projection space calibration model includes image depth information at different projection distances when the projection image is projected at a maximum projection range;

[0015] Alternatively, the projection image is orthographically projected onto a first target object at a first projection distance, so that the projected image is aligned with the first target object, and a picture scaling parameter required for image alignment and third depth information of the first target object are collected;

[0016] Performing orthographic projection of the projection image onto a first target object at a second projection distance, aligning the projected projection image with the first target object, and collecting fourth depth information of the first target object;

[0017] The front projection space calibration model is constructed according to the third depth information, the fourth depth information and the image scaling parameter. The front projection space calibration model includes image depth information at different projection distances when the projection image is projected with a maximum projection range.

[0018] In some embodiments of the present application, obtaining a target model surface corresponding to the projection distance from a preset front projection space calibration model includes:

[0019] Acquire a model surface that is at the projection distance from a vertex of the front projection space calibration model, and set the model surface as a target model surface; or

[0020] A model surface that is at the projection distance from the intersection of the extended lines of each edge of the front projection space calibration model is obtained, and the model surface is set as the target model surface.

[0021] In some embodiments of the present application, receiving the interaction operation and determining the target interaction position of the interaction operation based on the homography relationship and the target model surface includes:

[0022] Acquire an interactive operation triggered based on the corrected projection image, and obtain the coordinates of an interactive point of the interactive operation on the target model surface;

[0023] A target interaction position after correction of the interaction operation is obtained based on the interaction point coordinates and the homography relationship.

[0024] In some embodiments of the present application, acquiring the interactive operation triggered by the corrected projection image to obtain the coordinates of the interactive point of the interactive operation on the target model surface includes:

[0025] Acquire an interactive operation triggered based on the corrected projection image, and determine a first spatial coordinate of the interactive operation;

[0026] The first space coordinates are mapped to the target model surface based on the front projection space calibration model to obtain the interaction point coordinates of the interaction operation in the target model surface.

[0027] In some embodiments of the present application, obtaining and responding to a corrected target interaction position of the interactive operation based on the interaction point coordinates and the homography relationship includes:

[0028] Obtaining a coordinate ratio value of the interaction point coordinates in the target model surface;

[0029] Determining reference coordinates of the interaction point coordinates in the coordinate system of the projected image based on the coordinate scale value;

[0030] A target interaction position after correction of the interaction operation is obtained based on the reference coordinates and the homography relationship.

[0031] In some embodiments of the present application, obtaining the projection distance between the projection device and the projection surface includes:

[0032] The projection distance between the projection light machine along the optical axis and the projection surface corresponding to the projection image is detected by a depth sensor.

[0033] In some embodiments of the present application, before obtaining the homography between the original projected image and the projected image after trapezoidal correction when the projection device is projected sideways, the method includes:

[0034] Projecting a preset calibration image sideways onto a projection surface at a first position, and capturing the calibration image on the projection surface at a second position to obtain a captured image;

[0035] Establishing a homography between the original projection image and the projection image after trapezoidal correction based on the calibration image and the captured image;

[0036] Alternatively, based on multi-point depth information from the projection device to the projection surface and a preset correspondence relationship, a homography relationship between the original projection image and the projection image after trapezoidal correction is established.

[0037] In some embodiments of the present application, before receiving the interaction operation and determining the target interaction position of the interaction operation based on the homography relationship and the target model surface, the method includes:

[0038] Recognize the target user's gesture and determine whether the gesture triggers an interactive operation;

[0039] If the gesture of the target user triggers an interaction operation, the steps of receiving the interaction operation, determining a target interaction position of the interaction operation according to the homography relationship and the target model surface, and responding are performed.

[0040] On the other hand, the present application further provides a projection device, comprising:

[0041] one or more processors;

[0042] Memory; and

[0043] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps in the projection interaction method.

[0044] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in the projection interaction method.

[0045] The technical solution of the present application includes a projection interaction method, an apparatus, a projection device and a computer-readable storage medium; the projection interaction method includes: obtaining the homography relationship between the original projection image and the projection image after trapezoidal correction when the projection device is projected sideways; obtaining the projection distance between the projection device and the projection surface; obtaining the target model surface corresponding to the projection distance from a preset front projection space calibration model, and the front projection space calibration model is constructed according to the three-dimensional information of the projection image when projected frontally at different distances; receiving the interaction operation, and determining and responding to the interaction operation based on the homography relationship and the target model surface; the embodiment of the present application applies the homography relationship obtained by the trapezoidal correction of the projection image to the target model surface recognized by the depth sensor during side projection, thereby correcting the position of the interaction operation, so that the target interaction position is corrected, and the accuracy of the interaction operation during side projection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A schematic diagram of a scenario of the projection interaction method provided in an embodiment of the present application;

[0048] Figure 2 This is a flowchart of an embodiment of the projection interaction method in the embodiment of the present application;

[0049] Figure 3 A schematic diagram of an embodiment scenario for establishing a homography relationship in the projection interaction method in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of an embodiment scenario for establishing a homography relationship in the projection interaction method in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of an example scenario for establishing a front projection space calibration model in the projection interaction method in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of another embodiment of a scene for establishing a front projection space calibration model in the projection interaction method in an embodiment of the present application;

[0053] Figure 7 for Figure 6 Reference schematic diagram of the front projection space calibration model in the projection interaction method in the embodiment;

[0054] Figure 8 for Figure 6 Reference schematic diagram of the front projection space calibration model in the projection interaction method in the embodiment;

[0055] Figure 9 A schematic diagram of a scene of an embodiment of determining a target model surface in a projection interaction method provided in an embodiment of the present application;

[0056] Figure 10 A schematic flow chart of an embodiment of determining a target model surface in a projection interaction method provided in an embodiment of the present application;

[0057] Figure 11 A schematic flow chart of an embodiment of determining a target interaction position in a projection interaction method provided in an embodiment of the present application;

[0058] Figure 12 A schematic diagram of a scene of an embodiment of determining a target interaction position in a target model surface in a projection interaction method provided in an embodiment of the present application;

[0059] Figure 13 This is a schematic diagram of an embodiment of a flow chart for triggering an interactive operation in the projection interaction method provided in an embodiment of the present application;

[0060] Figure 14 This is a schematic structural diagram of an embodiment of a projection interaction device provided in an embodiment of the present application;

[0061] Figure 15 This is a schematic structural diagram of an embodiment of the projection device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of the present invention.

[0063] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0064] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to make and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art will recognize that the invention can be practiced without these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0065] The embodiments of the present application provide a projection interaction method, apparatus, device, and computer-readable storage medium, which are described in detail below.

[0066] The projection interaction method in an embodiment of the present invention is applied to a projection interaction device, which is arranged in a projection device. The projection device is provided with one or more processors, memories, and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the projection interaction method.

[0067] like Figure 1 As shown, Figure 1 This is a scene diagram of the projection interaction method in an embodiment of the present application. In the embodiment of the present invention, the projection interaction scene includes a projection device 100 (a projection interaction device is integrated in the projection device 100), and a computer-readable storage medium corresponding to the projection interaction is run in the projection device 100 to perform the steps of the projection interaction.

[0068] It is understandable that Figure 1 The projection device in the projection interaction scene shown, or the devices included in the projection device, does not constitute a limitation on the embodiments of the present invention. That is, the number of devices and types of devices included in the projection interaction scene, or the number and types of devices included in each device, do not affect the overall implementation of the technical solution in the embodiments of the present invention, and can all be regarded as equivalent replacements or derivatives of the technical solution claimed to be protected in the embodiments of the present invention.

[0069] In the embodiment of the present invention, the projection device 100 is mainly used to: obtain the homography relationship between the original projection image and the projection image after trapezoidal correction when the projection device is side-projected; obtain the projection distance between the projection device and the projection surface; obtain the target model surface corresponding to the projection distance from a preset front projection space calibration model, and the front projection space calibration model includes depth information of the projection image at different projection distances; receive interactive operations, and determine the target interaction position of the interactive operation based on the homography relationship and the target model surface.

[0070] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer projection devices shown in , or the projection device network connection relationship, such as Figure 1 Only one projection device is shown in the figure. It can be understood that the projection interaction scene can also include one or more other projection devices, which are not limited here. The projection device 100 can also include a memory.

[0071] In addition, in the projection interaction scenario of the present application, the projection device 100 can be provided with a display device, or the projection device 100 is not provided with a display device and is in communication with an external display device 200, and the display device 200 is used to output the results of the execution of the projection interaction method in the projection device. The projection device 100 can access a backend database 300 (the backend database can be in the local memory of the projection device, or the backend database can also be set up in the cloud), and the backend database 300 stores information related to the projection interaction.

[0072] It should be noted that Figure 1 The scene diagram of the projection interaction method shown is only an example. The projection interaction scene described in the embodiment of the present invention is to more clearly illustrate the technical solution of the embodiment of the present invention and does not constitute a limitation on the technical solution provided by the embodiment of the present invention.

[0073] Based on the above projection interaction scenario, an embodiment of a projection interaction method is proposed.

[0074] like Figure 2 As shown, Figure 2 This is a flowchart of an embodiment of the projection interaction method in the embodiment of the present application. The projection interaction method includes steps 201-204:

[0075] 201 , obtaining a homography relationship between an original projection image and a projection image after trapezoidal correction when the projection device is in side projection.

[0076] The projection interaction method in this embodiment is applied to a projection device, which includes but is not limited to a projection optical machine and a depth sensor. The projection device can be communicatively connected with a terminal or a server.

[0077] In this embodiment, the method for obtaining the homography relationship between the original projected image and the projected image after trapezoidal correction during side projection is not specifically limited. For example, the projection device can first measure the homography relationship through various means, and then adjust the projected image based on the homography relationship to achieve trapezoidal correction; or the four vertices of the projected image can be manually adjusted to achieve trapezoidal correction, and then the homography relationship is determined based on the pre-correction and post-correction projection images in the same coordinate system. In this embodiment, a specific implementation method for the projection device to obtain the homography relationship is provided, including:

[0078] 1. Projecting a preset calibration image onto a projection surface at a first position, and capturing the calibration image on the projection surface at a second position to obtain a captured image;

[0079] A homography relationship between the original projection image and the projection image after trapezoidal correction is established according to the calibration image and the captured image.

[0080] 2. Based on the multi-point depth information from the projection device to the projection surface and a preset correspondence relationship, a homography relationship between the original projection image and the trapezoidal-corrected projection image is established.

[0081] In the first method, the projector projects a preset calibration image onto the projection surface from a first position during side projection. The user then uses a camera to capture the calibration image on the projection surface from a second position facing the projection surface to obtain a captured image. The first position is where the projection axis of the projector is tilted relative to the projection surface, and the second position is where the user faces the projection surface. The projection surface can be, for example, a white wall. The calibration image can be an image with a common feature pattern, such as a QR code. The projector calculates the transformation relationship between each coordinate point in the calibration image and the captured image, and sets the transformation relationship for each coordinate point as a homography.

[0082] The second method is that the projection device directly uses a depth sensor to obtain the depth information of multiple points on the projection surface during side projection. Through the spatial geometric relationship and the preset correspondence, the projection device sets the transformation relationship of each coordinate point as a homography. Based on the depth information and spatial geometric relationship of multiple points, the inclination angle of the projection device relative to the projection surface and the projection distance can be determined, and the preset correspondence in the projection device includes the homography corresponding to these parameters. The projection device saves the homography as the homography between the original projection image before trapezoidal correction and the projection image after trapezoidal correction during side projection.

[0083] like Figure 3 and Figure 4 As shown, Figure 3 This is a schematic diagram of an embodiment of a scenario in which a homography relationship is established in a projection interaction method in an embodiment of the present application. Figure 3 This is a schematic diagram of the projected image being projected onto the wall. Figure 4 For Figure 3 The relationship between the original projection image and the corrected projection image inside the corresponding projection device. The projection device obtains the homography relationship required to correct the projection image on the wall from the trapezoid to the rectangle when the projection light machine is projected sideways. The specific form is such as a homography matrix H. That is, when the projection device is projected sideways, the original projection image (such as Figure 4 S1') will appear trapezoidal changes on the wall, such as Figure 3S1 in; therefore, when projecting sideways, the projection device first projects a calibration image with a characteristic pattern onto the projection surface (the calibration image preset in the projection device belongs to the original projection image), and the user shoots the calibration image with trapezoidal changes on the wall in the direction facing the wall to obtain a shooting picture; then, based on the characteristic pattern of the calibration image in the projection device and the vertex coordinates of the characteristic pattern of the calibration image in the shooting picture, the homography matrix H between the two images is calculated and established. The corresponding calculation method of the homography matrix H is a well-known technology in the projection field and will not be elaborated here. Through the homography matrix H, the projection device can change the original projected rectangular area to the projected trapezoidal area after trapezoidal correction, so that the final trapezoidal corrected image in the projection device is trapezoidal (such as Figure 4 S2' in , but it can be transformed back into a rectangle after being projected onto the wall, such as Figure 3 S2 in the image; based on the calculation of the homography matrix H, it is possible to know the corresponding position coordinates of the projection image after correction mapped from any coordinate in the original projection image in the projection optical machine. The use of a depth sensor to collect the distances of multiple points on the projection surface and combine them with the spatial geometric relationship to establish a homography relationship from the original projection surface to the side projection surface, which is similar to the homography relationship obtained by the above-mentioned image processing method. Based on the homography matrix H, the projection device performs trapezoidal correction on the side projection image, and projects the trapezoidally corrected projection image onto the projection surface, so that the projected projection image can present a matrix when viewed in the direction facing the projection surface, thereby facilitating viewing by users.

[0084] Correspondingly, combined with reference Figure 3 and Figure 4 The internal projection image of the projection device is S1' before correction. When the original projection image S1' is projected onto the projection surface during side projection, Figure 3 S1 in the figure; S2' is the projection image obtained after the original projection image S1' is corrected by the homography matrix H. When S2' is projected onto the projection surface during side projection, Figure 3 Based on the coordinate system of the original projected image S1', any coordinate in S1' can be mapped to the image area of ​​S2' through the homography matrix H, or any coordinate in the image area of ​​S2' can be mapped to S1' through the inverse matrix of the homography matrix H.

[0085] 202. Obtain a projection distance between the projection device and the projection surface.

[0086] The projection device obtains the projection distance between the projection light machine and the projection surface, wherein the method for the projection device to obtain the projection distance is not specifically limited, and can be: a depth sensor is set in the projection device, and the projection device detects the projection distance between the projection light machine along the optical axis of the light machine to the projection surface through the depth sensor. The type of depth sensor is not specifically limited, and the depth sensor can include a ToF depth sensor, a structured light depth sensor or other distance sensors.

[0087] Taking the example of setting up a ToF transmitter module in a projection device, the projection device obtains the projection distance from the optical axis direction of the ToF transmitter module to the projection surface, where the ToF (full name in English: Time of flight, Chinese name: Time of Flight method) ranging method is implemented. The ToF ranging method is a two-way ranging technology that mainly uses the flight time of the signal between two asynchronous transceivers to measure the distance between nodes. The projection distance can be the distance from the depth sensor along the central axis to the projection surface, or the distance from the projection optical machine to the projection surface, such as the distance from the projection optical machine to the bottom edge of the projected image on the projection surface, or the distance from the projection optical machine along the central axis of the light-emitting surface to the projection surface.

[0088] 203 : Acquire a target model surface corresponding to the projection distance from a preset front projection space calibration model, where the front projection space calibration model includes depth information of the projection image at different projection distances.

[0089] The projection device obtains a pre-built front projection space calibration model, which is constructed based on the three-dimensional information of the projected image when it is projected at different distances. Figure 5 In one embodiment, the steps of constructing the front projection space calibration model include:

[0090] 1.1. Orthographically project the projection image onto a first target object at a first projection distance within a maximum projection range, aligning the first target object with the projected projection image, and acquiring first depth information of the first target object;

[0091] 1.2. Then, orthographically projecting the projection image onto a second target object at a second projection distance within a maximum projection range, aligning the second target object with the projected projection image, and acquiring second depth information of the second target object, wherein the first projection distance and the second projection distance are different;

[0092] 1.3. Construct a front projection space calibration model based on the first depth information and the second depth information, where the front projection space calibration model includes image depth information at different projection distances when the projection image is projected at a maximum projection range.

[0093] This construction method requires replacing the screen of corresponding size according to the projection image size at different projection distances. That is, the first target object and the second target object are screens of different sizes, and the two correspond to different projection distances respectively. In the process of constructing the model, it is necessary to use a screen of the same size as the projection image size at the corresponding projection distance, and then use the depth sensor to detect the depth information of the four vertices of the screen to obtain the corresponding first depth information and second depth information. These depth information can reflect the spatial depth information of the maximum projection range of the projection image at the corresponding projection distance. At the same time, it can be determined that the vertex of the front projection space calibration model is the projection light point of the projection optical machine. The determination of the projection light point can ensure that the model surface of each projection distance in the front projection space calibration model accurately corresponds to the actual interaction area of ​​the user. Based on the projection distance and the spatial depth information of the maximum projection range of the projection image at the corresponding distance, the front projection space calibration model can be constructed.

[0094] like Figure 5 As shown, Figure 5 This is a schematic diagram of an embodiment scenario for establishing a front projection space calibration model in the projection interaction method in the embodiment of the present application. Two screens are placed in sequence at the first projection distance and the second projection distance in front of the projection device, and it is ensured that the projection image projected each time can just cover the screens at these two distances. When projecting on the screen at each projection distance, the depth sensor in the projection device is used to detect the depth information of the screen in front, that is, the projection device collects the first depth information of the first hard screen at the first projection distance through the depth sensor; then the projection device obtains the second depth information of the second hard screen object at the second projection distance. The first projection distance is different from the second projection distance. For example, the first projection distance is 2 meters and the second projection distance is 4 meters. The first depth information and the second depth information can be achieved by the ToF detection or structured light detection methods mentioned above.

[0095] The projection device obtains the depth information of the screen at different front projection distances, and then calculates the quadrangular pyramid model or quadrangular pyramid model of the projected image in the front projection scene as the front projection space calibration model and saves it. Figure 5 , the edge spatial coordinates of the screen with full-screen projection at a projection distance of 2 meters and the edge spatial coordinates of the screen with full-screen projection at a projection distance of 4 meters are measured by the depth sensor. The spatial coordinates of the vertex of the screen edge at 2 meters and the corresponding vertex spatial coordinates of the screen edge at 4 meters are connected to form a quadrangular pyramid or quadrangular pyramid-shaped spatial calibration model. Based on this model, the depth information of the projected image at any projection distance during front projection can be obtained.

[0096] refer to Figure 6 In one embodiment, the steps of constructing the front projection space calibration model may also include:

[0097] 2.1. Orthographically projecting the projection image onto a first target object at a first projection distance so that the projected image is aligned with the first target object, and collecting image scaling parameters required for image alignment and third depth information of the first target object;

[0098] 2.2. Performing orthographic projection of the projection image onto a first target object at a second projection distance, aligning the projected projection image with the first target object, and collecting fourth depth information of the first target object;

[0099] 2.3. Construct the front projection space calibration model based on the third depth information, the fourth depth information and the image scaling parameter, wherein the front projection space calibration model includes image depth information at different projection distances when the projection image is projected at the maximum projection range.

[0100] This construction method only needs to use the same screen for projection image sizes at different projection distances. At different projection distances, the projection image projected by the projection device is often larger than the size of the screen. Therefore, it is necessary to perform image scaling based on the screen scaling parameters to align the scaled projection image exactly with the screen, that is, the four vertex positions of the projection image coincide or basically coincide with the four vertices of the screen. Subsequently, based on the detected screen depth information and the screen scaling parameters, the spatial depth information of the maximum projection range of the projection image at the corresponding projection distance can be inferred. In 2.1, it is assumed that the size of the first target object at the first projection distance is smaller than the projection image projected by the projection device at the maximum projection range, so the projection image needs to be scaled; but in 2.2, there is also a situation where the size of the first target object at the second projection distance is smaller than the projection image projected by the projection device at the maximum projection range. Therefore, in some embodiments, it is also necessary to obtain the screen scaling parameters required for alignment of the projection image at the second projection distance, and then infer the spatial depth information of the maximum projection range of the projection image at the second projection distance. Based on the projection distance and the spatial depth information of the maximum projection range of the projection image at the corresponding distance, the following can be constructed: Figure 7 and Figure 8 The front projection space calibration model shown includes the image scaling parameters corresponding to the model surface at each projection distance.

[0101] It can be understood that the embodiment of the present application is explained by taking the example that the front projection space calibration model is pre-built by the projection device. In addition, the front projection space calibration model can also be stored in a mobile device such as a mobile phone and called through an application when needed.

[0102] refer to Figure 7 , Figure 7 for Figure 6 Schematic diagram of a reference scene converted into a plane coordinate system by establishing a front projection space calibration model in the projection interaction method in the embodiment, and Figure 8 This is a reference diagram of the front projection space calibration model in the three-dimensional space coordinate system. In this calibration model, from the perspective of the depth sensor, generally, the closer the projection distance is, the larger the relative size of the screen in the depth detection range (also called the detection array) at the corresponding distance, and the farther the screen is, the smaller the relative size of the detection range at the corresponding distance. That is, at different projection distances, the relative size of the projected image in the detection range of the depth sensor during front projection is different. In this way, when the user wants to interact with the same virtual button in the projected image, since the relative coordinate position of the virtual button position A0 in the detection array at different distances is not the same, there will be a misalignment problem in the plane rectangular coordinate system based on the array. However, when the projection image is front projected, the user interaction coordinates can be converted based on the measured projection distance and the space calibration model to ensure that users at different front projection distances can always trigger the virtual button position A0.

[0103] However, when the projection device is tilted relative to the projection surface to form a side projection, the projected image on the projection surface will become a trapezoid. The user often needs to perform trapezoidal correction on the screen to make the projected image on the projection surface return to a rectangle. However, this will undoubtedly cause the virtual key position A0 to be misaligned, and the user's actual interaction coordinates cannot be corrected solely by the projection distance and the spatial calibration model, resulting in interaction errors. In the embodiment of the present application, the projection device obtains the target model surface corresponding to the projection distance during side projection from the front projection spatial calibration model, and based on the homography relationship determined at the side projection position, the projection device can accurately find the virtual key for the user's interactive operation on the target model surface. Specifically:

[0104] 204 : Receive an interaction operation, and determine a target interaction position of the interaction operation according to the homography relationship and the target model surface.

[0105] The projection device receives an interactive operation, wherein the triggering method of the interactive operation is not specifically limited, that is, the interactive operation can be actively triggered by the user, for example, the user clicks on a virtual button displayed on the projection surface, and the projection device triggers the interactive operation after detecting the user's click operation through a depth sensor.

[0106] Through the above steps 201 to 204, the embodiment of the present application applies the homography relationship obtained by the trapezoidal correction of the projected image to the target model surface recognized by the depth sensor during side projection, thereby correcting the position of the interactive operation, so that the target interaction position improves the accuracy of the interactive operation during side projection.

[0107] In step 204, after side projection and performing trapezoidal correction on the projected image, the projection device receives the user's interactive operation through the depth sensor, and then uses the homography relationship to convert the interaction coordinates of the interactive operation detected on the target model surface into the coordinates of the target interaction position corresponding to the front projection, thereby triggering the touch operation at the corresponding front projection position. Specifically, the following steps are performed:

[0108] 1. Acquire an interactive operation triggered by the corrected projection image, and obtain the coordinates of the interactive point of the interactive operation on the target model surface;

[0109] 2. Based on the interaction point coordinates and the homography relationship, obtain the target interaction position after the interaction operation is corrected.

[0110] After performing trapezoidal correction on the projected image, the projection device obtains the interactive operation triggered by the user based on the trapezoidal-corrected projection image through the depth sensor, and obtains the interaction point coordinates of the interactive operation in the target model surface based on the target model surface obtained by the projection distance mentioned above; the projection device obtains the target interaction position after the interaction operation correction based on the interaction point coordinates and the homography relationship, and the projection device responds to the instructions corresponding to the target interaction position.

[0111] For reference Figure 9 , Figure 9 A schematic diagram of a scenario illustrating an embodiment of determining a target model surface in a projection interaction method provided in an embodiment of the present application. When a user clicks a virtual button displayed on a wall, the depth sensor identifies the first spatial coordinate of the interaction operation and also detects the current projection distance dm. The target model surface in the spatial calibration model, which is at a distance dm from the model vertex, is then determined. This target model surface is essentially equivalent to the plane on which the projection image is located at the projection distance dm. The plane on which the first spatial coordinate resides in the spatial calibration model is similar in shape to the target model surface. Based on simple geometric calculations, the coordinates of the interaction point mapped from the first spatial coordinate to the target model surface can be calculated. During side projection, the projected image is projected onto the wall and, based on the relative position of the wall and the projection device, the projection image undergoes a trapezoidal transformation, forming a rectangular image on the wall. The projection image and the target model surface correspond to the rectangular image of the wall. However, after the projection image undergoes trapezoidal correction, it no longer corresponds to the target model surface. Therefore, the user's interaction operations on the virtual buttons based on the trapezoidally corrected projection image will be misaligned relative to the same virtual buttons on the target model surface. Therefore, the embodiment of the present application applies the homography relationship obtained by the trapezoidal correction of the projected image to the target model surface recognized by the depth sensor during side projection, thereby correcting the position of the interactive operation, obtaining the target interaction position, and improving the accuracy of the interactive operation during side projection.

[0112] Reference Figure 10 , Figure 10It is a flow chart of an embodiment of determining the target model surface in the projection interaction method provided in the embodiment of the present application.

[0113] In some embodiments of the present application, step 203 obtains a target model surface corresponding to the projection distance from a preset front projection space calibration model. Two specific implementation methods are provided: implementation method 1 is as follows: step 301, and implementation method 2 is as follows: step 302:

[0114] 301, obtaining a model surface that is at the projection distance from the vertex of the front projection space calibration model, and setting the model surface as a target model surface;

[0115] The projection device sets the projection emission position information of the projection light machine in the preset front projection space calibration model as the vertex of the front projection space calibration model; the projection device obtains the model surface at the projection distance from the vertex of the front projection space calibration model, and the projection device sets the model surface as the target model surface, referring to Figure 9 , Figure 9 This is a scene diagram of an embodiment of determining the target model surface in the projection interaction method provided in the embodiment of the present application.

[0116] 302 : Acquire a model surface that is at the projection distance from the intersection of the extended lines of each edge of the front projection space calibration model, and set the model surface as a target model surface.

[0117] The projection device obtains a model surface at a projection distance from the intersection of the extended edges of the front projection space calibration model, and sets the model surface as the target model surface. In this embodiment, the projection device obtains a model surface corresponding to the projection distance from the front projection space calibration model as the target model surface. The projection device performs position conversion based on the target model surface, so that the user's actual interaction position coincides with the interaction target position in the corrected projected image, ensuring interaction accuracy and improving the operability of user interaction when the projector is projected sideways.

[0118] Reference Figure 11 , Figure 11 It is a flow chart of an embodiment of determining the target interaction position in the projection interaction method provided in the embodiment of the present application.

[0119] In some embodiments of the present application, since a general homography matrix corresponds to a coordinate system, a meaningful result can only be obtained by transforming the coordinates in the coordinate system. Therefore, when transforming the coordinates, it is necessary to first map the coordinates of the point to be transformed to the coordinate system to obtain the mapping coordinates of the point in the coordinate system, and then use the homography matrix to transform the mapping coordinates. The homography matrix is ​​generally calculated based on the coordinate system of the projected image, and each plane in the projected image and the spatial calibration model is a similar rectangle, that is, when the same interaction point is mapped to planes of different distances, its coordinates in each plane may be different, but the X-coordinate ratio and the Y-coordinate ratio are the same, so the mapping coordinates can be determined in the coordinate system of the projected image based on the coordinate ratio. Therefore, step 204 receives the interactive operation, and determines the target interaction position of the interactive operation and responds based on the homography relationship and the target model surface, specifically including steps 401-402:

[0120] 401. Acquire an interactive operation triggered based on the corrected projection image and determine a first spatial coordinate of the interactive operation; map the first spatial coordinate to the target model surface based on the orthographic space calibration model to obtain the coordinates of the interactive point of the interactive operation in the target model surface.

[0121] The projection device obtains an interactive operation triggered based on the corrected projection image, and the projection device obtains the first spatial coordinates of the interactive operation; the projection device maps the first spatial coordinates to the target model surface based on the front projection space calibration model to obtain the interaction point coordinates of the interactive operation in the target model surface.

[0122] That is, refer to Figure 7 and Figure 12 , Figure 12 This is a scene diagram of an embodiment of determining the target interaction position in the target model surface in the projection interaction method provided in the embodiment of the present application. Figure 12 The projection image, depth detection range, and target model surface as seen from the perspective of the corresponding projection device or depth detection sensor. At this perspective, the position and shape of the side-projected, trapezoidally corrected projection image within the front projection space calibration model are shown as S3. The S3 area is the trapezoidally corrected projection image projected onto the projection surface from the perspective of the projection device or depth detection sensor during side projection. Users visually interact based on the S3 area. A01 is the corresponding position of the virtual button position A0 on the target model surface S4 at a projection distance dm, and A21 is the corresponding position of the virtual button position A0 within the trapezoidally corrected projection image. In the S4 coordinate system, the coordinates of A21 and A01 are misaligned.

[0123] Combine Figure 4 and Figure 12, the original projection image S1' in the projection machine and the target model surface S4 are similar graphics; the projection image S2' after trapezoidal correction in the projection machine is also similar to the projection image S3 on the projection surface seen from the perspective of the projection machine. Therefore, the homography relationship between S1' and S2' can be applied to the coordinate transformation between S3 and S4. When the user's interactive operation on A21 in S3 is captured by the depth space sensor, the coordinates of A21 in the target model surface S4 coordinate system can be obtained, and then the coordinates of A21 in the target model surface S4 coordinate system are transformed based on the homography transformation matrix H: Figure 4 For example, when the homography matrix H is used to transform the coordinates in S1' to those in S2', then in the coordinate system of S4, the coordinates of A21 are calculated using the inverse matrix of the homography matrix H to obtain A01, which is the corresponding position A0 on the target model surface. When the homography matrix H is used to transform the coordinates in S2' to S1', then A01 is calculated based on the homography matrix H. After obtaining the corresponding A0 position on the target model surface, the interactive operation corresponding to the A0 position on the target model surface is responded to, thereby achieving accurate interaction.

[0124] 402, obtaining the coordinate ratio value of the interaction point coordinates in the target model surface; based on the coordinate ratio value, determining the reference coordinates of the interaction point coordinates in the coordinate system of the projected image; based on the reference coordinates and the homography relationship, obtaining the target interaction position after the interaction operation is corrected.

[0125] The projection device obtains the coordinate ratio value of the interaction point coordinate in the target model surface through the depth sensor. For example, based on the vertex of the target model surface as the origin, the ratio x / Xmax of the interaction point coordinate relative to the maximum coordinate value in the X direction and the ratio y / Ymax of the interaction point coordinate relative to the maximum coordinate value in the Y direction are calculated, (x, y) is the interaction point coordinate in the target model surface, Xmax and Ymax are the maximum x-axis coordinate and the maximum y-axis coordinate of the target model surface respectively; based on the coordinate ratio value, the projection device determines the reference coordinates of the interaction point coordinate in the coordinate system of the original projection image, that is, based on the maximum x-axis coordinate and the maximum y-axis coordinate of the original projection image, multiplying x / Xmax and y / Ymax accordingly to obtain the reference coordinates of the interaction point in the coordinate system of the original projection image; based on the reference coordinates and the homography relationship, the projection device obtains the target interaction position after the interaction operation is corrected. In this way, in the scenario of side projection and trapezoidal correction of the projection screen, the user interaction position detected by the depth detector at the side projection position can also be corrected to readapt the correspondence between the front projection space calibration model and the front projection screen, thereby realizing position conversion and obtaining the accurate target projection position, and then triggering the interactive operation based on the target projection position.

[0126] Reference Figure 13 , Figure 13 This is a flow chart of an embodiment of triggering an interactive operation in the projection interaction method provided in the embodiments of the present application.

[0127] In some embodiments of the present application, before step 204 receives the interaction operation and determines the target interaction position of the interaction operation based on the homography relationship and the target model surface, it specifically includes steps 501-502:

[0128] 501, recognizing a gesture of a target user and determining whether the gesture triggers an interactive operation;

[0129] The projection device recognizes the gesture of the target user and compares the gesture recognition result with each gesture in a pre-stored preset gesture set. If the preset gesture set includes the user's gesture, the projection device determines the interactive operation triggered by the gesture.

[0130] 502 , if the target user's gesture triggers an interaction operation, executing the steps of receiving the interaction operation and determining a target interaction position of the interaction operation based on the homography relationship and the target model surface.

[0131] In order to reduce invalid operations of the projection device in this embodiment, the projection device recognizes operations triggered by gestures of the target user in advance, thereby reducing the number of interactive responses of the projection device.

[0132] The embodiment of the present invention also provides a projection interaction device, such as Figure 14 As shown, Figure 14 It is a schematic structural diagram of an embodiment of the projection interaction device provided in the embodiments of the present application.

[0133] In this embodiment, the projection interaction device includes:

[0134] The first acquisition module 601 is used to obtain the homography relationship between the original projection image and the projection image after trapezoidal correction when the projection device is projected sideways;

[0135] The second acquisition module 602 is used to obtain the projection distance between the projection device 3 and the projection surface;

[0136] A third acquisition module 603 is configured to acquire a target model surface corresponding to the projection distance from a preset front projection space calibration model, wherein the front projection space calibration model includes depth information of the projection image at different projection distances;

[0137] The interaction response module 604 is configured to receive an interaction operation, and determine a target interaction position of the interaction operation based on the homography relationship and the target model surface, and respond thereto.

[0138] Furthermore, the projection interaction device in this embodiment further includes:

[0139] Performing orthographic projection of the projection image onto a first target object at a first projection distance, aligning the projected first projection image with the first target object, and collecting first depth information of the first target object;

[0140] orthographically projecting the projection image onto a second target object at a second projection distance, aligning the projected second projection image with the second target object, and collecting second depth information of the second target object, wherein the first projection distance is different from the second projection distance;

[0141] A front projection space calibration model is constructed according to the first depth information and the second depth information.

[0142] Furthermore, in this embodiment, the third acquisition module 603 in the projection interaction device further includes:

[0143] Acquire a model surface that is at the projection distance from a vertex of the front projection space calibration model, and set the model surface as a target model surface; or

[0144] A model surface that is at the projection distance from the intersection of the extended lines of each edge of the front projection space calibration model is obtained, and the model surface is set as the target model surface.

[0145] Furthermore, the interactive response module 604 in the projection interaction device in this embodiment includes:

[0146] Acquire an interactive operation triggered based on the corrected projection image, and obtain the coordinates of an interactive point of the interactive operation on the target model surface;

[0147] Based on the interaction point coordinates and the homography relationship, a target interaction position after correction of the interaction operation is obtained and responded to.

[0148] Furthermore, in this embodiment, the interactive response module 604 in the projection interaction device performs: obtaining the interactive operation triggered based on the corrected projection image, and obtaining the coordinates of the interactive point of the interactive operation on the target model surface, including:

[0149] Acquire an interactive operation triggered based on the corrected projection image, and determine a first spatial coordinate of the interactive operation;

[0150] The first space coordinates are mapped to the target model surface based on the front projection space calibration model to obtain the interaction point coordinates of the interaction operation in the target model surface.

[0151] Furthermore, in this embodiment, the interactive response module 604 in the projection interactive device executes: obtaining the target interactive position after the interactive operation is corrected based on the interactive point coordinates and the homography relationship and responding, including:

[0152] Obtaining a coordinate ratio value of the interaction point coordinates in the target model surface;

[0153] Determining reference coordinates of the interaction point coordinates in the coordinate system of the projected image based on the coordinate scale value;

[0154] Based on the reference coordinates and the homography relationship, a target interaction position after correction of the interaction operation is obtained and responded to.

[0155] Furthermore, the second acquisition module 602 in the projection interaction device in this embodiment includes:

[0156] The projection distance between the projection light machine along the optical axis of the light machine and the projection surface corresponding to the projection image is detected by a depth sensor.

[0157] Furthermore, the projection interaction device in this embodiment further includes:

[0158] Projecting a preset calibration image sideways onto a projection surface at a first position, and capturing the calibration image on the projection surface at a second position to obtain a captured image;

[0159] A homography relationship is established according to the calibration image and the captured image.

[0160] Furthermore, the projection interaction device in this embodiment further includes:

[0161] Recognize the target user's gesture and determine whether the gesture triggers an interactive operation;

[0162] If the gesture of the target user triggers an interaction operation, the steps of receiving the interaction operation, determining a target interaction position of the interaction operation according to the homography relationship and the target model surface, and responding are performed.

[0163] The projection interaction device in this embodiment obtains the homography relationship between the original projection image and the projection image after trapezoidal correction when the projection device is projected sideways; obtains the projection distance between the projection device and the projection surface; obtains the target model surface corresponding to the projection distance from a preset front projection space calibration model, and the front projection space calibration model is constructed according to the three-dimensional information of the projection image when it is projected at different distances; receives the interaction operation, and determines the target interaction position of the interaction operation based on the homography relationship and the target model surface; the embodiment of the present application applies the homography relationship obtained by the trapezoidal correction of the projection image to the target model surface recognized by the depth sensor during side projection, thereby correcting the position of the interaction operation, so that the target interaction position improves the accuracy of the interaction operation during side projection.

[0164] The embodiment of the present invention further provides a projection device, such as Figure 15 As shown, Figure 15 It is a schematic structural diagram of an embodiment of the projection device provided in the embodiments of the present application.

[0165] The projection device integrates any one of the projection interaction devices provided in the embodiments of the present invention, and the projection device includes:

[0166] one or more processors;

[0167] Memory; and

[0168] One or more applications, wherein the one or more applications are stored in the memory and are configured so that the processor executes the steps of the projection interaction method described in any of the above-mentioned projection interaction method embodiments.

[0169] Specifically, the projection device may include one or more processing core processors 701, one or more computer-readable storage media memories 702, a power supply 703, an input unit 704, and other components. Those skilled in the art will understand that Figure 15 The projection device structure shown in the figure does not constitute a limitation on the projection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0170] Processor 701 is the control center of the projection device. It connects all components of the projection device using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 702 and accessing data stored in memory 702, it performs various functions of the projection device and processes data, thereby providing overall monitoring of the projection device. Optionally, processor 701 may include one or more processing cores. Preferably, processor 701 integrates an application processor and a modem processor. The application processor primarily handles performance adjustment systems, user interfaces, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 701.

[0171] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the performance adjustment system, at least one application required for a function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created based on the use of the projection device, etc. In addition, the memory 702 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0172] The projection device also includes a power supply 703 for supplying power to various components. Preferably, the power supply 703 can be logically connected to the processor 701 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 703 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0173] The projection device may further include an input unit 704, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0174] Although not shown, the projection device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the projection device will load the executable files corresponding to one or more application processes into the memory 702 according to the following instructions, and the processor 701 will run the application stored in the memory 702 to implement various functions as follows:

[0175] Obtaining a homography relationship between an original projected image and a projected image after trapezoidal correction when the projection device is projected sideways;

[0176] Obtaining the projection distance between the projection device and the projection surface;

[0177] Acquire a target model surface corresponding to the projection distance from a preset front projection space calibration model, wherein the front projection space calibration model includes depth information of the projection image at different projection distances;

[0178] An interaction operation is received, and a target interaction position of the interaction operation is determined according to the homography relationship and the target model surface.

[0179] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0180] To this end, an embodiment of the present invention provides a computer-readable storage medium, which may include a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk. A computer program is stored on the computer-readable storage medium, and the computer program is loaded by a processor to execute the steps of any projection interaction method provided in an embodiment of the present invention. For example, the computer program loaded by the processor may execute the following steps:

[0181] Obtaining a homography relationship between an original projected image and a projected image after trapezoidal correction when the projection device is projected sideways;

[0182] Obtaining the projection distance between the projection device and the projection surface;

[0183] Acquire a target model surface corresponding to the projection distance from a preset front projection space calibration model, wherein the front projection space calibration model includes depth information of the projection image at different projection distances;

[0184] An interaction operation is received, and a target interaction position of the interaction operation is determined according to the homography relationship and the target model surface.

[0185] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above and will not be repeated here.

[0186] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to implement as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments and will not be repeated here.

[0187] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0188] The above is a detailed introduction to a projection interaction method provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A projection interaction method, characterized in that: The projection interaction method includes: Obtaining a homography relationship between an original projected image and a projected image after trapezoidal correction when the projection device is projected sideways; Obtaining the projection distance between the projection device and the projection surface; Acquire a target model surface corresponding to the projection distance from a preset front projection space calibration model, wherein the front projection space calibration model includes depth information of the projection image at different projection distances; receiving an interaction operation, and determining a target interaction position of the interaction operation according to the homography relationship and the target model surface; The front projection space calibration model is obtained by the following method: orthographically projecting the projection image onto a first target object at a first projection distance within a maximum projection range, aligning the first target object with the projected projection image, and acquiring first depth information of the first target object; orthographically projecting the projection image onto a second target object at a second projection distance within a maximum projection range, aligning the second target object with the projected projection image, and acquiring second depth information of the second target object, wherein the first projection distance is different from the second projection distance; Constructing a front projection space calibration model based on the first depth information and the second depth information, wherein the front projection space calibration model includes image depth information at different projection distances when the projection image is projected at a maximum projection range; Alternatively, the projection image is orthographically projected onto a first target object at a first projection distance, so that the projected image is aligned with the first target object, and a picture scaling parameter required for image alignment and third depth information of the first target object are collected; Performing orthographic projection of the projection image onto a first target object at a second projection distance, aligning the projected projection image with the first target object, and collecting fourth depth information of the first target object; The front projection space calibration model is constructed according to the third depth information, the fourth depth information and the image scaling parameter. The front projection space calibration model includes image depth information at different projection distances when the projection image is projected with a maximum projection range.

2. The projection interaction method according to claim 1, characterized in that: The step of obtaining a target model surface corresponding to the projection distance from a preset front projection space calibration model includes: Acquire a model surface that is at the projection distance from a vertex of the front projection space calibration model, and set the model surface as a target model surface; or A model surface that is at the projection distance from the intersection of the extended lines of each edge of the front projection space calibration model is obtained, and the model surface is set as the target model surface.

3. The projection interaction method according to claim 1, characterized in that: The receiving the interaction operation and determining a target interaction position of the interaction operation according to the homography relationship and the target model surface includes: Acquire an interactive operation triggered based on the corrected projection image, and obtain the coordinates of an interactive point of the interactive operation on the target model surface; A target interaction position after correction of the interaction operation is obtained based on the interaction point coordinates and the homography relationship.

4. The projection interaction method according to claim 3, characterized in that: The acquiring of the interactive operation triggered by the corrected projection image to obtain the coordinates of the interactive point of the interactive operation on the target model surface includes: Acquire an interactive operation triggered based on the corrected projection image, and determine the first spatial coordinates of the interactive operation; map the first spatial coordinates to the target model surface based on the orthographic space calibration model to obtain the interaction point coordinates of the interactive operation in the target model surface.

5. The projection interaction method according to claim 3, characterized in that: The obtaining, based on the interaction point coordinates and the homography relationship, a target interaction position after correction of the interaction operation includes: Obtaining a coordinate ratio value of the interaction point coordinates in the target model surface; Determining reference coordinates of the interaction point coordinates in the coordinate system of the original projection image based on the coordinate scale value; A target interaction position after correction of the interaction operation is obtained based on the reference coordinates and the homography relationship.

6. The projection interaction method according to claim 1, characterized in that: Before obtaining the homography relationship between the original projection image and the projection image after trapezoidal correction when the projection device is projected sideways, the method includes: Projecting a preset calibration image sideways onto a projection surface at a first position, and capturing the calibration image on the projection surface at a second position to obtain a captured image; Establishing a homography between the original projection image and the projection image after trapezoidal correction based on the calibration image and the captured image; Alternatively, based on multi-point depth information from the projection device to the projection surface and a preset correspondence relationship, a homography relationship between the original projection image and the projection image after trapezoidal correction is established.

7. The projection interaction method according to any one of claims 1 to 6, characterized in that: Before receiving the interaction operation and determining the target interaction position of the interaction operation based on the homography relationship and the target model surface, the method includes: Recognize the target user's gesture and determine whether the gesture triggers an interactive operation; If the target user's gesture triggers an interaction operation, the steps of receiving the interaction operation and determining a target interaction position of the interaction operation according to the homography relationship and the target model surface are performed.

8. A projection device, characterized in that: The projection device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps in the projection interaction method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the projection interaction method according to any one of claims 1 to 7.

Images