A high-precision touch interaction system and method for multi-surface splicing projection

Abstract

A high-precision touch interaction system and method for multi-faceted splicing projection, the system comprising at least two touch-sensitive first and second screens arranged at an angle, the first screen being equipped with a first infrared laser device and a first infrared camera device; the second screen being equipped with a second infrared laser device and a second infrared camera device; the first infrared laser device and the second infrared laser device having different wavelengths. The present invention also includes a high-precision touch interaction method for multi-faceted splicing projection. The present invention can realize a system and method for controlling a computer on a projection screen with low latency, high precision, and no touch blind spots for multi-faceted splicing projection.

Description

Technical Field

[0001] The invention relates to a multi-surface projection system, and in particular to a high-precision touch interaction system and method for multi-surface splicing projection. Background Art

[0002] The multi-surface projection system configures multiple projection surfaces on two or more non-parallel surfaces, and splices the images of multiple surfaces to create an immersive display effect. For example, the technical solutions of patent number CN103995429A, application number 201510209626.8, and patent number CN202837801U all disclose multi-surface projection systems. However, although these solutions have good immersive display effects, they cannot be interactive.

[0003] Traditional interactive projection technology uses sensors to capture and shoot target images (such as participants), which are then analyzed by an image analysis system to generate the movements of the captured objects. This movement data is combined with a real-time image interaction system to create a close interactive effect between the participants and the screen. This projection technology uses sensors to capture target impressions and is widely used, but its accuracy is very low and the delay is severe, making it difficult to produce good interactive effects.

[0004] By introducing an infrared straight laser transmitter in front of the display device (preferably, a projection screen), the laser transmitter emits a layer of laser curtain parallel to and close to the projection screen. The infrared camera can only see the reflected light of the object irradiated by the laser transmitter because a filter matching the laser transmitter is placed. When a finger or other object points to a specific object in front of the projection screen, it will intersect with the laser curtain in front of the projection screen, thereby forming a reflection. The infrared camera will only see this reflected light and cannot see other spectra, so it can accurately calculate the position touched by the finger. The technical solutions of application number 200810024947.0 and application number 200910058018.6 both disclose this method. The specific principle is: by placing an infrared laser transmitter at the edge of the projection screen, the infrared laser transmitter forms a laser curtain in the emission area, which is close to and parallel to the projection screen. When we need to operate the projection screen with our fingers or objects, the contact point will reflect the infrared laser, and the infrared camera will be able to capture the position of the point efficiently and accurately, and then calculate the position corresponding to the position in the computer coordinate system through the interactive host, so as to accurately control the computer on the projection screen and achieve the purpose of interacting with the computer.

[0005] The high-precision interaction of the multi-faceted projection system is that each projection can be controlled like a computer touch screen in the projection screen of the multi-faceted projection system. At present, there is no system or method for high-precision interaction of the multi-faceted splicing projection system. The present invention introduces a single-word laser into the multi-faceted projection system, and designs a method for constructing a high-precision interactive system for multi-faceted splicing projection and designing such a system.

[0006] The present invention considers that if the multi-faceted splicing projection (we take two screens as an example) is set at an angle, the angle between the two screens is usually less than 180 degrees. In particular, in many cases, the two vertical screens are placed at 90 degrees. Usually, the No. 1 laser screen on the No. 1 projection screen is parallel to the projection screen, and there is no laser in the projection screen; in actual processing, even if the projection screen is not operated, the infrared camera can see the laser spectrum reflected by the intersection of the laser screen and other objects outside the projection screen. The system can process the laser spectrum outside the screen and will not interfere with the system; if there is no angle, the No. 1 laser screen will not irradiate onto the No. 1 projection screen, nor onto the No. 2 projection screen, but will be parallel to the screen, so the entire screen can be operated. When there is an angle between the two screens, although the No. 1 laser screen is parallel to the No. 1 projection screen, the No. 2 laser will irradiate onto the No. 1 projection screen, which will interfere with the high-precision interaction of the multi-faceted projection, and the high-precision touch interaction of the multi-projection screen cannot be performed correctly. Summary of the invention

[0007] The purpose of the present invention is to provide a high-precision touch interaction system and method for multi-faceted splicing projection that is efficient, easy to use, feasible, stable, commercially available, low-latency, and has no touch blind spots.

[0008] The technical solution of the present invention is:

[0009] A high-precision touch interaction method for multi-surface splicing projection of the present invention comprises the following steps:

[0010] At least two first screens and second screens for two-sided splicing and projection are arranged at an angle, and a first infrared laser device and a first infrared camera device adapted to the first screen and a second infrared laser device and a second infrared camera device adapted to the second screen are arranged, and the wavelengths of the first infrared laser device and the second infrared laser device are different;

[0011] Turning on each infrared laser device to form a laser curtain parallel to the surfaces of the first screen and the second screen, and the first infrared laser device will emit a line of laser onto the second screen, and the second infrared laser device will also emit a line of laser onto the first screen;

[0012] Calculate the effective working area of ​​the infrared camera device;

[0013] When the user touches the first screen with a finger or other object, the laser curtain on the surface of the first screen will intersect with the finger or object to form a reflection. The emitted spectrum can just be captured by the first infrared camera device with the infrared filter in a closed state, and the position of the corresponding point in the computer coordinate system is obtained through calculation.

[0014] Furthermore, the connection method of each device is as follows: the computer is connected to the fusion device, the fusion device is connected to the first projector and the second projector, and the images of the projectors are seamlessly spliced ​​into the first screen and the second screen; the infrared camera device is adjusted so that the infrared camera device can see the entire image of the projection screen; if the content of the first screen is seen in the image of the first infrared camera device, it is represented by V11, and the entire image captured by the first infrared camera device is V1, then Similarly, the range of the second screen image within the second infrared camera image is V21, and the entire image captured by the first infrared camera image is V2, then The infrared laser device is adjusted to emit a laser curtain so that the laser curtain does not intersect with the corresponding screen.

[0015] Furthermore, the method for obtaining the effective working area of ​​the infrared camera device comprises the following steps:

[0016] A: Set the effective area of ​​the infrared camera device:

[0017] The processing unit on the computer controls the filter of the infrared camera device to close. At this time, the narrowband filter works and takes the ratio of the working threshold T W The value of small P is used as the binarization threshold of the effective area, that is, the effective area threshold T E =T W –P; then perform binarization processing, for but

[0018]

[0019] The set V1E with g(x,y)=0 is the effective area corresponding to the infrared camera device; where f(x,y) represents the grayscale value of the pixel at the position of row x and column y of pixel V1; V1 is the entire picture captured by the infrared camera device;

[0020] B: Calculate the working area of ​​the infrared camera device:

[0021] The processing unit on the computer controls the filter of the infrared camera device to open. At this time, the infrared camera device captures an unshielded image. The processing unit sets the display image, that is, the first screen is all white and the second screen is all black. By selecting a suitable threshold t, V1 is binarized. but

[0022]

[0023] The set V1W of g(x,y)=255 is the working area corresponding to the infrared camera device;

[0024] C: The intersection of V1E and V1W, V1EW, is the effective working area of ​​C1, that is:

[0025] V1EW=V1E∩V1W

[0026] Furthermore, the threshold t is selected by using the maximum inter-class variance method, and the grayscale of the image is divided into two groups using a certain assumed grayscale value t. When the inter-class variance of the two groups is the largest, this grayscale value t is the optimal threshold for image binarization.

[0027] Furthermore, the method for obtaining the position of the corresponding point in the computer coordinate system includes: when the infrared camera filter is turned on, setting the computer projection background to black and the foreground to multiple white points, according to the aforementioned working threshold T W The world coordinate positions of the center positions of multiple white dots in the infrared camera device are calculated, and the world coordinate positions are perspectively transformed with the computer coordinate system that generates these points to obtain the corresponding perspective transformation matrix, and a one-to-one mapping relationship between the world coordinate system in the screen and the computer coordinate system is established; the world coordinate positions of the above points in the infrared camera device are obtained by calculation, and then the corresponding coordinate positions of the above points in the computer coordinate system are calculated based on the perspective transformation matrix.

[0028] A high-precision touch interaction system for multi-faceted splicing projection of the present invention comprises at least two touch-sensitive first and second screens arranged at an angle, wherein the first screen is provided with a first infrared laser device and a first infrared camera device; the second screen is provided with a second infrared laser device and a second infrared camera device; the wavelengths of the first infrared laser device and the second infrared laser device are different.

[0029] Further, the wavelength of the first infrared camera device is the same as the wavelength of the first infrared laser device, and the wavelength of the second infrared camera device is the same as the wavelength of the second infrared laser device; or the wavelength of the first infrared laser device is within the wavelength range of the first infrared camera device, and the wavelength of the second infrared laser device is within the wavelength range of the second infrared camera device.

[0030] Furthermore, the invention also includes a computer, wherein the computer is connected to a fusion device, and the fusion device is connected to the first projector and the second projector, and the images of the projectors are seamlessly spliced ​​into the first screen and the second screen; the infrared laser device emits a laser curtain, and the laser curtain does not intersect with the corresponding screen; the infrared camera device can see the entire image of the corresponding screen; if the content of the first screen is seen in the image of the first infrared camera device, it is represented by V11, and the entire image captured by the first infrared camera device is V1, then Similarly, the range of the second screen image within the second infrared camera image is V21, and the entire image captured by the first infrared camera image is V2, then

[0031] Furthermore, the computer includes a processing unit for processing and analyzing each frame of image taken by the infrared camera device to identify and track the movement and position of each group of contacts.

[0032] Furthermore, the first infrared laser device and the second infrared laser device each include one or more infrared lasers; the first infrared camera device and the second infrared camera device each include one or more infrared cameras with an infrared filter switching function.

[0033] The beneficial effects of the present invention are as follows: by setting infrared laser devices of different wavelengths, even if the projection screen is irradiated by another infrared laser device, it can be recognized and photographed by one of the infrared camera devices of the corresponding wavelength, while the other infrared camera device that is not within the sensing wavelength range will not trigger shooting, thereby not interfering with the other screen; thereby, high-precision interaction can be achieved for multi-wall projection, forming an immersive interactive experience, and greatly improving the experience effect of multi-faceted projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 : is an overall structural diagram of a high-precision touch interaction system for multi-surface splicing projection according to an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of the composition of the interactive projection screen L1 according to an embodiment of the present invention;

[0036] Figure 3 It is the image seen when the filter of the infrared camera C1 of the embodiment of the present invention is closed (at this time, the filter works to filter all light except the corresponding wavelength, and only light near the corresponding wavelength can pass. Usually, the image emitted is generated after the laser curtain emitted by the infrared laser G1 intersects with the object. Other places are completely black. The two white lights seen are the infrared light reflected by the intersection of the laser curtain emitted by G1 with the ground and another screen L2);

[0037] Figure 4This is the effect picture seen by the infrared camera C1 after the processing unit of the embodiment of the present invention performs effective area setting. The gray area covered part is the ineffective area and will not be processed;

[0038] Figure 5 This is the effect diagram seen by the infrared camera C1 after the processing unit of the embodiment of the present invention performs the working area setting. The non-gray covered part is the working area, and the non-working area usually includes the non-valid area;

[0039] Figure 6 This is a rendering of the effective area completely overlapping with the projection screen L11 when the infrared camera C1 turns on the filter and the filter does not work after the processing unit of the embodiment of the present invention performs the working area setting;

[0040] Figure 7 It is the image seen when the filter of the infrared camera C1 of the embodiment of the present invention is closed and the bottom of the projection screen L11 is uneven (when the bottom of the projection screen L11 is uneven, since L11 and the laser curtain L12 are very close, the raised part of L11 will intersect with the laser curtain L12);

[0041] Figure 8 It is the effect diagram seen by the infrared camera C1 after the processing unit performs the effective working area setting when the bottom of the projection screen L11 is uneven in the embodiment of the present invention (in this case, the non-working area cannot include the non-effective area);

[0042] Fig. 9 This is a rendering of the infrared camera C1 when the processing unit of the embodiment of the present invention works normally (at this time, the filter is in a closed state, and when the finger touches the projection screen L1, the laser screen and the finger form a reflection. Such a picture has a clear contrast, and the processing efficiency is very high for the processing unit, and processing and controlling the computer can have a near real-time experience);

[0043] Fig.10 This is a diagram showing the effect seen by C1 when the infrared lasers G1 and G2 use the same wavelength when the filter of the infrared camera C1 is closed in the embodiment of the present invention (the white light on the vertical right is a line of infrared light reflected by the infrared laser G1 on the projection screen L2; the white light on the vertical left is the infrared light reflected by G2 and L1);

[0044] Fig.11 The embodiment of the present invention is Fig.10 The effect diagram seen by C1 after the processing unit executes the effective working area setting under the condition of ;

[0045] Fig.12 The embodiment of the present invention is Fig.10Under the condition of , the filter of C1 is turned on, and the effect seen by C1 after the filter does not work (it can be seen that part of the projection screen L11 is set as an ineffective area, and this part cannot be interactively touched).

[0046] Marking Description:

[0047] Computer------Generate images for display on projectors P1 and P2, and run processing units;

[0048] Processing unit---processes the images of infrared cameras C1 and C2 and controls the computer;

[0049] Fusion device - used to merge projectors P1 and P2 into a large screen without overlap;

[0050] P1, P2------projector, used to display the image in the computer;

[0051] L1, L2------touchable interactive projection screen, including projection screen L11 / L21 and laser screen L12 / L22;

[0052] G1, G2------infrared laser, used to emit a laser curtain, emitting a layer of infrared laser curtain for the projection screen L11 / L21, and the infrared laser wavelengths of G1 and G2 are different;

[0053] C1, C2------Infrared cameras with switchable infrared narrow-band filters, used to capture images of corresponding projection screens L11, L21, and together with the laser screens emitted by G1, G2, realize the touch function of interactive projection screens L1, L2; the wavelengths of the filters in infrared cameras C1 and C2 are different, the filter of C1 has the same wavelength as the laser emitted by G1; the filter of C2 has the same wavelength as the laser emitted by G2. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below with reference to specific embodiments.

[0055] A high-precision touch interaction system for a multi-surface projection system comprises at least two projection screens, namely a first screen and a second screen, wherein the first screen is equipped with a first infrared laser device and a first infrared camera device; the second screen is equipped with a second infrared laser device and a second infrared camera device; the first infrared laser device and the second infrared laser device have different wavelengths.

[0056] In this embodiment, since infrared laser devices with different wavelengths are provided, that is, the laser of the first infrared laser device irradiates the second projection screen, the image captured by the second infrared camera device will not be interfered by the laser of the first screen, because the second infrared camera device also cannot see the laser spectrum emitted by the first infrared laser device on the second screen, thus no interference will occur.

[0057] In this embodiment, the wavelength of the first infrared camera device is the same as the wavelength of the first infrared laser device, and the wavelength of the second infrared camera device is the same as the wavelength of the second infrared laser device; or the wavelength of the first infrared laser device is within the wavelength range of the first infrared camera device, and the wavelength of the second infrared laser device is within the wavelength range of the second infrared camera device. In this way, when the first projection screen is irradiated by the infrared lasers of other screens, due to the different wavelength ranges, the first infrared camera device will not see the infrared lasers irradiated to the first screen by the infrared laser devices corresponding to other screens, thereby not affecting the interactive effect of the first screen in the system, and thus can bring a complete interactive experience to the multi-faceted splicing projection.

[0058] In this embodiment, multi-screen splicing is achieved by multiple projectors, and the computer can output images with higher resolution.

[0059] In this embodiment, the first infrared camera device and the second infrared camera device are both connected to a computer, and the computer is connected to multiple projectors through a fusion device (including software fusion or hardware fusion device), and the images of multiple projectors are partially overlapped to form an ultra-large screen that can be turned. The computer is used to receive the image taken by the first infrared camera device or the second infrared camera device, and process and analyze each frame of the image taken to identify and track the movement and position of each group of contacts.

[0060] In this embodiment, the first infrared laser device and the second infrared laser device each include one or more infrared lasers; the first infrared camera device and the second infrared camera device each include one or more infrared cameras. That is, one infrared camera can be responsible for one or more screens, or multiple infrared cameras can be responsible for one screen. And, one screen can match one or more infrared lasers. Among them, the infrared camera of this embodiment is a camera with a switchable infrared narrow-band filter. When the filter is open, it is an ordinary camera, and when the filter is closed, it is an infrared camera.

[0061] In this embodiment, the user can use a finger or a stylus pen to perform touch operations on the screen.

[0062] The following is a preferred embodiment of the present invention:

[0063] A high-precision touch interactive system for a multi-surface projection system includes: two interactive projection screens L1 and L2, infrared lasers G1 and G2, infrared cameras C1 and C2, a computer, and projectors P1 and P2. The interactive projection screen includes two layers, such as Figure 2 As shown: the interactive projection screen L1 includes a projection screen L11 and a laser screen L12, and the laser screen L12 is located in front of the projection screen L11; L11 and L12 do not intersect and the thickness does not exceed 5mm. The screens L1 and L2 are arranged at an angle, for example, the interactive projection screen L1 is arranged on a wall, and the interactive projection screen L2 is arranged on another wall perpendicular to the wall, so that the interactive projection screens L1 and L2 are arranged at 90°. The infrared lasers G1 and G2 are used to emit laser screens L12 and L22 parallel to the projection screen L11 and the projection screen L21 respectively; the infrared laser G1 is arranged above the projection screen L11, and the infrared laser G2 is arranged above the projection screen L21. The wavelengths of the infrared lasers G1 and G2 are different, for example, the wavelength of the infrared laser G1 is 850nm; the wavelength of G2 is 940nm. Infrared cameras C1 and C2 are used to collect each frame of images of the user's touch (such as finger touch) on the interactive projection screen. The interactive projection screens L1 and L2 are located within the viewing angles of infrared cameras C1 and C2, and the infrared wavelength of infrared camera C1 matches the wavelength of infrared laser G1, and the infrared wavelength of infrared camera C2 matches the wavelength of infrared laser G2. For example: the wavelength of infrared camera C1 is a narrow-band filter of 850nm, and the wavelength of infrared camera C2 is a narrow-band filter of 940nm.

[0064] Preferably, C1 and C2 use black and white cameras.

[0065] A high-precision touch interaction method for a multi-surface projection system comprises the following steps:

[0066] Step 1: Connect all products, connect the computer to the fusion device, connect the fusion device to projectors P1 and P2, and seamlessly splice the images of the projectors into two projection screens L11 and L21; adjust the infrared camera C1 so that the infrared camera C1 can see the entire image of the projection screen L11. If the content of L11 is seen in the image of C1, it is represented by V11. The entire image seen by C1 is V1, so Similarly, the range of L21's screen within C2's screen is V21, and the entire screen of C2 is V2, so The infrared laser G1 emits a laser curtain L12, and the infrared laser G1 is adjusted so that the laser curtain L12 does not intersect with the projection screen L11. Similarly, the laser curtain L22 emitted by G2 does not intersect with the projection screen L21.

[0067] Step 2: Calibration.

[0068] (1) Set the effective area of ​​infrared camera C1

[0069] Calculate the effective area of ​​the infrared camera C1, and control the filter of the infrared camera C1 to close through the processing unit on the computer. At this time, the narrow-band filter works. Since the distinction between the black background and the laser irradiation area is very obvious, this embodiment can take the working threshold T W The smallest P value is used as the binarization threshold of the effective area, that is, the effective area threshold T E =T W –P. Then perform binarization processing and use f(x,y) to represent the pixel grayscale value at the position of the pixel x row and y column of V1, that is, for but:

[0070]

[0071] The set V1E where g(x,y)=0 is the effective area corresponding to the infrared camera C1. The system will only process the signals within the range of V1E for the infrared camera C1. In the actual working environment, it is impossible to require that the infrared camera C1 cannot see any laser emitted by the infrared laser G1. The use of the effective area V1E makes the system practical.

[0072] like Figure 3 As shown in the figure: This figure is an image before the effective area is executed. The lasers in the lower row are infrared lasers emitted by intersecting with the ground, and the lasers on the right are lasers generated by intersecting with the interactive projection screen L2. Figure 4 As shown: This figure shows the screen after the effective area setting is executed. The part covered by the gray pattern will be set as a blind area, and the processing unit running on the computer will not process the data in this area.

[0073] (2) Calculate the working area of ​​C1.

[0074] The processing unit on the computer controls the filter of the infrared camera C1 to open. At this time, the infrared camera sees an unshielded image. The processing unit sets the display image, that is, the interactive projection screen L1 is all white, and the interactive projection screen L2 is all black. In this way, the image V1 of the infrared camera C1 includes part or all of L1 and L2, as well as the background. In this embodiment, a suitable threshold t is selected to binarize V1, and the image portion V11 containing L1 is the effective working area of ​​the infrared camera C1. Let f(x, y) represent the grayscale value of the pixel at the position of the x row and y column of the pixel V1, that is, but:

[0075]

[0076] The set V1W of g(x,y)=255 is the working area corresponding to the infrared camera C1.

[0077] This embodiment selects a suitable threshold value, preferably using the "maximum inter-class variance method (Otsu)". The basic idea of ​​the Otsu algorithm is to use a certain assumed gray value t to divide the grayscale of the image into two groups. When the inter-class variance of the two groups is the largest, this gray value t is the optimal threshold value for image binarization. Of course, you can also specify this threshold value according to your own appropriate situation.

[0078] (3) The intersection of V1E and V1W, V1EW, is the effective working area of ​​C1. That is:

[0079] V1EW=V1E∩V1W

[0080] The processing unit will only process the points belonging to VIEW in the C1 screen, so that the interfering light sources outside the V11 screen will not affect the system. At the same time, the interfering light sources fixed in V11 will not affect the system, making the system very robust.

[0081] Figure 5 In order to complete the screen of the effective working area, the gray pattern covered part will be set as a blind area, and the processing unit will not process the data in this area. Figure 6 To open the filter, the infrared filter does not work. You can see that the effective working area (the part without gray shielding) is exactly the projection screen L11. As mentioned earlier, L12 does not intersect with L11, so under normal circumstances, all non-effective areas (such as Figure 3 The area intersecting with the ground and L2 in the laser curtain is outside L11, so the effective working area and the working area overlap. However, in the actual working environment, there are some environments where L1 is very uneven, which leads to the intersection of the laser curtain L12 and the wall, such as Figure 7 As shown, there is a foam block on L1, and the image is not set up as an effective working area; Figure 8 for Figure 7 Image of the effective area setting. In this case, the effective working area and the working area are different, which can improve the availability of the system.

[0082] Similarly, the effective working area of ​​C2 is obtained.

[0083] (3) Automatic calibration.

[0084] When the infrared camera C1 filter is turned on, the processing unit allows the computer to project a black background and a white dot on the projection screen L11. Calculate the coordinates of the center position of the white dot. Preferably, we select 7 rows and 9 columns of 63 coordinate points, a total of 35 quadrilaterals. Through C1, the coordinate positions of the center positions of the 63 points on L11 in the infrared camera C1 can be obtained and calculated, and the 63 points are divided into 35 quadrilateral areas. Perform a perspective transformation on the world coordinate positions in C1 of the 35 quadrilaterals and the computer coordinate system that generates these points to obtain the corresponding perspective transformation matrix, so that a one-to-one mapping relationship between the world coordinate system in V11 and the computer coordinate system is established.

[0085] Similarly, L21 is calibrated through C2.

[0086] After calibration, for C1, the system will only process the signal of the effective working area V1EW. When other reflective objects such as fingers or sticks approach, the objects will intersect with the laser curtain L12, thus forming a reflection. Fig. 9 is the position of the intersection point. From the image, it can be seen that except for the contact point, the background is all black. For this kind of image, according to the aforementioned working threshold T W We can quickly calculate the position of the above point in the world coordinate system of C1, and then quickly calculate the corresponding coordinate position of the point in the computer coordinate system according to the aforementioned perspective transformation matrix. The calculation time of the processing unit on an ordinary computer is usually within 3ms, so as to achieve economic efficiency and provide a near real-time user experience.

[0087] Since the wavelength of the filter of the infrared laser G2 and the infrared camera C2 is different from the wavelength of the filter of the infrared laser G1 and the infrared camera C1, the infrared laser G2 will not interfere with the infrared camera C1, and the infrared laser G1 will not interfere with the infrared camera C2. Similarly, the infrared camera C2 can also operate the computer by the same principle.

[0088] Preferably, this embodiment is described by a 2-sided splicing projection of 1920*1080. The upper left corner of the computer coordinate system corresponding to the projection screen L11 is (0,0), and the lower right corner is (1920-1,1080-1); the computer coordinate system of the projection screen L22 is: the upper left corner is (1920,0), and the lower right corner is (3840-1,1080-1). When our fingers operate on L1, we can operate the computer coordinate system corresponding to L11 according to the aforementioned perspective transformation matrix. Similarly, when we operate on L2 with objects or fingers, we can operate the computer coordinate system corresponding to L21, thereby perfectly realizing high-precision touch interaction for multi-sided splicing projections.

[0089] This embodiment compares the situation of using the same wavelength laser and filter in multi-surface splicing projection, such as Fig.10 As shown: the infrared laser G2 will irradiate into the projection screen L11, and the infrared laser G1 will also irradiate into the projection screen L21. As mentioned earlier, our system has strong stability and robustness while ensuring low latency. The system will set the part where the infrared laser G2 irradiates into L11 as a blind area, such as Fig.11 As shown. We open the filter of infrared camera C1, as shown Fig.12 As shown, it can be seen that part of the projection screen L11 is set as a blind area, and interaction is not possible.

[0090] In summary, the present invention realizes high-precision touch interaction for multi-surface spliced ​​projections, has good real-time performance and stability, and is industrially applicable.

Claims

1. A high-precision touch interaction method for multi-surface splicing projection, characterized in that: The following steps are involved: The first screen and the second screen of at least two spliced ​​projections are arranged at an angle, and a first infrared laser device and a first infrared camera device adapted to the first screen and a second infrared laser device and a second infrared camera device adapted to the second screen are arranged, and the wavelengths of the first infrared laser device and the second infrared laser device are different; the wavelength of the first infrared camera device is the same as the wavelength of the first infrared laser device, and the wavelength of the second infrared camera device is the same as the wavelength of the second infrared laser device; or The wavelength of the first infrared laser device is within the wavelength range of the first infrared camera device, and the wavelength of the second infrared laser device is within the wavelength range of the second infrared camera device; the first infrared camera device does not see the infrared laser irradiated to the first screen by the second infrared laser device, and the second infrared camera device does not see the infrared laser irradiated to the second screen by the first infrared laser device; Turning on each infrared laser device to form a laser curtain parallel to the surface of the first screen and the second screen respectively; Calculate the effective working area of ​​the infrared camera device; When the user touches the first screen with a finger or other object, the laser curtain on the surface of the first screen will intersect with the finger or object to form a reflection. The emitted spectrum can just be captured by the first infrared camera device with the infrared filter in a closed state, and the position of the corresponding point in the computer coordinate system is obtained through calculation.

2. The high-precision touch interaction method for multi-surface splicing projection according to claim 1 is characterized in that: The connection method of each device is as follows: the computer is connected to the fusion device, the fusion device is connected to the first projector and the second projector, and the images of the projectors are seamlessly spliced ​​into the first screen and the second screen; the infrared camera device is adjusted so that the infrared camera device can see the entire image of the projection screen; If the content of the first screen is seen in the picture of the first infrared camera device, it is represented by V11, and the entire picture captured by the first infrared camera device is V1, then Similarly, the range of the second screen image within the second infrared camera image is V21, and the entire image captured by the first infrared camera image is V2, then The infrared laser device is adjusted to emit a laser curtain so that the laser curtain does not intersect with the corresponding screen.

3. The high-precision touch interaction method for multi-surface splicing projection according to claim 1 or 2, characterized in that: The method for obtaining the effective working area of ​​the infrared camera device comprises the following steps: A: Set the effective area of ​​the infrared camera device: The processing unit on the computer controls the filter of the infrared camera device to close. At this time, the narrowband filter works and takes the ratio of the working threshold T W The value of small P is used as the binarization threshold of the effective area, that is, the effective area threshold T E =T W –P; then perform binarization processing, for but The set V1E with g(x,y)=0 is the effective area corresponding to the infrared camera device; where f(x,y) represents the grayscale value of the pixel at the position of row x and column y of pixel V1; V1 is the entire picture captured by the infrared camera device; B: Calculate the working area of ​​the infrared camera device: The processing unit on the computer controls the filter of the infrared camera device to open. At this time, the infrared camera device captures an unshielded image. The processing unit sets the display image, that is, the first screen is all white and the second screen is all black. By selecting a suitable threshold t, V1 is binarized. but The set V1W of g(x,y)=255 is the working area corresponding to the infrared camera device; C: The intersection of V1E and V1W, V1EW, is the effective working area of ​​C1, that is: V1EW=V1E∩V1W.

4. The high-precision touch interaction method for multi-surface splicing projection according to claim 3 is characterized in that: The threshold t is selected by using the maximum inter-class variance method, and the grayscale of the image is divided into two groups using a certain assumed grayscale value t. When the inter-class variance of the two groups is the largest, this grayscale value t is the optimal threshold for image binarization.

5. The high-precision touch interaction method for multi-surface splicing projection according to claim 1 or 2, characterized in that: The method for obtaining the position of the corresponding point in the computer coordinate system includes: when the filter of the infrared camera device is turned on, setting the computer projection background to black and the foreground to multiple white dots, calculating the world coordinate position of the center position of the multiple white dots in the infrared camera device, performing perspective transformation on the world coordinate position and the computer coordinate system that generates these points to obtain the corresponding perspective transformation matrix, and establishing a one-to-one mapping relationship between the world coordinate system in the screen and the computer coordinate system; obtaining the world coordinate position of the above-mentioned point in the infrared camera device by calculation, and then calculating the corresponding coordinate position of the above-mentioned point in the computer coordinate system according to the perspective transformation matrix.

6. A high-precision touch interaction system for multi-surface splicing projection, characterized in that: The invention comprises at least two touch-sensitive first and second screens which are arranged at an angle, wherein the first screen is provided with a first infrared laser device and a first infrared camera device; the second screen is provided with a second infrared laser device and a second infrared camera device; the first infrared laser device and the second infrared laser device have different wavelengths; the wavelength of the first infrared camera device is the same as that of the first infrared laser device, and the wavelength of the second infrared camera device is the same as that of the second infrared laser device; or the wavelength of the first infrared laser device is within the wavelength range of the first infrared camera device, and the wavelength of the second infrared laser device is within the wavelength range of the second infrared camera device; the first infrared camera device does not see the infrared laser irradiated to the first screen by the second infrared laser device, and the second infrared camera device does not see the infrared laser irradiated to the second screen by the first infrared laser device.

7. According to claim 6, a high-precision touch interaction system for multi-surface splicing projection, characterized in that: It also includes a computer, the computer is connected to the fusion device, the fusion device is connected to the first projector and the second projector, and the images of the projectors are seamlessly spliced ​​into the first screen and the second screen; the infrared laser device emits a laser curtain, and the laser curtain does not intersect with the corresponding screen; the infrared camera device can see the entire picture of the corresponding screen; if the content of the first screen is seen in the picture of the first infrared camera device, it is represented by V11, and the entire picture captured by the first infrared camera device is V1, then Similarly, the range of the second screen image within the second infrared camera image is V21, and the entire image captured by the first infrared camera image is V2, then 8. According to claim 7, a high-precision touch interaction system for multi-surface splicing projection is characterized in that: The computer includes a processing unit for processing and analyzing each frame of image taken by the infrared camera device to identify and track the movement and position of each group of contacts.

9. A high-precision touch interaction system for multi-surface splicing projection according to claim 6 or 7, characterized in that: The first infrared laser device and the second infrared laser device both include one or more infrared lasers; the first infrared camera device and the second infrared camera device both include one or more infrared cameras with an infrared filter switching function.

Images