TOF sensor-based projector touch recognition method, system and device

Abstract

The application provides a projector touch recognition method, system and device based on a TOF sensor, which comprises the following steps: image acquisition and preprocessing; background data acquisition: when the projector is turned on and the projection plane has no interactive unit, the amplitude image data and 3D point cloud data of the projection plane are collected, then the projection plane is fitted according to the 3D point cloud data and saved; interactive unit recognition: whether the interactive unit appears in the projection plane is judged; touch point recognition: whether the interactive unit touches the projection plane is judged, if yes, the position of the touch point of the interactive unit is recognized, the touch time is recorded, the touch flag is set, and the next step is continued; control action recognition: the recognition result of the current real-time image is compared with the recognition result of the previous real-time image, so that the control action of the interactive unit is judged. The application has the characteristics of fast speed, high accuracy, low cost, convenient use and the like.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a projector touch recognition method, system, and apparatus based on a TOF sensor. Background Technology

[0002] Nowadays, we frequently use projectors in our work, life, and study to project images or videos for viewing. However, most projectors on the market lack touch functionality and interactivity. Current technologies primarily utilize infrared touch technology and optical styluses, but both technologies have shortcomings, specifically:

[0003] Traditional infrared touch technology typically involves scanning along both the long and short sides in various directions to identify the blocked light paths. Then, based on the positions of the intersecting blocked light paths obtained from the scan, floating-point arithmetic is used to calculate the coordinates of the intersection points of the two intersecting blocked light paths to locate the touch point. Scanning all the blocked light paths is time-consuming, resulting in a long response time and slow response rate for the touchscreen.

[0004] When using an optical stylus, a built-in miniature infrared emitter sends a signal, which is captured by the infrared camera built into the projector. This signal is then processed by an image processor and converted into touchscreen coordinates, thus transforming the projected surface into a touchscreen. While an optical stylus can meet basic touch usage needs, the user experience is poor. It requires a dedicated stylus and projector, is easily lost, and is somewhat similar to older capacitive touchscreen phones where you have to physically touch the screen with a stylus. The touch experience is unsatisfactory, and they are expensive, with online sales starting at 4000 yuan.

[0005] Chinese patent application publication number CN208689559U discloses an external touch control plug-in for a projector, and Chinese patent application publication number CN109407901A discloses an external touch control system for a projector. The plug-in and the system work together. The plug-in is a self-designed housing with a storage cavity inside. Within the storage cavity is a light slot for placing an infrared lamp. A slit is opened on one side of the housing, connecting to the storage cavity. This slit forms a grid of infrared light emitted by the infrared lamp, which propagates through the slit and forms a grid arranged at a certain distance and angle from the projector screen. Each grid forms a diffractive light curtain covering the screen within a space at a corresponding set distance. The housing also has a grating, which consists of at least three types of groove-like structures formed at different angles to the horizontal direction on the housing. This creates a structure that allows reflected light beams parallel to each grid to enter each grid and reach the bottom of each grid. From this, we can see that the structure is complex, the workmanship requirements are high, and the plug-in is expensive. The external touch system consists of a light source emitting unit that emits light, a light receiving unit that receives reflected light parallel to a grid at a set angle after the light is blocked by an obstacle, and a signal processing unit that processes the unique and definite response signals generated by the sensors corresponding to each grid of reflected light and inputs them into a data processing unit. The data processing unit calculates the spatial coordinates of the touch point. The coordinate calculation method involves using three linear equations representing the spatial trajectories of the reflected light from each sensor at three angles, and the spatial coordinates of the intersection point calculated from the linear equations of the reflected light from at least two sensors are the spatial coordinates of the touch point. The calculation process is relatively complex and can only calculate the touch point; it cannot recognize conventional touch operations such as long presses and drags. Summary of the Invention

[0006] This application provides a projector touch recognition method, system, and device based on a TOF sensor, which can solve the problems of long response time and slow response rate in the current traditional infrared touch technology for touch point positioning, as well as the problems of expensive products, easy loss and inconvenience of using optical styluses.

[0007] According to a first aspect of this application, this application provides a projector touch recognition method based on a TOF sensor. The method includes: image acquisition and preprocessing: capturing real-time images of the projector's projection plane using a TOF sensor, acquiring corresponding amplitude and depth images, and then performing image preprocessing on the amplitude and depth images corresponding to the real-time images according to a preset method; collecting background data: when the projector is powered on and there are no interactive units on the projection plane, acquiring amplitude image data and 3D point cloud data of the projection plane, then fitting the projection plane based on the 3D point cloud data and saving it; identifying interactive units on the projection plane: determining whether an interactive unit appears on the projection plane. If yes, proceeding to the next step; otherwise, jumping to the image acquisition and preprocessing step; identifying touch points: determining whether the interactive unit touches the projection plane. If yes, identifying the position of the touch point of the interactive unit and recording the touch time, setting a touch flag, and proceeding to the next step; otherwise, considering that no touch is made on the projection plane, and jumping to the image acquisition and preprocessing step; identifying control actions: comparing the recognition result of the current real-time image with the recognition result of the previous real-time image to determine the control action of the interactive unit.

[0008] According to a second aspect of this application, this application provides a projector touch recognition system, the system comprising: an image acquisition and preprocessing module, used to capture real-time images of the projector's projection plane using a TOF sensor, acquire corresponding amplitude and depth images, and then perform image preprocessing on the amplitude and depth images corresponding to the real-time images according to a preset method; a background data acquisition module, used to acquire amplitude image data and 3D point cloud data of the projection plane when the projector is powered on and there is no interactive unit on the projection plane, and then fit the projection plane based on the 3D point cloud data and save it; an interactive unit recognition module, used to determine whether an interactive unit appears on the projection plane, if so, proceed to the next step, otherwise, jump to the image acquisition and preprocessing step; a touch point recognition module, used to determine whether the interactive unit touches the projection plane, if so, identify the position of the touch point of the interactive unit and record the touch time, and set a touch flag, and proceed to the next step, otherwise, consider that no touch is made on the projection plane, and jump to the image acquisition and preprocessing step; and a control action recognition module: comparing the recognition result of the current real-time image with the recognition result of the previous real-time image to determine the control action of the interactive unit.

[0009] According to a third aspect of this application, this application provides a projector touch recognition system, the system comprising: a projector for projecting an image; an interaction unit for touching the image to interact with the content of the image; a TOF sensor for capturing the image in real time, processing the obtained amplitude and depth images, and recognizing the interaction action; and a control unit for controlling the projector to respond to the action according to the interaction action.

[0010] The beneficial effects of this application are as follows: by capturing image data projected by the projector in real time using a TOF image sensor and performing image preprocessing, without the need for external objects, when interactive units such as fingers click, drag, or slide on the projection plane, the TOF image sensor can recognize the control action, and the system controls the projector to respond accordingly based on the control action. This application has the characteristics of high speed, high accuracy, low cost, and ease of use. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the projector touch recognition system of this application;

[0012] Figure 2 This is a structural framework diagram of the TOF sensor in this application;

[0013] Figure 3 This is an example of the projector touch recognition method based on a TOF sensor according to this application;

[0014] Figure 4 This is a flowchart of the projector touch recognition method based on a TOF sensor according to this application;

[0015] Figure 5 This is a schematic diagram of the fitted projection plane obtained by the acquisition projection plane step in this application;

[0016] Figure 6 This is an example schematic diagram of the steps for identifying the interactive unit of the projection plane in this application;

[0017] Figure 7 This is an example diagram illustrating the touch point identification steps of this application;

[0018] Figure 8 This is a flowchart of the identification and control action steps of this application. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0020] Please see Figure 1 , Figure 1 This is a schematic diagram of a projector touch recognition system, which includes a projector, a TOF sensor, an interaction unit, and a control unit.

[0021] The projector projects images onto a projection surface such as a screen or wall. The interactive unit (such as a finger or pen) is used to touch the image on the projection screen to interact with the content of the image. The TOF sensor is used to capture images in real time and recognize interactive actions. The control unit is used to control the projector to respond to the actions based on the interactive actions.

[0022] Please see Figure 2 The Time-of-Flight (TOF) sensor used in this invention mainly includes a microprocessor, a transmitting module, a receiving module, a memory, and a power supply unit. The transmitting module includes a light source driver, an infrared light source, and a diffuser; the receiving module includes a 3D sensor, a filter, and an optical lens. The microprocessor is the control and data processing center of the TOF sensor. Its main functions are: first, to control the light source to emit a specified modulation wave; and second, to process the depth data transmitted back by the TOF sensor and convert it into point cloud data. The microprocessor of the TOF sensor runs a touch detection program, which can obtain touch detection results in real time while taking pictures of the projection plane, such as whether there is a touch, single click, long press, drag, etc. It is worth mentioning that the infrared light source used in this TOF sensor is invisible light, with a wavelength range typically between 800nm ​​and 1550nm, and a filter is used to effectively reduce interference from ambient light and the light emitted by the projector. To achieve high-precision measurements, the light source output needs to have a good waveform. A VCSEL (Vertical Cavity Surface Emitting Laser) and an LED are used as the light-emitting devices. A VCSEL is a planar array light source; its emitted laser light is perpendicular to the top surface and features small size, easy integration, and ease of production and testing. In this embodiment, a VCSEL is selected as the light source and has undergone laser eye safety certification, ensuring that any harm to the eyes is controlled within a safe range, making it very suitable for use on a projector.

[0023] The TOF sensor is placed on the side of the projector, with the imaging area covering the projector's projection surface. It takes real-time pictures of the projector's projection surface, then processes the obtained amplitude and depth images, and finally identifies the control actions of the interactive unit and outputs the recognition results.

[0024] Please see Figure 3 and Figure 4 The projector touch recognition method based on a TOF sensor of this application includes the following steps:

[0025] Step S310: Image acquisition and preprocessing.

[0026] In this step, a real-time image of the projection plane of the projector is captured by a TOF sensor to obtain the corresponding amplitude and depth images. Then, the amplitude and depth images corresponding to the real-time images are preprocessed according to a preset method.

[0027] In step S310, the preset methods include: image filtering (median filtering, temporal filtering), distortion correction, and 3D correction.

[0028] Image filtering refers to suppressing noise in a target image while preserving as much detail as possible. Image filtering includes two methods: median filtering and temporal filtering.

[0029] Median filtering, based on order statistics theory, is a nonlinear signal processing technique that effectively suppresses noise. The basic principle of median filtering is to replace the value of a point in a digital image or sequence with the median value of all points in its neighborhood, making the surrounding pixel values ​​closer to the true value, thereby eliminating isolated noise points. Temporal median filtering is median filtering in the time domain, that is, median filtering between the previous few frames and the current frame, primarily preventing excessive jumps in depth data between frames.

[0030] The specific methods of 3D correction include:

[0031] Based on the relationship between image pixel coordinates and camera coordinates:

[0032]

[0033] We can obtain:

[0034]

[0035] Where (u0, v0) is the optical center of the TOF sensor, (f x f y ) are the focal lengths of the TOF sensor along the X and Y axes, respectively, and cosθ is the cosine of the angle between the TOF sensor coordinates and the pixel coordinates of the real-time image. The original depth value of the real-time image can be obtained by multiplying the cosine value by the original depth value of the real-time image.

[0036] Step S320: Collect background (projection plane) data.

[0037] Please see Figure 5 When the projector is powered on and there are no interactive units on the projection plane, the amplitude image data and 3D point cloud data of the projection plane are collected, and then the projection plane is fitted based on the 3D point cloud data and saved.

[0038] In the step of collecting background data, the method for fitting the projection plane based on the 3D point cloud data uses the least squares method, specifically including:

[0039] According to the general expression of the plane equation:

[0040]

[0041] The plane equation fitting can be obtained as follows:

[0042] For a series of n points (n≥3): (x i y i , z i ), i = 0, 1, ..., n-1, we need to use the point (x i y i , z i For each integer i = 0, 1, ..., n-1, fit the above plane equation to minimize S:

[0043] S=min∑(a0x i +a1y i +a2-z i ) 2

[0044] To minimize S, take the partial derivatives of a0, a1, and a2 respectively:

[0045]

[0046] Right now:

[0047]

[0048] We can obtain:

[0049]

[0050] Convert to matrix form:

[0051]

[0052] Solving the above system of linear equations yields a0, a1, and a2. Substituting these equations into formula (3-1) gives the plane equation.

[0053] Step S330: Identify the interactive units of the projection plane.

[0054] Determine whether an interactive unit appears in the projection plane. If yes, proceed to step S340; otherwise, proceed to step S310.

[0055] Please see Figure 6 In this embodiment, the interaction unit is a finger. In other embodiments, the interaction unit may also be a pen.

[0056] In step S330, the method for determining whether an interactive unit appears in the projection plane includes:

[0057] Step S331: Obtain the amplitude value of each pixel position in the real-time image obtained after step S310;

[0058] Step S332: Traverse the pixels of the real-time image, compare the difference between the amplitude values ​​of each pixel position in the real-time image and the projection plane. If there are pixels whose amplitude values ​​are greater than a first set threshold, it is considered that an interactive unit has appeared on the projection plane, and the position of these pixels is recorded. Otherwise, jump back to step S310.

[0059] In this embodiment, the first set threshold is 15. That is to say, when comparing the amplitude values ​​of the real-time image and the projection plane, if there are pixels whose amplitude values ​​differ by more than 15, it indicates that a finger appears in the real-time image, because the amplitude value changes due to the skin color of the finger and the illumination from the TOF image sensor.

[0060] Step S340: Identify the touch point.

[0061] Please see Figure 7 Determine whether the interactive unit touches the projection plane. If so, identify the position of the touch point of the interactive unit and record the touch time, and set a touch flag. Otherwise, assume that the interactive unit has not touched the projection plane and continue to step S350.

[0062] The step of determining whether the interactive unit touches the projection plane includes:

[0063] Step S341: Calculate the pixels whose amplitude difference is greater than the first set threshold, that is, the distance from the point cloud 3D coordinates (x, y, z) corresponding to the interactive unit to the projection plane. Here, the values ​​of A, B, C, and D are known, x, y, and z are the 3D coordinates of the pixel, and d is the distance from the pixel of the interactive unit to the projection plane. The specific calculation formula is as follows:

[0064]

[0065] Step S342: Retain pixels whose d is less than the second preset threshold. In this embodiment, the second preset threshold is 0.5cm. This has two advantages: first, it allows for the location of fingertips close to the projection plane; second, it eliminates noise and interference points.

[0066] Step S343: Count the number of neighboring points of these pixels that are not on the projection plane. If the number of these pixels meets the third preset threshold, it is considered that the interactive unit has touched the projection plane, and these pixels are used as the touch points.

[0067] In this embodiment, the third preset threshold is 9. Because a finger has a certain size, it forms a point cloud with a certain number of points. This step can identify the position of the fingertip, that is, the position of the touch point.

[0068] Step S350: Identify control actions.

[0069] The recognition result of the current real-time image is compared with the recognition result of the previous real-time image to determine the control action of the interactive unit.

[0070] Please see Figure 8 In step S350, the specific components include:

[0071] The result of step S340 identifying whether the interactive unit touches the projection screen is used as the input result;

[0072] If the interactive unit does not touch the projection plane, it determines whether a touch mark was set in the previous real-time image. If not, it waits for the recognition result of step S340. If yes, it determines whether the time between the current time point and the time when the touch mark appeared exceeds the set time. If the difference between the current time point and the time when the touch mark appeared exceeds the set time, the touch mark is cleared. Otherwise, the output control action is a single click.

[0073] If the interactive unit touches the projection plane, and it is determined that no touch mark has been set for the previous real-time image, then a touch mark is set for the current real-time image, and the position and time of the touch point are recorded.

[0074] If there are touch markers in the previous real-time image, when it is determined that the touch position of the current touch point has changed relative to the touch position of the previous real-time image with touch markers, the output control action is dragging, and the real-time touch coordinates are output.

[0075] If it is determined that the touch position of the current touch point has not changed relative to the touch position of the previously marked touch mark in the real-time image, then it is further determined whether the touch time point corresponding to the previous touch mark is greater than the set time. If so, the output control action is a long press; otherwise, it remains unchanged.

[0076] In this embodiment, the time is set to 3 seconds.

[0077] This application provides a projector touch recognition system, comprising: an image acquisition and preprocessing module, used to capture real-time images of the projector's projection plane using a TOF sensor, acquire corresponding amplitude and depth images, and then perform image preprocessing on the amplitude and depth images corresponding to the real-time images according to a preset method; a background data acquisition module, used to acquire amplitude image data and 3D point cloud data of the projection plane when the projector is powered on and there is no interactive unit on the projection plane, and then fit the projection plane based on the 3D point cloud data and save it; an interactive unit recognition module, used to determine whether an interactive unit appears on the projection plane. If so, proceed to the next step; otherwise, jump to the image acquisition and preprocessing step; a touch point recognition module, used to determine whether the interactive unit touches the projection plane. If so, identify the position of the touch point of the interactive unit and record the touch time, set a touch flag, and proceed to the next step; otherwise, consider that no touch is made on the projection plane, and jump to the image acquisition and preprocessing step; and a control action recognition module, which compares the recognition result of the current real-time image with the recognition result of the previous real-time image to determine the control action of the interactive unit.

[0078] This application provides a projector touch recognition device. The system includes: a projector for projecting an image; an interaction unit for touching the image to interact with the content of the image; a TOF sensor for capturing the image in real time, processing the obtained amplitude and depth images, and recognizing the interaction action; and a control unit for controlling the projector to respond to the action based on the interaction action.

[0079] The following is combined Figures 1 to 8 The working principle of this application will be explained below.

[0080] The interaction unit in this application is a finger, that is, interaction is achieved through a finger. First, when the projector is powered on and there is no finger on the projection surface, amplitude image data and 3D point cloud data of the projection surface are collected. Then, the projection surface is fitted based on the 3D point cloud data and saved.

[0081] When a user interacts with the content projected by the projector using their finger, the TOF image sensor captures image data of the projector in real time, identifies whether the user's action is dragging, clicking, or long-pressing, and outputs the recognition result to the terminal. The terminal then controls the projector to respond based on the recognition result. For example, Figure 3 In this process, the user taps the brightness mode on the projection plane with their finger. The TOF image sensor recognizes the user's action, and the terminal controls the projector to respond based on the user's action.

[0082] Verification has shown that this invention requires no additional objects (such as a stylus); touch operation is achieved simply by touching the finger, providing a realistic touch experience. It is convenient, simple to use, highly interactive, and low-cost. Furthermore, the detection algorithm uses a traditional approach, eliminating the need for neural network training, resulting in high speed, high accuracy, and a smooth user experience. This algorithm achieves a detection frame rate of 60fps on a PC (CPU: i7-7700, 3.6GHz) and 15fps on a TOF linear array microprocessor (CPU: 32-bit RISC-V CPU, 200MHz), demonstrating significant speed and cost advantages over traditional touch methods.

[0083] The beneficial effects of this application are as follows: by capturing image data projected by the projector in real time using a TOF image sensor and performing image preprocessing, without the need for external objects, when interactive units such as fingers click, drag, or slide on the projection plane, the TOF image sensor can recognize the control action, and the system controls the projector to respond accordingly based on the control action. This application has the characteristics of high speed, high accuracy, low cost, and ease of use.

[0084] Those skilled in the art will understand that all or part of the steps of the various methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk or optical disk, etc.

[0085] The above description, in conjunction with specific embodiments, provides a further detailed explanation of this application and should not be construed as limiting the specific implementation of this application to these descriptions. For those skilled in the art, several simple deductions or substitutions can be made without departing from the inventive concept of this application.

Claims

1. A method for projector touch recognition based on TOF sensor, characterized in that, The method includes: Image acquisition and preprocessing: The TOF sensor captures real-time images of the projection plane of the projector, and obtains corresponding amplitude and depth images. Then, the amplitude and depth images corresponding to the real-time images are preprocessed according to a preset method. Background data acquisition: When the projector is powered on and there is no interactive unit on the projection plane, the amplitude image data and 3D point cloud data of the projection plane are acquired. Then, the projection plane is fitted based on the 3D point cloud data and saved. The method of fitting the projection plane based on the 3D point cloud data adopts the least squares method. Identify interactive units on the projection plane: Determine whether an interactive unit appears on the projection plane. If so, proceed to the next step; otherwise, jump to the image acquisition and preprocessing step. The method for determining whether an interactive unit appears in the projection plane includes: acquiring the amplitude value of each pixel of the real-time image; traversing the pixels of the real-time image, comparing the difference between the amplitude values ​​of each pixel position in the real-time image and the projection plane, and if there are pixels whose difference between the amplitude values ​​is greater than a first set threshold, then it is considered that an interactive unit appears in the projection plane, and the position of these pixels is recorded; otherwise, the process jumps back to the image acquisition and preprocessing steps. Touch point identification: Determine whether the interactive unit touches the projection plane. If so, identify the position of the touch point of the interactive unit and record the touch time, set a touch flag, and continue to the next step. Otherwise, it is considered that the projection plane has not been touched, and jump to the image acquisition and preprocessing step. The step of determining whether the interactive unit touches the projection plane includes: Calculate the distance from the projection plane to the point cloud 3D coordinates (x, y, z) of the pixel whose difference in amplitude values ​​is greater than a first set threshold, which is the interactive unit. Here, the values ​​of A, B, C, and D are known, x, y, and z are the 3D coordinates of the pixel, and d is the distance from the pixel to the projection plane. The specific calculation formula is as follows: Retain pixels whose d is less than the second preset threshold; The number of neighboring pixels that are not on the projection plane is counted. If the number of pixels meets a third preset threshold, the interactive unit is considered to have touched the projection plane, and these pixels are used as the touch points. Recognize control actions: Compare the recognition result of the current real-time image with the recognition result of the previous real-time image to determine the control actions of the interaction unit; The step of identifying the control action includes: If the interactive unit does not touch the projection plane, it continues to determine whether the previous real-time image has the touch mark. If so, it determines whether the current time point and the time when the touch mark appeared before exceed a set time. If so, it clears the touch mark. If not, it outputs a click control action. If the interactive unit touches the projection plane, it continues to determine whether the previous real-time image already has a touch mark. If not, the touch mark is set, and the position and time of the touch point are recorded. If the previous real-time image already had a touch mark, then it is further determined whether the touch position of the current touch point has changed relative to the touch position of the previous real-time image marked with the touch mark. If so, the control action is output as drag, and the real-time touch coordinates are output. If it is determined that the touch position of the current touch point has not changed relative to the touch position of the real-time image marked with the touch mark, then it is further determined whether the touch time point corresponding to the previous touch mark is greater than the set time. If so, the control action is output as a long press; otherwise, the current state is maintained.

2. The identification method as described in claim 1, characterized in that, In the image acquisition and preprocessing steps, the image processing method using the preset method includes: image filtering, distortion correction, and 3D correction, wherein the 3D correction method specifically includes: According to the real-time image pixel coordinates S[ , ] and the coordinates of the TOF sensor [ , , ]relation: We can obtain: in,( , ) is the optical center of the TOF sensor, ( The focal lengths of the TOF sensor along the X and Y axes are respectively. The cosine value of the angle between the TOF sensor coordinates and the real-time image pixel coordinates is used to calculate the depth value of the corrected image by multiplying the original depth value of the real-time image by the cosine value.

3. A projector touch recognition system, characterized in that, The system includes: The image acquisition and preprocessing module is used to capture real-time images of the projection plane of the projector using a TOF sensor, acquire corresponding amplitude and depth images, and then perform image preprocessing on the amplitude and depth images corresponding to the real-time images according to a preset method. The background data acquisition module is used to acquire amplitude image data and 3D point cloud data of the projection plane when the projector is powered on and there is no interactive unit on the projection plane. Then, the projection plane is fitted based on the 3D point cloud data and saved. The method of fitting the projection plane based on the 3D point cloud data adopts the least squares method. The interaction unit identification module is used to determine whether an interaction unit appears in the projection plane. If so, proceed to the next step; otherwise, jump to the image acquisition and preprocessing step. The module for identifying the interaction unit is further configured to: acquire the amplitude value of each pixel in the real-time image; traverse the pixels of the real-time image, compare the difference between the amplitude values ​​of each pixel position in the real-time image and the projection plane, and if there are pixels whose difference between the amplitude values ​​is greater than a first set threshold, then it is considered that an interaction unit has appeared in the projection plane, and the position of these pixels is recorded; otherwise, it jumps back to the image acquisition and preprocessing steps. The touch point identification module is used to determine whether the interactive unit touches the projection plane. If so, it identifies the position of the touch point of the interactive unit and records the touch time, sets a touch flag, and continues to the next step. Otherwise, it is considered that the interactive unit has not touched the projection plane, and jumps to the image acquisition and preprocessing step. The touch point recognition module is further configured to: calculate the distance from the projection plane to the point cloud 3D coordinates (x, y, z) of the pixel whose amplitude difference is greater than a first preset threshold, i.e., the interactive unit, where the values ​​of A, B, C, and D are known, x, y, and z are the 3D coordinates of the pixel, and d is the distance from the pixel to the projection plane. The specific calculation formula is as follows: Retain pixels whose d is less than the second preset threshold; The number of neighboring pixels that are not on the projection plane is counted. If the number of pixels meets a third preset threshold, the interactive unit is considered to have touched the projection plane, and these pixels are used as the touch points. Identify control actions: Compare the identification result of the current real-time image with the identification result of the previous real-time image to determine the control actions of the interaction unit.

4. A projector touch recognition device, characterized in that, The device includes: A projector is used to project images onto a projection surface. An interactive unit is used to touch the image screen to interact with the content of the image screen; A Time-of-Flight (TOF) sensor is used to capture images in real time, process the resulting amplitude and depth images, and recognize interactive actions; and A control unit, which controls the projector to respond to actions based on interactive actions; The TOF sensor includes: microprocessor; A memory storing computer-readable instructions that, when executed by the processor, implement the projector touch recognition method as described in any one of claims 1 to 3; The transmitting module includes a light source driver, an infrared light source, and a diffuser. The receiving module includes a 3D sensor, a filter, and an optical lens.

5. The projector touch recognition device as described in claim 4, characterized in that, The TOF sensor is located on the side of the projector.

Images