Picture self-adaption and touch correction method and projection system
By acquiring and fitting the correction coordinates in the projector, the accurate identification of infrared stylus on the projection screen is achieved, which solves the problem of the lack of human-computer interaction and high-end projector gesture interaction in the mid- and low-end projectors, and simplifies operation and Improved accuracy.
Patent Information
- Application Number
- CN202410937154.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-15
AI Technical Summary
Mid- and low-end projectors lack human-computer interaction functions and are inconvenient to control. High-end projectors realize human-computer interaction through gestures, but learning is difficult, easy to misjudgment, and the control accuracy is reduced.
By obtaining the original coordinates and projection coordinates in the correction picture for fitting operations, obtaining adaptive parameters, realizing accurate recognition of the infrared stylus on the interactive projection screen, generating and executing interactive events, simplifying operations and improving accuracy.
It realizes direct human-computer interaction without memory gestures, is simple to operate, improves control accuracy, and reduces the difficulty of user learning.
Smart Images

Figure CN118963577A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of projection technology, and in particular relates to a screen adaptation and touch correction method and a projection system. Background Art
[0002] With the development of projection technology, the intelligence of projectors is getting higher and higher, and the touch technology in projection has also been developed accordingly. However, some low-end and mid-range projectors lack projection touch technology. When the screen projected on the screen needs to be changed (for example, switching PPT presentations) or the function keys in the screen need to be triggered (for example, clicking the shutdown button in the screen), the user needs to directly operate the computer device, which is cumbersome and the user cannot directly interact with the projection display device. Some high-end projectors can realize human-computer interaction, but the interaction method is relatively simple. For example, various gestures are pre-stored on the projection plane, each gesture corresponds to a different instruction, and then the gesture of the operator is photographed in real time through the camera. Gestures can be used to realize operations such as confirmation, cancellation, and page turning. However, the biggest defect of this projection system is that the number of instructions defined by gestures is limited. When the number of instructions increases, it will inevitably increase the difficulty of learning for users and it is difficult for ordinary users to accept it. On the other hand, if the shapes of gestures are similar, they are easily misjudged, resulting in operation errors, and the accuracy of control is reduced. Summary of the invention
[0003] (I) Purpose of the invention
[0004] In order to overcome the above shortcomings, the purpose of the present invention is to provide a screen adaptation and touch correction method and a projection system to solve the technical problems that the existing mid- and low-end projectors lack human-computer interaction functions and are inconvenient to operate, and high-end projectors need to use gestures to achieve human-computer interaction, which is difficult to learn and is easy to be misjudged and lead to operational errors, and the accuracy of control is reduced.
[0005] (II) Technical solution
[0006] To achieve the above objectives, the technical solutions provided by this application are as follows:
[0007] A method for image adaptation and touch correction, comprising:
[0008] Get the original coordinates of the calibration points in the calibration image;
[0009] After the calibration image is projected to form a calibration projection screen, the projection coordinates of the calibration points on the calibration projection screen are obtained;
[0010] Perform fitting operation on the original coordinates of the obtained calibration points and the projection coordinates, calculate the adaptation parameters and save them;
[0011] After the correction image is switched to the interactive interface and the interactive projection screen is projected, and after the user uses an infrared stylus pen to click on the interactive projection screen to emit a light spot for an input event, the projection coordinates of the input event are identified, the projection coordinates of the input event and the adaptation parameters are reversely calculated to obtain the original coordinates of the input event in the interactive interface, and an execution interactive event is generated;
[0012] The present application obtains the original coordinates of the correction points on the correction image in advance, and then obtains the projection coordinates of these correction points on the correction projection screen, and obtains the adaptation parameters between the correction image and the correction projection screen by fitting the two coordinates. After the correction is completed, you can switch to the interactive interface (the system's default UI interface) to project the interactive projection screen. When the user uses an infrared stylus to click on the interactive projection screen, the system can capture the light spot of the stylus on the interactive projection screen in real time according to the algorithm, and then obtain the original coordinates of the input event in the interactive interface through a reverse algorithm based on the adaptation parameters, and finally generate an InPut event and report it to the system for execution. Through the method of the present application, human-computer interaction can be achieved by simply clicking directly on the interactive projection screen. There is no need to remember various gestures, and the operation is simple. In addition, the calculation by coordinates is highly accurate.
[0013] In some embodiments, after obtaining the adaptation parameter and before saving, the adaptation parameter is further verified;
[0014] By setting the steps for verifying the adaptation parameters, the accuracy of the adaptation parameters can be ensured.
[0015] In some embodiments, there are four correction points and they are symmetrically distributed in pairs at the four corners of the correction image;
[0016] By setting four correction points and distributing them to the four corners of the correction image, the entire correction image can be covered, thereby improving the accuracy of the correction of the entire correction image.
[0017] In some embodiments, performing a fitting operation on the acquired original coordinates and the projected coordinates includes:
[0018] Substitute the original coordinates (x, y, z) of each correction point and its corresponding projection coordinates (x', y', z') into the matrix transformation formula to obtain the adaptation parameters, where:
[0019] The matrix transformation formula is:
[0020]
[0021] The adaptation parameters are:
[0022]
[0023] In some embodiments, the check point is located at a middle position in the calibration image, and checking the adaptation parameters includes:
[0024] The check point is located in the middle of the calibration image. Checking the adaptation parameters includes:
[0025] Get the original coordinates of the checkpoints in the calibrated image;
[0026] After the calibration image is projected to form a calibration projection screen, the projection coordinates of the calibration points on the calibration projection screen are obtained;
[0027] The original coordinates of the checkpoints are calculated with the adaptation parameters to obtain the comparison coordinates of the checkpoints;
[0028] Compare the check point comparison coordinates with the projection coordinates of the check point;
[0029] If the projection coordinates of the checkpoint are the same as the comparison coordinates of the checkpoint, the adaptation parameters are judged to be accurate;
[0030] Otherwise, it is determined that the adaptation parameters are inaccurate.
[0031] In some embodiments, after the correction image is projected to form a correction projection screen and before the projection coordinates of the correction point on the correction projection screen are obtained, it also includes: filtering the correction projection screen except for the infrared band;
[0032] By filtering out the infrared band, only infrared rays can be received later, filtering out other stray light interference and improving the anti-interference performance of the touch screen.
[0033] Another aspect of the present application provides a screen adaptive and touch projection system, comprising:
[0034] The main control device has a built-in interactive system and an adaptive and touch correction method, which can store the correction picture and obtain the original coordinates of the correction points in the correction picture;
[0035] A projection device, electrically connected to the main control device, for projecting a correction image to form a correction projection screen or an interactive interface in a projection interactive system to form an interactive projection screen, wherein the correction projection screen displays correction points;
[0036] A camera device, electrically connected to the main control device, for photographing a correction projection picture and sending the photographed first picture to the main control device so that the main control device can identify the projection coordinates of the correction point on the correction projection picture;
[0037] The infrared stylus pen emits infrared light spots when users click on the interactive projection screen to input input events;
[0038] The camera device also takes a picture of the interactive projection screen and sends the second picture taken to the main control device so that the main control device can identify the projection coordinates of the input event corresponding to the infrared band light spot on the interactive projection screen;
[0039] After receiving the projection coordinates of the correction point sent by the camera device, the main control device performs a fitting operation on the original coordinates of the correction point and the projection coordinates, obtains the adaptation parameters and saves them. When receiving the projection coordinates of the input event sent by the camera device, the projection coordinates of the input event and the adaptation parameters are reversely calculated to obtain the original coordinates of the input event in the interactive interface, and generate an execution interactive event.
[0040] In some embodiments, it also includes: a mode switching device arranged before the camera device and capable of switching to an infrared filtering mode. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is a hardware principle block diagram of the optimal projection system of the present invention;
[0042] Figure 2 It is to correct the projection picture in the picture adaptation and touch correction method of the present invention;
[0043] Figure 3 It is a corresponding diagram of coordinates between the correction image and the correction projection image in the image adaptation and touch correction method of the present invention;
[0044] Figure 4 It is a flow chart of the screen adaptation and touch correction method of the present invention. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.
[0046] The present invention provides a method for image adaptation and touch correction, comprising:
[0047] Get the original coordinates of the calibration points in the calibration image;
[0048] After the calibration image is projected to form a calibration projection screen, the projection coordinates of the calibration points on the calibration projection screen are obtained;
[0049] Perform fitting operation on the original coordinates of the obtained calibration points and the projection coordinates, calculate the adaptation parameters and save them;
[0050] After the correction image is switched to an interactive interface and an interactive projection screen is projected, when the user uses an infrared stylus pen to click on the interactive projection screen to input an input event, the light spot emitted by the tip of the infrared stylus pen will be captured by the system in real time, and the projection coordinates of the click input event will be identified through an algorithm.
[0051] The projection coordinates of the input event and the adaptation parameters are reversely calculated to obtain the original coordinates of the input event in the interactive interface, and generate an execution interactive event.
[0052] Specifically, the present application has two display screens, one is a calibration image, and the other is an interactive interface. The calibration image is projected to form a calibration projection image, and the interactive interface is projected to form an interactive projection image. The calibration points are marked on the calibration image, which are stored in advance and their pixel sizes are known. The coordinates of the calibration points on the calibration image are called the original coordinates of the calibration points, and the coordinates of the calibration points on the calibration projection image are the projection coordinates of the calibration points. Similarly, the pixel size of the interactive interface is also known in advance. The position coordinates of the input event clicked by the user on the interactive projection image are the projection coordinates of the input event, and the coordinates of the projection coordinates of the input event mapped to the interactive interface are the original coordinates of the input event.
[0053] Specifically, the touch correction method of the present application is applied to a projection system, and the projection system includes:
[0054] The main control device is equipped with a deeply optimized Android smart system, with built-in adaptive and touch correction methods, which can realize touch correction and touch interaction of the system Ui. The main control device also stores a correction picture with marked correction points, and the main control device can obtain the original coordinates of the correction points in the correction picture in advance;
[0055] The projection device is electrically connected to the main control device and is used for displaying the system screen. When performing touch calibration, the calibration image can be displayed to form a calibration projection screen. Specifically, the calibration point is displayed on the calibration projection screen.
[0056] A camera device, electrically connected to the main control device, for photographing a correction projection picture and sending the photographed first picture to the main control device so that the main control device can identify the projection coordinates of the correction point on the correction projection picture;
[0057] The infrared stylus pen allows users to click on the interactive projection screen to input events and emit infrared light spots.
[0058] Specifically, the projection device is an optical machine, the main control device is a smart device equipped with an Android smart system, which has built-in CPU, RAM, EMMC storage and a driving circuit for driving the optical machine, and the camera device is a camera.
[0059] Specifically, the main control device is equipped with an Android intelligent system, and a calibration picture is stored in the main control device. When the projection system is installed for the first time, the calibration picture is opened after the system is powered on to project the calibration picture to form a calibration projection screen. The camera device automatically shoots the calibration projection screen for automatic touch calibration. After the automatic touch calibration is completed, the calibration picture can be closed and switched to the interactive interface. The adaptation parameters are automatically saved in the main control device. The stored adaptation parameters can be directly applied the next time the system is turned on without the need for re-calibration.
[0060] Specifically, the principle of obtaining the original coordinates of the correction points in the correction image is as follows:
[0061] The calibration image is pre-stored in the main control device. The size of the calibration image is fixed and known. The calibration image is composed of multiple pixel points, so the original coordinates of the calibration points on the calibration image are known. For example, there are four calibration points. The calibration point in the upper left corner is 100 pixels away from the left edge of the calibration image and 100 pixels away from the upper edge. The original coordinates of the calibration point are (100, 100, 1). The original coordinates of other calibration points can be obtained in turn. The z-axis data defaults to 1.
[0062] Specifically, during the calibration process, the camera device takes a first photo of the calibration projection screen and transmits it to the main control device. The main control device can identify the position of the calibration point on the first photo based on the first photo, and then identify the projection coordinates of the calibration point on the calibration projection screen, and fit the original coordinates of the calibration point with the projection coordinates of the calibration point to obtain the adaptation parameters and save them.
[0063] Specifically, during the interaction process, the camera device captures the interactive projection screen in real time, and sends a second photo of the captured interactive projection screen to the main control device. The main control device identifies the position of the infrared band light spot on the second photo based on the second photo, and then identifies the projection coordinates of the input event on the interactive projection screen. The projection coordinates of the input event and the adaptation parameters are reversely calculated to calculate the original coordinates of the input event on the interactive interface. An InPut event is generated based on the original coordinates of the input event on the interactive interface, and is reported to the Android system in real time to execute this interactive event.
[0064] Preferably, the number of the correction points is four, and they are symmetrically distributed in pairs at the four corners of the correction image.
[0065] Specifically, the fitting calculation process of the four correction points is introduced in detail below:
[0066] The original coordinates of the four calibration points:
[0067] (x 1 ,y 1 , z 1 )
[0068] (x 2 ,y 2 , z 2 )
[0069] (x 3 ,y 3 , z 3 )
[0070] (x 4 ,y 4 , z 4 )
[0071] Projected coordinates of the four calibration points:
[0072] (x 1 ',y 1 ', z 1 ')
[0073] (x 2 ',y 2 ', z 2 ')
[0074] (x 3 ',y 3 ', z 3 ')
[0075] (x 4 ',y 4 ', z 4 ')
[0076] The original coordinates of each calibration point and its corresponding projection coordinates are grouped into four groups;
[0077] Substitute the four sets of coordinates of the correction points into the matrix transformation formula respectively:
[0078]
[0079] Eight equations can be obtained:
[0080] x 0 =p 31
[0081] y 0 =p 32
[0082] p 13 x 1 +x 1 =p 11 +p 31
[0083] p 13 y 1 +y 1 =p12 +p 32
[0084] p 13 x 2 +p 23 x 2 +x 2 =p 11 +p 21 +p 31
[0085] p 13 y 2 +p 23 y 2 +y 2 =p 12 +p 22 +p 32
[0086] p 23 x 3 +x 3 =p 21 +p 31
[0087] p 23 y 3 +y 3 =p 22 +p 32
[0088] Since x in the above eight equations 0 ~x 4 is known, y 0 ~y 4 is also known, so the coordinate transformation matrix can be obtained:
[0089]
[0090] Specifically, the coordinate transformation matrix is the adaptation parameter of this application, which mainly processes the x- and y-axis data, and the z-axis data defaults to 1.
[0091] Preferably, the present application also sets a check point in the middle of the correction image, so as to check the obtained adaptation parameters;
[0092] Specifically, the verification process is as follows:
[0093] First, get the original coordinates of the checkpoint (x 0 ,y 0 , z 0 ) and the projection coordinates (x 0 ',y 0 ', z 0 ');
[0094] The original coordinates of the checkpoint (x 0 ,y 0 , z 0 ) and adaptation parameters
[0095]
[0096] Multiply to find the coordinates of the checkpoint (x 0 1 ',y 0 1 ', z 0 1 ');
[0097] Compare this checkpoint to the coordinates (x 0 1 ',y 0 1 ', z 0 1 ') and the projection coordinates of the checkpoint (x 0 ',y 0 ', z 0 ') for comparison. If the error values of the two coordinates are judged to be within the set range, it can be considered that the variable adaptation parameters are accurate, that is, the test is successful.
[0098] Specifically, after the touch calibration is completed, when the user clicks the input event on the interactive projection screen to obtain the projection coordinates (x', y', z') of the input event, the projection coordinates (x', y', z') of the input event are divided by the adaptation parameter
[0099]
[0100] The original coordinates (x, y, z) of the input event on the interactive interface are obtained, and then an execution event is generated according to the original coordinates (x, y, z), which is reported to the system in real time, and the system executes the touch operation.
[0101] Preferably, the camera device is equipped with a control mode switching device that can control the light of different optical bands entering the camera device.
[0102] See also Figure 4Specifically, when the projection system is powered on and started, the projection image emitted by the projection device is in color, and the control mode switching device switches to the color mode. At this time, the projection device can perform adaptive adjustments such as automatic focus of the projection image, automatic trapezoidal correction of the projection image, and automatic obstacle avoidance of the projection image based on the built-in algorithm. These adaptive adjustments are conventional adjustment steps for projectors on the market. After the adaptive image adjustment is completed, the projection image presents a relatively clear and square projection image. After completing the above-mentioned adaptive adjustment, the step of automatic touch correction is performed. After completing the automatic touch correction, the control mode is controlled. The switching device switches to the infrared filter mode, which filters the visible light band, retains a specific infrared band, and reduces the interference of visible light. After completing the above steps, the camera device will detect and capture in real time whether there is an infrared band light spot emitted by the stylus in the interactive projection screen area; when the user uses the infrared stylus to click on the interactive projection screen, an infrared band light spot will be generated and photographed by the camera device. After the camera device photographs the light spot, it sends the photo to the main control device. The main control device analyzes the effectiveness of the light spot through the light spot judgment algorithm. If the light spot is judged to be valid, it will be ignored. If the light spot is judged to be invalid, it will be ignored.
[0103] Specifically, the method for judging the effectiveness of the light spot is:
[0104] Use the contour detection function in OpenCV to detect the contact contours in the captured image. Then calculate the area of each contour, and compare the area with the upper and lower limits of the set effective area. Those above the upper limit or below the lower limit are directly excluded, and those within the range can be judged as valid.
[0105] Another aspect of the present application provides a screen adaptive and touch projection system, comprising:
[0106] The main control device has a built-in interactive system and an adaptive and touch correction method, which can store the correction picture and obtain the original coordinates of the correction points in the correction picture;
[0107] A projection device, electrically connected to the main control device, for projecting a correction image to form a correction projection screen or an interactive interface in a projection interactive system to form an interactive projection screen, wherein the correction projection screen displays correction points;
[0108] A camera device, electrically connected to the main control device, for photographing a correction projection picture and sending the photographed first picture to the main control device so that the main control device can identify the projection coordinates of the correction point on the correction projection picture;
[0109] The infrared stylus pen emits infrared light spots when users click on the interactive projection screen to input input events;
[0110] The camera device also takes a picture of the interactive projection screen and sends the second picture taken to the main control device so that the main control device can identify the projection coordinates of the input event corresponding to the infrared band light spot on the interactive projection screen;
[0111] After receiving the projection coordinates of the correction point sent by the camera device, the main control device performs a fitting operation on the original coordinates of the correction point and the projection coordinates, obtains the adaptation parameters and saves them. When receiving the projection coordinates of the input event sent by the camera device, the main control device performs a reverse operation on the projection coordinates of the input event and the adaptation parameters, obtains the original coordinates of the input event in the interactive interface, generates an InPut event, and reports it to the Android system to execute this interactive event.
[0112] Preferably, a mode switching device capable of switching to an infrared filtering mode is also provided before the camera device, and the mode switching device is electrically connected to the main control device. Specifically, a customized narrow-band filter is used in the mode switching device, and by moving the narrow-band filter to a position aligned with the projection device, the visible light band can be filtered, retaining a specific infrared band, and reducing interference from visible light.
[0113] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.
Claims
1. A method for image adaptation and touch correction, characterized in that: include: Get the original coordinates of the calibration points in the calibration image; After the calibration image is projected to form a calibration projection screen, obtaining the projection coordinates of the calibration point on the calibration projection screen; Performing a fitting operation on the original coordinates of the calibration point obtained and the projection coordinates, obtaining the adaptation parameters and saving them; After the correction image is switched to an interactive interface and an interactive projection screen is projected, and after the user uses an infrared stylus to click on the interactive projection screen to emit a light spot as an input event, the projection coordinates of the input event are identified, the projection coordinates of the input event and the adaptation parameters are reversely calculated to obtain the original coordinates of the input event in the interactive interface, and generate an execution interactive event.
2. The screen adaptation and touch correction method according to claim 1, characterized in that: After determining the adaptation parameters and before saving them, the method further includes verifying the adaptation parameters.
3. The screen adaptation and touch correction method according to claim 1 or 2, characterized in that: There are four correction points, which are symmetrically distributed in pairs at the four corners of the correction image.
4. The screen adaptation and touch correction method according to claim 3, characterized in that: The fitting operation of the original coordinates of the obtained calibration points with the projection coordinates includes: fitting the original coordinates (x, y, z) of each calibration point with the corresponding projection coordinates (x ’ ,y ’ , z ’ ) are substituted into the matrix transformation formula to obtain the adaptation parameters, where: The matrix transformation formula is: The adaptation parameters are:
5. The screen adaptation and touch correction method according to claim 2, characterized in that: A check point is set at the middle position of the correction image, and the checking of the adaptation parameters includes: Obtaining the original coordinates of the check points in the calibration image; After the calibration image is projected to form a calibration projection picture, obtaining the projection coordinates of the verification point on the calibration projection picture; The obtained original coordinates of the checkpoint are calculated with the adaptation parameters to obtain the checkpoint comparison coordinates; Comparing the comparison coordinates of the check point with the projection coordinates of the check point; If the projection coordinates of the check point are the same as the check point comparison coordinates, then the adaptation parameters are determined to be accurate; Otherwise, it is determined that the adaptation parameter is inaccurate.
6. The screen adaptation and touch correction method according to any one of claims 1-2, 4-5, characterized in that: After the correction image is projected to form a correction projection screen and before the projection coordinates of the correction point on the correction projection screen are obtained, the method further includes: filtering the correction projection screen except for the infrared band.
7. A screen adaptive and touch projection system, characterized in that: include: The main control device has a built-in interactive system and an adaptive and touch correction method, which can store the correction picture and obtain the original coordinates of the correction points in the correction picture; A projection device, electrically connected to the main control device, for projecting the calibration image to form a calibration projection screen or an interactive interface in a projection interactive system to form an interactive projection screen, wherein the calibration point is displayed on the calibration projection screen; A camera device, electrically connected to the main control device, configured to capture the correction projection image to form a first photo and send the first photo to the main control device so that the main control device can identify the projection coordinates of the correction point on the correction projection image; An infrared stylus pen, for the user to click on the interactive projection screen to input an input event and emit an infrared band light spot; The camera device further captures the interactive projection screen to form a second photo and sends the second photo to the main control device so that the main control device can identify the projection coordinates of the input event on the interactive projection screen; After receiving the projection coordinates of the correction point sent by the camera device, the main control device performs a fitting operation on the original coordinates of the correction point and the projection coordinates, obtains and saves the adaptation parameters, and when receiving the projection coordinates of the input event sent by the camera device, performs a reverse operation on the projection coordinates of the input event and the adaptation parameters, obtains the original coordinates of the input event in the interactive interface, and generates an interactive event for execution.
8. The image adaptive and touch projection system according to claim 7, characterized in that: Also includes: A mode switching device capable of switching to an infrared filter mode is arranged before the camera device.
