A projection method, apparatus and system

Abstract

The application relates to a projection method, device and system, and relates to the technical field of projection. The method is applied to a processor in a projection system, the projection system further comprising a light machine, a detection device and a projection screen, the light machine being used for projecting a projection image onto the projection screen, and the detection device being used for detecting a picture on the projection screen to obtain a detection image. The method comprises the following steps: in each detection period, obtaining a cursor coordinate of a cursor in the projection image, generating the cursor in the projection image, and obtaining a cursor coordinate of the cursor in the detection image. According to a plurality of cursor coordinates of a plurality of projection images and a plurality of cursor coordinates of a plurality of detection images, a target mapping relationship between the cursor coordinate of the projection image and the cursor coordinate of the detection image is obtained. Thus, the problem that the mapping relationship between an image captured by a camera and the projection image is not considered in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of projection technology, and in particular to a projection method, apparatus and system. Background Technology

[0002] Projectors have become increasingly popular in recent years. Their high flexibility and large viewing screen size have made them the preferred choice for many families seeking a superior viewing experience. Projectors project images onto a screen, and users interact with them using a remote control.

[0003] Users can move the cursor step by step using the up, down, left, or right buttons on a traditional remote control. However, this requires multiple button presses when selecting playback resources or entering text, which is inconvenient.

[0004] Point-and-shoot remote control is a novel way to interact with projectors. Users can project a light spot onto the projection screen using a remote control with point-and-shoot functionality. The projector detects the position of the light spot and displays a cursor at that location. This allows for precise pointing and selection of the target without having to manually move the cursor step by step using buttons, offering high efficiency and convenience – what you point is what you get. Furthermore, remote controls with point-and-shoot functionality can perform functions that traditional remote controls cannot, such as writing, drawing, and annotation. Some smart projectors have a built-in camera that can be used to detect the light spot. However, this method does not consider the mapping relationship between the image captured by the camera and the projected image, meaning it does not account for the difference in viewing angle between the optical engine and the camera. Summary of the Invention

[0005] This application provides a projection method, apparatus, and system that solves the problem in the prior art that does not consider the mapping relationship between images captured by a camera and projected images.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, a projection method is provided. This method is applied to a processor in a projection system, which also includes an optical engine, a detection device, and a projection screen. The optical engine projects a projection image onto the projection screen, and the detection device detects the image on the projection screen to obtain a detection image. The method includes: acquiring the cursor coordinates of a cursor in the projection image at each detection time interval; generating a cursor in the projection image; and acquiring the cursor coordinates of the cursor in the detection image. Based on multiple cursor coordinates from multiple projection images and multiple cursor coordinates from multiple detection images, a target mapping relationship is obtained between the cursor coordinates of the projection images and the cursor coordinates of the detection images.

[0008] In the above technical solution, the mapping relationship between the projected image of the optical engine and the detected image of the detection device can be calculated by using the cursor coordinates of the projected image and the cursor coordinates of the detected image. That is, the viewing angle difference between the optical engine and the detection device is considered. Furthermore, the cursor is typically present in the projected image, eliminating the need for a projection calibration area and preventing interruption of the user's viewing experience. Users can also move the optical engine or detection device as needed without worrying about creating a calibration area that would interrupt their view, offering greater flexibility. Moreover, the optical engine or detection device does not need to be placed in a calibration position, there is no need to restrict the optical engine and detection device to share a common optical path, and there is no need to set up an array of light sensors on the projection screen. This results in lower projection system costs and more flexible placement of the projection system. Additionally, the cursor position is more obvious relative to boundary points, eliminating the need for the user to manually project light spots onto boundary points. This improves the accuracy of calculating the target mapping relationship, enhances the user experience, and reduces the difficulty of user operation.

[0009] In one possible implementation of the first aspect, the projection system further includes a remote control device for projecting a light spot onto the projection screen. The method also includes: acquiring the light spot coordinates in the detection image at each detection period; and, after acquiring the target mapping relationship, acquiring the cursor coordinates in the projection image, including: acquiring the cursor coordinates in the projection image based on the light spot coordinates in the detection image and the target mapping relationship. In the above possible implementations, when a user projects a light spot onto the projection screen using the remote control device, the projection system can accurately display the cursor at the light spot location on the projection screen according to the detected light spot and the target mapping relationship. This improves the user experience and avoids operational errors caused by inaccurate cursor display.

[0010] In one possible implementation of the first aspect, before obtaining the target mapping relationship, the cursor coordinates on the projected image are obtained, including: obtaining the cursor coordinates on the projected image based on the spot coordinates of the detected image and the preset mapping relationship. In the above possible implementation, a cursor can be generated according to the preset mapping relationship before obtaining the target mapping relationship, so that the user can operate the projected image by pointing at it with the remote control. This can save the user's time and improve the user experience.

[0011] In one possible implementation of the first aspect, before obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is a first distance. After obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is a second distance. The first distance is greater than the second distance. In the above possible implementation, before and after obtaining the target mapping relationship, the user can see that the distance between the light spot and the cursor on the projection screen becomes smaller, resulting in a better user experience.

[0012] In one possible implementation of the first aspect, obtaining the target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detection images based on the multiple cursor coordinates of the multiple projected images and the multiple cursor coordinates of the multiple detection images includes: determining the correspondence between the multiple cursors of the multiple projected images and the multiple cursors of the multiple detection images based on the similarity between a first curve formed by the multiple cursor coordinates of the multiple projected images and a second curve formed by the multiple cursor coordinates of the multiple detection images. The target mapping relationship is obtained based on the multiple cursor coordinates of the multiple projected images, the multiple cursor coordinates of the multiple detection images, and the correspondence. Exemplarily, the similarity includes at least one of the following: cosine similarity, Euclidean distance-based similarity, and curve slope-based similarity. In the above possible implementation, determining the correspondence between the multiple projected images and the multiple detection images through curve similarity avoids using incorrect data for calculation and can improve the accuracy of the target mapping relationship.

[0013] In one possible implementation of the first aspect, the target mapping relationship between the cursor coordinates of the projection images and the cursor coordinates of the detection images is obtained based on the multiple cursor coordinates of multiple projection images and the multiple cursor coordinates of multiple detection images. This includes: determining the correspondence between the multiple cursors in the multiple projection images and the multiple cursor coordinates in the multiple detection images based on the acquisition time of the multiple cursor coordinates in the multiple projection images, the acquisition time of the multiple cursor coordinates in the multiple detection images, and the time difference. The target mapping relationship is obtained based on the multiple cursor coordinates in the multiple projection images, the multiple cursor coordinates in the multiple detection images, and the correspondence. Here, the time difference is the time difference between the acquisition time of the cursor coordinates in the detection images and the acquisition time of the cursor coordinates in the corresponding projection images. In the above possible implementation, determining the correspondence between the multiple projection images and the multiple detection images through the time difference avoids using incorrect data for calculation and can improve the accuracy of the target mapping relationship.

[0014] In one possible implementation of the first aspect, obtaining a target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detected images based on multiple cursor coordinates of multiple projected images and multiple cursor coordinates of multiple detected images includes: receiving an indication signal; and, in response to the indication signal, obtaining the target mapping relationship based on the multiple cursor coordinates of the multiple projected images and the multiple cursor coordinates of the multiple detected images. In the above possible implementation, the processor can obtain the target mapping relationship under external indication, eliminating the need for processor computation and reducing processor computational overhead.

[0015] In one possible implementation of the first aspect, the indication signal is used to indicate a change in the relative position of the optical engine and the projection screen, or to indicate a change in the relative position of the detection device and the projection screen, or to indicate the activation of the optical engine and / or the detection device. In the above possible implementations, when any of the optical engine, detection device, and projection screen moves, or when the optical engine or detection device is activated, the indication signal can prompt the processor to acquire the target mapping relationship. Users can move or activate these devices as needed without worrying about affecting the user experience of the remote control operation.

[0016] Secondly, a projection device is provided, which is applied to a projection system. The projection system further includes an optical engine, a detection device, and a projection screen. The optical engine projects a projection image onto the projection screen, and the detection device detects the image on the projection screen to obtain a detection image. The projection device includes an acquisition unit and a calculation unit. The acquisition unit is used to acquire the cursor coordinates of a cursor in the projection image at each detection time period, generate a cursor in the projection image, and acquire the cursor coordinates of the cursor in the detection image. The calculation unit is used to acquire the target mapping relationship between the cursor coordinates of the projection images and the cursor coordinates of the detection images based on the multiple cursor coordinates of multiple projection images and the multiple cursor coordinates of multiple detection images.

[0017] In one possible implementation of the second aspect, the projection system further includes a remote control device for projecting a light spot onto the projection screen. The acquisition unit is further configured to acquire the light spot coordinates in the detection image at each detection period. After acquiring the target mapping relationship, the acquisition unit is specifically configured to acquire the cursor coordinates in the projected image based on the light spot coordinates in the detection image and the target mapping relationship.

[0018] In one possible implementation of the second aspect, before obtaining the target mapping relationship, an acquisition unit is specifically used to obtain the cursor coordinates of the cursor in the projected image based on the spot coordinates of the detected image and the preset mapping relationship.

[0019] In one possible implementation of the second aspect, before obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is a first distance. After obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is a second distance. The first distance is greater than the second distance.

[0020] In one possible implementation of the second aspect, the calculation unit is specifically configured to determine the correspondence between multiple cursors in multiple projected images and multiple cursors in multiple detected images based on the similarity between a first curve formed by multiple cursor coordinates in multiple projected images and a second curve formed by multiple cursor coordinates in multiple detected images. The calculation unit is further configured to obtain a target mapping relationship based on the multiple cursor coordinates in multiple projected images, the multiple cursor coordinates in multiple detected images, and the correspondence. Exemplarily, the similarity includes at least one of the following: cosine similarity, Euclidean distance-based similarity, and curve slope-based similarity.

[0021] In one possible implementation of the second aspect, the calculation unit is specifically used to determine the correspondence between multiple cursors in multiple projected images and multiple cursors in multiple detection images based on the acquisition times of multiple cursor coordinates in multiple projected images, the acquisition times of multiple cursor coordinates in multiple detection images, and the time difference. The calculation unit is further used to obtain a target mapping relationship based on the multiple cursor coordinates in multiple projected images, the multiple cursor coordinates in multiple detection images, and the correspondence. Here, the time difference is the time difference between the acquisition time of the cursor coordinates in the detection image and the acquisition time of the cursor coordinates in the corresponding projected image.

[0022] In one possible implementation of the second aspect, the computing unit is further configured to receive an indication signal. Specifically, the computing unit is configured to, in response to the indication signal, obtain a target mapping relationship based on multiple cursor coordinates of multiple projected images and multiple cursor coordinates of multiple detected images.

[0023] In one possible implementation of the second aspect, the indication signal is used to indicate a change in the relative position of the optical engine and the projection screen, or the indication signal is used to indicate a change in the relative position of the detection device and the projection screen, or the indication signal is used to indicate the start of the optical engine and / or the detection device.

[0024] Thirdly, a projection device is provided, the projection device including a processor and a memory, the memory storing instructions, the processor being used to invoke the instructions in the memory to execute the method provided by the first aspect or any possible implementation thereof.

[0025] In one possible implementation of the third aspect, the projection device further includes an optical engine and / or a detection device. The optical engine is used to project the projected image onto the projection screen. The detection device is used to detect the image on the projection screen to obtain a detection image.

[0026] Fourthly, a projection system is provided, the projection system including a remote control device and a projection device provided in the third aspect or any possible implementation thereof, the remote control device being used to project a light spot onto a projection screen.

[0027] In one possible implementation of the fourth aspect, the projection system also includes a projection screen.

[0028] Fifthly, a computer-readable storage medium is provided, wherein program code is stored therein, and the program code can be invoked by a processor to execute the method provided by the first aspect or any possible implementation thereof.

[0029] Sixthly, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to perform the method provided by the first aspect or any possible implementation thereof.

[0030] Understandably, the apparatus, system, computer storage medium, or computer program product of any of the projection methods provided above are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0031] Figure 1 A schematic diagram of a projection system provided in this application embodiment. Figure 1 ;

[0032] Figure 2 A schematic diagram of a cursor and a light spot provided for an embodiment of this application;

[0033] Figure 3 A schematic diagram of a projection system provided in this application embodiment. Figure 2 ;

[0034] Figure 4 A schematic diagram of a projector provided in this application embodiment. Figure 1 ;

[0035] Figure 5 A schematic diagram of a projector provided in this application embodiment. Figure 2 ;

[0036] Figure 6 A schematic diagram of a projection method provided in an embodiment of this application. Figure 1 ;

[0037] Figure 7 A schematic diagram of a projection method provided in an embodiment of this application. Figure 2 ;

[0038] Figure 8 A schematic diagram of a projection method provided in an embodiment of this application. Figure 3 ;

[0039] Figure 9A schematic diagram of a projection method provided in an embodiment of this application. Figure 4 ;

[0040] Figure 10 A schematic diagram of a projection image and a detection image provided in an embodiment of this application. Figure 1 ;

[0041] Figure 11 A schematic diagram of a projection image and a detection image provided in an embodiment of this application. Figure 2 ;

[0042] Figure 12 A schematic diagram of a projection image and a detection image provided in an embodiment of this application. Figure 3 ;

[0043] Figure 13 A schematic diagram of a projection method provided in an embodiment of this application. Figure 5 ;

[0044] Figure 14 A schematic diagram of a projection image and a detection image provided in an embodiment of this application. Figure 4 ;

[0045] Figure 15 A schematic diagram of a projection method provided in an embodiment of this application. Figure 6 ;

[0046] Figure 16 A schematic diagram of a projection device provided in an embodiment of this application. Figure 1 ;

[0047] Figure 17 A schematic diagram of a projection device provided in an embodiment of this application. Figure 2 ;

[0048] Figure 18 A schematic diagram of a projection device provided in an embodiment of this application. Figure 3 . Detailed Implementation

[0049] It should be noted that the terms "first" and "second" used in the embodiments of this application are only used to distinguish features of the same type and should not be construed as indicating relative importance, quantity, order, etc.

[0050] The terms "exemplary" or "for example" used in the embodiments of this application are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0051] The terms "coupling" and "connection" used in the embodiments of this application should be interpreted broadly. For example, they can refer to a physical direct connection or an indirect connection achieved through electronic devices, such as a connection achieved through resistors, inductors, capacitors or other electronic devices.

[0052] In the various embodiments of this application, the sequence number of each process does not imply the order of execution or the degree of importance. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0053] The embodiments of this application can be applied to a projection system, which can be used to implement projection functions. The specific structure of this projection system is illustrated below.

[0054] In some possible implementations, such as Figure 1 As shown, the projection system 1000 may include a projector 100, a projection screen 200, and a remote control 300. The projector 100 projects images onto the projection screen 200. The projected image refers to an image generated by the projector 100 that needs to be projected onto the projection screen 200 for display. The projection screen 200 can also be called a projection surface, and it can be used to display the projected image. Some projection screens 200 can provide clear, vibrant, and high-contrast image effects. In this embodiment, the projection screen 200 may be a specific screen made specifically for projection (e.g., a screen), or it may be an object whose surface can be used to display the projected image (e.g., a wall). The remote control 300 may be called a pointer, and it can be any of the following device types: remote control, mouse, smartphone, tablet computer, etc. The remote control 300 can also be other devices, and this embodiment does not limit this.

[0055] The projected image may include a cursor, which can be used to indicate the current operation position or selection state. In some cases, such as when the user is not operating on the projected image, the cursor may be hidden in the projected image. This application embodiment does not limit every frame of the image projected by the projector 100 to include a cursor. The projection system 1000 described in this application embodiment may have a pointing remote control function, and the remote control device 300 may be equipped with a light source, such as a visible light source, an invisible light source, etc. Figure 2 As shown, the user can project a light spot onto the projection screen 200 using the light source of the remote control device 300 (e.g., Figure 2 The projector 100 detects the position of the light spot (a dotted circular line) and displays a cursor (e.g., a dotted circular line) at the position of the light spot. Figure 2 The arrow controls the cursor's movement.

[0056] Point-and-control technology requires displaying a cursor at the location of the light spot; therefore, obtaining the location of the light spot is one of the key aspects of point-and-control technology. Several point-and-control methods will be illustrated below with examples.

[0057] In the first type of remote control pointing method, the position of the light spot can be obtained through the light sensor of the projection screen 200.

[0058] For example, such as Figure 3 As shown, the projection screen 200 includes a screen ( Figure 3 (Not shown) and multiple light sensors 210, which can be arranged in an array behind the screen. When the projector 100 projects a projected image onto the screen, at least one of the multiple light sensors 210 can determine the area information of the projected image based on the light intensity range. The projector 100 can establish a mapping relationship between the screen coordinate system and the projected image coordinate system based on this area information. When the remote control device 300 projects a light spot onto the screen, at least one of the multiple light sensors 210 can determine the coordinates of the light spot on the screen based on the light intensity range. The projector 100 can determine the coordinates of the light spot in the projected image based on the mapping relationship between the screen coordinate system and the projected image coordinate system, as well as the coordinates of the light spot on the screen. Thus, the projector 100 can display a cursor at the coordinates of the light spot on the projected image and perform corresponding operations based on the position of the cursor.

[0059] In this method, the projection screen 200 needs to be equipped with multiple light sensors 210, which is costly. Furthermore, the projector 100 can only project onto the projection screen 200 that has light sensors 210 installed on the back, and cannot project onto a separate screen or wall, which reduces its flexibility.

[0060] The projection system 1000 can use a detection device (such as a camera) instead of the light sensor 210 provided in the projection screen 200. Figure 4 As shown, Figure 1 The projector 100 may include a processor 110, an optical engine 120, and a detection device 130, both of which are coupled to the processor 110. Figure 5As shown, the optical engine 120 is used to project a projected image onto the projection screen 200. The detection device 130 is used to detect the image on the projection screen 200 to obtain a detection image. In this embodiment, detection can refer to capturing or acquiring an image; for example, the detection device 130 is a camera, and detection can refer to the process of the camera capturing an image of the projection screen 200. For example, the detection device 130 can be a camera or other image sensor. The orientation of the detection device 130 and the orientation of the optical engine 120 need to be coordinated so that the optical engine 120 can project onto the projection screen 200, and the detection device 130 can detect the image on the projection screen 200.

[0061] In the second pointing remote control method, when the remote control device 300 projects a light spot onto the projection screen 200, the detection device 130 can detect a detection image including the light spot and transmit the detection image to the processor 110. The processor 110 calculates the coordinates of the light spot in the detection image, uses the coordinates of the light spot in the detection image as the coordinates of the cursor corresponding to the light spot in the projection image, generates a projection image including the cursor, and sends it to the optical engine 120. The optical engine 120 projects the projection image onto the projection screen 200.

[0062] For example, such as Figure 6 As shown, the coordinates of the light spot detected by the detection device 130 in the detection image are (x, y). If the optical engine 120 wants to display a cursor at the position of the light spot on the projection screen 200, it should display the cursor at the coordinates (u, v) of the projected image. Due to the difference in viewing angle between the optical engine 120 and the detection device 130, there is a difference between the coordinate system of the detection image captured by the detection device 130 and the coordinate system of the projected image projected by the optical engine 120; (x, y) and (u, v) are different. If the cursor is displayed at the coordinates (x, y) in the projected image, then the position of the cursor on the projection screen 200 is different from the position of the light spot on the projection screen 200, resulting in an incorrect cursor position.

[0063] In this approach, the mapping relationship between the image captured by the detection device 130 and the projected image is not considered; that is, the viewing angle difference between the optical engine 120 and the detection device 130 is not taken into account. This may cause significant positional deviations of the light spot and cursor on the projection screen 200, leading to operational errors or a poor user experience.

[0064] In the third type of remote control pointing, the projector 100 obtains the mapping relationship H between the coordinate system of the detection image captured by the detection device 130 and the coordinate system of the projection image projected by the optical engine 120 in advance, thereby calibrating the coordinates of the cursor.

[0065] For example,

[0066] For example, such as Figure 7 As shown, processor 110 performs system calibration between optomechanical system 120 and detection device 130 to obtain mapping relationship H. Processor 110 performs feature extraction on the detection image sent by detection device 130 to obtain the coordinates of the light spot in the detection image. Based on mapping relationship H and the coordinates of the light spot in the detection image, processor 110 performs coordinate mapping to obtain the coordinates of the cursor in the projected image. Based on the program response, processor 110 generates a projected image including the cursor and sends it to optomechanical system 120.

[0067] according to Figure 6 It can be seen that the mapping relationship H changes with the relative position of the projector 100 and the projection screen 200.

[0068] In some examples, the projector 100 and the projection screen 200 are placed in calibrated positions, and the mapping relationship H is obtained through these calibrated positions. However, the projector 100 and the projection screen 200 must be placed in the calibrated positions, which reduces flexibility.

[0069] In other examples, processor 110 can generate a projected image with a calibration region and send it to optical engine 120. Optical engine 120 can project this image onto projection screen 200, and detection device 130 can detect the detection image with the calibration region. Processor 110 calculates the mapping relationship H based on the coordinates of the calibration region in the detection image and the coordinates of the calibration region in the projected image. However, displaying the calibration region interrupts the user's viewing, resulting in a poor user experience. To reduce the number of times the calibration region is displayed, the projector 100 and projection screen 200 should be kept as still as possible, which reduces flexibility.

[0070] In some other examples, the optical engine 120 and the camera share the same optical path, thereby eliminating the problem of viewing angle differences between the optical engine 120 and the detection device 130. However, this requires a custom optical path, resulting in a higher cost for the projector 100.

[0071] In the fourth type of remote control, the projector 100 obtains the mapping relationship H by using some fixed points in the projected image.

[0072] For example, the optical engine 120 can project a projection image onto the projection screen 200. The projection image includes multiple boundary points, and the detection device 130 can detect a detection image, which also includes these multiple boundary points. The processor 110 calculates a mapping relationship H based on the coordinates of these multiple boundary points in the detection image and the coordinates of these multiple boundary points in the projection image.

[0073] In this method, because the content of the projected image is uncertain, the boundary points of the projected image may not be obvious, leading to errors in the coordinate detection of the boundary points in the detection image, and thus errors in the calculation of the mapping relationship H. If the mapping relationship H is incorrect, there will be a large positional deviation between the light spot and the cursor on the projection screen 200, resulting in operation errors or a poor user experience.

[0074] In the fifth type of directional remote control method, the user manually uses the remote control device 300 to project light spots onto multiple boundary points in order to avoid the boundary points of the projected image being unclear.

[0075] For example, such as Figure 8 As shown, the user manually projects multiple light spots onto multiple boundary points using the remote control device 300. The detection device 130 detects and obtains a detection image, which includes multiple light spots at multiple boundary points. The processor 110 calculates a mapping relationship H based on the coordinates of each light spot in the detection image and the coordinates of each boundary point in the projected image. A projected image including a cursor is generated based on the mapping relationship H.

[0076] In this method, the user needs to manually project a light spot using the remote control 300 to obtain the mapping relationship, resulting in a poor user experience. Furthermore, the user needs to accurately project the light spot at the boundary points; otherwise, the calculated mapping relationship H will be inaccurate, making the operation too difficult for the user. To reduce the number of user operations, the projector 100 and projection screen 200 should be kept as still as possible, which reduces flexibility. Also, before obtaining the mapping relationship H, the user cannot operate the projected image by pointing the remote control, wasting the user's time.

[0077] Typically, the mapping relationship H is a matrix, therefore it needs to be obtained from multiple points in the projected image. As mentioned in the third, fourth, and fifth pointing remote control methods above, the mapping relationship H can be obtained from multiple points in a single frame. However, these multiple points are either interruptions to the user's viewing area of ​​the calibrated region or indistinct boundary points, resulting in a poor user experience.

[0078] In the sixth type of remote control pointing method, the mapping relationship H can be obtained through multiple points (such as the cursor) in multiple frames.

[0079] Based on this, embodiments of this application provide a projection method that can be applied to... Figure 1 The projection system 1000 shown may include... Figure 4The projection system 1000 is illustrated with a detection device 130, an optical engine 120, and a processor 110. This projection system 1000 may also include other circuits, devices, or equipment, which are not limited in this embodiment. The detection device 130, optical engine 120, and processor 110 may be located in three different devices. Alternatively, at least two of the detection device 130, optical engine 120, and processor 110 may be located in the same device, for example, the detection device 130, optical engine 120, and processor 110 may be jointly located in the projector 100. The projection method may be executed by the processor 110, or by a device including the processor 110 (e.g., the projector 100). This embodiment will now use the structure of the projector 100 as an example. Figure 4 The structure shown is used as an example to illustrate the detection method executed by processor 110.

[0080] The optical engine 120 can be used to project a projected image onto the projection screen 200. The detection device 130 can be used to detect the image on the projection screen 200 to obtain a detection image. The optical engine 120 projects one frame of a projected image, and the detection device 130 can detect that projected image to obtain one frame of a detection image; therefore, one frame of detection image corresponds to one frame of projected image. It is understood that not every frame of projected image needs to be detected; for example, detection can be performed every two frames of projected image. In practice, the logic function should determine whether each frame of projected image needs to be detected, and this embodiment does not impose such limitations.

[0081] like Figure 9 As shown, the projection method includes at least one or more of the following steps:

[0082] S110: During each detection period, the processor 110 obtains the cursor coordinates of the cursor in the projected image, generates the cursor in the projected image, and obtains the cursor coordinates of the cursor in the detection image.

[0083] In some examples, a detection period includes the time it takes for processor 110 to generate a frame of projected image, the time it takes for optical engine 120 to project that frame of projected image, and the time it takes for detection device 130 to detect that frame of projected image. In some cases, the duration of each detection period may be inconsistent. In other cases, the duration of each detection period may be consistent, and the detection period may also be referred to as a detection cycle.

[0084] For example, a detection period may include the following processes: Processor 110 calculates and obtains the cursor coordinates of the cursor in the projected image, and generates the cursor in the projected image. Typically, one cursor is displayed on a projected image. Processor 110 sends the projected image including the cursor to optical engine 120. Optical engine 120 projects the projected image onto projection screen 200. Detection device 130 detects the image on projection screen 200 and obtains a detection image corresponding to the projected image, which includes the cursor. Detection device 130 sends the detection image including the cursor to processor 110. Processor 110 obtains the cursor coordinates of the cursor in the detection image based on the detection image.

[0085] It can be seen that the processor 110 can obtain the coordinates of a cursor in the projected image and the coordinates of a cursor in the detection image. If there are multiple such cursors (that is, the projected image has multiple frames), then the processor 110 can calculate the above mapping relationship H.

[0086] S210: The processor 110 obtains the target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detected images based on the multiple cursor coordinates of the multiple projected images and the multiple cursor coordinates of the multiple detected images. The target mapping relationship is also the aforementioned mapping relationship H.

[0087] In some examples, these multiple projected images and multiple detection images correspond to multiple detection time periods.

[0088] For example, such as Figure 10 As shown, during the Mth detection period, the processor 110 generates the Mth projection image (including the cursor M) and sends it to the optical engine 120. The detection device 130 captures the Mth projection image on the projection screen 200, obtains the Mth detection image (including the cursor M), and sends it to the processor 110.

[0089] During the M+1th detection period, the processor 110 generates the M+1th projection image (including cursor M+1) and sends it to the optical engine 120. The detection device 130 captures the M+1th projection image on the projection screen 200, obtains the M+1th detection image (including cursor M+1), and sends it to the processor 110.

[0090] During the M+2 detection period, the processor 110 generates the M+2 projection image (including cursor M+2) and sends it to the optical engine 120. The detection device 130 captures the M+2 projection image on the projection screen 200, obtains the M+2 detection image (including cursor M+2), and sends it to the processor 110.

[0091] During the M+3 detection period, the processor 110 generates the M+3rd projection image (including cursor M+3) and sends it to the optical engine 120. The detection device 130 captures the M+3rd projection image on the projection screen 200, obtains the M+3rd detection image (including cursor M+3), and sends it to the processor 110.

[0092] Multiple detection time periods can refer to the four detection time periods from the Mth detection time period to the M+3rd detection time period. Multiple projection images can refer to the four projection images from the Mth projection image to the M+3rd projection image. Multiple detection images can refer to the four detection images from the Mth detection image to the M+3rd detection image.

[0093] Processor 110 can obtain four sets of point pairs. The first set of point pairs refers to the cursor coordinates of cursor M in the Mth projected image and the cursor coordinates of cursor M in the Mth detection image. The second set of point pairs refers to the cursor coordinates of cursor M+1 in the M+1 projected image and the cursor coordinates of cursor M+1 in the M+1 detection image. The third set of point pairs refers to the cursor coordinates of cursor M+2 in the M+2 projected image and the cursor coordinates of cursor M+2 in the M+2 detection image. The fourth set of point pairs refers to the cursor coordinates of cursor M+3 in the M+3 projected image and the cursor coordinates of cursor M+3 in the M+3 detection image.

[0094] The processor 110 can obtain the target mapping relationship based on these four sets of point pairs. In practice, the number of point pairs can be more or less, and this embodiment does not limit this.

[0095] In this embodiment, the mapping relationship between the projected image of the optical engine 120 and the detected image of the detection device 130 can be calculated using the cursor coordinates of the projected image and the cursor coordinates of the detected image. That is, the viewing angle difference between the optical engine 120 and the detection device 130 is taken into account. Furthermore, since the cursor is typically present in the projected image, there is no need to project a calibration area, thus not interrupting the user's viewing. The user can also move the projector 100 as needed without worrying about the calibration area reappearing, offering greater flexibility. Moreover, the projector 100 does not need to be placed in a calibration position, there is no need to limit the optical engine 120 and the detection device 130 to share an optical path, and there is no need to set up a light sensor 210. The projection system 1000 has a lower cost and more flexible placement. Additionally, the cursor position is more obvious relative to the boundary point, eliminating the need for the user to manually project a light spot onto the boundary point. This improves the accuracy of calculating the target mapping relationship, enhances the user experience, and reduces the difficulty of user operation.

[0096] In some possible implementations, the processor 110 also needs to know the one-to-one correspondence between the multiple projected images and the multiple detected images in order to calculate the target mapping relationship. Two methods by which the processor 110 obtains this one-to-one correspondence will be described below.

[0097] In the first approach, the processor 110 obtains the one-to-one correspondence mentioned above through curve similarity.

[0098] In some examples, S210 may specifically include: the processor 110 determining the correspondence between multiple cursors in the multiple projected images and multiple cursors in the multiple detection images based on the similarity between a first curve formed by multiple cursor coordinates in the multiple projected images and a second curve formed by multiple cursor coordinates in the multiple detection images. The processor 110 then obtains the target mapping relationship based on the multiple cursor coordinates in the multiple projected images, the multiple cursor coordinates in the multiple detection images, and the correspondence.

[0099] For example, multiple projected images are merged into a single image, such that the single image includes multiple cursors forming a first curve, or a first trajectory. Multiple detected images are merged into a single image, such that the single image includes multiple cursors forming a second curve, or a second trajectory.

[0100] For example, such as Figure 11 As shown, the Mth to M+3rd projected images are merged into the left image, and the four cursors are connected to form the first curve. The Mth to M+3rd detected images are merged into the right image, and the four cursors are connected to form the second curve. Based on the similarity between the first and second curves, it can be seen that the cursors in the first row of the left image correspond to the cursors in the first row of the right image, the cursors in the second row of the left image correspond to the cursors in the second row of the right image, and so on. The processor 110 can calculate the target mapping relationship, such as the homography matrix, based on the corresponding set of point pairs.

[0101] Furthermore, by way of example, the aforementioned similarity includes at least one of the following: cosine similarity, Euclidean distance-based similarity, and curve slope-based similarity. Other similarities may also be included, and this application embodiment does not limit this.

[0102] In this method, the correspondence between multiple projected images and multiple detected images is determined by curve similarity, avoiding the use of incorrect data for calculation and improving the accuracy of target mapping.

[0103] In the second method, the processor 110 obtains the one-to-one correspondence mentioned above through timestamps.

[0104] In some examples, S210 may specifically include: the processor 110 determining the correspondence between the multiple cursors in the multiple projected images and the multiple cursors in the multiple detection images based on the acquisition times of the multiple cursor coordinates in the multiple projected images, the acquisition times of the multiple cursor coordinates in the multiple detection images, and the time difference. The processor 110 obtains the target mapping relationship based on the multiple cursor coordinates in the multiple projected images, the multiple cursor coordinates in the multiple detection images, and the correspondence. The time difference is the time difference between the acquisition time of the cursor coordinates in the detection image and the acquisition time of the cursor coordinates in the corresponding projected image.

[0105] For example, the time when the processor 110 acquires or generates the projected image can be regarded as the time when the cursor coordinates of the projected image are acquired; the time when the processor 110 receives the detection image can be regarded as the time when the cursor coordinates of the detection image are acquired. The time difference can be a time difference pre-configured by the processor 110 at the factory. When the processor 110 generates a frame of projected image, after this time difference, the processor 110 controls the detection device 130 to capture and return the detection image.

[0106] For example, such as Figure 12 As shown, processor 110 records the timestamp of generating each projected image and the timestamp of receiving each detected image. Processor 110 obtains projected images generated at times 1, 3, 5, and 7, and detected images acquired at times 2, 4, 6, and 8. The time difference is pre-configured to 1, so the cursor of the projected image at time 1 corresponds to the cursor of the detected image at time 2, the cursor of the projected image at time 3 corresponds to the cursor of the detected image at time 4, and so on. Processor 110 can calculate target mapping relationships, such as homography matrices, based on the corresponding set of point pairs.

[0107] In this method, the correspondence between multiple projected images and multiple detected images is determined by the time difference, avoiding the use of incorrect data for calculation, which can improve the accuracy of target mapping.

[0108] After explaining how to obtain the target mapping relationship, we will now give an example of how the projector 100 uses the mapping relationship for pointing and remote control.

[0109] In some possible implementations, the remote control device 300 is used to project a light spot onto the projection screen 200. For example... Figure 13 As shown, the method may further include S100: in each detection period, the processor 110 acquires the spot coordinates in the detection image.

[0110] For example, in each detection period, a light spot is projected onto the projected image, so the detection image includes not only the cursor generated by the processor 110, but also the light spot projected by the remote control device 300. For the same detection period, the detection image in S100 and the detection image in S110 can be the same detection image. For example, as Figure 14 As shown, each detection image includes a cursor and a light spot. The light spot in each detection period can be used to generate the cursor for the next detection period.

[0111] For example, the execution time of S100 and S110 is not limited to before or after the execution of S210. That is, before obtaining the target mapping relationship, the processor 110 can obtain the cursor coordinates of the projected image and the cursor coordinates and spot coordinates of the corresponding detection image in each detection period. After obtaining the target mapping relationship, the processor 110 can also obtain the cursor coordinates of the projected image and the cursor coordinates and spot coordinates of the corresponding detection image in each detection period.

[0112] After obtaining the target mapping relationship, S110 may specifically include S220: the processor 110 obtains the cursor coordinates of the cursor in the projected image based on the spot coordinates of the detected image and the target mapping relationship.

[0113] For example, such as Figure 13 As shown, S220 can specifically refer to: in each detection period after obtaining the target mapping relationship, the processor 110 obtains the cursor coordinates of the cursor in the projection image of the current detection period based on the spot coordinates of the detection image of the previous detection period and the target mapping relationship, generates the cursor in the current projection image, and obtains the cursor coordinates of the cursor in the current detection image.

[0114] For example, such as Figure 14 As shown, after the processor 110 obtains the target mapping relationship, during the M+4th detection period, the processor 110 generates the M+4th projection image (including cursor M+4) and sends it to the optical engine 120. The detection device 130 captures the M+4th projection image on the projection screen 200, obtaining the M+4th detection image (including cursor M+4 and light spot M+4) and sends it to the processor 110. During the M+5th detection period, the processor 110 obtains the coordinates of cursor M+5 in the M+5th projection image based on the coordinates of light spot M+4 in the M+4th detection image and the target mapping relationship. The processor 110 generates the M+5th projection image and sends it to the optical engine 120. The M+5th projection image includes cursor M+5. The processor 110 receives the M+5th detection image from the detection device 130. The M+5th detection image includes cursor M+5 and light spot M+5.

[0115] Understandably, the time difference between the user projecting the light spot and the time the processor 110 calculates the cursor based on the light spot is very small and difficult for the user to perceive. Therefore, the impact on the user's viewing experience is negligible.

[0116] In the example above, the acquisition time of the detection image in S220 is usually after the acquisition of the target mapping relationship. However, in some special cases, the acquisition time of the detection image in S220 can also be before the acquisition of the target mapping relationship.

[0117] For example, the aforementioned M+3th detection image also includes spot M+3. When processor 110 acquires the M+3th detection image, it has not yet calculated the target mapping relationship. If processor 110 calculates the target mapping relationship quickly, and calculates the target mapping relationship before the M+4th projection image is generated, then the coordinates of the cursor M+4 in the M+4th projection image can be calculated based on the target mapping relationship and spot M+3.

[0118] In this embodiment, when the user projects a light spot onto the projection screen 200 via the remote control device 300, the projector 100 can accurately display a cursor at the light spot location on the projection screen 200 based on the detected mapping relationship between the light spot and the target. This improves the user experience and avoids operational errors caused by inaccurate cursor display.

[0119] In some possible implementations, before the target mapping relationship is acquired, multiple cursors can be moved by button operation of a conventional remote control device 300, or multiple cursors can be moved by pointing remote control technology. Since the target mapping relationship has not yet been acquired, cursors can be generated through other mapping relationships.

[0120] In some examples, before obtaining the target mapping relationship, S110 may specifically include S200: the processor 110 obtains the cursor coordinates of the cursor in the projected image based on the spot coordinates of the detected image and the preset mapping relationship.

[0121] For example, such as Figure 13 As shown, S200 can specifically refer to: in each detection period before obtaining the target mapping relationship, the processor 110 obtains the cursor coordinates of the cursor in the projection image of the current detection period based on the spot coordinates of the detection image of the previous detection period and the preset mapping relationship, generates the cursor in the current projection image, and obtains the cursor coordinates of the cursor in the current detection image.

[0122] For example, such as Figure 14As shown, before the processor 110 obtains the target mapping relationship, the aforementioned Mth detection image also includes a light spot M. The processor 110 calculates the cursor coordinates of the cursor M+1 in the M+1th projection image based on the coordinates of the light spot M in the Mth detection image and the preset mapping relationship.

[0123] For example, the preset mapping relationship can be a mapping relationship pre-configured by the processor 110 at the factory, and the preset mapping relationship is not necessarily accurate. The target mapping relationship is a mapping relationship calculated based on the actual projection situation, and the target mapping relationship is more accurate.

[0124] For another example, the projector 100 can refresh the mapping relationship in real time, and the preset mapping relationship can be the target mapping relationship obtained during the last refresh. If the relative positions of the optical engine 120, the detection device 130, and the projection screen 200 do not change during the refresh, then the refreshed target mapping relationship is consistent with the preset mapping relationship, and the preset mapping relationship is relatively accurate. If the relative positions of at least two of the optical engine 120, the detection device 130, and the projection screen 200 change during the refresh, then the refreshed target mapping relationship is inconsistent with the preset mapping relationship, and the preset mapping relationship is no longer accurate.

[0125] In this embodiment, before acquiring the target mapping relationship, the projector 100 can generate a cursor based on a preset mapping relationship, allowing the user to operate the projected image by pointing at it with the remote control. This saves the user's time and improves the user experience. Although the preset mapping relationship is slightly less accurate than the target mapping relationship, the time required to acquire the target mapping relationship is shorter, thus having a smaller impact on the user experience.

[0126] In some possible implementations, taking the inconsistency between the preset mapping relationship and the target mapping relationship as an example, the distance between the cursor and the light spot on the projection screen 200 decreases before and after obtaining the target mapping relationship. Specifically, before obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen 200 and the position of the cursor corresponding to that light spot on the projection screen 200 is a first distance; after obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen 200 and the position of the cursor corresponding to that light spot on the projection screen 200 is a second distance; the first distance is greater than the second distance.

[0127] For example, the cursor can be an icon, such as a mouse pointer icon, a finger icon, or a circular icon, etc. The light spot can be a point, a circle, a crosshair, or other spatially modulated shapes. The position of the light spot on the projection screen 200 can refer to a preset point of the light spot, such as the center point or the boundary point. The position of the cursor on the projection screen 200 can refer to a preset point of the cursor, such as the center point or the boundary point.

[0128] Ideally, the second distance is 0, and the cursor and the light spot coincide. That is, before and after obtaining the target mapping relationship, the cursor and the light spot on the projection screen 200 go through a process from not coinciding to coinciding.

[0129] In practice, the cursor calculated based on the target mapping relationship may still have a slight error, so the second distance is not zero. The cursor and the light spot do not completely overlap, but rather achieve a basically overlapping, substantially overlapping, or approximately overlapping effect. Alternatively, in order for the user to see both the light spot and the cursor simultaneously and obtain a clearer visual experience, the cursor calculated based on the target mapping relationship needs to have a slight positional difference from the light spot, so the second distance is not zero.

[0130] In this embodiment, before and after obtaining the target mapping relationship, the user can see that the distance between the light spot on the projection screen 200 and the cursor becomes smaller, resulting in a better user experience.

[0131] After explaining how to apply target mapping relationships and preset mapping relationships, we will introduce when to refresh the mapping relationships.

[0132] In some possible implementations, S210 may specifically include: the processor 110 receiving an indication signal. In response to the indication signal, the processor 110 obtains a target mapping relationship based on multiple cursor coordinates of multiple projected images and multiple cursor coordinates of multiple detected images. In this way, the processor 110 can obtain the target mapping relationship under external indication without requiring computation by the processor 110, thus reducing the computational overhead of the processor 110.

[0133] For example, the indication signal is used to indicate a change in the relative position of the optical engine 120 and the projection screen 200, or the indication signal is used to indicate a change in the relative position of the detection device 130 and the projection screen 200, or the indication signal is used to indicate the activation of the optical engine 120 and / or the detection device 130. A sensor may be provided on the optical engine 120, the detection device 130, or the projector 100, and the indication signal may originate from that sensor.

[0134] In this embodiment, when any of the optical engine 120, detection device 130, and projection screen 200 is moved, or when the optical engine 120 or detection device 130 is activated, an indication signal can prompt the processor 110 to obtain the target mapping relationship. Users can move or activate these devices as needed without affecting the user experience of the remote control operation.

[0135] In other possible implementations, the processor 110 can use each acquired detection image to calculate the mapping relationship. When the new mapping relationship (i.e., the target mapping relationship) is consistent with the old mapping relationship (i.e., the preset mapping relationship) or the difference is small, it indicates that the relative positions of the optical engine 120 and the projection screen 200, and the relative positions of the detection device 130 and the projection screen 200, have not changed, and there is no need to refresh the mapping relationship. When the new mapping relationship (i.e., the target mapping relationship) is inconsistent with the old mapping relationship (i.e., the preset mapping relationship) or the difference is large, it indicates that at least one of the above-mentioned relative positions has changed, and the mapping relationship needs to be refreshed. The processor 110 then uses the new mapping relationship to generate the cursor.

[0136] To facilitate understanding, the following examples illustrate possible implementations of this projection method in various application scenarios. For instance... Figure 15 As shown, the user projects a light spot onto the projection screen 200 using a remote control. The camera of the projector 100 captures a detection image carrying the light spot. The processor 110 of the projector 100 maps the light spot from the coordinate system of the detection image to the coordinate system of the projected image based on the mapping relationship F, obtaining the cursor coordinates in the projected image. At this time, the mapping relationship F is a preset mapping relationship. Based on the cursor coordinates in the projected image, the processor 110 issues a command to draw a projected image carrying the cursor and sends it to the optical engine 120 of the projector 100. The processor 110 determines whether the mapping parameters need to be refreshed. If not, the previous steps are repeated. If so, the processor 110 adds the cursor coordinates of the projected image to trajectory 1. The camera captures a detection image carrying the aforementioned cursor. The processor 110 adds the cursor coordinates of the detection image to trajectory 2. The processor 110 determines whether the number of cursors in trajectory 1 and trajectory 2 is sufficient. If not, the previous steps are repeated. If sufficient, the processor 110 calculates the target mapping relationship based on trajectory 1 and trajectory 2. The processor 110 updates the mapping relationship F to the target mapping relationship.

[0137] The foregoing mainly describes the projection system 1000, projector 100, processor 110, and projection method. It is understood that the projection system 1000, projector 100, or processor 110 includes corresponding hardware structures and / or software modules to perform the aforementioned functions. Those skilled in the art should readily recognize that, based on the structures and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] This application embodiment can divide functional modules according to the projection system 1000, projector 100, or processor 110 corresponding to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0139] In the case of dividing each function into corresponding modules, embodiments of this application also provide a projection device, which can be applied to... Figure 1 The projection system 1000 shown may include Figure 4 The processor 110, optical engine 120, and detection device 130 are shown. The optical engine 120 projects the image onto the projection screen 200. The detection device 130 detects the image on the projection screen 200 to obtain a detection image. Figure 16 As shown, the projection device 400 includes an acquisition unit 410 and a calculation unit 420.

[0140] The acquisition unit 410 can be used to acquire the cursor coordinates of the cursor in the projected image at each detection time period, generate a cursor in the projected image, and acquire the cursor coordinates of the cursor in the detection image.

[0141] The calculation unit 420 can be used to obtain the target mapping relationship between the cursor coordinates of the projection images and the cursor coordinates of the detection images based on the multiple cursor coordinates of the multiple projection images sent by the acquisition unit 410 and the multiple cursor coordinates of the multiple detection images.

[0142] In some possible implementations, the projection system 1000 further includes a remote control device 300 for projecting a light spot onto the projection screen 200. The acquisition unit 410 is further configured to acquire the light spot coordinates in the detection image during each detection period. After acquiring the target mapping relationship, the acquisition unit 410 is specifically configured to acquire the cursor coordinates in the projected image based on the light spot coordinates in the detection image and the target mapping relationship.

[0143] In some possible implementations, the acquisition unit 410 is further configured to acquire the spot coordinates in the detection image at each detection period. Before acquiring the target mapping relationship, the acquisition unit 410 is specifically configured to acquire the cursor coordinates in the projection image based on the spot coordinates in the detection image and a preset mapping relationship.

[0144] In some possible implementations, before acquiring the target mapping relationship, the distance between the position of the light spot on the projection screen 200 and the position of the cursor corresponding to the light spot on the projection screen 200 is a first distance. After acquiring the target mapping relationship, the distance between the position of the light spot on the projection screen 200 and the position of the cursor corresponding to the light spot on the projection screen 200 is a second distance. The first distance is greater than the second distance.

[0145] In some possible implementations, the calculation unit 420 is specifically used to determine the correspondence between multiple cursors in multiple projected images and multiple cursors in multiple detection images based on the similarity between a first curve formed by multiple cursor coordinates of multiple projected images sent by the acquisition unit 410 and a second curve formed by multiple cursor coordinates of multiple detection images. The calculation unit 420 is also specifically used to obtain a target mapping relationship based on the multiple cursor coordinates of multiple projected images, the multiple cursor coordinates of multiple detection images, and the correspondence, all sent by the acquisition unit 410. The similarity may include at least one of the following: cosine similarity, Euclidean distance-based similarity, and curve slope-based similarity. Other similarity methods are also possible, and this application embodiment does not limit this.

[0146] In some possible implementations, the calculation unit 420 is specifically used to determine the correspondence between the multiple cursors in the multiple projection images and the multiple cursors in the multiple detection images based on the acquisition times of the multiple cursor coordinates in the multiple projection images sent by the acquisition unit 410, the acquisition times of the multiple cursor coordinates in the multiple detection images, and the time difference. The calculation unit 420 is also specifically used to obtain a target mapping relationship based on the multiple cursor coordinates in the multiple projection images sent by the acquisition unit 410, the multiple cursor coordinates in the multiple detection images, and the correspondence. Here, the time difference is the time difference between the acquisition time of the cursor coordinates in the detection image and the acquisition time of the cursor coordinates in the corresponding projection image.

[0147] In some possible implementations, the calculation unit 420 is further configured to receive an indication signal. Specifically, the calculation unit 420 is configured to, in response to the indication signal, acquire a target mapping relationship based on multiple cursor coordinates of multiple projected images sent by the acquisition unit 410 and multiple cursor coordinates of multiple detected images.

[0148] In some possible implementations, the indication signal is used to indicate a change in the relative position of the optical engine 120 and the projection screen 200, or the indication signal is used to indicate a change in the relative position of the detection device 130 and the projection screen 200, or the indication signal is used to indicate the activation of the optical engine 120 and / or the detection device 130.

[0149] Taking the projection device 400 applied to the aforementioned processor 110 as an example, such as Figure 17 As shown, the calculation unit 420 of the projection device 400 can also be called the mapping parameter calculation unit 421. The acquisition unit 410 of the projection device 400 may specifically include a detection unit 411, a coordinate mapping unit 412, and a cursor control unit 413.

[0150] The detection unit 411 can be used to receive the detection image from the detection device 130 in each detection period, obtain the cursor coordinates and spot coordinates in the detection image, send the cursor coordinates in the detection image to the mapping parameter calculation unit 421, and send the spot coordinates in the detection image to the coordinate mapping unit 412.

[0151] Before obtaining the target mapping relationship, the coordinate mapping unit 412 can be used to obtain the cursor coordinates of the cursor in the projection image based on the spot coordinates of the detection image sent by the detection unit 411 and the preset mapping relationship, and send the cursor coordinates of the projection image to the cursor control unit 413 and the mapping parameter calculation unit 421.

[0152] After obtaining the target mapping relationship, the coordinate mapping unit 412 can be used to obtain the cursor coordinates of the cursor in the projection image based on the spot coordinates of the detection image sent by the detection unit 411 and the target mapping relationship sent by the mapping parameter calculation unit 421, and send the cursor coordinates of the projection image to the cursor control unit 413 and the mapping parameter calculation unit 421.

[0153] The cursor control unit 413 can be used to acquire the cursor coordinates of the projected image sent by the coordinate mapping unit 412 in each detection period, generate a cursor in the projected image, and send the projected image to the optical engine 120.

[0154] The mapping parameter calculation unit 421 can be used to obtain the target mapping relationship between the cursor coordinates of the projection image and the cursor coordinates of the detection image based on the multiple cursor coordinates of the multiple projection images sent by the coordinate mapping unit 412 and the multiple cursor coordinates of the multiple detection images sent by the detection unit 411, and send the target mapping relationship to the coordinate mapping unit 412.

[0155] It is understood that each component of the projection device 400 can be used to implement the corresponding steps in the aforementioned projection method embodiments. Since each step has been described in detail in the aforementioned projection method embodiments, it will not be repeated here.

[0156] The above description of one projection device in this application embodiment is from the perspective of modular functional entities. The following description of another projection device in this application embodiment is from the perspective of hardware processing. For example... Figure 18As shown, the projection device 500 includes a processor 110 and a memory 520. The memory 520 stores instructions, and the processor 110 is used to invoke the instructions in the memory 520 to execute one or more steps in the aforementioned projection method embodiments. In some possible embodiments, the projection device 500 may further include an optical engine 120 and / or a detection device 130. The optical engine 120 is used to project a projection image onto the projection screen 200. The detection device 130 is used to detect the image on the projection screen 200 to obtain a detection image.

[0157] It is understood that each component of the projection device 500 can be used to implement the corresponding steps in the aforementioned projection method embodiments. Since each step has been described in detail in the aforementioned projection method embodiments, it will not be repeated here.

[0158] This application embodiment also provides a projection system 1000, which includes a remote control device 300 and... Figure 18 The projection device 500 and remote control device 300 shown are used to project a light spot onto the projection screen 200. The structure of the projection system 1000 can be referenced from [reference needed]. Figure 1 The structure shown. In some possible implementations, the projection system 1000 may also include a projection screen 200.

[0159] It is understood that each component of the projection system 1000 can be used to implement the corresponding steps in the aforementioned projection method embodiments. Since each step has been described in detail in the aforementioned projection method embodiments, it will not be repeated here.

[0160] This application also provides a computer-readable storage medium storing program code. When the medium is run on a device (e.g., a microcontroller, chip, computer, or processor), the program code can be invoked by the processor to execute one or more steps in the above method embodiments.

[0161] Based on this understanding, this application also provides a computer program product containing instructions. The technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or its processor to execute all or part of the steps of the methods described in the various embodiments of this application.

[0162] The processor involved in the embodiments of this application can be a chip. For example, it can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0163] The memory involved in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0164] It is understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0165] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A projection method, characterized in that, The method is applied to a processor in a projection system, which further includes an optical engine, a detection device, and a projection screen. The optical engine projects a projection image onto the projection screen, and the detection device detects the image on the projection screen to obtain a detection image. The method includes: In each detection period, the cursor coordinates of the cursor in the projected image are obtained, the cursor is generated in the projected image, and the cursor coordinates of the cursor in the detection image are obtained. Based on multiple cursor coordinates of multiple projected images and multiple cursor coordinates of multiple detected images, a target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detected images is obtained.

2. The projection method according to claim 1, characterized in that, The projection system further includes a remote control device for projecting a light spot onto the projection screen; the method further includes: In each detection period, the coordinates of the light spot in the detection image are obtained; After obtaining the target mapping relationship, obtaining the cursor coordinates of the cursor in the projected image includes: Based on the spot coordinates of the detected image and the target mapping relationship, the cursor coordinates of the cursor in the projected image are obtained.

3. The projection method according to claim 2, characterized in that, Before obtaining the target mapping relationship, obtaining the cursor coordinates of the cursor in the projected image includes: Based on the spot coordinates of the detected image and the preset mapping relationship, the cursor coordinates of the cursor in the projected image are obtained.

4. The projection method according to claim 3, characterized in that, Before obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is the first distance; After obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is the second distance; The first distance is greater than the second distance.

5. The projection method according to any one of claims 1-4, characterized in that, The step of obtaining the target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detected images based on the multiple cursor coordinates of the multiple projected images and the multiple cursor coordinates of the multiple detected images includes: Based on the similarity between a first curve formed by multiple cursor coordinates of multiple projected images and a second curve formed by multiple cursor coordinates of multiple detection images, the correspondence between multiple cursors of multiple projected images and multiple cursors of multiple detection images is determined. The target mapping relationship is obtained based on the multiple cursor coordinates of the multiple projected images, the multiple cursor coordinates of the multiple detected images, and the correspondence.

6. The projection method according to claim 5, characterized in that, The similarity includes at least one of the following: cosine similarity, Euclidean distance-based similarity, and curve slope-based similarity.

7. The projection method according to any one of claims 1-4, characterized in that, The step of obtaining the target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detected images based on the multiple cursor coordinates of the multiple projected images and the multiple cursor coordinates of the multiple detected images includes: Based on the acquisition time of multiple cursor coordinates of multiple projected images, the acquisition time of multiple cursor coordinates of multiple detection images, and the time difference, the correspondence between multiple cursors of multiple projected images and multiple cursors of multiple detection images is determined. The target mapping relationship is obtained based on multiple cursor coordinates of multiple projected images, multiple cursor coordinates of multiple detected images, and the correspondence relationship. Wherein, the time difference is the time difference between the acquisition time of the cursor coordinates of the detection image and the acquisition time of the cursor coordinates of the corresponding projection image.

8. The projection method according to any one of claims 1-7, characterized in that, The step of obtaining the target mapping relationship between the cursor coordinates of the projected images and the cursor coordinates of the detected images based on the multiple cursor coordinates of the multiple projected images and the multiple cursor coordinates of the multiple detected images includes: Receive indication signal; In response to the indication signal, the target mapping relationship is obtained based on multiple cursor coordinates of multiple projected images and multiple cursor coordinates of multiple detected images.

9. The projection method according to claim 8, characterized in that, The indication signal is used to indicate a change in the relative position of the optical engine and the projection screen, or the indication signal is used to indicate a change in the relative position of the detection device and the projection screen, or the indication signal is used to indicate the activation of the optical engine and / or the detection device.

10. A projection device, characterized in that, The projection device is applied to a projection system, which further includes an optical engine, a detection device, and a projection screen. The optical engine is used to project a projection image onto the projection screen, and the detection device is used to detect the image on the projection screen to obtain a detection image. The projection device includes an acquisition unit and a calculation unit. The acquisition unit is used to acquire the cursor coordinates of the cursor in the projected image at each detection time period, generate the cursor in the projected image, and acquire the cursor coordinates of the cursor in the detection image. The calculation unit is used to obtain the target mapping relationship between the cursor coordinates of the projection image and the cursor coordinates of the detection image based on the multiple cursor coordinates of the multiple projection images and the multiple cursor coordinates of the multiple detection images.

11. The projection device according to claim 10, characterized in that, The projection system also includes a remote control device for projecting light spots onto the projection screen. The acquisition unit is further configured to acquire the light spot coordinates of the light spot in the detection image during each detection period; After obtaining the target mapping relationship, the obtaining unit is specifically used to obtain the cursor coordinates of the cursor in the projected image based on the spot coordinates of the detected image and the target mapping relationship.

12. The projection device according to claim 11, characterized in that, Before obtaining the target mapping relationship, the obtaining unit is specifically used to obtain the cursor coordinates of the cursor in the projected image based on the spot coordinates of the detected image and the preset mapping relationship.

13. The projection device according to claim 12, characterized in that, Before obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is the first distance; After obtaining the target mapping relationship, the distance between the position of the light spot on the projection screen and the position of the cursor corresponding to the light spot on the projection screen is the second distance; The first distance is greater than the second distance.

14. The projection device according to any one of claims 10-13, characterized in that, The calculation unit is specifically used to determine the correspondence between multiple cursors in multiple projected images and multiple cursors in multiple detected images based on the similarity between a first curve formed by multiple cursor coordinates in multiple projected images and a second curve formed by multiple cursor coordinates in multiple detected images. The calculation unit is further configured to obtain the target mapping relationship based on the multiple cursor coordinates of the multiple projected images, the multiple cursor coordinates of the multiple detected images, and the correspondence relationship.

15. The projection device according to claim 14, characterized in that, The similarity includes at least one of the following: cosine similarity, Euclidean distance-based similarity, and curve slope-based similarity.

16. The projection device according to any one of claims 10-13, characterized in that, The calculation unit is specifically used to determine the correspondence between multiple cursors in multiple projected images and multiple cursors in multiple detected images based on the acquisition time of multiple cursor coordinates in multiple projected images, the acquisition time of multiple cursor coordinates in multiple detected images, and the time difference. The calculation unit is further configured to obtain the target mapping relationship based on the multiple cursor coordinates of the multiple projected images, the multiple cursor coordinates of the multiple detected images, and the correspondence relationship; Wherein, the time difference is the time difference between the acquisition time of the cursor coordinates of the detection image and the acquisition time of the cursor coordinates of the corresponding projection image.

17. The projection device according to any one of claims 10-16, characterized in that, The computing unit is also specifically used to receive indication signals; The computing unit is specifically used to, in response to the indication signal, obtain the target mapping relationship based on multiple cursor coordinates of multiple projected images and multiple cursor coordinates of multiple detected images.

18. The projection device according to claim 17, characterized in that, The indication signal is used to indicate a change in the relative position of the optical engine and the projection screen, or the indication signal is used to indicate a change in the relative position of the detection device and the projection screen, or the indication signal is used to indicate the activation of the optical engine and / or the detection device.

19. A projection device, characterized in that, The projection device includes a processor and a memory, the memory storing instructions, and the processor being used to invoke the instructions in the memory to execute the projection method as described in any one of claims 1-9.

20. The projection device according to claim 19, characterized in that, The projection device also includes an optical engine and / or a detection device; The optical engine is used to project the projected image onto the projection screen; The detection device is used to detect the image on the projection screen to obtain a detection image.

21. A projection system, characterized in that, The projection system includes a remote control device and a projection device as described in claim 19 or 20, wherein the remote control device is used to project a light spot onto the projection screen.

22. The projection system according to claim 21, characterized in that, The projection system also includes the projection screen.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method described in any one of claims 1 to 9.

24. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-9.

Images