Projection picture screen alignment method and holder projection equipment

Abstract

The invention relates to a projection picture screen alignment method and a pan-tilt projection device, and the method comprises the steps: determining the first three-dimensional information of a projection picture in a space based on the sensor data of the pan-tilt projection device and a projection parameter corresponding to the projection picture; determining second three-dimensional information of the projection curtain in the space according to a shot image of the projection curtain and the sensor data; according to the first three-dimensional information and the second three-dimensional information, the holder projection equipment is driven to rotate in the space, so that the projection picture is projected to the projection curtain; and performing picture adjustment on the projection picture, so that the projection picture and the projection curtain realize screen alignment. By means of the method, the position of the curtain can be automatically and accurately recognized in the real-time projection process of the projection equipment, and accurate screen alignment is achieved according to the position of the projection picture and the position of the curtain.

Description

Technical Field

[0001] The present application relates to the field of projection technology, and in particular to a method for projecting images onto a screen and a pan-tilt projection device. Background Art

[0002] In the related projection screen alignment method, usually after the projection device projects a calibration image, the calibration image is captured by the camera and then aligned with the screen, which interrupts the user's viewing and affects the user's viewing experience. Summary of the invention

[0003] The present application discloses a method for aligning projection images and a pan-tilt projection device, which can align projection images without projecting a calibration image, will not affect the user's viewing experience, and improve the user's viewing experience.

[0004] In a first aspect, the present application relates to a method for aligning projection images, comprising: determining first three-dimensional information of the projection image in space based on sensor data of a pan-tilt projection device and projection parameters corresponding to the projection image; determining second three-dimensional information of the projection screen in space based on a captured image of the projection screen and the sensor data; driving the pan-tilt projection device to rotate in space based on the first three-dimensional information and the second three-dimensional information so that the projection image is projected onto the projection screen; and adjusting the projection image so that the projection image and the projection screen are aligned.

[0005] Optionally, the above-mentioned driving the pan-tilt projection device to rotate in space so that the projection picture is projected onto the projection screen includes: using the pan-tilt to adjust the projection direction and / or projection angle of the pan-tilt projection device so that the projection picture of the adjusted pan-tilt projection device covers the projection screen.

[0006] Optionally, the projection screen alignment method also includes: determining the frame information of the projection screen based on the second three-dimensional information; if the projection screen is determined to be an incomplete projection screen based on the frame information, using the pan-tilt projection device to adjust the projection direction and / or projection angle so that the data collection range of the sensor device covers the complete projection screen.

[0007] Optionally, before adjusting the projection image so that the projection image is aligned with the projection screen, the projection image alignment method also includes: determining a first ratio of a current projection distance to a current projection image width; if the first ratio is less than a preset ratio, adjusting the position of the pan-tilt projection device so that the current projection image covers the projection screen.

[0008] Optionally, before adjusting the projection image so that the projection image and the projection screen are aligned, the projection image alignment method also includes: determining a first ratio of a current projection distance to a current projection image width; if the first ratio is less than a preset ratio, adjusting the lens parameters of the pan-tilt projection device so that the current projection image covers the projection screen.

[0009] Optionally, the projection screen alignment method further includes: if the projection screen is smaller than the display area of ​​the projection screen, adjusting the first vertex coordinates of the projection screen, and scaling the projection screen according to the shape of the projection screen.

[0010] Optionally, the projection screen matching method also includes: if the projection screen includes multiple projection screens corresponding to multiple pan-tilt projection devices, determining the third three-dimensional information of each projection screen in the space where the multiple pan-tilt projection devices are located; based on the third three-dimensional information and the second three-dimensional information, using the pan-tilt of each pan-tilt projection device to correct the corresponding projection screen, so that the corrected multiple projection screens are spliced ​​and displayed on the projection screen.

[0011] Optionally, the above-mentioned adjustment of the projection picture so that the projection picture and the projection screen are aligned includes: determining the second vertex coordinates of the projection screen in the projector chip coordinate system according to the first three-dimensional information and the second three-dimensional information, and determining the display area of ​​the projection picture according to the second vertex.

[0012] Optionally, the above picture adjustment includes picture edge correction, picture brightness adjustment and / or picture color temperature adjustment.

[0013] In a second aspect, the present disclosure also provides a pan-tilt projection device, comprising: one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the projection screen method of the first aspect mentioned above.

[0014] In a third aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the projection image screen alignment method.

[0015] The present application relates to a method for aligning a projection screen, which obtains the first three-dimensional information of the projection screen in space by combining the three-dimensional sensor data of the pan-tilt projection device with the projection parameters; and determines the second three-dimensional information of the projection screen based on the captured image and the three-dimensional sensor data of the pan-tilt projection device; based on the first three-dimensional information of the projection screen and the second three-dimensional information of the projection screen, drives the pan-tilt projection device to rotate in space to change the projection direction so that the projection screen can be projected onto the projection screen; and further optimizes and adjusts the projection screen to achieve alignment between the projection screen and the projection screen. Compared with the alignment solution that requires the user to manually adjust the projection position, the above method can automatically and accurately identify the screen position without the user's perception during the real-time projection process of the projection device, and achieves accurate alignment based on the position of the projection screen and the screen position, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 This is an application scenario diagram of the projection screen method provided in an embodiment of the present application.

[0018] Figure 2 This is an example of the data collection range of the sensor device provided in the embodiment of the present application. Figure 1 .

[0019] Figure 3 This is an example of the data collection range of the sensor device provided in the embodiment of the present application. Figure 2 .

[0020] Figure 4 It is a flow chart of the projection screen alignment method provided in an embodiment of the present application.

[0021] Figure 5 This is an example diagram of a projection image provided by an embodiment of the present application projected onto a projection screen.

[0022] Figure 6 This is an example of the projection screen and the projection screen provided in the embodiment of the present application. Figure 1 .

[0023] Figure 7 This is an example diagram of the maximum projection screen area determined based on sensor data provided in an embodiment of the present application.

[0024] FIG8( a ) is an example diagram of the center point of the projection image and the projection screen provided in an embodiment of the present application.

[0025] FIG8( b ) is an example diagram showing that the center point of the projection image and the projection screen coincide with each other as provided in an embodiment of the present application.

[0026] Fig. 9 This is an example diagram of a projection screen covered by a projection image provided in an embodiment of the present application.

[0027] Fig.10 This is an example of the projection screen and the projection screen provided in the embodiment of the present application. Figure 2 .

[0028] Fig.11 This is an example diagram of displaying a projection image in the center of a projection screen provided in an embodiment of the present application.

[0029] Figure 12(a)-Figure 12(b) This is an example diagram of multiple projection images and a projection screen provided in an embodiment of the present application.

[0030] Fig.13 It is a structural schematic diagram of the pan-tilt projection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0032] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0033] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0034] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0035] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0036] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0037] Related projection screen technologies still face many challenges in practical applications. These technologies have obvious shortcomings in terms of ease of use, degree of automation, correction accuracy, and environmental adaptability.

[0038] For example Figure 1 , which is an application scenario diagram of the projection image-to-screen method provided in an embodiment of the present application. Figure 1 The projection screen (eg, Figure 1 The projection image 102 in the image and the projection screen (eg Figure 1 The projection screen 103 in the projection image has a large position deviation, for example, the distance between the center point of the projection screen and the center point of the projection image is greater than a preset distance (for example, the preset distance is half the width of the full projection screen). The coordinate system OXYZ can represent the spatial coordinate system corresponding to the projection device, such as the projector optical center coordinate system of the projection device, that is, a three-dimensional coordinate system established with the projector optical center as the origin. Among them, the style of the projection device in the lower right corner is only an example, and is only used to better demonstrate the coordinate system OXYZ.

[0039] In this case, users are usually required to manually adjust the projection position of the projection device so that the projection image can be projected onto the screen. This is not only time-consuming and labor-intensive, but also prone to correction errors due to human factors, making it difficult to fully meet users' needs for efficient and accurate projection experience.

[0040] In addition, the relevant projection equipment is highly dependent on the position of the projection screen and is usually only suitable for projection scenarios where the screen position is fixed. It is difficult to apply to scenarios where the screen position changes, resulting in users still having to manually adjust the projection equipment after moving the screen or changing the projection angle to ensure that the picture is on the screen.

[0041] In the related art, it is usually assumed that the projection image and the screen are in the same plane, and the projection image covers the screen, and the relative relationship between the two only needs to be determined based on the captured image. In addition, it is usually necessary to project a template image with a preset mark (such as a preset pattern image), and realize the screen matching based on the captured image of the projection image corresponding to this template image. Therefore, it is difficult to achieve seamless screen matching on the user side, which affects the user experience.

[0042] To solve the above problems, the embodiment of the present application provides a method for aligning the projection screen. By combining the three-dimensional sensor data of the pan-tilt projection device with the projection parameters, the first three-dimensional information of the projection screen in space is obtained; and the second three-dimensional information of the projection screen is determined according to the captured image and the three-dimensional sensor data of the pan-tilt projection device; based on the first three-dimensional information of the projection screen and the three-dimensional information of the projection screen, the pan-tilt projection device is driven to rotate in space to change the projection direction so that the projection screen can be projected onto the projection screen; the projection screen is further optimized and adjusted to achieve alignment between the projection screen and the projection screen. The method can automatically and accurately identify the screen position during the real-time projection process of the projection device, and achieve accurate alignment according to the position of the projection screen and the screen position, so as to achieve alignment of the projection screen without the user's perception, thereby improving the user's experience.

[0043] In this embodiment, the projection screen matching method can be applied to a pan-tilt projection device and a camera device that are electrically or communicatively connected to each other, and can also be directly applied to a pan-tilt projection device including a camera device, and the camera device can be used to capture an image including a projection screen. In one example, the captured image 1030 can include a complete projection screen 103, for example Figure 2-Figure 3 As shown. For a pan-tilt projection device that needs to perform screen alignment for projection images, the function for screen alignment for projection images provided by the method of the present application can be directly integrated on the pan-tilt projection device, or run on the pan-tilt projection device in the form of a software development kit (SDK). In other embodiments, the camera device can also be a camera device of a terminal device, such as a camera of a mobile phone, etc., and the function for screen alignment for projection images provided by the method of the present application can also be directly integrated on the terminal device.

[0044] In one embodiment, the pan-tilt projection device may include a driving device (e.g., a pan-tilt) and a projection device (or referred to as a projection device, such as a projector). The driving device may respond to instructions from a control unit of the pan-tilt projection device to drive the pan-tilt projection device to rotate in space, so that the pan-tilt projection device may move in a yaw direction (e.g., Figure 1 The projection direction of the PTZ projection device is changed; and / or the PTZ projection device can be rotated in the pitch direction (for example, Figure 1 Shown Figure 1 As shown, the projection device rotates around the X-axis direction, so that the projection angle of the pan-tilt projection device changes.

[0045] In another embodiment, the pan-tilt projection device may further include a sliding device, such as a pulley. The sliding device may respond to instructions from the control unit of the pan-tilt projection device to drive the pan-tilt projection device to slide in a preset guide rail, or may respond to manual adjustment of the pan-tilt projection device by a user to enable the pan-tilt projection device to move in the front and rear directions (e.g. Figure 1 The projection distance of the PTZ projection device in the front and rear directions is adjusted by moving the PTZ projection device along the Y-axis direction as shown in the figure. In the absence of other adjustments, the closer the projection distance, the smaller the size of the projection picture, and the farther the projection distance, the larger the size of the projection picture.

[0046] In another embodiment, the pan-tilt projection device may further include one or more sensor devices, such as a depth sensor, a time of flight (TOF) sensor, and the sensor device may be installed at a fixed position of the pan-tilt projection device. The sensor data obtained by the sensor device can be used to accurately determine the first three-dimensional information of the projection image and the second three-dimensional information of the projection screen. For example, the sensor data may include depth data indicating the distance between the sensor device and the projection image and / or the projection screen. Among them, the data acquisition range of the sensor device can cover the entire projection image.

[0047] In one example, for example Figure 2 As shown, the data collection range 1010 of the sensor device can cover the entire projection screen 103. Therefore, the projection screen in some embodiments of the present application represents the entire projection screen determined based on the sensor data.

[0048] In another example, for example Figure 3 As shown, the data collection range 1010 of the sensor device may be limited by the projection position of the pan-tilt projection device and cannot cover the entire projection screen 103. Therefore, the projection screen in other embodiments of the present application represents the projection screen 1031 determined according to the sensor data, which may not be the entire projection screen 103.

[0049] The data collection range of the sensor device is related to the projection distance. Without other adjustments, the closer the projection distance, the smaller the data collection range of the sensor device, and the farther the projection distance, the larger the data collection range of the sensor device.

[0050] In one embodiment, the sensor device and the optical machine of the pan-tilt projection device can be calibrated to determine the conversion relationship (e.g., the first conversion relationship) between the coordinate system of the sensor device and the optical machine coordinate system (e.g., the optical center coordinate system of the projector), and the sensor data can be converted to the optical center coordinate system of the projector based on the first conversion relationship. Therefore, the sensor data of the pan-tilt projection device in some embodiments of the present application may include depth data in the optical center coordinate system of the projector.

[0051] In one embodiment, the sensor device and the camera device can be calibrated to determine the conversion relationship (for example, a second conversion relationship) between the coordinate system of the sensor device and the captured image coordinate system of the camera device of the pan-tilt projection device, and the sensor data can be converted to the captured image coordinate system based on the second conversion relationship, so that the sensor data corresponding to any pixel point in the captured image can be determined.

[0052] The following will take the projection image screen method applied to a pan-tilt projection device including a camera device, a driving device, a projection device, a sliding device and a sensor device as an example. In order to facilitate a clear and complete description of the embodiment of the present application, the following will combine Figure 4 This chapter introduces the specific process of projected image alignment. Figure 4 As shown, it is a flowchart of the projection screen alignment method provided in an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0053] Step S401: Determine first three-dimensional information of the projection picture in space based on sensor data of the pan-tilt projection device and projection parameters corresponding to the projection picture.

[0054] In one embodiment, the method for aligning the projection image to the projection screen is essentially to ensure that the PTZ projection device can accurately project the projection image onto the projection screen and keep the projection image aligned with the border of the projection screen. To this end, the key step of aligning the projection image to the projection screen is to determine the relative position of the projection image and the projection screen in space, and then adjust the projection position of the projection device (such as the projection direction, projection angle, and projection distance, etc.) according to the relative position, so that the projection image is projected onto the projection screen, and further optimize and adjust the projection image to achieve accurate alignment.

[0055] In one example, combining Figure 1 , Figure 5 and Figure 6 This section explains how to align the projection screen. Figure 1 The relative positions of the projection image 102 and the projection screen 103 are shown, and the projection position of the projection device 101 is adjusted so that the projection image 102 is projected onto the projection screen 103, for example Figure 5 Further Figure 5 The projection image 102 shown is optimized and adjusted to achieve accurate screen alignment, for example Figure 6 The projection image 102 and the projection screen 103 are precisely overlapped.

[0056] In one embodiment, the projection image and the projection screen are located in the same space. To determine the relative position of the projection image and the projection screen in the space, the position of the projection image in the space and the position of the projection screen in the space can be determined, and the relative position can be determined based on the relationship between the two positions. The spatial coordinate system of the space where the projection image and the projection screen are located can be determined, thereby determining the position of the projection image and the projection screen in the spatial coordinate system.

[0057] In one example, the projector optical center coordinate system of the pan-tilt projection device can be used as the spatial coordinate system (eg Figure 1 Alternatively, the sensor device coordinate system may be used as the spatial coordinate system. The present application does not impose any specific restrictions on the spatial coordinate system, and the spatial coordinate system will be taken as the projector optical center coordinate system as an example below.

[0058] In one embodiment, sensor data in the projector optical center coordinate system can be obtained based on the conversion relationship between the sensor device and the projector optical center coordinate system (e.g., the first conversion relationship), such as accurate depth data for indicating the distance between the projector optical center and the projection screen. The first three-dimensional information of the projection screen in space can be determined in combination with the sensor data and the projection parameters, wherein the projection parameters may include but are not limited to one or more of the following known parameters: the trapezoidal correction vertex coordinates obtained after the projection screen is trapezoidally corrected based on the trapezoidal correction function of the projector, and the projection ratio of the projector (i.e., the ratio between the projection distance and the width of the projection screen). Based on the trapezoidal correction vertex coordinates and the projection ratio of the projection screen, the area where the projection screen is located (e.g., the area surrounded by the trapezoidal correction vertex coordinates), the two-dimensional plane coordinates of each pixel point in the area where the projection screen is located, and the depth data within the range of the area where the projection screen is located can be determined. The two-dimensional plane coordinates of each pixel point in the area where the projection screen is located are fused with the three-dimensional depth data of each pixel point in the area where the projection screen is located to obtain the first three-dimensional information of the projection screen in space.

[0059] For example, the first three-dimensional information may include but is not limited to three-dimensional information of multiple feature points of the projection screen (such as the coordinates of the four vertices of the projection screen, the coordinates of the center point of the projection screen), the size of the projection screen, and the angle of the projection screen relative to the projector, such as the angle of the center point of the projection screen relative to the coordinate origin of the spatial coordinate system.

[0060] Through the above embodiments, the depth data obtained by the sensor device and the projection parameters of the projector can be integrated to realize the accurate three-dimensional information construction of the projection picture in space, thereby improving the accuracy of the three-dimensional coordinates of the projection picture, and further improving the accuracy of the projection picture to the screen.

[0061] In other embodiments, the method for determining the first three-dimensional information of the projection screen may also include but is not limited to a combination of one or more of the following methods: Method 1: By collecting multiple captured images containing the projection image, the first three-dimensional information of the projection screen is determined based on the multiple captured images using image matching technology. Method 2: The positioning and tracking of the projection screen is achieved by integrating a visual inertial sensor (Visual-Inertial Odometry, VIO). VIO combines the data of the camera and the inertial measurement unit (IMU) to provide accurate position information in dynamic scenes. Method 3: Use a laser ranging sensor to accurately measure the distance between the projection screen and the curtain, and calculate the first three-dimensional information of the projection screen based on the measured distance information. The method for determining the first three-dimensional information of the projection screen can be selected according to the actual situation, and this application does not impose specific restrictions on this.

[0062] Step S402: Determine second three-dimensional information of the projection screen in space according to the captured image of the projection screen and the sensor data.

[0063] In one embodiment, a camera device may be used to collect a captured image (e.g., an RGB image), and position information of the projection screen in the captured image may be determined based on the captured image, for example, the positions of the screen vertices and borders of the projection screen in the captured image. Based on the position information of the projection screen in the captured image, the sensor data is fused to determine the three-dimensional information of the projection screen in a spatial coordinate system (e.g., a projector optical center coordinate system) as the second three-dimensional information.

[0064] In one example, determining the position information of the projection screen in the captured image based on the captured image may include: improving the image quality of the captured image by preprocessing the captured image, such as image noise reduction processing, brightness adjustment processing, etc. Recognizing the position information of the projection screen in the preprocessed captured image based on an image recognition algorithm, the image recognition algorithm may include but is not limited to a neural network model, such as a convolutional neural network model.

[0065] In one example, fusing sensor data based on position information of the projection screen in the captured image to determine the second three-dimensional information of the projection screen in the projector optical center coordinate system may include: determining the sensor data corresponding to any pixel point in the captured image based on a conversion relationship (e.g., a second conversion relationship) between the coordinate system of the sensor device and the coordinate system of the captured image; determining the area where the projection screen is located in the captured image according to the position information of the projection screen in the captured image; and fusing the sensor data to the corresponding pixel points in the area where the projection screen is located in the captured image to obtain the second three-dimensional information of the projection screen.

[0066] In one example, sensor data in the projector optical center coordinate system can be obtained based on the conversion relationship (e.g., the first conversion relationship) between the sensor device and the projector optical center coordinate system. The sensor data may include depth data in a spatial coordinate system (e.g., the projector optical center coordinate system), such as precise depth data indicating the distance between the projector optical center and the projection screen.

[0067] Through the above-mentioned embodiment, the image of the projection screen is collected in real time, and the position information of the projection screen is extracted through image preprocessing and recognition algorithms, and the three-dimensional position of the projection screen in space is determined efficiently and accurately in combination with sensor data, thereby providing a data basis for the subsequent projection screen alignment process.

[0068] In one embodiment, for example Figure 2 As shown, if both the captured image and the sensor data contain a complete projection screen, the second three-dimensional information of the complete projection screen in space can be determined based on the captured image and the sensor data.

[0069] In one embodiment, the projection screen can be identified based on an image recognition algorithm to determine a candidate area corresponding to the projection screen. The candidate areas are screened based on the depth data, and the candidate areas that meet the preset conditions are selected as the projection screen. The second three-dimensional information of the projection screen is determined based on the depth data corresponding to the projection screen.

[0070] For example, the second three-dimensional information may include but is not limited to three-dimensional information of multiple feature points of the projection screen (such as the coordinates of the four vertices of the projection screen, the coordinates of the center point of the projection screen), the size of the projection screen, and the angle of the projection screen relative to the projector, for example, the angle of the center point of the projection screen relative to the coordinate origin of the spatial coordinate system.

[0071] In one embodiment, identifying the projection screen based on an image recognition algorithm and determining the candidate area corresponding to the projection screen may include: collecting training data, such as sample image data of the projection screen, using the training data to train a deep neural network model (such as a convolutional neural network model), and using the trained model to recognize the captured image to determine the candidate area corresponding to the projection screen.

[0072] In one embodiment, screening candidate areas based on depth data, and selecting candidate areas that meet preset conditions as projection screens may include: determining candidate areas in the candidate areas that have screen features, are flat, perpendicular to the ground, and within a preset distance range from the projection device based on the depth data. The preset distance range may be set according to actual needs, and the present application does not limit this. Screening candidate areas determined based on depth data, and selecting candidate areas that meet preset conditions as projection screens may include: screening candidate areas of the screen identified by the model through a series of geometric constraints, and the geometric constraints may include but are not limited to: area constraints, flatness constraints, symmetry constraints, and shape constraints.

[0073] Among them, the area of ​​the projection screen is usually large, and the candidate areas with smaller areas can be filtered out based on the area constraint. The surface of the projection screen is usually relatively flat, and the flatness of the candidate area can be determined by plane fitting the depth data of the candidate area, and the candidate areas that do not meet the flatness requirements can be excluded based on the flatness constraint. The projection screen usually has a certain symmetry (for example, the projection screen usually has the symmetry of a rectangle or square), and the symmetry characteristics of the candidate area can be evaluated, and the candidate areas whose symmetry characteristics do not meet the requirements can be filtered out based on the symmetry constraint. The projection screen is usually a convex quadrilateral, and the candidate areas whose shapes are not convex quadrilaterals can be filtered out based on the shape constraint by evaluating whether the shape formed by the depth data of the candidate area is a convex quadrilateral.

[0074] Through the above embodiments, the depth sensor data and the deep neural network model can be combined to accurately identify and determine the second three-dimensional information of the projection screen in space. The candidate area with screen features is identified by training the model, and multiple geometric constraints such as area, flatness, symmetry and shape are used for fine screening, effectively eliminating non-screen areas and ensuring the accuracy of coordinate positioning.

[0075] In one embodiment, if the data collection range of the sensor device covers the entire projection screen, the second three-dimensional information of the entire projection screen can be determined.

[0076] In another embodiment, if the data collection range of the sensor device cannot cover the entire projection screen, in order to allow the projection image to be aligned with the complete projection screen, the frame information of the projection screen can be determined based on the second three-dimensional information. Figure 3 As shown in 1031), the projection direction and / or projection angle of the pan-tilt projection device is adjusted by using the pan-tilt projection device so that the data collection range of the sensor device covers the entire projection screen (e.g. Figure 2 As shown 1010 covers 103).

[0077] In one example, the frame of the projection screen may be determined according to the coordinates of the four vertices of the projection screen, and frame information may be determined according to the frame. For example, the frame information may include whether the frame coincides with the boundary of the data collection range of the sensor device.

[0078] Among them, the four vertices of the projection screen include the upper left vertex, the upper right vertex, the lower left vertex and the lower right vertex of the projection screen, and the projection screen may include frames in multiple directions, for example, frames at the top, bottom, left and right.

[0079] When determining the border of the projection screen according to the coordinates of the four vertices of the projection screen, the upper border of the projection screen can be determined according to the upper left vertex and the upper right vertex, the lower border of the projection screen can be determined according to the lower left vertex and the lower right vertex, the left border of the projection screen can be determined according to the upper left vertex and the lower left vertex, and the right border of the projection screen can be determined according to the upper right vertex and the lower right vertex.

[0080] When determining the frame information based on the frame, it can be determined whether the frame coincides with the boundary of the data collection range of the sensor device based on the depth data corresponding to the frame, for example Figure 3 As shown, it can be determined that the upper frame and the right frame coincide with the boundary of the data collection range of the sensor device.

[0081] In one example, if the frame information indicates that there is a frame that overlaps with the boundary of the data collection range of the sensor device, it can be determined that the projection screen is an incomplete projection screen. According to the orientation of the frame that overlaps with the boundary of the data collection range of the sensor device, the projection direction and / or projection angle and / or projection distance of the pan-tilt projection device can be adjusted so that the data collection range of the sensor device covers the complete projection screen.

[0082] For example Figure 3 As shown, the upper frame of the projection screen 1031 coincides with the upper boundary of the data collection range 1010 of the sensor device, and the pan-tilt projection device can be driven by the pan-tilt projection device to rotate upward in the pitch direction, thereby adjusting the projection angle of the pan-tilt projection device, so that the upper boundary of the data collection range of the sensor device can be adjusted upward. For example, the projection angle of the pan-tilt projection device can be adjusted multiple times at a preset angle until the data collection range of the sensor device can cover the upper boundary of the entire projection screen.

[0083] For example Figure 3As shown, the right side frame of the projection screen 1031 coincides with the right side boundary of the data collection range 1010 of the sensor device, and the pan-tilt projection device can be driven by the pan-tilt projection device to rotate rightward along the yaw direction, thereby adjusting the projection direction of the pan-tilt projection device, so that the right side boundary of the data collection range of the sensor device can be adjusted to the right. For example, the projection direction of the pan-tilt projection device can be adjusted multiple times at a preset angle until the data collection range of the sensor device can cover the right side boundary of the entire projection screen. Among them, the preset angle can be set according to actual needs, and this application does not impose specific restrictions on this.

[0084] In another example, the projection distance of the pan-tilt projection device can be adjusted so that the pan-tilt projection device can move backward in the front-to-back direction to increase the projection distance, thereby increasing the data collection range of the sensor device. For example, the projection distance of the pan-tilt projection device can be adjusted multiple times at a preset distance until the data collection range of the sensor device can cover the entire projection screen (e.g. Figure 2 As shown, 1010 covers 103). The preset distance can be set according to actual needs, and this application does not make any specific restrictions on this.

[0085] Through the above-mentioned embodiments, the frame information of the projection screen can be accurately identified, and the direction and angle (even the projection distance) of the projection device can be automatically optimized and adjusted based on the frame information until the data acquisition range of the sensor device fully covers the projection screen, ensuring that the projection picture can be accurately aligned with the complete projection screen, achieving high-quality, flawless projection display effects, and significantly improving the user experience.

[0086] In other embodiments, due to the installation location of the cloud sensor, the equipment limitation of the pan-tilt head, etc., the following situation may occur: no matter how the pan-tilt head projection device is driven to rotate or move in space, the data collection range of the sensor device cannot cover the entire projection screen. The maximum projection screen area that can be covered by the data collection range of the sensor device can be used as the projection screen for the projection image in the subsequent process.

[0087] For example Figure 7 As shown, the data collection range 1010 of the sensor device can only cover Figure 7 The maximum projection screen area 1032 in the image can be used to realize screen matching of projection images based on the maximum projection screen area 1032.

[0088] In other embodiments, the method for determining the second three-dimensional information of the projection screen may also include, but is not limited to: using an infrared sensor array to identify the position of the screen, and by detecting the reflection of the infrared signal, the system can accurately locate the edge of the screen and adjust the position of the projection image. The method for determining the second three-dimensional information of the projection screen can be selected according to actual conditions, and this application does not impose specific restrictions on this.

[0089] Step S403: According to the first three-dimensional information and the second three-dimensional information, the pan-tilt projection device is driven to rotate in space so that the projection picture is projected onto the projection screen.

[0090] In one embodiment, the relative position between the projection image and the projection screen can be determined based on the first three-dimensional information and the second three-dimensional information, and the pan-tilt projection device is driven to rotate in space based on the relative position, so that the projection image is projected onto the projection screen. The relative position may include the relative direction of the yaw direction, the relative angle of the pitch direction, and the relative distance in the front-back direction.

[0091] In one embodiment, the projection direction and / or projection angle of the pan-tilt projection device can be adjusted by using the pan-tilt projection device so that the projection image of the adjusted pan-tilt projection device covers the projection screen. For example, the projection image is adjusted to cover the projection screen with the minimum loss of the projection image as a constraint. The specific implementation logic includes: according to the first three-dimensional information and the second three-dimensional information, the angles to be rotated when each vertex of the four vertices of the projection image rotates to the corresponding vertex of the projection screen are calculated in turn, the median of the four angles obtained is taken, and the pan-tilt projection device is driven to rotate according to the median.

[0092] In another embodiment, the projection direction and / or projection angle of the pan-tilt projection device can be adjusted by using the pan-tilt projection device so that the center point of the projection picture of the adjusted pan-tilt projection device coincides with the center point of the projection screen.

[0093] For example, as shown in FIG8(a), the center point of the projection screen can be determined based on the connection between the vertices of the projection screen, and the center point of the projection screen can be determined based on the connection between the vertices of the projection screen. The relative position between the center point of the projection screen and the center point of the projection screen is determined by the first three-dimensional information and the second three-dimensional information, and the projection direction and / or projection angle of the pan-tilt projection device are adjusted according to the relative position, so that the center point of the projection screen of the pan-tilt projection device after adjustment coincides with the center point of the projection screen, as shown in FIG8(b).

[0094] Through the above embodiments, the three-dimensional information position relationship between the projection image and the center point of the screen can be calculated, and the projection direction and / or angle of the pan-tilt projection device can be adjusted by using the flexibility of the pan-tilt projection device to ensure that the center point of the projection image can be accurately aligned with the center point of the projection screen. This can effectively avoid projection offset and help to achieve subsequent projection alignment.

[0095] In one embodiment, before adjusting the projection picture, the size of the projection picture may be adjusted so that the projection picture covers the projection screen, wherein the size of the projection picture may be adjusted in a variety of ways.

[0096] In one example, a first ratio of the current projection distance to the current projection screen width may be determined; if the first ratio is less than a preset ratio, the position of the pan-tilt projection device may be adjusted so that the current projection screen covers the projection screen.

[0097] The first ratio of the current projection distance to the current projection screen width can be determined according to the first three-dimensional information, and if the first ratio is less than the preset ratio, it means that the current projection screen is small. The preset ratio can be set according to actual needs, and this application does not impose specific restrictions on this.

[0098] When adjusting the position of the PTZ projection device, the PTZ projection device can be moved backward in the front-to-back direction to increase the projection distance, so that the range of the projection image becomes larger. For example, the projection distance of the PTZ projection device can be adjusted multiple times at a preset distance until the current projection image covers the projection screen (for example, Fig. 9 shown).

[0099] Through the above embodiments, the ratio of the current projection distance to the projection screen width can be intelligently determined and compared with the preset ideal ratio. Once it is determined that the ratio is too small, it means that the projection screen is too small, and the PTZ projection device is automatically triggered to adjust its position backward in the front-to-back direction, gradually increasing the projection distance, thereby effectively expanding the coverage of the projection screen until the predetermined projection screen is completely covered. It can achieve precise control and automatic optimization of the projection effect, significantly improving the user experience.

[0100] In another example, a first ratio of the current projection distance to the current projection picture width may be determined; if the first ratio is less than a preset ratio, the lens parameters of the pan-tilt projection device are adjusted so that the current projection picture covers the projection screen.

[0101] When adjusting the lens parameters of the pan-tilt projection device, the zoom ring of the zoom lens can be driven to rotate in a direction to increase the projection image (for example, rotate rightward), thereby achieving optical zoom and increasing the range of the projection image.

[0102] In another example, for a pan-tilt projection device that supports digital zoom, the digital zoom parameters can also be adjusted to enlarge the projection image through a software algorithm.

[0103] Through the above embodiment, when it is detected that the ratio of the current projection distance to the projection screen width is lower than the preset ratio, the pan-tilt projection device can respond intelligently by adjusting the lens parameters of the pan-tilt projection device to effectively expand the coverage of the projection screen and ensure that the projection content can completely cover the projection screen.

[0104] In other embodiments, when the same environment is used multiple times, the PTZ projection device can memorize the frequently used projection screen positions. The user only needs to select a preset scene, and the PTZ projection device can quickly project the projection image to the projection screen position, thereby realizing quick image positioning and saving time.

[0105] In other embodiments, if it is determined that the relative position of the projection screen or the PTZ projection device has changed, the PTZ projection device can adjust the projection position in real time and dynamically track the position of the projection screen to ensure that the image always maintains the correct screen state. It can be applied to scenes where the projection device or screen needs to be frequently moved, such as mobile exhibitions or interactive displays.

[0106] Step S404: adjust the projection image so that the projection image and the projection screen are aligned.

[0107] In one embodiment, after the pan-tilt projection device is adjusted so that the projection image is projected onto the projection screen, the projection image and the projection screen are substantially aligned in spatial position, for example Fig. 9 In order to achieve accurate alignment between the projection image and the projection screen, the projection image can also be adjusted so that the vertices of the projection image can coincide with the vertices of the projection screen area, so that the projection image and the projection screen can be aligned, for example Fig.10 shown.

[0108] In one embodiment, when adjusting the projection picture, the second vertex coordinates of the projection screen in the projector chip coordinate system can be determined according to the first three-dimensional information and the second three-dimensional information, and the display area of ​​the projection picture can be determined according to the second coordinates.

[0109] Among them, by comparing the first three-dimensional information of the projection image and the second three-dimensional information of the projection screen, the second vertex coordinates of the projection screen in the projector chip coordinate system can be determined, and the second vertex coordinates can be used to indicate how the projection image should be adjusted to be correctly displayed on the projection screen. The display area of ​​the projection image can be adjusted according to the second vertex coordinates, for example, the size, position, shape, etc. of the projection image can be adjusted to ensure that it completely covers the effective display area of ​​the projection screen while avoiding exceeding the screen boundary or leaving a blank area.

[0110] Through the above embodiment, the first three-dimensional information of the projection image and the second three-dimensional information data of the projection screen can be used to accurately calculate and determine the plane position of the projection screen in the projector chip coordinate system. Based on this plane coordinate, the display area of ​​the projection image can be finely adjusted to ensure that the vertices of the projection image and the vertices of the screen area are precisely coincident, thereby perfectly covering the effective display area of ​​the screen, without overflow or blank space, achieving precise screen alignment, and significantly improving the viewing experience and projection effect.

[0111] In one embodiment, the image adjustment performed on the projected image may include but is not limited to image edge correction, image brightness adjustment and / or image color temperature adjustment. For example, image edge correction can be used to correct geometric distortion problems that may occur at the edge of the projected image, such as keystone distortion, barrel distortion or pincushion distortion. Image brightness adjustment can adapt to different viewing environments and projection screen characteristics to ensure that the brightness of the projected image is neither overexposed nor too dark. Image color temperature adjustment can change the color balance of the projected image to make it closer to the color performance under natural light or a specific light source.

[0112] Through the above embodiments, the quality of the projected image can be optimized, the viewing experience can be improved, and the projected content can be accurately displayed.

[0113] In other embodiments, if the projection picture is smaller than the display area of ​​the projection screen, the projection picture is displayed in the center. In one example, it can be determined whether the projection picture is smaller than the display area of ​​the projection screen based on the first three-dimensional information and the second three-dimensional information. When the projection picture is smaller than the display area of ​​the projection screen, if the size of the projection picture cannot be adjusted, or if the display effect of the current projection picture already meets the requirements, the projection picture can be displayed in the center of the display area of ​​the projection screen. Among them, the shape of the projection picture can be adjusted, for example, the shape of the projection picture can be adjusted based on trapezoidal correction to improve the display effect of the projection picture.

[0114] In another example, it is preferred to keep the ratio of the projected image unchanged to prevent the image content from being distorted. Fig.11 As shown, the projection screen 102 can be displayed in the center of the projection screen 1032, leaving a uniform blank area at the outer edge of the projection screen. If the projection screen is large, the edge of the screen can be appropriately cropped to ensure that the main content is still within the projection screen area.

[0115] In another example, the user may be more concerned about certain important content in the projection image. Intelligent analysis can be used to identify the main content area in the projection image, and the main content area can be prioritized to be aligned with the projection screen to ensure that the important information is fully displayed within the visible range.

[0116] In another example, if the projection image is smaller than the display area of ​​the projection screen, the first vertex coordinates of the projection image are adjusted, and the projection image is scaled according to the shape of the projection screen. For example, the aspect ratio of the projection image is determined according to the aspect ratio of the projection screen, and the first vertex coordinates of the projection image are adjusted based on the determined aspect ratio of the projection image, and the shape of the projection image can be adjusted based on the first vertex coordinates to improve the display effect of the projection image.

[0117] In one embodiment, if the projection screen includes multiple projection screens corresponding to multiple pan-tilt projection devices, the third three-dimensional information of each projection screen in the space where the multiple pan-tilt projection devices are located is determined. According to the third three-dimensional information and the second three-dimensional information, the corresponding projection screen is corrected using the pan-tilt of each pan-tilt projection device, so that the corrected multiple projection screens are spliced ​​and displayed on the projection screen. Among them, the range of the corresponding projection screen area can be allocated to each projection screen according to the importance of the screen content of each projection screen. The splicing position of adjacent projection screens can also be determined according to the degree of correlation between the screen contents of the projection screens.

[0118] For example Figure 12(a)-Figure 12(b) As shown, the multiple projection images include projection images A, B, C, and D. The corresponding projection images can be corrected according to actual needs, so that the corrected multiple projection images are spliced ​​and displayed on the projection screen.

[0119] Through the above embodiments, for multi-projection scene, the third 3D information of each projection image can be accurately obtained and compared with the second 3D information of the projection screen, and the projection image can be finely corrected by using the pan-tilt function of the pan-tilt projection device to ensure that multiple projection images are seamlessly spliced ​​on the projection screen. The projection area can also be intelligently allocated and the splicing position optimized according to the importance and relevance of the image content, thereby significantly improving the visual effect.

[0120] The embodiment of the present application provides a method for aligning a projection image with a screen, which can automatically and accurately identify the position of a screen during real-time projection by a projection device, and achieve precise alignment according to the position of the projection image and the position of the screen, thereby achieving alignment of the projection image with the screen without the user's perception, thereby improving the user experience.

[0121] It should be understood that the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, but these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0122] In some embodiments, the above-mentioned projection image alignment method can be implemented in the form of a computer program. The computer program can be used in Fig.13 The following is combined with the PTZ projection device shown in the figure. Fig.13 , the architecture of the hardware device for realizing the projection image screen matching method is introduced.

[0123] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0124] See also Fig.13 , which is a schematic diagram of the structure of the pan-tilt projection device provided in an embodiment of the present application.

[0125] The pan-tilt projection device 200 includes a projection unit 210 and a driving unit 220 for driving the projection unit 210. The projection unit 210 can form an optical image and project the optical image onto an imaging medium SC.

[0126] The projection unit 210 includes a light source unit 211 , a light modulator 212 , and an optical system 213 . The driving unit 220 includes a light source driving unit 221 and a light modulator driving unit 222 .

[0127] The light source unit 211 may include a solid light source such as a light emitting diode (LED), a laser, a pump lamp, etc. The light source unit 211 may include optical elements such as a lens and a polarizing plate for improving the optical characteristics of the projected light, and a dimming element for adjusting the light flux.

[0128] The light source driving unit 221 can control the light source in the light source unit 211 to operate, including lighting and extinguishing, according to the instruction of the control unit 250 .

[0129] The light modulator 212 includes a display panel 215 , which may be a transmissive liquid crystal display (LCD) panel, a reflective liquid crystal on silicon (LCOS) panel, or a digital micromirror device (DMD).

[0130] The light modulator 212 is driven by a light modulator driving unit 222 , and the light modulator driving unit 222 is connected to the image processing unit 245 .

[0131] The image processing unit 245 inputs image data to the light modulator driving unit 222. The light modulator driving unit 222 converts the input image data into a data signal suitable for the operation of the display panel 215. The light modulator driving unit 222 applies a voltage to each pixel of each display panel 215 according to the converted data signal, and draws an image on the display panel 215.

[0132] The optical system 213 includes a lens or a reflector for forming an image of the incident image light PLA on the imaging medium SC. The optical system 213 may also include a zoom mechanism for magnifying or reducing the image projected on the imaging medium SC and a focus adjustment mechanism for adjusting the focus.

[0133] The pan-tilt projection device 200 further includes an operation unit 231, a signal receiving unit 233, an input interface 235, a storage unit 237, a data interface 241, an interface unit 242, a frame memory 243, an image processing unit 245, and a control unit 250. The input interface 235, the storage unit 237, the data interface 241, the interface unit 242, the image processing unit 245, and the control unit 250 can communicate data with each other via the internal bus 207.

[0134] The operation unit 231 can generate corresponding operation signals according to the operation of various buttons and switches on the housing surface of the pan-tilt projection device 200, and output them to the input interface 235. The input interface 235 includes a circuit that outputs the operation signal input from the operation unit 231 to the control unit 250.

[0135] After receiving a signal (such as an infrared signal or a Bluetooth signal) sent from the control device 5 (such as a remote controller), the signal receiving unit 233 can decode the received signal to generate a corresponding operation signal. The signal receiving unit 233 outputs the generated operation signal to the input interface 235. The input interface 235 outputs the received operation signal to the control unit 250.

[0136] The storage unit 237 may be a magnetic recording device such as a hard disk drive (HDD) or a storage device using a semiconductor storage element such as a flash memory. The storage unit 237 stores programs executed by the control unit 250, data processed by the control unit 250, image data, and the like.

[0137] The data interface 241 includes a connector and an interface circuit, and can be wired to other electronic devices 100. The data interface 241 may be a communication interface for performing communication with other electronic devices 100. The data interface 241 receives sensor data, image data, sound data, etc. from other electronic devices 100. In this embodiment, the other electronic devices 100 may include a sensor device, the sensor data may be depth data, and the image data may be a content image.

[0138] The interface unit 242 is a communication interface for communicating with other electronic devices 100 according to the Ethernet standard. The interface unit 242 includes a connector and an interface circuit for processing a signal sent by the connector. The interface unit 242 is an interface substrate including a connector and an interface circuit and is connected to the main substrate of the control unit 250, which is a substrate on which a processor 253 and other components are mounted. The connector and the interface circuit constituting the interface unit 242 are mounted on the main substrate of the control unit 250. The interface unit 242 can receive setting information or instruction information sent by other electronic devices 100.

[0139] The control unit 250 includes a memory 251 and a processor 253. The control unit 250 issues instructions to the driving device 6, and the driving device 6 drives the pan-tilt projection device 200 to rotate in space in response to the instructions from the control unit 250.

[0140] The memory 251 is a storage device that stores programs and data executed by the processor 253 in a non-volatile manner. The memory 251 is composed of a semiconductor storage element such as a magnetic storage device, a flash read-only memory (ROM), or other types of non-volatile storage devices. The memory 251 may also include a random access memory (RAM) that constitutes a work area of ​​the processor 253. The memory 251 stores data processed by the control unit 250 and a control program executed by the processor 253.

[0141] The processor 253 may be composed of a single processor or a combination of multiple processing groups. The processor 253 executes a control program to control various parts of the pan-tilt projection device 200. For example, the processor 253 performs the corresponding interactive projection screen matching according to the operation signal generated by the operation unit 231, and outputs the parameters used in the interactive projection screen matching (such as parameters for performing trapezoidal correction on the image) to the image processing unit 245. In addition, the processor 253 can control the light source in the light source unit 211 to light up, turn off or adjust the brightness by controlling the light source driving unit 221.

[0142] The image processing unit 245 and the frame memory 243 may be composed of integrated circuits. Integrated circuits include large-scale integrated circuits (LSI), application-specific integrated circuits (ASIC), and programmable logic devices (PLD), wherein the PLD may include a field-programmable gate array (FPGA). Among them, the integrated circuit may also include a portion of an analog circuit, or a combination of a processor and an integrated circuit. The combination of a processor and an integrated circuit is called a microcontroller unit (MCU), a system on chip (SoC), a system LSI, a chipset, etc.

[0143] The image processing unit 245 may store the image data received from the data interface 241 in the frame memory 243. The frame memory 243 includes a plurality of memory banks, each of which includes a storage capacity that can write image data of one frame. The frame memory 243 may be composed of a synchronous dynamic random access memory (SDRAM) or a dynamic random access memory (DRAM).

[0144] The image processing unit 245 can perform interactive projection screen correction on the image data stored in the frame memory 243, including resolution conversion, size adjustment, distortion correction, shape correction, digital zoom, image tone adjustment, image brightness adjustment, etc.

[0145] The image processing unit 245 can also convert the input frame frequency of the vertical synchronization signal into a drawing frequency and generate a vertical synchronization signal having the drawing frequency. The generated vertical synchronization signal is called an output synchronization signal. The image processing unit 245 then outputs the output synchronization signal to the optical modulator driving unit 222.

[0146] In some embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program, and the computer program is loaded by a processor, so that the processor executes the steps of the above-mentioned data migration method. The steps of the data migration method here can be the steps of the projection screen alignment method in each of the above-mentioned embodiments.

[0147] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

[0148] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0149] Although the subject matter has been described in the language of specific method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A projection screen alignment method, applied to a pan-tilt projection device, characterized in that: include: Determine first three-dimensional information of the projection picture in space based on sensor data of the pan-tilt projection device and projection parameters corresponding to the projection picture; Determine second three-dimensional information of the projection screen in space according to the captured image of the projection screen and the sensor data; According to the first three-dimensional information and the second three-dimensional information, driving the pan-tilt projection device to rotate in space so that the projection picture is projected onto the projection screen; The projection picture is adjusted so that the projection picture and the projection screen are aligned with each other.

2. The projection screen alignment method according to claim 1, characterized in that: The step of driving the pan-tilt projection device to rotate in space so that the projection image is projected onto the projection screen comprises: The projection direction and / or projection angle of the pan-tilt projection device is adjusted by using the pan-tilt projection device so that the projection picture of the adjusted pan-tilt projection device covers the projection screen.

3. The projection screen alignment method according to claim 1, characterized in that: The method further comprises: Determining frame information of the projection screen according to the second three-dimensional information; If it is determined according to the frame information that the projection screen is an incomplete projection screen, the projection direction and / or projection angle of the pan-tilt projection device is adjusted using the pan-tilt platform so that the data collection range of the sensor device covers the complete projection screen.

4. The projection screen alignment method according to claim 1, characterized in that: Before adjusting the projection picture so that the projection picture and the projection screen are aligned, the method further includes: Determine a first ratio of a current projection distance to a current projection screen width; If the first ratio is less than a preset ratio, the position of the pan-tilt projection device is adjusted so that the current projection image covers the projection screen.

5. The projection screen alignment method according to claim 1, characterized in that: Before adjusting the projection picture so that the projection picture and the projection screen are aligned, the method further includes: Determine a first ratio of a current projection distance to a current projection screen width; If the first ratio is less than a preset ratio, the lens parameters of the pan-tilt projection device are adjusted so that the current projection image covers the projection screen.

6. The projection screen alignment method according to claim 1, characterized in that: The method further comprises: If the projection image is smaller than the display area of ​​the projection screen, the first vertex coordinates of the projection image are adjusted, and the projection image is scaled according to the shape of the projection screen.

7. The projection screen alignment method according to claim 1, characterized in that: The method further comprises: If the projection picture includes multiple projection pictures corresponding to multiple pan-tilt projection devices, determining third three-dimensional information of each projection picture in the space where the multiple pan-tilt projection devices are located; According to the third three-dimensional information and the second three-dimensional information, the corresponding projection picture is corrected using the pan-tilt of each pan-tilt projection device, so that the corrected multiple projection pictures are spliced ​​and displayed on the projection screen.

8. The projection screen alignment method according to claim 1, characterized in that: The step of adjusting the projection picture so that the projection picture and the projection screen are aligned with each other includes: The second vertex coordinates of the projection screen in the projector chip coordinate system are determined according to the first three-dimensional information and the second three-dimensional information, and the display area of ​​the projection image is determined according to the second vertex coordinates.

9. The projection screen alignment method according to claim 1, characterized in that: The picture adjustment includes picture edge correction, picture brightness adjustment and / or picture color temperature adjustment.

10. A pan-tilt projection device, characterized in that: The PTZ projection device comprises: one or more processors; A memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the projection screen method according to any one of claims 1 to 9.

Images