Projection device and projection picture splicing method

Abstract

This application relates to the field of projection equipment technology, providing a projection device and a method for stitching projected images. This method involves capturing images of a first image card projected onto a projection surface by a first optical engine based on first optical engine coordinates, and at least one second image card projected onto the projection surface by a second optical engine from the projection device based on second optical engine coordinates. This allows for the determination of the first and second projection coordinates on the projection surface, corresponding to the first and second optical engine coordinates, respectively. Then, based on the first and second projection coordinates, the stitched projected image corresponding to the intersection height of the first and second image cards is determined. Based on the stitched projected image, the first and second optical engine coordinates are adjusted. Furthermore, when the horizontal boundaries of the first and second image cards differ significantly, it is not necessary to adjust the positions of each projection device, thereby reducing the difficulty of stitching multiple projected images.

Description

Technical Field

[0001] This application belongs to the field of projection equipment technology, and in particular relates to a projection device and a method for splicing projected images. Background Technology

[0002] When using multiple projection devices, it is usually necessary to merge the multiple projected images so that they can be stitched together into a complete image displayed on the screen.

[0003] Currently, when blending projected images, the top and bottom horizontal boundaries of the two images are usually aligned to achieve image stitching. However, when the initial projected images have a significant height difference, the height adjustment range of the optical engine coordinates within the projector is limited. Therefore, it is necessary to move the projector or adjust it by rotating a pan-tilt platform to reduce the height difference of the initial projected images before performing image blending.

[0004] However, the above methods require adjusting the actual position of the projection device first, or using external equipment, and then making secondary adjustments to the optical engine coordinates. In this case, adjusting in two dimensions will increase the difficulty of stitching the projected image. Summary of the Invention

[0005] This application provides a projection device and a method for splicing projected images, which can solve the problem of difficulty in splicing projected images when the heights of the initially projected images differ significantly.

[0006] In a first aspect, embodiments of this application provide a projection device, including:

[0007] The optical engine is configured to project the image onto the projection surface;

[0008] The camera is configured to capture images of a first image card projected onto a projection surface by a first optical engine in the projection device, and at least one second image card projected onto a projection surface by a second optical engine in the projection device.

[0009] The controller is configured as follows:

[0010] Obtain the first and second picture cards;

[0011] Determine the first optical engine coordinates of the first optical engine on the first projection coordinates of the projection plane;

[0012] Based on the first and second map cards, determine the second optical engine coordinates of the second optical engine on the projection plane and the second projection coordinates of the second optical engine.

[0013] Based on the first projection coordinates and the second projection coordinates, the stitched projection image is determined; the height of the image in the stitched projection image is the intersection height of the first and second image cards;

[0014] Based on the spliced ​​projection image, the first optical engine coordinates and the second optical engine coordinates are adjusted respectively to obtain the third optical engine coordinates corresponding to the first optical engine and the fourth optical engine coordinates corresponding to the second optical engine;

[0015] Among them, the third optical-mechanical coordinate is used as the new optical-mechanical coordinate of the first optical-mechanical system, and the fourth optical-mechanical coordinate is used as the new optical-mechanical coordinate of the second optical-mechanical system.

[0016] Secondly, embodiments of this application provide a method for splicing projected images, the method comprising:

[0017] Obtain a first image card and a second image card; the first image card is an image card projected onto the projection surface by a first optical engine in the projection device, and the second image card is at least one image card projected onto the projection surface by a second optical engine in the projection device;

[0018] Determine the first optical engine coordinates of the first optical engine on the first projection coordinates of the projection plane;

[0019] Based on the first and second map cards, determine the second optical engine coordinates of the second optical engine on the projection plane and the second projection coordinates of the second optical engine.

[0020] Based on the first projection coordinates and the second projection coordinates, the stitched projection image is determined; the height of the image in the stitched projection image is the intersection height of the first and second image cards;

[0021] Based on the spliced ​​projection image, the first optical engine coordinates and the second optical engine coordinates are adjusted respectively to obtain the third optical engine coordinates corresponding to the first optical engine and the fourth optical engine coordinates corresponding to the second optical engine;

[0022] Among them, the third optical-mechanical coordinate is used as the new optical-mechanical coordinate of the first optical-mechanical system, and the fourth optical-mechanical coordinate is used as the new optical-mechanical coordinate of the second optical-mechanical system.

[0023] Thirdly, embodiments of this application provide another projection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0025] Fifthly, embodiments of this application provide a computer program product that, when run on a projection device, causes the projection device to execute the method described in the first aspect.

[0026] The beneficial effects of the embodiments of this application compared with the prior art are:

[0027] In the above technical solution, a first image card projected onto the projection surface by a first optical engine based on its first optical engine coordinates, and at least one second image card projected onto the projection surface by a second optical engine from a projection device based on its second optical engine coordinates, are captured by a camera. This allows for the determination of the first projection coordinates of the first optical engine on the projection surface, and the second projection coordinates of the second optical engine on the projection surface, based on the first and second image cards. Then, a stitched projection image is determined based on the first and second projection coordinates. In this stitched projection image, the image height is the intersection height of the first and second image cards. Therefore, even when the horizontal boundaries of the first and second image cards differ significantly, it is not necessary to adjust the positions of each projection device. Furthermore, after adjusting the first and second optical engine coordinates based on the stitched projection image, the horizontal boundary of the projection image projected by the projection device based on the adjusted third optical engine coordinates can coincide as closely as possible with the horizontal boundary of the projection image projected by the projection device based on the adjusted fourth optical engine coordinates. This reduces the difficulty of stitching multiple projection images.

[0028] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 The following are schematic diagrams illustrating projection scenarios of projection devices in some embodiments of this application;

[0031] Figure 2 The optical path of the projection device in some embodiments of this application is shown in the diagram;

[0032] Figure 3 This application shows schematic diagrams of the lens structure of a projection device provided in some embodiments;

[0033] Figure 4This application shows schematic diagrams of the distance sensor and image acquisition device structure of a projection device provided in some embodiments;

[0034] Figure 5 A system block diagram of a projection device provided in some embodiments of this application is shown;

[0035] Figure 6 This application illustrates a schematic diagram of an application scenario where a projection device, according to some embodiments of the present application, performs projection image splicing.

[0036] Figure 7 The timing interaction diagram of a projection screen splicing method in a projection device in some embodiments of this application is shown;

[0037] Figure 8 This application shows a schematic diagram illustrating an application scenario of the first image card in the projection device for splicing projected images according to some embodiments of the present application;

[0038] Figure 9 This application shows a schematic diagram illustrating an application scenario of the second card in the projection device for splicing projected images, according to some embodiments of the present application.

[0039] Figure 10 This application illustrates a schematic diagram of an application scenario where a projection device, according to some embodiments of the present application, splices projection images.

[0040] Figure 11 This application illustrates a schematic diagram of an application scenario where a projection device, according to some embodiments of the present application, implements another spliced ​​projection image in the splicing of projection images;

[0041] Figure 12 This application illustrates a schematic diagram of an application scenario where a projection device determines a target splicing method in the process of splicing projected images, according to some embodiments of this application.

[0042] Figure 13 This application illustrates a schematic diagram of an application scenario where a projection device, according to some embodiments of the present application, performs a further splicing of projected images.

[0043] Figure 14 This application illustrates a schematic diagram of an application scenario where a projection device implements another spliced ​​projection screen in the splicing of projection screens according to some embodiments of the present application;

[0044] Figure 15 The timing interaction diagram of another projection screen splicing method in a projection device in some embodiments of this application is shown;

[0045] Figure 16 This application illustrates a schematic diagram of an application scenario where a projection device implements a fine grid in projected image stitching, according to some embodiments of the present application.

[0046] Figure 17The timing interaction diagrams of various components in the main projection device and the slave projection device in some embodiments of this application are shown. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0048] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0049] Specific details, such as particular system architectures and techniques, are set forth for illustrative purposes and not for limitation, to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted to avoid unnecessary detail that could obscure the description of this application.

[0050] The embodiments of this application can be applied to various types of projection devices. A projection device is a device that can project images or videos onto a screen. A projection device can project laser light of a specific color onto a screen to form a specific image through the refraction of optical components. During the projection process, a certain distance can be maintained between the projection device and the screen so that the image formed on the screen conforms to the focal length range of the optical components, thereby obtaining a clear image. The following will use a projector as an example to describe the projection device and projection control method.

[0051] Taking a projector as an example, a projector can be connected to computers, cable TV networks, the internet, VCD (Video Compact Disc), DVD (Digital Versatile Disc Recordable), game consoles, DV camcorders, etc., through different interfaces to play corresponding video signals. Projectors are widely used in homes, offices, schools, and entertainment venues.

[0052] Figure 1The following are schematic diagrams illustrating projection scenarios of projection devices in some embodiments of this application. Figure 2 A schematic diagram of the optical path of a projection device in some embodiments of this application is shown.

[0053] like Figure 1-2 As shown, the projection scenario provided in this application includes a projection screen 1 and a projection device 2. The projection screen 1 is fixed at a first position on the projection surface. For example, a wall can be used as the projection surface, and a screen can be fixed at the first position on the projection surface. The projection device 2 is placed at a second position so that the image it projects matches the projection screen 1. The projection device 2 includes a light source 100, an optical engine 200, and a lens 300. The light source 100 provides illumination for the optical engine 200, which modulates the light beam and outputs it to the lens 300 for imaging, projecting it onto the projection surface 400 to form a projected image. Since the light source 100, the optical engine 200, and the lens 300 are used together to emit projection light to project the projected image, in some embodiments of this application, the light source 100, the optical engine 200, and the lens 300 are collectively referred to as the light-emitting assembly.

[0054] In some embodiments, the light source 100 of the projection device 2 includes a light emitter assembly 110 and an optical lens assembly 120. The light beam emitted by the light emitter assembly 110 can pass through the optical lens assembly 120 to provide illumination for the optical engine 200.

[0055] It should be noted that in the embodiments of this application, the light source 100 can be a laser light source, an LED light source, or a Liquid Crystal Display (LCD) light source; this application does not specifically limit this. For example, taking the light source 100 as a laser light source, in some embodiments, the optical engine 200 of the projection device 2 can be implemented to include a blue optical engine, a green optical engine, a red optical engine, and may also include a heat dissipation system, a circuit control system, etc.

[0056] Figure 3 A schematic diagram of the lens structure of a projection device provided in some embodiments of this application is shown.

[0057] To support the automatic focusing process of projection device 2, such as Figure 3 As shown, the lens 300 of the projection device 2 may also include an optical component 310 and a drive motor 320. The optical component 310 is a lens group consisting of one or more lenses, which can refract the light emitted by the optical engine 200, so that the light emitted by the optical engine 200 can be transmitted onto the projection surface 400 to form a transmitted content image.

[0058] The optical assembly 310 may include a lens barrel and multiple lenses disposed within the lens barrel. Depending on whether the lenses are movable, the lenses in the optical assembly 310 can be divided into movable lenses 311 and fixed lenses 312. By changing the position of the movable lens 311, the distance between the movable lens 311 and the fixed lens 312 is adjusted, thereby changing the overall focal length of the optical assembly 310. Therefore, the drive motor 320 can connect to the movable lens 311 in the optical assembly 310, driving the movable lens 311 to move its position, thus achieving an automatic focusing function.

[0059] It should be noted that the focusing process described in some embodiments of this application refers to changing the position of the moving lens 311 by driving the motor 320, thereby adjusting the distance between the moving lens 311 and the fixed lens 312, that is, adjusting the image plane position. Therefore, in the imaging principle of the lens combination in the optical component 310, the adjustment of focal length is actually the adjustment of image distance. However, in terms of the overall structure of the optical component 310, adjusting the position of the moving lens 311 is equivalent to adjusting the overall focal length of the optical component 310.

[0060] When the distance between the projection device 2 and the projection surface 400 varies, the lens of the projection device 2 needs to be adjusted to different focal lengths to project a clear image onto the projection surface 400. During projection, the distance between the projection device 2 and the projection surface 400 will vary depending on the user's placement, requiring different focal lengths. Therefore, to adapt to different usage scenarios, the projection device 2 needs to adjust the focal length of the optical component 310.

[0061] Figure 4 The diagram shows a schematic representation of the distance sensor and image acquisition device of a projection device provided in some embodiments of this application.

[0062] like Figure 4As shown, the projection device 2 can also have a built-in or external image acquisition device 700. The image acquisition device 700 can be a camera, capable of capturing images of the screen projected by the projection device 2 to obtain the projected content image. The projection device determines whether the current lens focal length is suitable by performing a sharpness detection on the projected content image, and adjusts the focal length if it is not suitable. When automatically focusing based on the projected content image captured by the image acquisition device 700, the projection device can continuously adjust the lens position and take pictures, and find the focus position by comparing the sharpness of the images before and after, thereby adjusting the moving lens 311 in the optical component to a suitable position. For example, the controller 500 can first control the drive motor 320 to gradually move the moving lens 311 from the focusing starting position to the focusing ending position, continuously acquiring projected content images through the camera during this period. Then, by performing a sharpness detection on multiple projected content images, the position with the highest sharpness is determined, and finally the drive motor 320 is controlled to adjust the moving lens 311 from the focusing end position to the position with the highest sharpness, completing the automatic focusing.

[0063] The distance sensor 600 can be a time-of-flight (TOF) based sensor device such as lidar or infrared radar, capable of detecting target distances. It can detect the distance between the projection surface 400 and the optical engine 200. The distance sensor 600 can be positioned at the optical engine 200, including a signal transmitter and a receiver. During distance detection, the transmitter of the distance sensor 600 can emit a wireless signal towards the projection surface. After contacting the projection surface, the wireless signal is reflected back to the receiver of the distance sensor 600. The signal flight time is calculated based on the time it takes for the transmitter to emit the signal and the time it takes for the receiver to receive the signal. Combined with the flight speed, the actual flight distance of the wireless signal can be obtained.

[0064] Figure 5 A system block diagram of a projection device provided in some embodiments of this application is shown.

[0065] like Figure 5 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0066] In some embodiments, at least one application runs in the application layer. These applications can be Windows programs included with the operating system, system settings programs, or clock programs, etc. For example, a media center that can interact with the user. Applications can also be applications developed by third-party developers. For example, a fusion application that can interact with a fusion application from a projection device. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0067] The application includes, but is not limited to, on / off control functions for controlling the operation of the projection device, attitude detection functions for detecting the attitude of the projection device, device networking functions for networking with the projection device, geometric correction functions for adjusting the projection graphics card, configuration graphics functions for configuring the projection graphics card, splicing control functions for controlling the splicing of the projected image, graphics card display functions for displaying the graphics card, an interactive user interface, application communication functions for communicating with the projection device, progress blending functions for displaying the projected image blending progress, and playback control functions for controlling the projected image.

[0068] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0069] like Figure 5 As shown, the framework layer provides the application programming interface (API) and programming framework for the application. The application framework layer includes some predefined functions. The application framework layer acts as a processing center, determining the actions taken by the applications in the application layer. Through the API interface, applications can access system resources and obtain system services during execution. Figure 5 As shown, the services provided by the framework layer include, but are not limited to, the Concerto service for communication, the Power Manager for power management, the Input Manager for managing input information, the UK Player for providing audio and video playback, the algorithm service for focusing the projected image and calculating and correcting the various optical engine coordinates and projection coordinates when stitching the projected image, and the OSD blending service for achieving projection image blending.

[0070] In some embodiments, middleware is used to connect various parts of the framework layer and the driver layer or different applications to achieve data forwarding. For example, the data includes, but is not limited to, optomechanical parameters and driver parameters during geometric correction. Optomechanical parameters can be optomechanical coordinates, and driver parameters can be motor parameters.

[0071] In some embodiments, the driver layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 5 As shown, the driver layer can be configured with hardware drivers, and the drivers included in the driver layer can be at least one of the following: DLP optical engine driver, motor driver, camera driver, time of flight (TOF) sensor driver, inertial measurement unit (IMU) driver, etc.

[0072] The interaction between the layers can be as follows: Figure 5 The interactive arrows shown are not described in detail here.

[0073] In some embodiments, the projection device can be used individually as described in the examples above, or it can be used in conjunction with other projection devices to project onto a single screen, with the projected images stitched together to form a complete, high-resolution projected image.

[0074] However, when using multiple projection devices, the different positions of each device result in variations in their projection angles and distances from the projection surface, leading to significant differences in the initial height of the projected images. In this situation, because the optical engine coordinates of each projection device have limited adjustment range, it is impossible to achieve perfect alignment of the upper and lower horizontal boundaries of the projected image without adjusting the device's position or projection angle.

[0075] For example, refer to Figure 6 , Figure 6 The illustration shows application scenarios for projected image splicing using projection devices according to some embodiments of this application. Specifically, when the height h of the projected images differs significantly, due to the limited adjustment range of the optical engine coordinates, without adjusting the position or projection angle of the projection device, it will be impossible to make boundary a and boundary b coincide, thus failing to correct the projected image.

[0076] However, if the projection angle of the projector is adjusted by rotating the cloud platform to reduce the height h before stitching the projected images, external equipment is required. Furthermore, if the position of the projector is adjusted, followed by a secondary adjustment of the optical engine coordinates, these two adjustments will increase the difficulty of stitching the projected images.

[0077] Based on this, in order to achieve overlap of the upper and lower horizontal boundaries of multiple projected images without moving the projection equipment and reducing the difficulty of splicing projected images, embodiments of this application provide a projection equipment and a method for splicing projected images. This method first captures a first image card projected onto the projection surface by a first optical engine based on its first optical engine coordinates, and at least one second image card projected onto the projection surface by a second optical engine from the projection equipment based on its second optical engine coordinates. Thus, the first projection coordinates of the first optical engine on the projection surface are determined, and the second projection coordinates of the second optical engine on the projection surface are determined based on the first and second image cards. Based on the first and second projection coordinates, the spliced ​​projected image is determined. In this spliced ​​projected image, the image height will be the intersection height of the first and second image cards. Therefore, even when the upper and lower horizontal boundaries of the first and second image cards differ significantly, it is not necessary to adjust the positions of the various projection equipment. Furthermore, after adjusting the first and second optical engine coordinates based on the spliced ​​projection images, the horizontal boundary of the projection image when the projection device projects based on the adjusted third optical engine coordinates can coincide as closely as possible with the horizontal boundary of the projection image when the projection device projects based on the adjusted fourth optical engine coordinates. This reduces the difficulty of splicing multiple projection images.

[0078] To facilitate a further understanding of the technical solutions in some embodiments of this application, the technical solutions of the projection device and the projection screen splicing method, and how these technical solutions solve the aforementioned technical problems, are described in detail below with reference to some specific embodiments and accompanying drawings. The embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0079] The projection device includes an optical engine, a camera, and a main controller, with the main controller connected to both the camera and the optical engine. The optical engine is configured to project a pattern card onto a projection surface. The camera is configured to capture images of a first pattern card projected onto the projection surface by a first optical engine in the projection device, and at least one second pattern card projected onto the projection surface by a second optical engine in the projection device.

[0080] Figure 7 This application illustrates a timing interaction diagram of a projection screen splicing method in a projection device according to some embodiments, such as... Figure 7 As shown, the controller is configured to perform the following steps:

[0081] S710, obtain the first and second map cards.

[0082] The remaining projection devices also include structures such as optical engines, cameras, and controllers. To distinguish them from the projection devices that perform the projection image stitching method, the optical engine of the aforementioned projection devices can be considered a second optical engine, the camera a slave camera, and the controller a slave controller. Furthermore, the projection device that performs the projection image stitching method can be considered a main projection device, the optical engine of the main projection device can be considered a first optical engine, the camera a main camera, and the controller a main controller.

[0083] In some embodiments, the projection surface can be considered as the projection screen described above, the first optical engine is the optical engine in the projection device, and the image card projected by the first optical engine based on the current first optical engine coordinates can be considered as the first image card described above. After the camera captures a picture of the first image card, it can send the first image card to the main controller.

[0084] The first map can be a map without any markers or a map with a first preset marker; there is no limitation on this. For example, refer to... Figure 8 , Figure 8 This illustration shows an application scenario of the first image card in the projection device for splicing projected images according to some embodiments of this application. The aforementioned black area can be considered as a first preset marker point. The number of first preset marker points can be one or more; and the first preset marker points can be marker points marked with a special color or a special shape, without limitation.

[0085] In some embodiments, the main controller can adjust the projection angle of the first optical engine to its maximum angle so that the first optical engine can project the first image card with the largest possible range, thereby maximizing the size of the stitched projection image obtained subsequently. In this case, the optical engine coordinates corresponding to the maximum angle in the first optical engine are the aforementioned first optical engine coordinates.

[0086] It is understood that when the camera captures an image of the first image card, the resulting image includes at least the first image card. In another embodiment, the image may also include a projection screen. In this embodiment, the captured image is used as the first image card for illustration.

[0087] In some embodiments, similar to the first image card, the second image card is an image card captured by a camera and projected from the second optical engine in the projection device based on the current second optical engine coordinates. The structure of the secondary projection device can be the same as that of the primary projection device. Furthermore, the main controller can also send a maximum projection command to the second optical engine in the secondary projection device, instructing the second optical engine to adjust its projection angle to the maximum angle, enabling the second optical engine to project the maximum range of the second image card, thereby maximizing the size of the subsequently stitched projection image. In this case, the optical engine coordinates corresponding to the maximum angle in the second optical engine are the second optical engine coordinates.

[0088] It should be noted that when stitching together the projected images from multiple projection devices, one of the projection devices can be selected as the master projection device to execute the above-described image stitching method. In this case, the other projection devices can be considered as slave projection devices, used to respond to the control commands of the master projection device to stitch the projected images.

[0089] As an example, a projection device can respond to a user's first setup command to determine itself as the master projection device for executing the projected image stitching method. At this time, the master projection device can send slave device setup commands to the other projection devices to indicate that the other projection devices are slave projection devices and respond to the control commands of the master projection device.

[0090] In another embodiment, when multiple projection devices are stitching together a screen, these devices need to be connected to the same local area network (LAN) for communication. Each projection device can then discover itself using its own configured stitching application. For any target projection device, upon discovering other projection devices, the target projection device can close the stitching application client to prevent other projection devices from still being able to discover it. The main controller can then designate the target projection device as the primary projection device and the discovered projection devices as secondary projection devices.

[0091] Alternatively, the target projection device and the discovered projection device can be used for device position calibration. When the main controller of the target projection device determines that the target projection device is to the left of the discovered projection device, the target projection device is designated as the main projection device, and the discovered projection device is designated as the slave projection device. Conversely, when the main controller of the target projection device determines that the target projection device is to the right of the discovered projection device, the discovered projection device is designated as the executing main projection device, and the target projection device is designated as the slave projection device. In this embodiment, the method of determining the main projection device and the slave projection device from multiple projection devices is not limited.

[0092] Once the main and secondary projection devices are identified, they can communicate with each other via Concerto. Furthermore, the main controller in the main projection device uses corresponding control commands to control the main and secondary projection devices to stitch the projected images together.

[0093] Understandably, when it's necessary to photograph the second image card, the main controller can first send a projection command to the second optical engine of the projection device to control the second optical engine to project the second image card onto the projection surface. Then, after detecting that the second optical engine has completed the projection command, the slave controller can send a message indicating that projection is complete to the main controller. Finally, the main controller can generate a shooting command to the camera, instructing the camera to photograph the second image card.

[0094] In some embodiments, when the main controller sends various commands to the second optical engine and the slave camera, it can first send a slave control command. Then, the slave control command controls the corresponding second optical engine and slave camera to respond to the command.

[0095] It should be noted that before photographing the first and second image cards, both the main projection device and the slave projection device should first perform a seamless focusing process to ensure that the images of the first and second image cards projected are clear.

[0096] The second map may be similar to the first map, which will not be described in detail. For example, the second map may also be a map with a second preset marker point.

[0097] It should be noted that since both the first and second maps are projected onto the projection surface and need to be captured by the camera of the projection device, in order to distinguish between the first and second maps, they can be maps with different marker points. For example, refer to... Figure 9 , Figure 9 This illustration shows an application scenario of the second image card in the projection device for projected image splicing according to some embodiments of this application. The black area mentioned above can be considered as the second preset marker point. (Refer to...) Figure 8 and Figure 9 It can be seen that the first preset marker point and the second preset marker point are different.

[0098] In another embodiment, to further facilitate the differentiation between the first and second image cards, and to subsequently establish a mapping relationship between the first and second optical engines based on the first and second image cards, the main controller can control the projection device to turn off its screen when the camera captures the first image card. That is, the captured first image card will only contain the first image card and not the second image card. Similarly, when the camera captures the second image card, the main controller can control the projection device to turn off its screen. That is, the captured second image card will only contain the second image card and not the first image card.

[0099] S720. Determine the first optical engine coordinates of the first optical engine on the first projection coordinates of the projection plane.

[0100] In some embodiments, the aforementioned first optical engine coordinates are used to describe the display area when the first optical engine projects the first drawing card onto the projection surface. For example, the aforementioned first optical engine coordinates can be multiple first optical engine corner point coordinates of the first optical engine. When the first optical engine projects based on the first optical engine corner point coordinates, the first corner point coordinates of the first drawing card on the projection surface correspond one-to-one with the first optical engine corner point coordinates.

[0101] Based on the above explanation, it can be understood that when it is necessary to stitch together the projected image of the first optical engine, the coordinates of the first optical engine (e.g., the coordinates of the first optical engine corner point) are adjusted so that when the first optical engine projects again based on the adjusted first optical engine coordinates, the horizontal boundary of the projected image can coincide with the horizontal boundary of the projected image from the projection device.

[0102] In another embodiment, the aforementioned first optical engine coordinates can also be the coordinates of a first center point, and the relative positional relationships between the first center point coordinates and each of the first optical engine corner points. In this case, the first center point coordinates and their relative positional relationships can be considered to describe the display area when the first optical engine projects the first card onto the projection surface. Adjusting the first optical engine coordinates can then be considered as adjusting the position of the first center point coordinates and their relative positional relationships.

[0103] In this embodiment, the method of setting the first optomechanical coordinates is not limited. For ease of explanation, the first optomechanical coordinates can be considered as the coordinates of the four first optomechanical corner points.

[0104] Here, the first optical engine coordinates are the coordinates in the first optical engine coordinate system, and the first projection coordinates corresponding to the projection onto the projection surface are the coordinates in the projection surface coordinate system. Based on this, the main controller can map the first optical engine coordinates according to the preset mapping relationship between the first optical engine and the projection surface to obtain the first projection coordinates corresponding to each first optical engine corner point coordinate.

[0105] In another embodiment, the main controller may also determine the first projection coordinates in response to user input, without limitation.

[0106] It should be noted that the purpose of performing the projection image stitching method is to ensure that the horizontal boundaries of the multiple projected images ultimately projected onto the projection surface coincide. Therefore, performing stitching based on the first projection coordinates on the projection surface makes the stitching effect of multiple projected images more intuitive.

[0107] S730. Based on the first and second map cards, determine the second optical engine coordinates of the second optical engine on the projection plane.

[0108] In some embodiments, the main controller can establish a first mapping relationship between the first and second optomechanical coordinate systems based on the positions of the first preset marker points in the first map and the first optical engine, and the positions of the second preset marker points in the second map and the second optical engine, respectively. Then, the second optomechanical coordinates are mapped based on the first mapping relationship to obtain the second projected coordinates.

[0109] In some embodiments, since the first preset marker point is pre-set in the first optical engine, the first optical engine position of the first preset marker point in the first optical engine coordinate system can be considered known. Furthermore, since the first preset marker point is a marker point on the first image card, the first image position of the first marker point in the camera coordinate system can also be determined when the camera captures the first image card.

[0110] For example, the main controller can use an image recognition algorithm to identify the first image card and determine the position of the first marker point in the first image card. Then, based on the positions of the first preset marker point in the first image card and the first optical engine, the main controller can directly establish a second mapping relationship between the camera coordinate system and the first optical engine coordinate system of the projection device. That is, the main controller can establish a second mapping relationship between the camera coordinate system and the first optical engine coordinate system according to the position of the first optical engine and the position of the first image.

[0111] Similar to the second mapping relationship established above, the main controller can establish a third mapping relationship between the camera coordinate system and the second optical engine coordinate system based on the positions of the second preset marker points in the second map card and the second optical engine, respectively. This will not be described in detail.

[0112] The second preset marker point, located at the second optical engine position within the second optical engine, can be a position pre-set within the second optical engine. Therefore, the projection device can send the second optical engine position to the main controller.

[0113] It is understandable that the second mapping relationship is the mapping relationship between the camera coordinate system of the main projection device and the first optical-mechanical coordinate system, and the third mapping relationship is the mapping relationship between the camera coordinate system of the main projection device and the second optical-mechanical coordinate system. Based on this, the first mapping relationship between the first and second optical-mechanical coordinate systems can be obtained according to the first and second mapping relationships.

[0114] Based on this, the main controller can first determine the position of the second optomechanical coordinates in the first optomechanical coordinate system based on the first mapping relationship. Then, based on the preset mapping relationship between the first optomechanical coordinate system and the projection surface, the mapped optomechanical coordinate positions are mapped again to obtain the second projection coordinates corresponding to the second optomechanical coordinates.

[0115] In another embodiment, the projection device may also determine the second projection coordinates of the second optical-mechanical coordinates on the projection surface based on the mapping relationship between itself and the projection surface, and then send the second projection coordinates to the main projection device. In this embodiment, the method of determining the second projection coordinates corresponding to the second optical-mechanical coordinates is not limited.

[0116] S740. Based on the first projection coordinates and the second projection coordinates, determine the spliced ​​projection screen; the screen height in the spliced ​​projection screen is the intersection height of the first and second graphics cards.

[0117] In some embodiments, the spliced ​​projection image can be the image corresponding to the first intersection area of ​​the first and second graphics cards, or the area covered by the intersection height of the first and second graphics cards.

[0118] For example, refer to Figure 10 and Figure 11 , Figure 10 This application illustrates schematic diagrams of application scenarios for splicing projected images using projection devices according to some embodiments of the present application; and, Figure 11 This illustration shows an application scenario of a projection device implementing another spliced ​​projection screen in some embodiments of this application.

[0119] Among them, such as Figure 10 As shown, Figure 10 Region 1 in the diagram represents the first intersection area between the first and second graphics cards, which is the spliced ​​projection image. And, as... Figure 11 As shown, Figure 11 Region 2 in the diagram is the area covered by the intersection height of the first and second diagrams.

[0120] It is understandable that regardless of the type of spliced ​​projection image, its corresponding height is always the intersection height of the first and second graphics cards. Therefore, when adjusting the first and second optical engine coordinates based on the spliced ​​projection image corresponding to the intersection height, the horizontal boundaries of the projected images can be made to overlap as much as possible when the first and second optical engines project based on the adjusted optical engine coordinates. Thus, even when the height adjustment range of the optical engine coordinates is limited, splicing of the projected images can still be achieved.

[0121] In some embodiments, since there are multiple possible spliced ​​projection images generated by image stitching, the main controller can also respond to the user's stitching command to determine the target stitching method. Then, the spliced ​​projection image is generated based on the determined target stitching method.

[0122] As an example, the main controller can be based on, for example... Figure 7 Steps S7401-S7403 shown determine the stitched projection image. Details are as follows:

[0123] S7401 determines the target splicing method in response to the user's splicing command.

[0124] The aforementioned splicing instructions can be generated after the user makes a selection on the display device. For example, the main projection device can display multiple preset splicing methods on the display interface of the main projection device via its output interface. Then, in response to the user's selection, the main projection device can generate a splicing instruction corresponding to the target splicing method and send it to the main projection device.

[0125] In another embodiment, the main projection device may also include a display screen for interacting with the user. For example, it may receive and respond to splicing instructions to determine the target splicing method.

[0126] For example, refer to Figure 12 , Figure 12 This illustration shows an application scenario diagram of determining the target splicing method in the projection screen splicing process of some embodiments of this application. The display device or display screen can display, for example... Figure 12 The selection window shown allows users to choose between a combined or stacked layout as the target splicing method.

[0127] S7402. If the target splicing method is a stacking method, then the first intersection area between the first and second image cards is determined based on the first projection coordinates and the second projection coordinates, and the first intersection area is determined as the spliced ​​projection image.

[0128] In some embodiments, when the target splicing method is determined to be a stacking method, the main controller can determine the coordinates of each projection corner point in the first intersection area based on the first projection coordinates and the second projection coordinates, so as to describe the spliced ​​projection screen using the coordinates of each projection corner point.

[0129] For example, refer to Figure 13 , Figure 13 The illustration shows a schematic diagram illustrating an application scenario where a projection device, according to some embodiments of this application, performs a further splicing of projected images. Among them, Figure 13 The region enclosed by A1, B1, C1, and D1 can be considered as the first map projected by the first optical engine onto the projection surface, and the region enclosed by A2, B2, C2, and D2 can be considered as the second map projected by the second optical engine onto the projection surface. The first intersection region can be considered as the region enclosed by A2, B3, C1, and D3.

[0130] In this embodiment, taking the first optical-mechanical coordinates as the coordinates of the four first optical-mechanical corner points as an example, the corresponding first projection coordinates will also be the coordinates of the four first projection corner points of the first map (i.e., A1, B1, C1, D1). At this time, when determining the coordinates of each projection corner point in the first intersection region, the main controller can use A2 and C1 as the coordinates of two projection corner points in the first intersection region. Then, the ordinate of the second projection corner point coordinates corresponding to A2 can be subtracted from the intersection height to obtain the ordinate of the projection corner point coordinates corresponding to D3, and the abscissa of the second projection corner point coordinates corresponding to A2 can be determined as the abscissa of the projection corner point coordinates corresponding to D3. The coordinates of the B3 projection corner point in the first intersection region can also be determined in the same way, which will not be described in detail here.

[0131] Based on the above explanation, once the coordinates of each projection corner point in the first intersection region are obtained, the projected stitched image can be determined.

[0132] The intersection height can be determined based on the ordinates of the first and second projected coordinates corresponding to the two intersecting boundaries in the first and second maps, respectively. (Refer to...) Figure 13 We can consider the two intersecting boundaries as C1D1 and A2B2. The main controller can calculate the difference between the ordinates in C1 and A2, and use this difference as the intersection height in the calculation.

[0133] S7403. If the target splicing method is a combined splicing method, the first projected image corresponding to the intersection height in the first image card is determined based on the first projection coordinates, and the second projected image corresponding to the intersection height in the second image card is determined based on the second projection coordinates, and the first coverage area of ​​the first projected image and the second projected image is determined as the spliced ​​projected image.

[0134] Based on the above explanation of determining the first intersection area as the projected stitched image, refer to... Figure 13 The first projection surface can be considered as the area enclosed by the coordinates of projection corner points A3, B3, C1, and D1, and the second projection surface can be considered as the area enclosed by the coordinates of A2, B2, C3, and D3. Therefore, the first coverage area of ​​the first and second projection images can be considered as the area enclosed by the coordinates of projection corner points A3, B2, C3, and D1.

[0135] When determining the coordinates of the projection corner point corresponding to A3, the main controller can subtract the ordinate of the first projection corner point corresponding to D1 from the intersection height to obtain the ordinate of A3, and then determine the abscissa of the first projection corner point corresponding to D1 as the abscissa of A3. The coordinates of the remaining projection corner points in the first and second projection images can also be determined in the same way, which will not be explained in detail here.

[0136] Based on the above explanation, after obtaining the coordinates of each projection corner point corresponding to the first and second projection images respectively, the above-mentioned projection splicing image can be determined.

[0137] S750: Based on the spliced ​​projection image, the first optical engine coordinates and the second optical engine coordinates are adjusted to obtain the third optical engine coordinates corresponding to the first optical engine and the fourth optical engine coordinates corresponding to the second optical engine.

[0138] In some embodiments, the third optical engine coordinates are used as the new optical engine coordinates for the first optical engine, and the fourth optical engine coordinates are used as the new optical engine coordinates for the second optical engine. The purpose of adjusting the first and second optical engine coordinates based on the stitched projection image is to enable the projection image of the first optical engine based on the adjusted third optical engine coordinates to be stitched together with the projection image of the second optical engine based on the adjusted third optical engine coordinates to form the aforementioned stitched projection image.

[0139] As an example, the main controller can first determine the first projection screen corresponding to the projection device and the second projection screen corresponding to the projection device from the spliced ​​projection screen. Then, it determines the third optical engine coordinates based on the third projection coordinates corresponding to the first projection screen and sets the third optical engine coordinates as the optical engine coordinates of the first optical engine; and it determines the fourth optical engine coordinates based on the fourth projection coordinates corresponding to the second projection screen and sets the fourth optical engine coordinates as the optical engine coordinates of the second optical engine.

[0140] Specifically, when the target splicing method is a stacking method, the spliced ​​projection image can be considered as the first projection image corresponding to the projection device and the second projection image corresponding to the projection device. Furthermore, when the target splicing method is a combined splicing method, as explained in step S704 above, the spliced ​​projection image corresponding to the combined splicing method is the first coverage area of ​​the first projection image and the second projection image.

[0141] In some embodiments, the third projection coordinates describe the display area of ​​the first projection image on the projection surface. In order for the image projected by the first optical engine to coincide with the first projection image, the third projection coordinates need to be mapped to the third optical engine coordinates in the first optical engine coordinate system and set as the optical engine coordinates of the first optical engine.

[0142] As explained above, the main controller can map the coordinates of the first optical engine according to the preset mapping relationship between the first optical engine and the projection plane, thereby obtaining the first projected coordinates corresponding to each corner point of the first optical engine. Based on this, the main controller can also map the third projected coordinates according to the preset mapping relationship, thereby obtaining the third optical engine coordinates.

[0143] Similar to determining the third optomechanical coordinates, the main controller can map the fourth projected coordinates based on the aforementioned preset mapping relationship to obtain intermediate optomechanical coordinates represented by the first optomechanical coordinate system. Then, based on the first mapping relationship between the first and second optomechanical coordinate systems, the intermediate optomechanical coordinates can be mapped again to obtain the fourth optomechanical coordinates represented by the second optomechanical coordinate system, and set as the optomechanical coordinates of the second optomechanical system.

[0144] In this embodiment, a camera captures images of a first image card projected onto the projection surface by a first optical engine based on its first optical engine coordinates, and at least one second image card projected onto the projection surface by a second optical engine from a projection device based on its second optical engine coordinates. This allows for the determination of the first projection coordinates of the first optical engine on the projection surface, and the second projection coordinates of the second optical engine on the projection surface, based on the first and second image cards. Then, a stitched projection image is determined based on these first and second projection coordinates. In this stitched projection image, the image height is the intersection height of the first and second image cards. Therefore, even when the horizontal boundaries of the first and second image cards differ significantly, it is not necessary to adjust the positions of the individual projection devices. Furthermore, after adjusting the first and second optical engine coordinates based on the stitched projection image, the horizontal boundary of the projection image projected by the projection device based on the adjusted third optical engine coordinates can be made to coincide as closely as possible with the horizontal boundary of the projection image projected by the projection device based on the adjusted fourth optical engine coordinates. This reduces the difficulty of stitching multiple projection images.

[0145] It should be noted that when the first optical engine projects the image based on the third optical engine coordinate system, the horizontal boundary of its projected image may not completely coincide with the horizontal boundary of the projected image projected by the second optical engine based on the fourth optical engine coordinate system. In other words, there may still be some error in the stitching of the projected images. This error may stem from errors in establishing the aforementioned mapping relationships, causing the horizontal boundaries of the projected images to not completely coincide when the first and second optical engines project based on their corresponding optical engine coordinate systems.

[0146] For example, refer to Figure 14 , Figure 14 This illustration shows an application scenario of a projection device implementing projection screen splicing in some embodiments of this application. The area enclosed by the coordinates of projection corner points A3, B3, C1, and D1 represents the first image card actually projected onto the projection screen by the first optical engine based on the coordinates of the third optical engine. The area enclosed by the coordinates of projection corner points A2, B2, C3, and D3 represents the second image card actually projected onto the projection screen by the second optical engine based on the coordinates of the fourth optical engine. Figure 14As can be seen from this, there are small errors between A2B2 and A3B3, and between C1D1 and C3D3.

[0147] Therefore, in order to reduce the error when splicing multiple projected images and improve the splicing effect, the main controller can adjust the third and fourth optical engine coordinates again based on the projected image of the first optical engine based on the third optical engine coordinates and the projected image of the second optical engine based on the fourth optical engine coordinates.

[0148] However, it should be noted that since the first and second optical engine coordinates have already been adjusted once, the projected image projected by the first optical engine based on the adjusted third optical engine coordinates, while having some error compared to the projected image projected by the second optical engine based on the adjusted fourth optical engine coordinates, is likely to have a small error. Therefore, simultaneously adjusting both the third and fourth optical engine coordinates may lead to over-adjustment, resulting in stitching errors between the projected images projected by the first and second optical engines based on further adjusted coordinates. Furthermore, simultaneously adjusting both the first and second optical engines is relatively complex.

[0149] Therefore, in order to quickly reduce the error in projected image stitching, the main controller can, according to, such as Figure 15 The steps shown adjust the fourth optical engine coordinates of the second optical engine. Details are as follows:

[0150] S1510, control the projection device to capture the third image card projected onto the projection surface by the first optical engine based on the third optical engine coordinates, and the fourth image card projected onto the projection surface by the second optical engine based on the fourth optical engine coordinates.

[0151] In some embodiments, the master controller may send adjustment commands to the slave projection device to instruct the slave controller in the slave projection device to perform optical-mechanical coordinate adjustment.

[0152] As explained above, the structure of the secondary projection device is the same as that of the primary projection device. Therefore, when photographing the third and fourth image cards, the primary projection device can first control the first optical engine to project the third image card based on the coordinates of the third optical engine, and then send a shooting command and a screen-off command to the secondary projection device. The second optical engine in the secondary projection device can respond to the screen-off command to turn off the screen, and the secondary camera can respond to the shooting command to photograph the third image card.

[0153] Afterwards, the slave controller can detect when the camera has finished capturing an image and send a notification of completion to the master controller. Upon receiving this notification, the master controller can turn off the screen of the master projection device and send a notification of screen shutdown to the slave projection device. Upon receiving this notification, the slave projection device can control the second optical engine to project the fourth map card based on the coordinates of the fourth optical engine. Finally, it controls the slave camera to capture an image of the fourth map card.

[0154] S1520: Control the projection device to adjust the fourth optical engine coordinates based on the third and fourth graphics cards to obtain the fifth optical engine coordinates.

[0155] The third and fourth diagrams mentioned above can be similar to the first and second diagrams mentioned above, except that the optical engine coordinates corresponding to the first and second optical engines change when they are projected, which will not be explained in detail.

[0156] In some embodiments, the projection device can determine the third image card as the standard image card, and adjust the fourth image card so that the horizontal boundary of the adjusted fourth image card coincides with the horizontal boundary of the third image card. Then, based on the projection coordinates of the projected image corresponding to the adjusted fourth image card, the fourth optical engine coordinates can be adjusted to obtain the fifth optical engine coordinates.

[0157] As an example, the main controller can control the projector to determine the desired projection coordinates based on the fifth projection coordinates of the third card and the sixth projection coordinates of the fourth card. Then, it controls the projector to adjust the fourth optical-mechanical coordinates based on the desired projection coordinates to obtain the fifth optical-mechanical coordinates.

[0158] Specifically, it can be assumed that the overlap between the projected image corresponding to the expected projection coordinates and the horizontal boundary of the third card is higher than the overlap between the horizontal boundary of the fourth card and the third card. Furthermore, only the optical engine coordinates of the secondary projection device need to be adjusted; the optical engine coordinates of the primary projection device do not need to be adjusted.

[0159] As a specific example, the main controller can control the slave projection device to perform the following steps to adjust the fourth optical-mechanical coordinates. Details are as follows: The slave projection device can determine the fifth projection coordinates corresponding to the third card and the sixth projection coordinates corresponding to the fourth card. Then, based on the fifth projection coordinates, the tilt rate of the third card is determined. Finally, based on the tilt rate and the abscissa in the sixth projection coordinates, the desired ordinate is determined when projecting from the slave projection device, and the abscissa and desired ordinate are defined as the desired projection coordinates. Alternatively, based on the tilt rate and the ordinate in the sixth projection coordinates, the desired abscissa is determined when projecting from the slave projection device, and the ordinate and desired abscissa are defined as the desired projection coordinates. Finally, the slave projection device is controlled to adjust the fourth optical-mechanical coordinates based on the desired projection coordinates to obtain the fifth optical-mechanical coordinates.

[0160] The method by which the slave controller of the projection device determines the fifth projection coordinate of the third card and the sixth projection coordinate of the fourth card is similar to the method by which the master controller of the main projection device determines the first projection coordinate of the first card and the second projection coordinate of the second card, and will not be described in detail.

[0161] In some embodiments, the aforementioned inclination rate can be considered as the inclination rate of the horizontal boundary of the third drawing card relative to a standard horizontal line or a standard vertical line. Therefore, when determining the inclination rate, it can be calculated based on the coordinates of the projected corner points of the horizontal boundary in the third drawing card.

[0162] For example, let's take the fifth projection coordinate as the projection corner point coordinates (A3, B3, C1, D1) of the third drawing card. (Refer to...) Figure 14 The linear equation of the line segment passing through A3B3 can be calculated based on the coordinates of the projected corner point corresponding to A3B3. In this case, the slope in the linear equation can be considered as the aforementioned inclination rate.

[0163] When determining the desired projection coordinates corresponding to the sixth projection coordinate system, the desired corner coordinates corresponding to each projection corner point in the sixth projection coordinate system can be calculated separately. For example, for the desired corner coordinates corresponding to the A2 projection corner point coordinates, the abscissa of the A2 projection corner point coordinates can be input into the linear equation corresponding to the inclination rate (the linear equation corresponding to A3B3) to obtain the desired ordinate of the A2 projection corner point coordinates. At this point, the desired ordinate and the abscissa of A2 are the desired corner coordinates corresponding to A2. That is, as shown... Figure 14 In section A4, following this method, the desired corner coordinates for each corner point can be obtained. All desired corner coordinates constitute the aforementioned desired projected coordinates.

[0164] It is understandable that when the expected corner coordinates corresponding to A2 satisfy the linear equation corresponding to A3B3, the expected corner coordinates A4 corresponding to A2 can be considered to be located on line segment A3B3 (i.e., the boundary of the third diagram). Furthermore, the boundary formed by the expected corner coordinates corresponding to B2 and A2, determined in the above manner, will coincide with line segment A3B3. Similarly, the boundary formed by the expected corner coordinates corresponding to C3D3, determined in the above manner, will also coincide with line segment C1D1.

[0165] In the above example, the fifth and sixth projection coordinates are both calculated using the projection corner coordinates of the third and fourth maps, respectively. In another embodiment, the fifth and sixth projection coordinates can also be calculated using the projection coordinates of preset marker points of the third and fourth maps, respectively. This will not be described in detail.

[0166] In another embodiment, the main controller can also control the projection device to determine the desired abscissa when projecting from the projection device based on the tilt rate and the ordinate in the sixth projection coordinate, and determine the ordinate and the desired abscissa as the desired projection coordinate.

[0167] For example, for the desired corner coordinates corresponding to the A2 projected corner coordinates, the ordinate of the A2 projected corner coordinates can be input into the linear equation corresponding to the slope (the linear equation corresponding to A3B3) to obtain the desired abscissa of the A2 projected corner coordinates. At this point, the desired abscissa of the corner and the ordinate corresponding to A2 are the desired corner coordinates corresponding to A2. In this way, the desired corner coordinates corresponding to each corner can be obtained. All the desired corner coordinates are the aforementioned desired projected coordinates.

[0168] However, in real-world scenarios, when the ordinate of the A2 projection corner point is input into the linear equation corresponding to A3B3, the expected x-coordinate of the corner point may not fall within the range of the x-coordinates corresponding to A3B3. This could result in the adjusted fourth card of the second optical engine projection having no intersection with the third card, thus preventing the projected images from being stitched together.

[0169] Therefore, in order to ensure that the fourth map of the adjusted second optical engine projection has an intersection area with the third map and that the upper and lower horizontal boundaries coincide, the horizontal coordinate in the sixth projection coordinate system can be used in the above processing.

[0170] In some embodiments, after obtaining the desired projection coordinates, the desired projection coordinates can be mapped based on the mapping relationship between the projection plane and the second optomechanical coordinate system to obtain the fifth optomechanical coordinates.

[0171] However, as explained above, errors may exist in establishing the various mapping relationships, causing the horizontal boundaries of the projected images of the first and second optical engines to not completely coincide when projecting based on their corresponding optical engine coordinates. Therefore, the method of mapping the desired projection coordinates to obtain the fifth optical engine coordinates and indirectly adjusting the optical engine coordinates of the second optical engine may result in the final projected image of the second optical engine still not coinciding with the horizontal boundary of the projected image of the first optical engine.

[0172] Based on this, in order to improve the splicing effect of the projected image, the main controller can directly control the slave projection device to determine the adjustment direction of the fourth optical engine coordinate based on the desired projection coordinate and the sixth projection coordinate, so as to control the slave projection device to directly adjust the fourth optical engine coordinate along the adjustment direction to obtain the fifth optical engine coordinate.

[0173] For example, refer to Figure 14When the desired corner coordinates of A4 are determined to be below the projected corner coordinates of A2, the adjustment direction of the fourth optical-mechanical corner coordinates corresponding to the projected corner coordinates of A2 in the second optical-mechanical coordinate system can be determined from the projection device; this adjustment direction is also downward. That is, the ordinate of the fourth optical-mechanical corner coordinates corresponding to A2 after adjustment is less than the ordinate of the fourth optical-mechanical corner coordinates corresponding to A2 before adjustment. In this manner, the adjustment directions of the remaining second optical-mechanical corner coordinates in the second optical-mechanical coordinate system can be determined sequentially. All adjusted fourth optical-mechanical corner coordinates correspond to the fifth optical-mechanical coordinate system.

[0174] It should be added that when adjusting the fourth optical engine corner point coordinates along the adjustment direction, a preset value can be added or subtracted from the ordinate of the fourth optical engine coordinates each time to obtain the fourth optical engine corner point coordinates composed of new ordinates. The preset value can be set according to actual conditions and is not limited thereto. In this embodiment, to improve the splicing accuracy of the projected image, the preset value can be 1 pixel. Furthermore, by gradually adjusting the fourth optical engine coordinates to adjust the projected image of the second optical engine, the horizontal boundary of the projected image can gradually coincide with the horizontal boundary of the third image card projected by the main projection device, improving the splicing effect of the projected image.

[0175] It should be noted that after the fifth optical engine coordinates are obtained from the fourth optical engine coordinates by the projection device in the above manner, the horizontal boundary of the map projected by the second optical engine based on the fifth optical engine coordinates may still not coincide with the horizontal boundary of the third map projected by the first optical engine based on the third optical engine coordinates.

[0176] Based on this, in order to improve the splicing effect of the projected image, the main controller can control the projector to set the fifth optical engine coordinate to the fourth optical engine coordinate, and repeat the second step and the steps after the second step until the sixth projection coordinate coincides with the desired projection coordinate.

[0177] The second step includes: controlling the projection device to capture a third image card projected onto the projection surface by the first optical engine based on the third optical engine coordinates, and a fourth image card projected onto the projection surface by the second optical engine based on the fourth optical engine coordinates.

[0178] In another embodiment, after determining that the sixth projection coordinates coincide with the desired projection coordinates from the projection device, the fourth optical-mechanical coordinate adjustment process can be terminated, and an adjustment termination command can be sent to the main controller of the main projection device. At this time, the main projection device can again project the third card based on the third optical-mechanical coordinates and display it on the display screen as shown below. Figure 16 The fine grid A is shown. Simultaneously, in response to the user's fine-tuning operation on fine grid A, the third optical engine coordinates are adjusted, and the projected third chart is adjusted accordingly. Furthermore, the fifth chart is projected from the projection device based on the fifth optical engine coordinates and displayed on the display screen as shown. Figure 16The fine grid B is shown. Simultaneously, the control system responds to the user's fine-tuning operations on the fine grid B, adjusting the coordinates of the fifth optical engine and correspondingly adjusting the fifth projection card. This allows the horizontal boundaries of the final third and fifth projection cards to further overlap, improving the stitching effect of the projected image.

[0179] After the main controller of the main projector detects the end of the fine grid adjustment process, the playback control of the slave projectors can also be controlled by the main controller of the main projector. For example, the main controller can control both the main projector and the slave projectors to enter blended projection mode to play blended images. Furthermore, upon ending playback, it sends an exit command to the slave projectors to instruct them to exit the blended projection mode.

[0180] In some embodiments, based on the above-described process of splicing and blending projected images, it is known that when playing the blended image, there is at least a partial overlap between the projected image from the main projection device and the projected image from the secondary projection device. Since both the main and secondary projection devices project onto this overlap area, the brightness of this overlap area will increase. Therefore, in order to improve the playback effect of the projected image, gamma brightness processing technology can be used to process the brightness of the spliced ​​projected image to ensure a more balanced overall brightness in the played blended image.

[0181] In real-world scenarios, accidental contact or improper placement of the main and secondary projection devices may cause them to move. This can lead to the first and second optical engines projecting images based on their respective optical engine coordinates, resulting in the projection image boundaries not aligning. In such cases, the main projection device can re-execute the aforementioned projection image stitching method to achieve image stitching.

[0182] In some embodiments, the main controller is further configured to: when any projection device is detected to move, capture images of a third image card projected onto the projection surface by the first optical engine based on the third optical engine coordinates, and a fourth image card projected onto the projection surface by the second optical engine based on the fourth optical engine coordinates. Then, based on the third and fourth image cards, determine a second intersection area corresponding to the stitched projection image when the stitching method is stacking, and a second coverage area corresponding to the stitched projection image when the stitching method is merging. Finally, determine a target stitching method based on the second intersection area and the second coverage area, and adjust the first and second optical engine coordinates based on the stitched projection image corresponding to the target stitching method.

[0183] Each of the main and slave projection devices can be equipped with an inertial measurement unit (IMU) or a gyroscope sensor to detect movement. Upon detecting movement, the slave controller in one of the slave projection devices can send movement information to the main controller of the main projection device. The main controller of the main projection device can also receive position information from its own IMU or gyroscope sensor to determine if the main projection device has moved. Finally, upon determining that the main projection device has moved, or upon receiving movement information from any slave controller, the aforementioned optomechanical adjustment process is executed to stitch the projected images together.

[0184] The method for determining the second intersection region and the second coverage region based on the third and fourth map cards can refer to the method for determining the first intersection region and the first coverage region in the example above, and will not be described in detail here.

[0185] It should be noted that, in order to reduce the adjustment range of the optical engine coordinates of the first and second optical engines during the projection image stitching process, the main controller can determine the ratio of the second intersection area and the second coverage area. Then, when the ratio is greater than a preset value, it can be assumed that there is a large second intersection area between the third and fourth graphics cards. Based on this, the stacking method can be determined as the target stitching method. Conversely, when the ratio is less than or equal to the preset value, it can be assumed that there is a small intersection area between the third and fourth graphics cards. Based on this, the merging method can be determined as the target stitching method.

[0186] It should be added that, in the absence of a second overlapping area, the third and fourth graphics cards can be considered to have no overlap. In this case, the main projection device displays a prompt message to instruct the user to adjust the position of the main or secondary projection device to ensure overlap between the third graphics cards.

[0187] The method of adjusting the first and second optical engine coordinates based on the spliced ​​projection image corresponding to the target splicing method can be referred to the example above, which shows how to adjust the first and second optical engine coordinates based on the spliced ​​projection image to obtain the third optical engine coordinate corresponding to the first optical engine and the fourth optical engine coordinate corresponding to the second optical engine. This will not be explained further.

[0188] In summary, the above embodiments describe how the main controller of the main projection device adjusts the optical engine coordinates of the first and second optical engines to stitch together the projected images from the main and slave projection devices. To describe in detail the functions of each component in the main and slave projection devices, the following will illustrate... Figure 17 The embodiments shown are described in detail below. Figure 17 The following diagram illustrates the timing interaction of various components in the main projection device and the slave projection device in some embodiments of this application. Details are as follows:

[0189] The main projection device includes a main controller, a main camera, and a first optical engine, and the secondary projection device includes a secondary controller, a secondary camera, and a second optical engine.

[0190] Initially, the main controller generates a projection command for the largest possible image size and sends it to the first optical engine. The first optical engine responds to this projection command and adjusts the projection angle to the maximum angle to project the largest possible area of ​​the first image card. At this point, the optical engine coordinates corresponding to the maximum angle are the first optical engine coordinates.

[0191] Furthermore, the main controller can also send a projection command to the second optical engine in the projection device, maximizing the projection area. The second optical engine can also respond to this projection command and adjust the projection angle to the maximum angle to project the second image card with the largest possible area. At this time, the optical engine coordinates corresponding to the maximum angle in the second optical engine are the second optical engine coordinates.

[0192] Afterwards, the main controller can control the main projection device to complete the focusing process, and simultaneously send a focusing command to the slave controller of the slave projection device to instruct the slave controller to control the slave projection device to complete the focusing process. After the slave projection device has finished focusing, it can send a focusing completion message to the main controller.

[0193] The main controller can send a screen-off command to the second optical engine to stop projecting the second image card and to control the main camera to capture images of the first image card. Then, it sends a projection command to the second optical engine to control it to project the second image card, to control the first optical engine to stop projecting the first image card, and to control the main camera to capture images of the second image card.

[0194] Then, the main controller can determine the first projection coordinates of the first optical engine coordinates on the projection plane, and the second projection coordinates of the second optical engine coordinates on the projection plane. Furthermore, based on the first and second projection coordinates and the corresponding target stitching method, the stitched projection image is determined. Subsequently, based on the stitched projection image, the first and second optical engine coordinates are initially adjusted to obtain the third optical engine coordinates corresponding to the first optical engine, and the fourth optical engine coordinates corresponding to the second optical engine.

[0195] The target splicing method is determined by the main controller in response to the splicing command given by the user on the display screen of the main projection device.

[0196] After completing the initial adjustments to the optical engine coordinates of each projection device, the horizontal boundary of the image projected by the first optical engine based on the third optical engine coordinates may not completely coincide with the horizontal boundary of the image projected by the second optical engine based on the fourth optical engine coordinates. Therefore, the main controller needs to further adjust the third and fourth optical engine coordinates. In this embodiment, the adjustment of the fourth optical engine coordinates is taken as an example.

[0197] The main controller can first send a projection command to the first optical engine to project the third image card based on the coordinates of the third optical engine. Simultaneously, the main controller can control the second optical engine to turn off its screen and control the camera to capture images of the third image card. Then, the main controller can send a screen-off command to the first optical engine to stop projection. Simultaneously, the main controller can control the second optical engine to project the fourth image card based on the coordinates of the fourth optical engine and control the camera to capture images of the fourth image card. Finally, the main controller can send an adjustment command to the slave controller to instruct it to complete the adjustment process.

[0198] Adjustment process: The controller can adjust the fourth optomechanical coordinates based on the third and fourth graph cards to obtain the fifth optomechanical coordinates.

[0199] During the adjustment of the fourth optical engine coordinates, if the desired projection coordinates from the projection device coincide with the sixth projection coordinates, the adjustment process can be exited. Otherwise, if the desired projection coordinates do not coincide with the sixth projection coordinates, the adjustment process is repeated by the controller.

[0200] Furthermore, when the main controller detects movement of any projection device, it can again control the main camera to capture images of the third image (projected onto the projection surface by the first optical engine based on the third optical engine coordinates) and the fourth image (projected onto the projection surface by the second optical engine based on the fourth optical engine coordinates), based on the aforementioned interaction process of capturing the first and second image cards. Then, based on the third and fourth image cards, the target stitching method is determined, and the third and fourth optical engine coordinates are automatically adjusted according to the stitched projection image corresponding to the target stitching method.

[0201] The method of adjusting the third and fourth optical-mechanical coordinates based on the spliced ​​projection image corresponding to the target splicing method is similar to the method of adjusting the first and second optical-mechanical coordinates based on the spliced ​​projection image, and will not be explained further. For details, please refer to... Figure 17 The content corresponding to the dashed box.

[0202] It should be noted that during the adjustment of the fourth optical engine coordinates, the target stitching method can be determined based on the ratio of the second intersection area to the second coverage area.

[0203] In summary, the above description illustrates the timing interaction diagram of the various components in the main and secondary projection devices. By using the above method to stitch the projected images, even when the horizontal boundaries of the first and second projection cards differ significantly, it is possible to avoid adjusting the positions of the individual projection devices and improve the stitching effect of the projected images.

[0204] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or certain parts of the embodiments of the present invention.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0206] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of embodiments suitable for specific application considerations.

Claims

1. A projection device, characterized in that, include: The optical engine is configured to project the image onto the projection surface; A camera is configured to capture images of a first image card projected onto the projection surface by a first optical engine in the projection device, and at least one second image card projected onto the projection surface by a second optical engine in the projection device. The controller is configured as follows: Obtain the first image card and the second image card; Determine the first optical engine coordinates on the projection plane and their first projection coordinates; Based on the first and second maps, determine the second optical engine coordinates of the second optical engine on the projection plane and their second projection coordinates. Based on the first projection coordinates and the second projection coordinates, the stitched projection image is determined; the image height in the stitched projection image is the intersection height of the first image card and the second image card; Based on the spliced ​​projection image, the first optical engine coordinates and the second optical engine coordinates are adjusted respectively to obtain the third optical engine coordinates corresponding to the first optical engine and the fourth optical engine coordinates corresponding to the second optical engine; The third optical-mechanical coordinates are used as the new optical-mechanical coordinates of the first optical-mechanical system, and the fourth optical-mechanical coordinates are used as the new optical-mechanical coordinates of the second optical-mechanical system.

2. The projection device according to claim 1, characterized in that, The controller is also configured to: When the camera is taking a picture of the first image card, the screen of the projection device is turned off; and, When the camera is taking a picture of the second image card, the projection device is controlled to turn off its screen.

3. The projection device according to claim 1, characterized in that, The controller, based on the first projection coordinates and the second projection coordinates, determines the stitched projection image and is configured as follows: The target splicing method is determined in response to the input splicing command; If the target splicing method is a stacking method, then the first intersection area in the first image card and the second image card is determined based on the first projection coordinates and the second projection coordinates, and the first intersection area is determined as the spliced ​​projection image; If the target splicing method is a merging method, then the first projected image corresponding to the intersection height in the first image card is determined based on the first projection coordinates, and the second projected image corresponding to the intersection height in the second image card is determined based on the second projection coordinates, and the first coverage area of ​​the first projected image and the second projected image is determined as the spliced ​​projected image.

4. The projection device according to claim 1, characterized in that, Based on the spliced ​​projection image, the controller adjusts the first and second optical engine coordinates respectively to obtain the third optical engine coordinates corresponding to the first optical engine and the fourth optical engine coordinates corresponding to the second optical engine, which are configured as follows: From the spliced ​​projection image, determine the first projection image corresponding to the projection device and the second projection image corresponding to the projection device. The third optical-mechanical coordinates are determined based on the third projection coordinates corresponding to the first projected image, and the third optical-mechanical coordinates are set as the optical-mechanical coordinates of the first optical engine; and... The fourth optical engine coordinates are determined based on the fourth projection coordinates corresponding to the second projection image, and the fourth optical engine coordinates are set as the optical engine coordinates of the second optical engine.

5. The projection device according to claim 1, characterized in that, The controller is also configured to: The system controls the projection device to capture images of a third image card projected from the first optical engine onto the projection surface based on the third optical engine coordinates, and a fourth image card projected from the second optical engine onto the projection surface based on the fourth optical engine coordinates. The control system adjusts the fourth optical-mechanical coordinates based on the third and fourth graphics cards to obtain the fifth optical-mechanical coordinates.

6. The projection device according to claim 5, characterized in that, The controller controls the projection device to adjust the fourth optical-mechanical coordinates based on the third and fourth graphics cards to obtain the fifth optical-mechanical coordinates, which is configured as follows: The control unit determines the fifth projection coordinates corresponding to the third drawing card and the sixth projection coordinates corresponding to the fourth drawing card from the projection device. The control device determines the tilt rate of the third chart based on the fifth projection coordinates; The control system determines the desired ordinate of the projection device during projection based on the tilt rate and the abscissa in the sixth projection coordinate system, and defines the abscissa and the desired ordinate as the desired projection coordinates. The system controls the projection device to adjust the fourth optical-mechanical coordinates based on the desired projection coordinates to obtain the fifth optical-mechanical coordinates.

7. The projection device according to claim 6, characterized in that, The controller controls the projection device to adjust the fourth optical-mechanical coordinates based on the desired projection coordinates to obtain the fifth optical-mechanical coordinates, and is configured as follows: The control device determines the adjustment direction of the fourth optical-mechanical coordinates based on the desired projection coordinates and the sixth projection coordinates; The fourth optical-mechanical coordinate is adjusted by controlling the projection device along the adjustment direction to obtain the fifth optical-mechanical coordinate.

8. The projection device according to claim 1, characterized in that, The controller is also configured to: When any projection device is detected to be moving, the third image card of the first optical engine projected onto the projection surface based on the third optical engine coordinates, and the fourth image card of the second optical engine projected onto the projection surface based on the fourth optical engine coordinates are captured respectively. Based on the third and fourth diagrams, the second intersection area corresponding to the spliced ​​projection screen is determined when the splicing method is stacking, and the second coverage area corresponding to the spliced ​​projection screen is determined when the splicing method is merging. The target stitching method is determined based on the second intersection region and the second coverage region; Based on the splicing projection image corresponding to the target splicing method, the first optical-mechanical coordinates and the second optical-mechanical coordinates are adjusted respectively.

9. The projection device according to claim 8, characterized in that, The controller determines the target stitching method based on the second intersection region and the second coverage region, and is configured as follows: Determine the ratio of the second intersection region to the second coverage region; If the ratio is greater than a preset value, then the stacking method is determined as the target splicing method; If the ratio is less than or equal to the preset value, then the merging method is determined as the target merging method.

10. A method for splicing projected images, characterized in that, include: Obtain a first image card and a second image card; the first image card is an image card projected onto a projection surface by a first optical engine in a projection device, and the second image card is at least one image card projected onto the projection surface by a second optical engine in a projection device; Determine the first optical engine coordinates on the projection plane and their first projection coordinates; Based on the first and second maps, determine the second optical engine coordinates of the second optical engine on the projection plane and their second projection coordinates. Based on the first projection coordinates and the second projection coordinates, the stitched projection image is determined; the image height in the stitched projection image is the intersection height of the first image card and the second image card; Based on the spliced ​​projection image, the first optical engine coordinates and the second optical engine coordinates are adjusted respectively to obtain the third optical engine coordinates corresponding to the first optical engine and the fourth optical engine coordinates corresponding to the second optical engine; The third optical-mechanical coordinate is used as the new optical-mechanical coordinate of the first optical-mechanical system, and the fourth optical-mechanical coordinate is used as the new optical-mechanical coordinate of the second optical-mechanical system.

Images