A picture splicing processing method and device, equipment and storage medium

Abstract

The application discloses a picture splicing processing method and device, equipment and a storage medium, and the method comprises the following steps: in the case that a first projection picture intersects with a second projection picture from a second projection device, a target area is planned in a region formed by the first projection picture and the second projection picture based on a preset proportion; the target area is divided into a first area and a second area through a target division line; the target division line is in the first projection picture and the second projection picture at the same time; the first projection picture and the second projection picture are respectively corrected based on projection coordinates of the first area and projection coordinates of the second area, so that the first projection picture and the second projection picture are spliced together in a square form, and the overall display effect of the spliced picture is improved.

Description

Technical Field

[0001] This application relates to image stitching technology, and more particularly to an image stitching processing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, projection devices with multi-screen splicing capabilities are typically professional-grade and rarely equipped with automatic splicing functionality. Even those devices that offer automatic splicing often limit themselves to basic splicing operations and cannot automatically correct the squareness of each spliced ​​image simultaneously. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for image splicing processing.

[0004] The technical solution of this application is implemented as follows:

[0005] Firstly, a method for image stitching is provided, the method comprising:

[0006] When the first projection image intersects with the second projection image from the second projection device, a target area is planned in the area formed by the first projection image and the second projection image based on a preset ratio;

[0007] The target area is divided into a first area and a second area by a target dividing line; wherein the target dividing line is simultaneously within the first projection screen and the second projection screen;

[0008] Based on the projection coordinates of the first region and the projection coordinates of the second region, the first projection image and the second projection image are corrected respectively, so that the first projection image and the second projection image are stitched together.

[0009] Secondly, a video stitching processing device is provided, characterized in that the device comprises:

[0010] The processing unit is configured to, when the first projection image intersects with the second projection image from the second projection device, plan a target area in the area formed by the first projection image and the second projection image based on a preset ratio;

[0011] The processing unit is further configured to divide the target region into a first region and a second region by means of a target dividing line; wherein the target dividing line is simultaneously within the first projection screen and the second projection screen;

[0012] The correction unit is used to correct the first projected image and the second projected image based on the projection coordinates of the first region and the projection coordinates of the second region, so that the first projected image and the second projected image are stitched together.

[0013] Thirdly, an electronic device is provided, comprising: a processor and a memory configured to store a computer program capable of running on the processor, wherein the processor is configured to perform the steps of the method of the first aspect when running the computer program.

[0014] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method of the first aspect.

[0015] Fifthly, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method of the first aspect.

[0016] This application provides a method, apparatus, device, and storage medium for image splicing. The method includes: when a first projected image intersects with a second projected image from a second projection device, planning a target area within the region formed by the first and second projected images based on a preset ratio; dividing the target area into a first area and a second area using a target dividing line; wherein the target dividing line lies simultaneously within both the first and second projected images; and correcting the first and second projected images based on the projection coordinates of the first and second areas respectively, thereby completing the splicing of the first and second projected images. This achieves a square-shaped splicing of the first and second projected images, improving the overall display effect of the spliced ​​image. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the image stitching processing method in the embodiments of this application. Figure 1 ;

[0018] Figure 2 This is a schematic diagram illustrating the intersection of a first projected image and a second projected image, as exemplified by an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of an area composed of a first projection screen and a second projection screen, as exemplified by an embodiment of this application.

[0020] Figure 4 This is a schematic diagram of a target area as exemplified in an embodiment of this application;

[0021] Figure 5 A schematic diagram of a target dividing line as an example of an embodiment of this application. Figure 1 ;

[0022] Figure 6 This is a flowchart illustrating the image stitching processing method in the embodiments of this application. Figure 2 ;

[0023] Figure 7 A schematic diagram of a target dividing line as an example of an embodiment of this application. Figure 2 ;

[0024] Figure 8 A schematic diagram of a target dividing line as an example of an embodiment of this application. Figure 3 ;

[0025] Figure 9 A schematic diagram of a target dividing line as an example of an embodiment of this application. Figure 4 ;

[0026] Figure 10 This is a flowchart illustrating the image stitching processing method in the embodiments of this application. Figure 3 ;

[0027] Figure 11 This is a flowchart illustrating the image stitching processing method in the embodiments of this application. Figure 4 ;

[0028] Figure 12 This is a schematic diagram illustrating a gap between a first feature map and a second feature map, as exemplified by an embodiment of this application.

[0029] Figure 13 This is a schematic diagram illustrating an example of an embodiment of this application where a first feature map and a second feature map overlap.

[0030] Figure 14 This is a schematic diagram of the structure of the image splicing device in the embodiments of this application;

[0031] Figure 15 This is a schematic diagram of the structure of the electronic equipment in the embodiments of this application. Detailed Implementation

[0032] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing the embodiments only and is not intended to limit the scope of this application.

[0034] In the following description, references to "some embodiments," "this embodiment," "this embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0035] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0036] In this embodiment, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0037] This application provides a method for image stitching processing. Figure 1 This is a flowchart illustrating the image stitching processing method in the embodiments of this application. Figure 1 ,like Figure 1 As shown, the image stitching process includes the following steps:

[0038] S101: When the first projection screen intersects with the second projection screen from the second projection device, a target area is planned in the area formed by the first projection screen and the second projection screen based on a preset ratio.

[0039] In this embodiment of the application, a first projection device projects a first source image onto a target wall, and a second projection device projects a second source image onto the target wall. Thus, the target wall displays a first projected image corresponding to the first source image and a second projected image corresponding to the second source image. For example... Figure 2 As shown, the first and second projected images on the target wall intersect.

[0040] In this embodiment, both the first and second projection devices include a Time-of-Flight (TOF) sensor. Based on existing mature projection distance modeling modules, the mapping relationship H1 between the source coordinate system and the projection coordinate system of the first projection device, and the mapping relationship H2 between the source coordinate system and the projection coordinate system of the second projection device, can be measured. The first projection device calculates the projection coordinates of the four vertices of the first projected image using H1 and the source coordinates of the four vertices of the first source image, and then projects the first projected image onto the target wall. The second projection device calculates the projection coordinates of the four vertices of the second projected image using H2 and the source coordinates of the four vertices of the second source image, and then projects the second projected image onto the target wall. Combined with... Figure 2 The first and second projected images are stitched together horizontally. If the right border a1b1 of the first projected image is located to the right of the left border a2b2 of the second projected image, then the first and second projected images are considered to intersect. For example, taking the top left vertex of the target wall as the origin, the horizontal axis as X, and the vertical axis as Y, when the maximum value of the projection coordinate X of the right border a1b1 of the first projected image is greater than the minimum value of the projection coordinate X of the left border a2b2 of the second projected image, and the Y coordinate ranges of the two borders intersect, the first and second projected images are considered to intersect.

[0041] The source image refers to the image that the projection device needs to project. The projected image refers to the image projected onto the wall by the projection device.

[0042] The preset ratio refers to a ratio set by developers based on experience or experimentation, or it can be any ratio. For example, the preset ratio could be 16:9 or 4:3.

[0043] The target area is a regular, square quadrilateral.

[0044] For example, any point in the area formed by the first and second projected images is taken as the top-left vertex of the area to be planned. Based on a preset ratio, the bottom-left and top-right vertices are found along the horizontal and vertical axes respectively, thus finding the corresponding area. Then, the area of ​​this area is calculated. The target area refers to the area with the largest area.

[0045] like Figure 3 As shown, this is the area composed of the first projected image and the second projected image.

[0046] like Figure 4 As shown, the target areas P1P2P3P4 are planned within the area formed by the first projection screen and the second projection screen.

[0047] S102: Divide the target area into a first area and a second area by using a target dividing line; wherein the target dividing line is simultaneously within the first projection screen and the second projection screen.

[0048] like Figure 5 As shown, the target dividing line P5P6 is simultaneously in the first projection screen and the second projection screen. The target area P1P2P3P4 is divided into the first area P1P5P6P4 and the second area P5P2P3P6 by the target dividing line P5P6.

[0049] S103: Based on the projection coordinates of the first region and the projection coordinates of the second region, the first projection image and the second projection image are corrected respectively, so that the first projection image and the second projection image are stitched together.

[0050] In this embodiment, after the target region is planned, the projected coordinates of the four vertices of the target region can be obtained. The projected coordinates of the two vertices of the target dividing line can also be directly obtained. Therefore, the projected coordinates of the four vertices of the first region P1P5P6P4 and the four vertices of the second region P5P2P3P6 can be known.

[0051] It should be noted that this image stitching method can be applied to the first projection device. Based on this, the first projection device sends the projection coordinates of the four vertices P5, P2, P3, and P6 of the second region to the second projection device. Further, the first projection device corrects the first projected image based on the projection coordinates of the four vertices P1, P5, P6, and P4 of the first region. The second projection device corrects the second projected image based on the projection coordinates of the four vertices P5, P2, P3, and P6 of the second region. This allows the first and second projected images to be stitched together.

[0052] It should be noted that this image stitching method can also be applied to a second projection device. Based on this, the second projection device sends the projection coordinates of the four vertices P1, P5, P6, and P4 of the first region to the first projection device. Further, the second projection device corrects the second projected image based on the projection coordinates of the four vertices P5, P2, P3, and P6 of the second region. The first projection device corrects the first projected image based on the projection coordinates of the four vertices P1, P5, P6, and P4 of the first region. This allows the first and second projected images to be stitched together.

[0053] It should be noted that this image stitching method can also be applied to a standalone calibration device. Based on this, the calibration device sends the projected coordinates of the four vertices of the first region (P1, P5, P6, P4) to the first projection device, and the projected coordinates of the four vertices of the second region (P5, P2, P3, P6) to the second projection device. This allows the first projection device to correct the first projected image based on the projected coordinates of the four vertices of the first region (P1, P5, P6, P4), and the second projection device to correct the second projected image based on the projected coordinates of the four vertices of the second region (P5, P2, P3, P6). This allows the first and second projected images to be stitched together.

[0054] It should be noted that there is at least one first projected image and at least one second projected device. Correspondingly, there is at least one first projected image and at least one second projected image. For example, if one first projected device and two second projected devices are included, the first projected image can be spliced ​​to the left of the first second projected image, and the second second projected image can be spliced ​​to the right of the first second projected image. Alternatively, the first projected image can be spliced ​​above the first second projected image, and the second second projected image can be spliced ​​below the first second projected image. Other splicing methods are also possible, depending on the requirements, and will not be elaborated further.

[0055] The above solution enables the first and second projected images to be stitched together in a square shape, improving the overall display effect of the stitched image.

[0056] In some embodiments of this application, dividing the target region into a first region and a second region by a target dividing line includes the following steps:

[0057] S601: Based on the splicing method, select a dividing line from the preset position of the target area.

[0058] In this embodiment, the splicing method can be left-right splicing, and the preset position can be the left, middle, or right. If the preset position is the left, the right dividing line adjacent to the left boundary of the target area is used as the first selected dividing line. If the preset position is the middle, the middle dividing line of the target area is used as the first selected dividing line. If the preset position is the right, the left dividing line adjacent to the right boundary of the target area is used as the first selected dividing line. It should be noted that the dividing line can be parallel to the left and right borders of the target area, or it can be non-parallel to the left and right borders of the target area.

[0059] In this embodiment, the splicing method can also be top-to-bottom splicing, and the preset position can be the top, middle, or bottom. If the preset position is the top, the bottom dividing line immediately adjacent to the top boundary of the target area is used as the first selected dividing line. If the preset position is the middle, the middle dividing line of the target area is used as the first selected dividing line. If the preset position is the bottom, the top dividing line immediately adjacent to the bottom boundary of the target area is used as the first selected dividing line. It should be noted that the dividing line can be parallel to the top and bottom borders of the target area, or it can be non-parallel to the top and bottom borders of the target area.

[0060] S602: When the dividing line is simultaneously within the first projection screen and the second projection screen, the target area is divided into the first area and the second area by the target dividing line; wherein, the target dividing line is the dividing line.

[0061] For example, if all points on the dividing line are on the first projected image (including the boundary), then the dividing line is considered to be within the first projected image. If all points on the dividing line are on the second projected image (including the boundary), then the dividing line is considered to be within the second projected image. Here, the projected image can be a regular quadrilateral or an irregular quadrilateral.

[0062] For example, the first projected image and the second projected image are as follows: Figure 5 In the intersection scenario shown, the endpoint P6 of the dividing line is on the lower boundary of the first projection screen, the remaining points of the dividing line are within the first projection screen, and all points of the dividing line are within the second projection screen. The dividing line is parallel to the left and right borders of the target area.

[0063] For example, two projected images are as follows: Figure 7 In the intersection shown, all points of the dividing line are within both projected images. The dividing line is not parallel to the left and right borders of the target area.

[0064] For example, two projected images are as follows: Figure 8 In the intersection scenario shown, all points of the dividing line lie on the right boundary of the left projection image, and all points of the dividing line lie on the left boundary of the right projection image. The dividing line is not parallel to the left or right borders of the target area.

[0065] For example, two projected images are as follows: Figure 9 In the intersection shown, the two endpoints of the dividing line are on the upper and lower boundaries of the left projection image, respectively, while the remaining points of the dividing line are within the left projection image. One endpoint of the dividing line is on the lower boundary of the right projection image, while the remaining points of the dividing line are within the right projection image. The dividing line is parallel to the left and right borders of the target area.

[0066] S603: When the dividing line is in the first projection screen but not in the second projection screen, adjust the dividing line to shift towards the second projection screen until the dividing line is simultaneously in both the first and second projection screens.

[0067] For example, when the first and second projected images are spliced ​​side-by-side, the dividing line is adjusted to shift to the right (i.e., shifted towards the second projected image) until the dividing line is simultaneously within both the first and second projected images.

[0068] For example, when the first and second projected images are stitched together vertically, the dividing line is adjusted to be shifted downwards (i.e., shifted towards the second projected image) until the dividing line is simultaneously within both the first and second projected images.

[0069] S604: If the dividing line is in the second projection screen but not in the first projection screen, adjust the dividing line to shift towards the first projection screen until the dividing line is simultaneously in both the first and second projection screens.

[0070] For example, when the first and second projected images are spliced ​​side-by-side, the dividing line is adjusted to shift to the left (i.e., shifted towards the first projected image) until the dividing line is simultaneously within both the first and second projected images.

[0071] For example, when the first and second projected images are stitched together vertically, the dividing line is adjusted to be shifted upwards (i.e., shifted towards the first projected image) until the dividing line is simultaneously within both the first and second projected images.

[0072] In some embodiments of this application, the step of correcting the first projected image and the second projected image based on the projection coordinates of the first region and the projection coordinates of the second region includes the following steps:

[0073] S1001: Based on the first mapping relationship, determine the source coordinates corresponding to the projection coordinates of the first region; wherein, the first mapping relationship is the mapping relationship between the source coordinate system and the projection coordinate system of the first projection device.

[0074] S1002: Based on the source coordinates of the first region, the source coordinates of the first source image corresponding to the first projected image are corrected to correct the first projected image.

[0075] S1003: Based on the second mapping relationship, determine the source coordinates corresponding to the projection coordinates of the second region, wherein the second mapping relationship is the mapping relationship between the source coordinate system and the projection coordinate system of the second projection device.

[0076] S1004: Based on the source coordinates of the second region, the source coordinates of the second source image corresponding to the second projected image are corrected in order to correct the second projected image.

[0077] Here, the first projected image is the image projected onto the target wall by the first projection device from the first source image. The second projected image is the image projected onto the target wall by the second projection device from the second source image.

[0078] In this embodiment, when the image stitching method is applied to a first projection device, the first projection device corrects the source coordinates of the first source image corresponding to the first projected image based on the source coordinates of the first region, obtaining the corrected source coordinates of the first source image. Based on the first mapping relationship and the corrected source coordinates of the first source image, the first projection device obtains the corrected projection coordinates of the first projected image, and then projects the first projected image onto the target wall based on these corrected projection coordinates. The first projection device sends the projection coordinates of the second region to the second projection device, causing the second projection device to correct the source coordinates of the second source image corresponding to the second projected image based on the source coordinates of the second region, obtaining the corrected source coordinates of the second source image. Based on the second mapping relationship and the corrected source coordinates of the second source image, the second projection device obtains the corrected projection coordinates of the second projected image, and then projects the second projected image onto the target wall based on these corrected projection coordinates.

[0079] In this embodiment, when the image stitching method is applied to a second projection device, the second projection device corrects the source coordinates of the second source image corresponding to the second projected image based on the source coordinates of the second region, obtaining the corrected source coordinates of the second source image. The second projection device then obtains the corrected projection coordinates of the second projected image based on the second mapping relationship and the corrected source coordinates of the second source image, and subsequently projects the second projected image onto the target wall based on these corrected projection coordinates. Alternatively, the second projection device sends the projection coordinates of the first region to the first projection device, enabling the first projection device to correct the source coordinates of the first source image corresponding to the first projected image based on the source coordinates of the first region, obtaining the corrected source coordinates of the first source image. The first projection device then obtains the corrected projection coordinates of the first projected image based on the first mapping relationship and the corrected source coordinates of the first source image, and subsequently projects the first projected image onto the target wall based on these corrected projection coordinates.

[0080] In this embodiment of the application, when the image stitching method is applied to a separate calibration device, the calibration device sends the projection coordinates of the first region to the first projection device and the projection coordinates of the second region to the second projection device, so that the first projection device completes the calibration of the first projected image and the second projection device completes the calibration of the second projected image.

[0081] In some embodiments of this application, the method further includes the following steps:

[0082] S1101: Use a camera to capture a first feature image and a second feature image projected onto the target wall; wherein the first feature image is projected by a first projection device and the second feature image is projected by a second projection device.

[0083] In this embodiment, after the first projection device corrects the first projected image based on the projection coordinates of the four vertices of the first region, it projects a pre-stored first feature map onto the target wall. The first feature map is an image composed of multiple pixels. Simultaneously, after the second projection device corrects the second projected image based on the projection coordinates of the four vertices of the second region, it projects a pre-stored second feature map onto the target wall. The second feature map is also an image composed of multiple pixels. Further, the first projection device uses a camera to capture the first and second feature maps projected onto the target wall. Alternatively, the second projection device uses a camera to capture the first and second feature maps projected onto the target wall. Alternatively, the calibration device uses a camera to capture the first and second feature maps projected onto the target wall.

[0084] S1102: If the first splicing boundary of the first feature map is not aligned with the second splicing boundary of the second feature map, adjust the first feature map and / or the second feature map until the first splicing boundary is aligned with the second splicing boundary.

[0085] In this embodiment of the application, when the first feature image and the second feature image are joined horizontally, it is determined whether the right boundary (i.e., the first joining boundary) of the first feature image and the left boundary (i.e., the second joining boundary) of the second feature image are aligned. If the upper vertex of the right boundary of the first feature image and the upper vertex of the left boundary of the second feature image are not the same point, and / or the lower vertex of the right boundary of the first feature image and the lower vertex of the left boundary of the second feature image are not the same point, it indicates that the first joining boundary of the first feature image and the second joining boundary of the second feature image are not aligned. Conversely, if the upper vertex of the right boundary of the first feature image and the upper vertex of the left boundary of the second feature image are the same point, and the lower vertex of the right boundary of the first feature image and the lower vertex of the left boundary of the second feature image are the same point, it indicates that the first joining boundary of the first feature image and the second joining boundary of the second feature image are aligned.

[0086] In this embodiment of the application, when the image stitching method is applied to a first projection device, if the first stitching boundary of the first feature image is not aligned with the second stitching boundary of the second feature image (including deviation and overlap), the first projection device adjusts the first feature image, and / or the first projection device sends second adjustment information to the second projection device, so that the second projection device adjusts the second feature image based on the adjustment information until the first stitching boundary of the first feature image is aligned with the second stitching boundary of the second feature image.

[0087] In this embodiment of the application, when the image stitching method is applied to a second projection device, if the first stitching boundary of the first feature image is not aligned with the second stitching boundary of the second feature image (including deviation and overlap), the second projection device adjusts the second feature image, and / or the second projection device sends first adjustment information to the first projection device, so that the first projection device adjusts the first feature image based on the adjustment information until the first stitching boundary of the first feature image is aligned with the second stitching boundary of the second feature image.

[0088] In this embodiment of the application, when the image stitching method is applied to a calibration device, if the first stitching boundary of the first feature image and the second stitching boundary of the second feature image are not aligned (including deviation and overlap), the calibration device sends first adjustment information and second adjustment information to the first projection device and the second projection device respectively, so that the first projection device adjusts the first feature image based on the first adjustment information and the second projection device adjusts the second feature image based on the second adjustment information, until the first stitching boundary of the first feature image and the second stitching boundary of the second feature image are aligned.

[0089] S1103: Based on the source coordinates of the first feature map before and after adjustment and / or the source coordinates of the second feature map before and after adjustment, adjust the source coordinates of the first source image and / or the second source image so that the first projection image and the second projection image are aligned and stitched together.

[0090] For example, based on the source coordinates of the first feature map before and after adjustment, it is known that the first feature map has been shifted 3 pixels to the right. Therefore, the first source image is shifted 3 pixels to the right to obtain the new source coordinates of the first source image. Further, based on the first mapping relationship and the new source coordinates of the first source image, the new projection coordinates of the first projected image are obtained, and then the first projected image is presented on the target wall. This allows the first projected image and the second projected image to be aligned and stitched together.

[0091] In other words, the first source image is adjusted accordingly based on the adjustment method of the first feature map.

[0092] In some embodiments of this application, adjusting the first feature map and / or the second feature map until the first stitching boundary aligns with the second stitching boundary when the first stitching boundary of the first feature map is not aligned with the second stitching boundary includes the following steps:

[0093] If there is a gap between the first splicing boundary and the second splicing boundary, adjust the first feature map closer to the second feature map, and / or adjust the second feature map closer to the first feature map, until the first splicing boundary and the second splicing boundary are aligned;

[0094] like Figure 12 As shown, there is a gap between the right boundary (i.e., the first splicing boundary) of the first feature map and the left boundary (i.e., the second splicing boundary) of the second feature map.

[0095] For example, if the first feature map and the second feature map are stitched together horizontally, and there is a gap between the first stitching boundary of the first feature map and the second stitching boundary of the second feature map, the first feature map can be gradually shifted to the right by one pixel unit, so that the first feature map is closer to the second feature map, until the first stitching boundary is aligned with the second stitching boundary. And / or, the second feature map can be gradually shifted to the left by one pixel unit, so that the second feature map is closer to the first feature map, until the first stitching boundary is aligned with the second stitching boundary.

[0096] If the first splicing boundary overlaps with the second splicing boundary, adjust the first feature map to deviate from the second feature map, and / or adjust the second feature map to deviate from the first feature map, until the first splicing boundary aligns with the second splicing boundary.

[0097] like Figure 13 As shown, the right boundary of the first feature map (i.e., the first splicing boundary) overlaps with the left boundary of the second feature map (i.e., the second splicing boundary).

[0098] For example, if the first feature map and the second feature map are stitched together horizontally, and the first stitching boundary of the first feature map overlaps with the second stitching boundary of the second feature map, the first feature map can be gradually shifted to the left by one pixel unit, causing the first feature map to deviate from the second feature map until the first stitching boundary aligns with the second stitching boundary. And / or, the second feature map can be gradually shifted to the right by one pixel unit, causing the second feature map to deviate from the first feature map until the first stitching boundary aligns with the second stitching boundary.

[0099] In some embodiments of this application, the method further includes:

[0100] If the first projected image and the second projected image do not intersect, a prompt message is output until the first projected image and the second projected image intersect; wherein, the prompt message is used to indicate that the first projection device and / or the second projection device need to be moved.

[0101] For example, the prompt message can be projected onto the target wall, or the user can be prompted by voice to move the first projection device and / or the second projection device until the first projection image intersects with the second projection image.

[0102] Based on the above embodiments, this application specifically illustrates a screen stitching processing method. It includes two projection devices, namely a first projection device and a second projection device. Both devices include a TOF sensor. The first projection device also includes an RGB sensor, i.e., a camera. The second projection device may or may not include an RGB sensor.

[0103] Taking the application of this image stitching processing method to the first projection device as an example, the specific steps include:

[0104] S1: The first projection device, based on an existing mature projection distance modeling module, measures the mapping relationship H1 between the source coordinate system and the projection coordinate system of the first projection device. The second projection device, based on an existing mature projection distance modeling module, measures the mapping relationship H2 between the source coordinate system and the projection coordinate system of the second projection device.

[0105] S2: The first projection device calculates the projection coordinates of the four vertices of the first projection image using H1 and the source coordinates of the four vertices of the first source image, and then projects the first projection image onto the target wall. The second projection device calculates the projection coordinates of the four vertices of the second projection image using H2 and the source coordinates of the four vertices of the second source image, and then projects the second projection image onto the target wall.

[0106] S3: Based on the projection coordinates of the four vertices of the first projection screen and the projection coordinates of the four vertices of the second projection screen, determine whether the first projection screen and the second projection screen intersect.

[0107] S4: If the first projected image and the second projected image do not intersect, output a prompt message to remind the user that the first projection device and / or the second projection device need to be moved to ensure that the image boundaries are related.

[0108] S5: The first projection screen intersects with the second projection screen (e.g., Figure 2 As shown), based on a preset ratio, target regions P1P2P3P4 are planned within the area formed by the first and second projected images (e.g., ...). Figure 4 (As shown).

[0109] S6: Select a dividing line in the target area and determine whether the dividing line is simultaneously in the first projection screen and the second projection screen.

[0110] S7: When the dividing line is simultaneously within the first and second projection screens, the target region P1P2P3P4 is divided into the first region P1P5P6P4 and the second region P5P2P3P6 by the target dividing line P5P6 (e.g., ...). Figure 5 (As shown).

[0111] S8: If the dividing line is in the first projection screen but not in the second projection screen, adjust the dividing line to move towards the second projection screen until the dividing line is simultaneously in both the first and second projection screens; if the dividing line is in the second projection screen but not in the first projection screen, adjust the dividing line to move towards the first projection screen until the dividing line is simultaneously in both the first and second projection screens.

[0112] S9: The first projection device, based on H1, determines the source coordinates corresponding to the projection coordinates of the four vertices P1P5P6P4 in the first region. Based on the source coordinates of the four vertices P1P5P6P4 in the first region, it corrects the source coordinates of the first source image corresponding to the first projected image, so as to correct the first projected image.

[0113] The projection coordinates of the four vertices P5P2P3P6 in the second region are sent to the second projection device. The second projection device determines the source coordinates corresponding to the projection coordinates of the four vertices P5P2P3P6 in the second region based on H2. Then, based on the source coordinates of the four vertices P5P2P3P6 in the second region, the source coordinates of the second source image corresponding to the second projection image are corrected to correct the second projection image.

[0114] It should be noted that due to variations between different projector models, overlaps or gaps may appear in the projected image stitching area after calibration. These issues can be addressed by taking a picture of the projected image with a camera for verification and adjustment. Verification and adjustment methods include:

[0115] S10: The first projection device projects a first feature image onto the target wall. The second projection device projects a second feature image onto the target wall.

[0116] S11: The first projection device uses a camera to capture the first feature image and the second feature image projected onto the target wall.

[0117] S12: If there is a gap between the first stitching boundary of the first feature map and the second stitching boundary of the second feature map, adjust the first feature map closer to the second feature map until the first stitching boundary is aligned with the second stitching boundary. If there is an overlap between the first stitching boundary of the first feature map and the second stitching boundary of the second feature map, adjust the first feature map away from the second feature map until the first stitching boundary is aligned with the second stitching boundary.

[0118] S13: Based on the source coordinates of the first feature map before and after adjustment, adjust the source coordinates of the first source image so that the first projected image and the second projected image are aligned and stitched together.

[0119] To implement the method of the embodiments of this application, based on the same inventive concept, the embodiments of this application also provide a screen stitching processing device. Figure 14 This is a schematic diagram of the structure of the image splicing processing device in the embodiments of this application, as shown below. Figure 14 As shown, the image splicing processing device 140 includes:

[0120] Processing unit 1401 is configured to, when the first projection image intersects with the second projection image from the second projection device, plan a target area in the area formed by the first projection image and the second projection image based on a preset ratio.

[0121] The processing unit 1401 is further configured to divide the target area into a first area and a second area by means of a target dividing line; wherein the target dividing line is simultaneously within the first projection screen and the second projection screen;

[0122] The correction unit 1402 is used to correct the first projected image and the second projected image based on the projection coordinates of the first region and the projection coordinates of the second region, so that the first projected image and the second projected image are stitched together.

[0123] The above solution enables the first and second projected images to be stitched together in a square shape, improving the overall display effect of the stitched image.

[0124] In some embodiments of this application, the processing unit 1401 is further configured to select a dividing line from a preset position of the target area based on the splicing method; when the dividing line is simultaneously within the first projection screen and the second projection screen, the target area is divided into the first area and the second area by the target dividing line; wherein, the target dividing line is the dividing line.

[0125] In some embodiments of this application, the processing unit 1401 is further configured to, when the dividing line is in the first projection screen but not in the second projection screen, adjust the dividing line to translate towards the second projection screen until the dividing line is simultaneously in both the first projection screen and the second projection screen; and when the dividing line is in the second projection screen but not in the first projection screen, adjust the dividing line to translate towards the first projection screen until the dividing line is simultaneously in both the first projection screen and the second projection screen.

[0126] In some embodiments of this application, the correction unit 1402 is further configured to determine the source coordinates corresponding to the projection coordinates of the first region based on a first mapping relationship; wherein the first mapping relationship is a mapping relationship between the source coordinate system and the projection coordinate system of the first projection device; based on the source coordinates of the first region, correct the source coordinates of the first source image corresponding to the first projected image to correct the first projected image; and determine the source coordinates corresponding to the projection coordinates of the second region based on a second mapping relationship; wherein the second mapping relationship is a mapping relationship between the source coordinate system and the projection coordinate system of the second projection device; based on the source coordinates of the second region, correct the source coordinates of the second source image corresponding to the second projected image to correct the second projected image.

[0127] In some embodiments of this application, the processing unit 1401 is further configured to use a camera to capture a first feature image and a second feature image projected onto a target wall; wherein the first feature image is projected by the first projection device and the second feature image is projected by the second projection device; if the first stitching boundary of the first feature image is not aligned with the second stitching boundary of the second feature image, the first feature image and / or the second feature image are adjusted until the first stitching boundary is aligned with the second stitching boundary; based on the source coordinates of the adjusted first feature image and / or the source coordinates of the second feature image before and after adjustment, the source coordinates of the first source image and / or the second source image are adjusted so that the first projected image and the second projected image are aligned and stitched.

[0128] In some embodiments of this application, the processing unit 1401 is further configured to, when there is a gap between the first splicing boundary and the second splicing boundary, adjust the first feature image closer to the second feature image, and / or adjust the second feature image closer to the first feature image, until the first splicing boundary and the second splicing boundary are aligned; and when the first splicing boundary and the second splicing boundary overlap, adjust the first feature image to deviate from the second feature image, and / or adjust the second feature image to deviate from the first feature image, until the first splicing boundary and the second splicing boundary are aligned.

[0129] In some embodiments of this application, the processing unit 1401 is further configured to output a prompt message when the first projection screen and the second projection screen do not intersect, until the first projection screen and the second projection screen intersect; wherein the prompt message is used to indicate that the first projection device and / or the second projection device need to be moved.

[0130] This application also provides another electronic device. Figure 15 This is a schematic diagram of the structure of the electronic devices in the embodiments of this application, such as... Figure 15 As shown, the electronic device 150 includes: a processor 1501 and a memory 1502 configured to store a computer program capable of running on the processor; the electronic device may be a first projection device or a second projection device or a separate calibration device;

[0131] When the processor 1501 is configured to run a computer program, it executes the method steps described in the foregoing embodiments.

[0132] Of course, in practical applications, such as Figure 15 As shown, the various components in the electronic device 150 are coupled together via a bus system 1503. It is understood that the bus system 1503 is used to enable communication between these components. In addition to a data bus, the bus system 1503 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 15 The general labeled all buses as Bus System 1503.

[0133] In practical applications, the aforementioned processor can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the functions of the aforementioned processor can also be other types, and the embodiments of this application do not specifically limit this.

[0134] The aforementioned memory can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0135] In an exemplary embodiment, this application also provides a computer-readable storage medium for storing a computer program.

[0136] Optionally, the computer-readable storage medium can be applied to any of the methods in the embodiments of this application, and the computer program causes the computer to execute the corresponding processes implemented by the processor in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0137] For example, embodiments of this application also provide a computer program product, including a computer program that can be executed by a processor of an electronic device to perform the steps of any of the foregoing methods.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0139] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0142] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0143] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0144] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for image stitching, characterized in that, The method includes: When the first projection image intersects with the second projection image from the second projection device, a target area is planned in the area formed by the first projection image and the second projection image based on a preset ratio; The target area is divided into a first area and a second area by a target dividing line; wherein the target dividing line is simultaneously within the first projection screen and the second projection screen; Based on the projection coordinates of the first region and the projection coordinates of the second region, the first projection image and the second projection image are corrected respectively, so that the first projection image and the second projection image are stitched together.

2. The method according to claim 1, characterized in that, The step of dividing the target region into a first region and a second region using target dividing lines includes: Based on the splicing method, a dividing line is selected from a preset position in the target area; When the dividing line is simultaneously within both the first projected image and the second projected image, the target region is divided into the first region and the second region by the target dividing line; wherein, the target dividing line is the dividing line.

3. The method according to claim 2, characterized in that, The method further includes: If the dividing line is in the first projection screen but not in the second projection screen, adjust the dividing line to be translated towards the second projection screen until the dividing line is simultaneously in both the first projection screen and the second projection screen; If the dividing line is in the second projection screen but not in the first projection screen, then the dividing line is adjusted to be shifted towards the first projection screen until the dividing line is simultaneously in both the first projection screen and the second projection screen.

4. The method according to any one of claims 1 to 3, characterized in that, The step of correcting the first projected image and the second projected image based on the projection coordinates of the first region and the projection coordinates of the second region includes: Based on the first mapping relationship, the source coordinates corresponding to the projection coordinates of the first region are determined; wherein, the first mapping relationship is the mapping relationship between the source coordinate system and the projection coordinate system of the first projection device; Based on the source coordinates of the first region, the source coordinates of the first source image corresponding to the first projected image are corrected in order to correct the first projected image. Based on the second mapping relationship, the source coordinates corresponding to the projection coordinates of the second region are determined; wherein, the second mapping relationship is the mapping relationship between the source coordinate system and the projection coordinate system of the second projection device; Based on the source coordinates of the second region, the source coordinates of the second source image corresponding to the second projected image are corrected to correct the second projected image.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: A camera is used to capture a first feature image and a second feature image projected onto a target wall; wherein the first feature image is projected by the first projection device and the second feature image is projected by the second projection device. If the first stitching boundary of the first feature map is not aligned with the second stitching boundary of the second feature map, adjust the first feature map and / or the second feature map until the first stitching boundary is aligned with the second stitching boundary. Based on the source coordinates of the first feature map before and after adjustment and / or the source coordinates of the second feature map before and after adjustment, adjust the source coordinates of the first source image and / or the second source image so that the first projected image and the second projected image are aligned and stitched together.

6. The method according to claim 5, characterized in that, The step of adjusting the first feature map and / or the second feature map until the first splicing boundary aligns with the second splicing boundary when the first splicing boundary of the first feature map is not aligned with the second splicing boundary includes: If there is a gap between the first splicing boundary and the second splicing boundary, adjust the first feature map closer to the second feature map, and / or adjust the second feature map closer to the first feature map, until the first splicing boundary and the second splicing boundary are aligned; If the first splicing boundary overlaps with the second splicing boundary, the first feature map is adjusted to deviate from the second feature map, and / or the second feature map is adjusted to deviate from the first feature map, until the first splicing boundary aligns with the second splicing boundary.

7. The method according to any one of claims 1 to 3, characterized in that, The method further includes: If the first projected image and the second projected image do not intersect, a prompt message is output until the first projected image and the second projected image intersect; wherein, the prompt message is used to indicate that the first projection device and / or the second projection device need to be moved.

8. A video splicing processing device, characterized in that, The device includes: The processing unit is configured to, when the first projection image intersects with the second projection image from the second projection device, plan a target area in the area formed by the first projection image and the second projection image based on a preset ratio; The processing unit is further configured to divide the target region into a first region and a second region by means of a target dividing line; wherein the target dividing line is simultaneously within the first projection screen and the second projection screen; The correction unit is used to correct the first projected image and the second projected image based on the projection coordinates of the first region and the projection coordinates of the second region, so that the first projected image and the second projected image are stitched together.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory configured to store computer programs capable of running on the processor. Wherein, when the processor is configured to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 7.

Images