Projection mapping correction system and method and non-transient computer readable storage medium thereof
By combining a projection device, an image capture device, and a processor, the position of the projection device is automatically calculated and adjusted, solving the problem that projection image correction in existing technologies requires manual labor and is prone to errors, and achieving efficient and accurate projection mapping correction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-11
- Publication Date
- 2026-03-10
AI Technical Summary
In existing technologies, projection image calibration requires a lot of manpower and on-site adjustments are prone to errors, resulting in calibration failures.
By employing a combination of a projection device, an image capture device, and a processor, the system automatically performs projection mapping correction by calculating the difference between the feature points of the projected target object and the light-sculpted feature points. This includes calculating the correction matrix and adjusting the position of the projection device.
It achieves efficient projection mapping correction without human intervention, improving projection accuracy and reducing correction costs.
Smart Images
Figure CN121644776A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a projection mapping correction system, method, and a non-transitory computer-readable storage medium thereof. Specifically, this invention relates to a projection mapping correction system, method, and a non-transitory computer-readable storage medium capable of correctly performing projection image correction. Background Technology
[0002] Generally speaking, when performing architectural light sculpting, in order for the projection device to accurately project the image onto the building, it is necessary to perform an image alignment operation to align the projected image with multiple edges and corners of the building.
[0003] In existing technologies, the alignment of projected images is usually pre-calculated by software, and then on-site professionals make on-site adjustments to the projected images based on the actual projection conditions.
[0004] However, the aforementioned on-site adjustment of projected images requires a large number of professional personnel, increasing costs significantly. Furthermore, because on-site projection conditions often change over time, errors may occur during on-site adjustments, preventing proper projection mapping correction.
[0005] Therefore, providing a projection mapping correction technology that can correctly perform projection image correction is a goal that the industry urgently needs to strive for. Summary of the Invention
[0006] One object of this disclosure is to provide a projection mapping correction system. The projection mapping correction system includes a projection device, an image capture device, and a processor, the processor being communicatively connected to the projection device and the image capture device. The image capture device captures an image frame corresponding to a projection target object. The processor calculates multiple feature points in the image frame corresponding to the projection target object. The projection device projects a light-sculpted image onto the projection target object. The image capture device captures a light-sculpted image frame corresponding to the light-sculpted image. The processor calculates multiple light-sculpted feature points in the light-sculpted image frame corresponding to the projection target object. Based on the multiple feature points and a difference value of each of the multiple light-sculpted feature points, the processor generates a projection mapping correction corresponding to the light-sculpted image.
[0007] In one embodiment of this disclosure, the plurality of feature points includes a plurality of edge points corresponding to the projected target object.
[0008] In one embodiment of this disclosure, the light-sculpted image further includes a correspondence between the plurality of edge points, and generating the projection mapping correction includes the following operation: the processor generates the projection mapping correction corresponding to the light-sculpted image based on the correspondence and the plurality of difference values.
[0009] In one embodiment of this disclosure, the projection mapping correction system further performs the following operations: the projection device projects a corrected light sculpture image onto the projection target object based on the light sculpture image and the projection mapping correction.
[0010] In one embodiment of this disclosure, the projection mapping correction system further performs the following operations: the image capturer captures the LightSculpture image frame corresponding to the corrected LightSculpture image; the processor calculates a projection accuracy of the corrected LightSculpture image based on the LightSculpture image frame; and in response to the projection accuracy being lower than a preset accuracy, the projection device is moved to a new projection position based on the projection mapping correction, wherein the projection device performs a LightSculpture image projection operation corresponding to the projection target object at the new projection position.
[0011] In one embodiment of this disclosure, generating the projection mapping correction includes the following operations: the processor generates at least one correction matrix based on the difference values of the plurality of feature points and the plurality of sculpted feature points respectively; and the processor generates the projection mapping correction based on the at least one correction matrix.
[0012] In one embodiment of this disclosure, generating the at least one correction matrix includes the following operations: the processor cuts a region in the light-sculpted image into multiple parts; and the processor calculates a portion of the correction matrix corresponding to each of the multiple parts based on the multiple feature points and the multiple light-sculpted feature points corresponding to each of the multiple parts of the region to generate the at least one correction matrix.
[0013] In one embodiment of this disclosure, the projection mapping correction system further performs the following operations: the projection device projects a corrected projection image onto the projection target object based on the projection image and the projection mapping correction, wherein the corrected projection image corresponding to the region is generated based on the plurality of partial correction matrices.
[0014] In one embodiment of this disclosure, dividing the light-sculpted image into the plurality of parts includes the following operation: the processor divides the region in the light-sculpted image frame into the plurality of parts based on a preset number of edge points, wherein the number of edge points corresponding to each of the plurality of parts is not greater than the preset number of edge points.
[0015] In one embodiment of this disclosure, the preset number of edge points is lower than the maximum number of edge points in the region of the projected target object.
[0016] Another object of this disclosure is to provide a projection mapping correction method for an electronic system. The electronic system includes a projection device, an image capturer, and a processor. The image capturer is used to capture an image frame corresponding to a projection target object. The projection mapping correction method includes the following steps: calculating a plurality of feature points in the image frame corresponding to the projection target object; projecting a light-sculpted image onto the projection target object; capturing a light-sculpted image frame corresponding to the light-sculpted image; calculating a plurality of light-sculpted feature points in the light-sculpted image frame corresponding to the projection target object; and generating a projection mapping correction corresponding to the light-sculpted image based on the plurality of feature points and a difference value of each of the plurality of light-sculpted feature points.
[0017] Another object of this disclosure is to provide a non-transitory computer-readable storage medium storing a computer program containing multiple program instructions. After being loaded into an electronic system, the computer program executes a projection mapping correction method. The electronic system includes a projection device, an image capturer, and a processor. The image capturer is used to capture an image frame corresponding to a projection target object. The projection mapping correction method includes the following steps: calculating multiple feature points in the image frame corresponding to the projection target object; projecting a light sculpture image onto the projection target object; capturing a light sculpture image frame corresponding to the light sculpture image; calculating multiple light sculpture feature points in the light sculpture image frame corresponding to the projection target object; and generating a projection mapping correction corresponding to the light sculpture image based on the multiple feature points and a difference value of each of the multiple light sculpture feature points.
[0018] The projection mapping correction technology provided in this disclosure (including at least a system, method, and its non-transitory computer-readable storage medium) acquires image frames and light-sculpted image frames corresponding to the projected target object using an image capture device. It then generates a projection mapping correction corresponding to the light-sculpted image by using the difference values between multiple feature points in the image frames and multiple light-sculpted feature points in the light-sculpted image frames. This projection mapping correction technology can automatically perform the correction operation using computer vision technology, efficiently generating projection mapping corrections without human intervention. Therefore, the projection mapping correction technology provided in this disclosure can quickly generate projection mapping corrections, improving the accuracy of projection operations performed by the projection device and reducing the cost of correction operations.
[0019] The following detailed description of the technology and implementation methods of this disclosure, in conjunction with the accompanying drawings, enables those skilled in the art to understand the technical features of the claimed invention. Attached Figure Description
[0020] Figure 1 A schematic diagram of the architecture of the projection mapping correction system according to the first embodiment is shown;
[0021] Figure 2 A schematic diagram illustrating the architecture of an image capturer in some embodiments is shown;
[0022] Figure 3 A schematic diagram showing multiple feature points and multiple light-sculpted feature points in certain embodiments;
[0023] Figure 4 A schematic diagram illustrating a light-sculpted image frame comprising multiple parts in certain embodiments; and
[0024] Figure 5 A partial flowchart of the projection mapping correction method according to the second embodiment is shown.
[0025] Symbol explanation:
[0026] 1: Projection mapping correction system
[0027] PD: Projection device
[0028] ICD: Image Capturer
[0029] PS: Processor
[0030] PTO: Projected target object
[0031] 21: Send / Receive Interface
[0032] 23: Image Capture Unit
[0033] 25: Processor
[0034] 300: Schematic diagram
[0035] R1: Region
[0036] FP1, FP2: Feature points
[0037] LSFP1, LSFP2: Light Sculpting Feature Points
[0038] 400: Schematic diagram
[0039] R2: Region
[0040] S1, S2: Partial
[0041] S1_LSFP1, S1_LSFP2, S1_LSFP3: Light Sculpting Feature Points
[0042] S2_LSFP1, S2_LSFP2, S2_LSFP3: Light Sculpting Feature Points
[0043] 500: Projection Mapping Correction Method
[0044] S501, S503, S505, S507, S509: Steps Detailed Implementation
[0045] The following description, through embodiments, explains the projection mapping correction system, method, and non-transitory computer-readable storage medium provided in this disclosure. However, these embodiments are not intended to limit the implementation of this disclosure to any of the environments, applications, or methods described in the embodiments. Therefore, the description of the embodiments is for illustrative purposes only and is not intended to limit the scope of this disclosure. It should be understood that in the following embodiments and drawings, elements not directly related to this disclosure have been omitted and are not shown, and the dimensions of each element and the proportions between elements are merely illustrative and not intended to limit the scope of this disclosure.
[0046] Firstly, the first embodiment of this disclosure is a projection mapping correction system 1, the schematic diagram of which is shown in... Figure 1 In this embodiment, the projection mapping correction system 1 includes a projection device PD, at least one image capture device ICD, and a processor PS. The processor PS is communicatively connected to the projection device PD and the image capture device ICD. The projection device PD projects a LightSculpt image onto a projection target object PTO (e.g., a building). The image capture device ICD can be positioned around the projection target object PTO to capture image frames corresponding to the projection target object PTO.
[0047] It should be noted that the image capture device (ICD) can be any image capture device with image capture capabilities (e.g., a depth camera lens). The image capture device (ICD) is used to generate real-time images corresponding to a field of view (FOV). For example, the image capture device (ICD) can be positioned in front of the projected target object (PTO) to capture image frames containing the front of the projected target object (PTO).
[0048] In some embodiments, in order to avoid image occlusion and improve the accuracy of correction, the projection mapping correction system 1 may set multiple auxiliary image capture devices (i.e., at least one image capture device ICD) at multiple angle positions of the projection target object PTO to perform correction operations.
[0049] In this embodiment, a schematic diagram of the image capture ICD architecture is shown. Figure 2 The image capture device (ICD) includes a transceiver interface 21, an image capturing unit 23, and a processor 25. The processor 25 is electrically connected to the transceiver interface 21 and the image capturing unit 23, and the transceiver interface 21 is communicatively connected to the processor PS.
[0050] It should be noted that the projection device PD can be any device with LightScribe image projection capabilities. The transceiver interface 21 is an interface capable of receiving and transmitting data, or other interfaces capable of receiving and transmitting data known to those skilled in the art to which this application pertains. The transceiver interface can receive data from sources such as external devices, external web pages, external applications, etc. The image capturing unit 23 can be any camera with image capturing capabilities. The processor PS and processor 25 can be various processing units, central processing units (CPUs), microprocessors, or other computing devices known to those skilled in the art to which this application pertains.
[0051] It should be noted that this disclosure does not limit the number of image captures included in the projection mapping correction system 1. The projection mapping correction system 1 can perform correction operations by using two or more image captures with mutual positioning capabilities (e.g., obtaining its own position and the positions of other image captures through computer vision analysis). In addition, multiple image captures can be set at different positions of the projection mapping correction system 1 (e.g., the left and right sides of the projection device PD) and generate corresponding image frames with different image capturing angles.
[0052] In some embodiments, to save on device costs, the processor PS may be located in other devices or combined with a device with computing capabilities (e.g., a processor using an image capture device ICD). In some embodiments, the projection mapping correction system 1 may also be implemented directly by a projection device PD having an image capture device and a processor (e.g., both the image capture device ICD and the processor PS are integrated into the projection device PD).
[0053] For ease of understanding, the following description will use a projection mapping correction system 1 that includes one image capture device (i.e., an image capture device ICD). Those skilled in the art should be able to understand other implementations with different numbers of image capture devices based on the content of this disclosure, which will not be elaborated here.
[0054] First, in this embodiment, the image capture unit (ICD) captures an image frame corresponding to the projection target object (PTO). The image frame contains part or all of the projection target object (PTO) (e.g., the area to be lightsculpted). In some embodiments, the projection mapping correction system 1 can capture multiple image frames corresponding to different projection target object (PTO) areas using multiple image capture units (ICDs).
[0055] Next, in this embodiment, the processor PS calculates multiple feature points corresponding to the projected target object (PTO) in the image frame, for example, through edge detection or feature detection operations. In some embodiments, the multiple feature points include multiple edge points corresponding to the projected target object (PTO), where an edge point is the intersection of two edges (i.e., a corner point).
[0056] In some implementations, the processor PS may pre-store a three-dimensional model of the projected target object PTO, and calculate the plurality of feature points corresponding to the projected target object PTO using the three-dimensional model and edge computing.
[0057] Next, in this embodiment, the projection device PD projects a LightScribe image onto the projection target object PTO. For example, the projection device PD can receive a LightScribe image and an activation control signal from the processor PS, and the projection device PD initiates the LightScribe projection operation based on the activation control signal.
[0058] In some implementations, the processor PS can generate a light sculpture image in advance based on the content to be projected and a 3D model of the target object PTO, through software calculations, such as using software like madmapper, resolume-arena, etc.
[0059] Next, in this embodiment, the image capture device ICD captures a LightSculpt image frame corresponding to the LightSculpt image, the LightSculpt image frame containing the image of the LightSculpt image projected onto the projection target object PTO.
[0060] Next, in this embodiment, the processor PS calculates multiple light-sculpted feature points corresponding to the projected target object PTO in the light-sculpted image frame. For example, the processor PS can detect feature points (e.g., edge points) of the image actually projected onto the projected target object PTO through edge detection or feature detection operations.
[0061] Finally, in this embodiment, the processor PS generates a projection mapping correction corresponding to the light-sculpted image based on the plurality of feature points and a difference value of each of the plurality of light-sculpted feature points.
[0062] For easier understanding, please refer to Figure 3 . Figure 3 A schematic diagram 300 illustrates the plurality of feature points and the plurality of light-sculpted feature points. In this example, schematic diagram 300 illustrates feature points FP1 and FP2 located in region R1 (the region indicated by the dashed line) of the projected target object PTO. Additionally, schematic diagram 300 illustrates light-sculpted feature points LSFP1 and LSFP2 (corresponding to feature points FP1 and FP2, respectively) generated by the actual projection of the light-sculpted image onto the projected target object PTO.
[0063] In this example, the processor PS calculates the difference between feature point FP1 and sculpted feature point LSFP1 (e.g., difference in distance, depth, direction, etc.), and calculates the difference between feature point FP2 and sculpted feature point LSFP2, generating a projection mapping correction corresponding to the sculpted image.
[0064] In some implementations, the processor PS can calculate a correction matrix corresponding to the light-sculpted image using the difference values of feature points at the same location (e.g., feature point FP1 and light-sculpted feature point LSFP1). Specifically, the processor PS generates at least one correction matrix based on the plurality of feature points and the difference values of each of the plurality of light-sculpted feature points. Then, the processor PS generates the projection mapping correction based on the at least one correction matrix.
[0065] It should be noted that the plurality of sculpted feature points can be corrected using the at least one correction matrix (e.g., differences in distance, depth, direction, etc.) to adjust the projection position to the position of the plurality of feature points. Specifically, the at least one correction matrix may include at least one of a translation transformation matrix, an Euclidean transformation matrix, a similarity transformation matrix, an affine transformation matrix, and a projective transformation matrix.
[0066] In some implementations, the projection device PD or processor PS may adjust the LightSculpture image based on the projection mapping correction, and project the corrected LightSculpture image onto the projection target object PTO. Specifically, the projection device PD projects a corrected LightSculpture image onto the projection target object PTO based on the LightSculpture image and the projection mapping correction.
[0067] In some implementations, the processor PS can control the projection device PD to turn on and off repeatedly to perform the correction operation of generating projection mapping correction until the projection error value is lower than a preset value.
[0068] In some embodiments, the initial projection position of the projection device PD may be unfavorable (e.g., the projection angle is too small or there are obstructions), which may prevent the projection accuracy of the corrected LightScribe image from being improved. In this case, the projection mapping correction system 1 can actively move the position of the projection device PD.
[0069] Specifically, the image capture unit (ICD) captures the LightSculpture image frame corresponding to the corrected LightSculpture image. Next, the processor (PS) calculates a projection accuracy of the corrected LightSculpture image based on the LightSculpture image frame. Finally, in response to the projection accuracy being lower than a preset accuracy, the projection device (PD) is moved to a new projection position based on the projection mapping correction, wherein the projection device (PD) performs a LightSculpture image projection operation corresponding to the projection target object (PTO) at the new projection position.
[0070] For example, the projection mapping correction system 1 can calculate an adjustment displacement value based on the at least one correction matrix via a processor PS. Based on the adjustment displacement value, the processor PS generates a control signal to move the projection device PD.
[0071] In some implementations, the processor PS can transmit control signals to an automated robotic arm to move the projection device PD.
[0072] For example, the projection device PD is mounted on a movable pan-tilt unit. In response to the projection accuracy of the corrected LightSculpture image being lower than a preset value, the processor PS calculates an adjustment displacement value based on the at least one correction matrix and transmits a control signal to the movable pan-tilt unit. The movable pan-tilt unit then moves to the new projection position based on the adjustment displacement value.
[0073] In some implementations, the projection target object (PTO) may contain surfaces with inconsistent depth (e.g., internal and external stairs), causing distortion in the light sculpture image. To mitigate this problem, the projection mapping correction system 1 can segment the region of the light sculpture image into smaller segments, allowing each segment to use its own transformation matrix (e.g., an affine transformation matrix).
[0074] Specifically, the processor PS divides a region in the light-sculpted image into multiple parts. Then, based on the multiple feature points corresponding to each of the multiple parts of the region and the multiple light-sculpted feature points, the processor PS calculates a portion of the correction matrix corresponding to each of the multiple parts to generate the at least one correction matrix.
[0075] It should be noted that since a single matrix may not be sufficient for accurate projection correction, the processor PS can calculate a correction matrix for each part. For example, the processor PS can find a transformation matrix that matches each pair of edge points in the two calculated sets of four edge points.
[0076] For example, the affine transformation matrix can be calculated by the following equation (1):
[0077]
[0078] For an affine transformation matrix, the processor PS can use 3 pairs of xy to find a solution for 6 elements (i.e., variables a to f).
[0079] For example, the projection transformation matrix can be calculated by the following equation (2):
[0080]
[0081] For the projection transformation matrix, the processor PS can use 4 pairs of xy to find a solution with 8 elements (variable i is restricted to 1).
[0082] For example, a transformation matrix can also be composed of translation and transformation matrices, such as the following equation (3):
[0083]
[0084] x′=A 00 x+A 01 y+b0
[0085] y′=A 10 x+A 11 y+b1
[0086] In some embodiments, the light-sculpted image projected onto the projection target object PTO is generated by adjusting the projection image based on multiple partial correction matrices (i.e., the projection image is generated by combining the adjusted projection image portions). Specifically, the projection device PD projects a corrected light-sculpted image onto the projection target object PTO based on the light-sculpted image and the projection mapping correction, wherein the corrected light-sculpted image corresponding to the region is generated based on the multiple partial correction matrices.
[0087] In some implementations, the processor PS divides the region in the light-sculpted image into multiple parts based on a preset number of edge points, wherein the number of edge points corresponding to each of the multiple parts is not greater than the preset number of edge points.
[0088] In some embodiments, a number of the plurality of partial correction matrices corresponds to a number of the plurality of parts, and the at least one correction matrix comprises the plurality of partial correction matrices.
[0089] In some implementations, the preset number of edge points is less than the maximum number of edge points in the region of the projected target object PTO. For example, when the maximum number of edge points in the region is 6, the processor PS divides the region into multiple parts with 3 or fewer edge points (i.e., each part consists of fewer edge points).
[0090] For easier understanding, please refer to Figure 4 . Figure 4 A schematic diagram 400 illustrates the plurality of feature points and the plurality of light-sculpted feature points. In this example, schematic diagram 400 illustrates a region R2 (containing 6 edge points) located on the projected target object PTO, from which the processor PS cuts out multiple portions.
[0091] Taking a region R2 divided into parts S1 and S2 as an example, part S1 contains three edge points S1_LSFP1, S1_LSFP2, and S1_LSFP3, and part S2 contains three edge points S2_LSFP1, S2_LSFP2, and S2_LSFP3. In this example, the processor PS calculates the differences between the edge points S1_LSFP1, S1_LSFP2, and S1_LSFP3 in part S1 and the original feature points to generate a partial correction matrix for the corresponding part S1. Similarly, in this example, the processor PS calculates the differences between the edge points S2_LSFP1, S2_LSFP2, and S2_LSFP3 in part S2 and the original feature points to generate a partial correction matrix for the corresponding part S2.
[0092] In some implementations, the processor PS can reduce the number of cut-out portions by calculating the regions that require the least adjustment for merging.
[0093] In some implementations, the pre-generated light-sculpted image can directly record the correspondence (e.g., positional relationship) of each edge point to accelerate the processor's comparison calculation time. Specifically, the light-sculpted image further includes a correspondence corresponding to the plurality of edge points, and the processor PS generates the projection mapping correction corresponding to the light-sculpted image based on the correspondence and the plurality of difference values.
[0094] In some embodiments, the processor PS may transmit a first control signal to cause the projection device PD to project a LightScribe image onto the projection target object PTO. Then, the processor PS transmits a second control signal to cause the image capture device ICD to capture a LightScribe image frame corresponding to the LightScribe image. The aforementioned switching operation is repeated, and the correction operation is repeatedly performed until the projection error is less than a preset value.
[0095] As described above, the projection mapping correction system 1 provided in this disclosure acquires image frames and light-sculpted image frames corresponding to the projected target object using an image capture device, and calculates the difference values of the plurality of feature points in the image frames and the plurality of light-sculpted feature points in the light-sculpted image frames using a processor, thereby generating a projection mapping correction corresponding to the light-sculpted image. The projection mapping correction system 1 provided in this disclosure can automatically perform the correction operation using computer vision technology, requiring no human intervention and efficiently generating projection mapping corrections. Therefore, the projection mapping correction system 1 provided in this disclosure can quickly generate projection mapping corrections, improving the accuracy of the projection device's projection operation and reducing the cost of the correction operation.
[0096] The second embodiment of this disclosure is a projection mapping correction method, the flowchart of which is shown below. Figure 5 The projection mapping correction method 500 is applicable to an electronic system, such as the projection mapping correction system 1 described in the first embodiment. The electronic system includes a projection device, an image capture device, and a processor, such as the projection device PD, image capture device ICD, and processor PS described in the first embodiment. The image capture device is used to capture an image frame corresponding to a projection target object. The projection mapping correction method 500 generates a projection mapping correction through steps S501 to S509.
[0097] In step S501, the electronic system calculates multiple feature points in the image frame corresponding to the projected target object. Then, in step S503, the electronic system projects a light sculpture image onto the projected target object.
[0098] Subsequently, in step S505, the electronic system captures a light-sculpted image frame corresponding to the light-sculpted image. Next, in step S507, the electronic system calculates multiple light-sculpted feature points in the light-sculpted image frame corresponding to the projected target object.
[0099] Finally, in step S509, the electronic system generates a projection mapping correction corresponding to the light-sculpted image based on the plurality of feature points and a difference value of each of the plurality of light-sculpted feature points.
[0100] In some embodiments, the plurality of feature points include a plurality of edge points corresponding to the projected target object.
[0101] In some embodiments, the light-sculpted image further includes a correspondence between the plurality of edge points, and generating the projection mapping correction includes the following steps: generating the projection mapping correction corresponding to the light-sculpted image based on the correspondence and the plurality of difference values.
[0102] In some embodiments, the projection mapping correction method 500 further includes the following steps: based on the light sculpture image and the projection mapping correction, projecting a corrected light sculpture image onto the projection target object.
[0103] In some embodiments, the projection mapping correction method 500 further includes the following steps: capturing the light-sculpted image frame corresponding to the corrected light-sculpted image; calculating a projection accuracy of the corrected light-sculpted image based on the light-sculpted image frame; and in response to the projection accuracy being lower than a preset accuracy, moving the projection device to a new projection position based on the projection mapping correction, wherein the projection device performs a light-sculpted image projection operation corresponding to the projection target object at the new projection position.
[0104] In some embodiments, generating the projection mapping correction includes the steps of: generating at least one correction matrix based on the difference values of the plurality of feature points and the plurality of sculpted feature points respectively; and generating the projection mapping correction based on the at least one correction matrix.
[0105] In some embodiments, generating the at least one correction matrix includes the steps of: dividing a region in the light-sculpted image into multiple parts; and calculating a portion of the correction matrix corresponding to each of the multiple parts and the multiple light-sculpted feature points to generate the at least one correction matrix.
[0106] In some embodiments, the projection mapping correction method 500 further includes the following steps: based on the light sculpture image and the projection mapping correction, projecting a corrected light sculpture image onto the projection target object, wherein the corrected light sculpture image corresponding to the region is generated based on the plurality of partial correction matrices.
[0107] In some embodiments, dividing the light-sculpted image into the plurality of parts includes the following steps: dividing the region in the light-sculpted image frame into the plurality of parts based on a preset number of edge points, wherein the number of edge points corresponding to each of the plurality of parts is not greater than the preset number of edge points.
[0108] In addition to the steps described above, the second embodiment can also perform all the operations and steps of the projection mapping correction system 1 described in the first embodiment, have the same function, and achieve the same technical effect. Those skilled in the art to which this disclosure pertains can directly understand how the second embodiment performs these operations and steps based on the first embodiment described above, has the same function, and achieves the same technical effect, so it will not be described in detail here.
[0109] The calculation method described in the second embodiment can be implemented by a computer program having multiple instructions. Each computer program can be a file that can be transmitted over a network, or it can be stored in a non-transitory computer-readable storage medium. For each computer program, after the multiple instructions it contains are loaded into an electronic system (e.g., projection mapping correction system 1), the computer program executes the projection mapping correction method described in the second embodiment. The non-transitory computer-readable storage medium can be an electronic product, such as a read-only memory (ROM), a flash memory, a floppy disk, a hard disk, a compact disk (CD), a USB flash drive, a database accessible via a network, or any other storage medium known to those skilled in the art and having the same function.
[0110] In summary, the projection mapping correction technology provided in this disclosure (including at least a system, method, and its non-transitory computer-readable storage medium) acquires image frames and light-sculpted image frames corresponding to the projected target object using an image capture device, and calculates the difference values of the plurality of feature points in the image frames and the plurality of light-sculpted feature points in the light-sculpted image frames using a processor, thereby generating a projection mapping correction corresponding to the light-sculpted image. The projection mapping correction technology provided in this disclosure can automatically perform the correction operation using computer vision technology, requiring no human intervention and efficiently generating projection mapping corrections. Therefore, the projection mapping correction technology provided in this disclosure can quickly generate projection mapping corrections, improving the accuracy of projection operations performed by the projection device and reducing the cost of correction operations.
[0111] The above embodiments are merely illustrative of some implementations of this disclosure and for explaining its technical features, and are not intended to limit the scope and extent of protection of this disclosure. Any changes or equivalent arrangements that can be easily made by those skilled in the art to which this disclosure pertains are within the scope of this disclosure, and the scope of protection of this disclosure is determined by the claims.
Claims
1. A projection mapping correction system, characterized by, A projection device; An image capturer configured to capture an image frame corresponding to a projection target object; and A processor communicatively coupled to the projection device and the image capturer; The projection mapping correction system is configured to perform the following operations: The processor calculates a plurality of feature points corresponding to the projection target object in the image frame; The projection device projects a light sculpture image onto the projection target object; The image capturer captures a light sculpture image frame corresponding to the light sculpture image; The processor calculates a plurality of light sculpture feature points corresponding to the projection target object in the light sculpture image frame; and The processor generates a projection mapping correction corresponding to the light sculpture image based on a plurality of difference values of the plurality of feature points and the plurality of light sculpture feature points. The plurality of feature points include a plurality of edge points corresponding to the projection target object.
2. The projection mapping correction system of claim 1, wherein, The light sculpture image further includes a corresponding relationship corresponding to the plurality of edge points, and generating the projection mapping correction includes the following operations:
3. The projection mapping correction system of claim 2, wherein, The processor generates the projection mapping correction corresponding to the light sculpture image based on the corresponding relationship and the plurality of difference values. The projection mapping correction system is further configured to perform the following operations:
4. The projection mapping correction system of claim 1, wherein, The projection device projects a corrected light sculpture image onto the projection target object based on the light sculpture image and the projection mapping correction. The projection mapping correction system is further configured to perform the following operations:
5. The projection mapping correction system of claim 4, wherein, The image capturer captures the light sculpture image frame corresponding to the corrected light sculpture image; The processor calculates a projection accuracy of the corrected light sculpture image based on the light sculpture image frame; and In response to the projection accuracy being lower than a preset accuracy, the projection device is moved to a new projection position based on the projection mapping correction, wherein the projection device performs a light sculpture image projection operation corresponding to the projection target object at the new projection position. Generating the projection mapping correction includes the following operations:
6. The projection mapping correction system of claim 1, wherein, The processor generates at least one correction matrix based on the difference values of the plurality of feature points and the plurality of light sculpture feature points; and The processor generates the projection mapping correction based on the at least one correction matrix. Generating the at least one correction matrix includes the following operations:
7. The projection mapping correction system of claim 6, wherein, The processor cuts a region in the light sculpture image into a plurality of parts; and The processor calculates a part correction matrix corresponding to each of the plurality of parts based on the plurality of feature points and the plurality of light sculpture feature points corresponding to each of the plurality of parts to generate the at least one correction matrix. The projection mapping correction system is further configured to perform the following operations:
8. The projection mapping correction system of claim 7, wherein, The projection device projects a corrected light sculpture image onto the projection target object based on the light sculpture image and the projection mapping correction, wherein the corrected light sculpture image corresponding to the region is generated based on the plurality of part correction matrices. Cutting the light sculpture image into the plurality of parts includes the following operations:
9. The projection mapping correction system of claim 7, wherein, The processor cuts the region in the light sculpture image frame into the multiple parts based on a preset edge point number, wherein each of the multiple parts corresponds to an edge point number not greater than the preset edge point number.
10. The projection mapping correction system of claim 9, wherein, The preset edge point number is lower than a highest edge point number of the region of the projection target object.
11. A method of projection mapping correction, characterized by, A method for a projection mapping correction is provided. The method includes the following steps: calculating a plurality of feature points in an image frame corresponding to a projection target object; projecting a light sculpture image onto the projection target object; capturing a light sculpture image frame corresponding to the light sculpture image; calculating a plurality of light sculpture feature points in the light sculpture image frame corresponding to the projection target object; and generating a projection mapping correction corresponding to the light sculpture image based on a difference value of each of the plurality of feature points and the plurality of light sculpture feature points.
12. A non-transitory computer-readable storage medium, comprising: A non-transitory computer readable storage medium stores a computer program including a plurality of program instructions. The computer program performs a projection mapping correction method after being loaded into an electronic system. The electronic system includes a projection device, an image capturer, and a processor. The image capturer is used to capture an image frame corresponding to a projection target object. The projection mapping correction method includes the following steps: calculating a plurality of feature points in an image frame corresponding to a projection target object; projecting a light sculpture image onto the projection target object; capturing a light sculpture image frame corresponding to the light sculpture image; calculating a plurality of light sculpture feature points in the light sculpture image frame corresponding to the projection target object; and generating a projection mapping correction corresponding to the light sculpture image based on a difference value of each of the plurality of feature points and the plurality of light sculpture feature points.