Projection calibration processing method and related device

Abstract

A projection calibration processing method and a related device relate to the technical field of projection. The method comprises the following steps: controlling a first projection device to project one or more first geometric figures to a first area; and controlling the first projection device to project the second geometric figure to the first area. And performing projection calibration on the first projection device based on the first image. The first geometry includes at least four vertexes. The second geometric figure is a geometric figure without a vertex. The first image is an image obtained by shooting a first geometric figure and a second geometric figure projected in the first area by the image acquisition device. By adopting the method, the computing power cost can be reduced while the robustness of the calibration algorithm is ensured.

Description

Technical Field

[0001] The present application relates to the field of projection technology, and in particular to a projection calibration processing method and related devices. Background Art

[0002] Projection calibration refers to the calibration of a projection device (such as a projector or a vehicle's pixel lights, high-precision projection light modules, etc.) to ensure that the position and direction of the projected image in space are accurate. The purpose of projection calibration is to improve the quality and accuracy of the projected image and reduce image deformation and distortion. In the projection calibration method, a pattern can be projected by a projection device, and an image acquisition device captures the pattern projected by the projection device. Then, based on the captured pattern, the perspective relationship between the image acquisition device plane and the projection plane is solved to complete the calibration. However, the entire calibration process involves a large amount of computation and a high computing power cost. Therefore, how to reduce the computing power cost while ensuring the robustness of the calibration algorithm requires further research. Summary of the Invention

[0003] The present application provides a projection calibration processing method and related devices, which can reduce computing power costs while ensuring the robustness of the calibration algorithm.

[0004] In a first aspect, the present application provides a projection calibration processing method, the method comprising:

[0005] Controlling the first projection device to project one or more first geometric figures onto the first area; the first geometric figures include at least four vertices;

[0006] Controlling the first projection device to project a second geometric figure onto the first area; the second geometric figure is a geometric figure without vertices;

[0007] The first projection device is projected and calibrated based on the first image; the first image is an image obtained by an image acquisition device shooting the first geometric figure and the second geometric figure projected in the first area.

[0008] In the above scheme, the pattern used for projection calibration includes a first geometric figure with at least four vertices and a second geometric figure without vertices, and the pattern design is simple. At least four vertices of the first geometric figure in the first image obtained by capture can be used as the corner points required for calibration, and there is no need to detect corner points in a complex pattern, thereby reducing the amount of calculation for calibration corner point detection and reducing computing power costs. In addition, the second geometric figure is used to assist in achieving precise calibration, and more accurate calibration may not be achieved using only a few corner points of the first geometric figure. Therefore, the second geometric figure can be used to further obtain corner points for calibration. The corner points obtained by combining the first geometric figure and the second geometric figure can achieve more accurate calibration. And improve the robustness of the algorithm. In addition, the second geometric figure has no vertices to distinguish it from the first geometric figure, so that the first geometric figure can be quickly identified from the pattern and the corner points of the first geometric figure can be detected, further reducing the amount of calculation for calibration corner point detection.

[0009] In a possible implementation, one or more first geometric figures and at least four second geometric figures are simultaneously projected onto the first area, and the at least four second geometric figures surround the first geometric figures; the first image includes the first geometric figures and the at least four second geometric figures.

[0010] In the above scheme, when the first and second geometric figures are projected simultaneously, the second geometric figure can surround the first geometric figure. The number of second geometric figures is at least four. The greater the number of second geometric figures, the more points detected subsequently for calculating the transformation matrix, the higher the error tolerance, and the higher the accuracy of the calculated transformation matrix, which in turn improves calibration accuracy.

[0011] In one possible implementation, one or more first geometric figures and at least four second geometric figures are projected successively into the first area; the first image includes a second image and a third image, the second image is an image obtained by an image acquisition device photographing the one or more first geometric figures projected onto the first area, and the third image is an image obtained by an image acquisition device photographing the at least four second geometric figures projected onto the first area.

[0012] In the above scheme, multiple geometric figures can be projected separately and sequentially, so that multiple images (such as the second image and the third image) can be captured sequentially. For example, one or more first geometric figures can be projected first. The projected first geometric figures can be captured to obtain a second image. Then, the at least four second geometric figures can be projected again. The projected second geometric figures can be captured to obtain a third image. This implementation method can also achieve subsequent accurate projection calibration.

[0013] In a possible implementation, the method further includes: controlling the first projection device to project one or more third geometric figures onto the first area, wherein the one or more third geometric figures are nested inside the first geometric figure.

[0014] In the above scheme, one or more third geometric figures can be nested within the first geometric figure, allowing the first geometric figure to be accurately identified in the first image with the help of the third geometric figures. For example, in a scenario with ground reflections or the intrusion of interfering objects, the first image may include multiple first geometric figures, but only one first geometric figure is projected by the first projection device. Based on this, if a third geometric figure that can be distinguished from the other figures is embedded in the projected first geometric figure, the projected first geometric figure can be accurately identified, thereby ensuring the accuracy of subsequent calibration results.

[0015] In one possible implementation, the above method includes:

[0016] Controlling the second projection device to project one or more fourth geometric figures onto the first area; the fourth geometric figures include at least four vertices;

[0017] Controlling the second projection device to project a fifth geometric figure onto the first area; the fifth geometric figure is a geometric figure without vertices;

[0018] The second projection device is projected and calibrated based on the fourth image; the fourth image is an image obtained by the image acquisition device shooting the fourth geometric figure and the fifth geometric figure projected in the first area.

[0019] In the above solution, for scenes including multiple projection devices, each projection device can be accurately calibrated, so that the multiple projection devices can achieve better fusion projection and reduce the situation where the image fusion effect is poor in the fusion projection.

[0020] In a possible implementation, the method further includes: controlling the second projection device to project one or more sixth geometric figures onto the first area, wherein the one or more sixth geometric figures are nested inside the fourth geometric figure.

[0021] In the above scheme, one or more sixth geometric figures can be nested within the fourth geometric figure, so that the fourth geometric figure can be accurately identified in the fourth image with the help of the sixth geometric figure. For example, in a scenario with ground reflections or interference, the fourth image may include multiple fourth geometric figures of the same or similar shape, but only one fourth geometric figure is projected by the second projection device. Based on this, if the projected fourth geometric figure is embedded with a sixth geometric figure that can be distinguished from the other figures, the projected fourth geometric figure can be accurately identified. This ensures the accuracy of subsequent calibration results.

[0022] In one possible implementation, the performing projection calibration on the first projection device based on the first image includes:

[0023] Obtaining a first matrix based on a first geometric figure in the first image; the first matrix is ​​a transformation relationship matrix from a first preset coordinate system to a second preset coordinate system; the first preset coordinate system is a coordinate system corresponding to the image acquisition device, and the second preset coordinate system is a coordinate system corresponding to the projection;

[0024] Obtaining a second matrix based on the first matrix and the first image; the second matrix is ​​an optimized relationship matrix transformed from the first preset coordinate system to the second preset coordinate system;

[0025] A first transformation matrix is ​​determined according to the second matrix; the first transformation matrix is ​​a transformation relationship matrix from the first preset coordinate system to the second preset coordinate system that is finally calibrated.

[0026] In the above scheme, the initial transformation matrix (i.e., the first matrix) from the first preset coordinate system to the second preset coordinate system is first obtained based on the first geometric figure in the first image. Then, the transformation matrix is ​​further optimized using the first matrix and the captured first image, thereby improving the accuracy of the calibration.

[0027] In one possible implementation, obtaining the first matrix according to the first geometric figure in the first image includes:

[0028] Acquire a first coordinate set according to a first geometric figure in the first image; the first coordinate set includes coordinates of a plurality of vertices of the first geometric figure in a first preset coordinate system;

[0029] A first matrix is ​​determined according to a first coordinate set and a second coordinate set; the second coordinate set includes coordinates of a plurality of vertices of the first geometric figure in a second preset coordinate system.

[0030] In the above solution, the first matrix can be solved by using the coordinates of the multiple points detected in the first image in the first preset coordinate system and the known coordinates of the multiple points in the second preset coordinate system during projection.

[0031] In one possible implementation, obtaining the first coordinate set according to the first geometric figure in the first image includes:

[0032] Acquire a first image region of a first image; the first image region includes a first geometric figure;

[0033] Preprocessing the first image region; the preprocessing includes one or more of the following operations: grayscale processing, denoising processing, and pixel stretching processing;

[0034] A first coordinate set is obtained according to the preprocessed first image region.

[0035] In the above solution, a graphic region containing the first geometric figure is selected from the first image for processing, reducing image search time. Furthermore, the selected image region is preprocessed. Grayscaling and denoising processes can reduce noise captured by the image acquisition device. Pixel stretching can normalize projection content under varying lighting conditions, reducing interference from strong light environments. This improves subsequent calibration accuracy.

[0036] In one possible implementation, obtaining the first coordinate set according to the preprocessed first image region includes:

[0037] enhancing the contrast between the foreground and background of the preprocessed first image region;

[0038] A first coordinate set is obtained according to the first image region after the contrast between the foreground and the background is enhanced.

[0039] In the above solution, the accuracy of corner point coordinate acquisition is improved by enhancing the contrast between the foreground and background of the image and reducing interference from light sources or color and texture of the projection surface.

[0040] In a possible implementation, one or more third geometric figures are nested within the first geometric figure;

[0041] If the first image also includes one or more figures with the same or similar shapes as the first geometric figure, before obtaining the first matrix according to the first geometric figure in the first image, the method further includes:

[0042] From a plurality of graphics with the same or similar shapes in the first image, a graphic having one or more third geometric graphics nested therein is determined as a first geometric graphic.

[0043] In the above scheme, in scenarios such as ground reflection or intrusion of interference objects, the target figure can still be accurately identified through the third geometric figure nested inside the first geometric figure, thereby effectively ensuring the accuracy of subsequent calibration.

[0044] In a possible implementation, the method further includes: acquiring a third coordinate set based on one or more third geometric figures in the first image; the third coordinate set includes coordinates of center points of the one or more third geometric figures in the first preset coordinate system;

[0045] Acquiring a first matrix according to a first geometric figure in the first image includes: acquiring the first matrix according to the first geometric figure in the first image and a third coordinate set.

[0046] In the above scheme, since there is a third coordinate set corresponding to the third geometric figure, even if the vertices of the first geometric figure are blocked or the vertex detection of the first geometric figure is inaccurate, the first matrix can still be calculated well, thereby enhancing the robustness of the algorithm.

[0047] In one possible implementation, the step of obtaining the second matrix based on the first matrix and the first image includes:

[0048] transforming the first image into a first corrected image represented in a second preset coordinate system according to the first matrix;

[0049] A second matrix is ​​obtained based on the first geometric figure in the first image and at least one second geometric figure in the first corrected image.

[0050] In the above scheme, the preliminary relationship matrix (i.e., the first matrix mentioned above) is used to correct and optimize the first image so that more complex and difficult to distinguish feature points with high positioning accuracy (i.e., the feature points of the second geometric figure) can be more accurately detected from the image, thereby achieving a more accurate relationship matrix acquisition and improving calibration accuracy.

[0051] In a possible implementation, the determining of the first transformation matrix according to the second matrix includes: acquiring the first transformation matrix according to the second matrix and the first corrected image.

[0052] In the above solution, a more accurate change relationship matrix can be further obtained based on the second matrix and the first corrected image to further improve the accuracy of calibration.

[0053] In a possible implementation, the method further includes: controlling the first projection device to project the image onto the first area according to the first transformation matrix.

[0054] In the above solution, after the first transformation matrix is ​​obtained, the first projection device can be controlled to project an image according to the calibrated transformation matrix. The distortion of the projected image is greatly improved compared to the image projected before calibration.

[0055] In one possible implementation, obtaining the second matrix according to the first geometric figure in the first image and at least one second geometric figure in the first corrected image includes:

[0056] Divide the first correction pattern into regions according to the region where each second geometric figure is located in the second preset coordinate system when the first pattern is projected, to obtain one or more sub-regions; a sub-region includes a region where a second geometric figure is located in the second preset coordinate system when the first pattern is projected;

[0057] In a case where a second geometric figure meeting a preset condition is detected in each of the at least one sub-area, a second matrix is ​​acquired according to the first geometric figure in the first image and the at least one detected second geometric figure meeting the preset condition.

[0058] In the above scheme, the second geometric figure that meets the conditions can be screened first, thereby proving that the calibration can be continued and ensuring the accuracy of the calibration.

[0059] In a second aspect, the present application provides a projection calibration processing device, which includes a control unit and a processing unit, wherein:

[0060] A control unit, configured to control the first projection device to project one or more first geometric figures onto the first area; the first geometric figures include at least four vertices;

[0061] The control unit is further configured to control the first projection device to project a second geometric figure onto the first area; the second geometric figure is a geometric figure without vertices;

[0062] The processing unit is used to perform projection calibration on the first projection device based on the first image; the first image is an image obtained by the image acquisition device shooting the first geometric figure and the second geometric figure projected in the first area.

[0063] In a possible implementation, one or more first geometric figures and at least four second geometric figures are simultaneously projected onto the first area, and the at least four second geometric figures surround the first geometric figures; the first image includes the first geometric figures and the at least four second geometric figures.

[0064] In one possible implementation, one or more first geometric figures and at least four second geometric figures are projected successively into the first area; the first image includes a second image and a third image, the second image is an image obtained by an image acquisition device photographing the one or more first geometric figures projected onto the first area, and the third image is an image obtained by an image acquisition device photographing the at least four second geometric figures projected onto the first area.

[0065] In a possible implementation, the control unit is further configured to control the first projection device to project one or more third geometric figures onto the first area, wherein the one or more third geometric figures are nested inside the first geometric figure.

[0066] In a possible implementation, the control unit is further configured to: control the second projection device to project one or more fourth geometric figures onto the first area; the fourth geometric figure includes at least four vertices;

[0067] The control unit is further configured to: control the second projection device to project a fifth geometric figure onto the first area; the fifth geometric figure is a geometric figure without vertices;

[0068] The processing unit is further used to: perform projection calibration on the second projection device based on the fourth image; the fourth image is an image obtained by the image acquisition device shooting the fourth geometric figure and the fifth geometric figure projected in the first area.

[0069] In a possible implementation, the control unit is further configured to control the second projection device to project one or more sixth geometric figures onto the first area, wherein the one or more sixth geometric figures are nested inside the fourth geometric figure.

[0070] In one possible implementation, the processing unit is specifically configured to:

[0071] Obtaining a first matrix based on a first geometric figure in the first image; the first matrix is ​​a transformation relationship matrix from a first preset coordinate system to a second preset coordinate system; the first preset coordinate system is a coordinate system corresponding to the image acquisition device, and the second preset coordinate system is a coordinate system corresponding to the projection;

[0072] Obtaining a second matrix based on the first matrix and the first image; the second matrix is ​​an optimized relationship matrix transformed from the first preset coordinate system to the second preset coordinate system;

[0073] A first transformation matrix is ​​determined according to the second matrix; the first transformation matrix is ​​a transformation relationship matrix from the first preset coordinate system to the second preset coordinate system that is finally calibrated.

[0074] In one possible implementation, the processing unit is specifically configured to:

[0075] Acquire a first coordinate set according to a first geometric figure in the first image; the first coordinate set includes coordinates of a plurality of vertices of the first geometric figure in a first preset coordinate system;

[0076] A first matrix is ​​determined according to a first coordinate set and a second coordinate set; the second coordinate set includes coordinates of a plurality of vertices of the first geometric figure in a second preset coordinate system.

[0077] In one possible implementation, the processing unit is further configured to:

[0078] Acquire a first image region of a first image; the first image region includes a first geometric figure;

[0079] Preprocessing the first image region; the preprocessing includes one or more of the following operations: grayscale processing, denoising processing, and pixel stretching processing;

[0080] A first coordinate set is obtained according to the preprocessed first image region.

[0081] In one possible implementation, the processing unit is further configured to:

[0082] enhancing the contrast between the foreground and background of the preprocessed first image region;

[0083] A first coordinate set is obtained according to the first image region after the contrast between the foreground and the background is enhanced.

[0084] In one possible implementation, one or more third geometric figures are nested inside the first geometric figure; if the first image also includes one or more figures with the same or similar shape as the first geometric figure, the processing unit is further used to determine the figure with one or more third geometric figures nested inside as the first geometric figure from the multiple figures with the same or similar shapes in the first image before obtaining the first matrix based on the first geometric figure in the first image.

[0085] In one possible implementation, the processing unit is also used to: obtain a third coordinate set based on one or more third geometric figures in the first image; the third coordinate set includes the coordinates of the center points of one or more third geometric figures in the first preset coordinate system; the processing unit is also specifically used to: obtain a first matrix based on the first geometric figure and the third coordinate set in the first image.

[0086] In one possible implementation, the processing unit is further specifically configured to:

[0087] transforming the first image into a first corrected image represented in a second preset coordinate system according to the first matrix;

[0088] A second matrix is ​​obtained based on the first geometric figure in the first image and at least one second geometric figure in the first corrected image.

[0089] In one possible implementation, the processing unit is further specifically configured to:

[0090] A first transformation matrix is ​​obtained according to the second matrix and the first corrected image.

[0091] In a possible implementation, the control unit is further configured to: control the first projection device to project the image onto the first area according to the first transformation matrix.

[0092] In one possible implementation, the processing unit is further configured to:

[0093] Divide the first correction pattern into regions according to the region where each second geometric figure is located in the second preset coordinate system when the first pattern is projected, to obtain one or more sub-regions; a sub-region includes a region where a second geometric figure is located in the second preset coordinate system when the first pattern is projected;

[0094] In a case where a second geometric figure meeting a preset condition is detected in each of the at least one sub-area, a second matrix is ​​acquired according to the first geometric figure in the first image and the at least one detected second geometric figure meeting the preset condition.

[0095] In a third aspect, embodiments of the present application provide a projection calibration processing device, comprising a processor. The processor may be coupled to a memory. When the processor executes a computer program or computer instructions stored in the memory, it may implement any of the methods described in the first aspect. Exemplarily, the device may also include a communication interface for communicating with other devices. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0096] In one possible implementation, the apparatus may include:

[0097] Memory for storing computer programs or computer instructions;

[0098] The processor is configured to: control a first projection device to project one or more first geometric figures onto a first area; control the first projection device to project a second geometric figure onto the first area; and perform projection calibration on the first projection device based on a first image. The first geometric figure includes at least four vertices; the second geometric figure is a geometric figure without vertices; and the first image is an image captured by an image acquisition device capturing the first and second geometric figures projected onto the first area.

[0099] It should be noted that the computer programs or computer instructions in the memory of this application can be pre-stored or downloaded from the Internet and stored when the device is used. This application does not specifically limit the source of the computer programs or computer instructions in the memory. The coupling in the embodiments of this application is an indirect coupling or connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.

[0100] In a fourth aspect, an embodiment of the present application provides a projection system, which includes a control device and a projection module, the projection module includes a first projection device, or the projection module includes a first projection device and a second projection device, and the control device is used to execute the method described in any one of the first aspects above.

[0101] In a fifth aspect, an embodiment of the present application provides a vehicle, which includes the projection system described in the fourth aspect above.

[0102] In a sixth aspect, the present application provides a computer-readable storage medium, which stores a computer program or computer instructions, and the aforementioned computer program or computer instructions are executed by a processor to implement any method of the above-mentioned first aspect.

[0103] In a seventh aspect, the present application provides a computer program product. When the computer program product is executed by a processor, any method of the above-mentioned first aspect will be implemented.

[0104] In an eighth aspect, the present application provides a chip, comprising a logic circuit and an interface, wherein the logic circuit and the interface are coupled, the interface being used to input and / or output information, and the logic circuit being used to execute any of the methods of the first aspect.

[0105] In a ninth aspect, the present application provides a projection method, the method comprising:

[0106] Controlling the first projection device and the second projection device to project the same first picture in the first area; the first picture projected by the first projection device and the first picture projected by the second projection device do not overlap;

[0107] Controlling a first projection device to project a first pattern onto a first area; the first pattern includes a first geometric figure and a second geometric figure, the first geometric figure includes at least four vertices, the first geometric figure is surrounded by at least four second geometric figures, and the second geometric figure is a geometric figure without vertices; the first pattern is used to calibrate the first projection device;

[0108] Controlling the second projection device to project a second pattern onto the first area; the second pattern includes a fourth geometric figure and a fifth geometric figure, the fourth geometric figure includes at least four vertices, the fourth geometric figure is surrounded by at least four fifth geometric figures, and the fifth geometric figure is a geometric figure without vertices; the second pattern is used to calibrate the second projection device;

[0109] After the first projection device and the second projection device are calibrated, the first projection device and the second projection device are controlled to project the same second picture in the first area; the second picture projected by the first projection device and the second picture projected by the second projection device overlap.

[0110] In the above solution, for a scene including at least two projection devices, each projection device can be accurately calibrated, so that multiple projection devices can achieve better fusion projection and reduce the situation where the image fusion effect in the fusion projection is poor.

[0111] The solutions provided in the second to ninth aspects are used to implement or cooperate with the methods provided in the first aspect, and therefore can achieve the same or corresponding beneficial effects as the methods corresponding to the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Figure 1 and Figure 2 Shown is a schematic diagram of the system architecture provided in an embodiment of the present application.

[0113] Figure 3 The figure shows a schematic diagram of the relationship between the projection plane and the imaging plane of the image acquisition device provided in an embodiment of the present application.

[0114] Figure 4 and Figure 5 Shown is a schematic diagram of the projection calibration process provided in an embodiment of the present application.

[0115] Figure 6 Shown is a schematic diagram of the fusion projection provided in an embodiment of the present application.

[0116] Figure 7 Shown is a schematic flow chart of the method provided in an embodiment of the present application.

[0117] Figure 8 Shown is a projection schematic diagram of the projection device provided in an embodiment of the present application.

[0118] Figures 9 to 11 A schematic diagram of the projection pattern provided in an embodiment of the present application.

[0119] Figure 11A A schematic diagram of the projection pattern provided in an embodiment of the present application.

[0120] Figures 12 to 13 A schematic diagram of the projection pattern provided in an embodiment of the present application.

[0121] Figure 14 A schematic diagram of a process for detecting graphics by enhancing image contrast provided in an embodiment of the present application.

[0122] Figure 15 A schematic diagram of a projection device provided in an embodiment of the present application projecting a pattern in a ground reflective scene.

[0123] Figure 16 An image schematic diagram provided for an embodiment of the present application.

[0124] Figure 17 A schematic diagram of dividing the sub-area where the second geometric figure is located provided in an embodiment of the present application.

[0125] Figure 18 A schematic diagram of a process for detecting graphics by enhancing image contrast provided in an embodiment of the present application.

[0126] Figure 19 A schematic diagram of dividing the sub-area where the second geometric figure is located provided in an embodiment of the present application.

[0127] Figure 20 A schematic diagram of the projection pattern provided in an embodiment of the present application.

[0128] Figure 21 and Figure 22 A schematic diagram of the device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0129] In the embodiment of the present application, "multiple" refers to two or more. In the embodiment of the present application, "and / or" is used to describe the association relationship of associated objects, indicating three relationships that can exist independently. For example, A and / or B can be expressed as follows: A exists alone, B exists alone, or A and B exist at the same time. The description methods such as "at least one of a1, a2, ... and an" used in the embodiment of the present application include the situation where any one of a1, a2, ... and an exists alone, and also include any combination of any multiple of a1, a2, ... and an, each of which can exist alone; for example, the description method of "at least one of a, b and c" includes the situation where a is alone, b is alone, c is alone, a and b combination, a and c combination, b and c combination, or abc combination.

[0130] In this application, the terms "first," "second," and the like are used to distinguish between identical or similar items having substantially the same function or effect. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another.

[0131] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0132] The following is an exemplary introduction with reference to the accompanying drawings.

[0133] First, we will introduce the projection processing system that may be applicable to the embodiments of the present application. For example, see Figure 1 . Figure 1 A possible projection processing system is shown as an example, which may include a control device 101, a first projection device 102, and an image acquisition device 103. The first projection device 102 and the image acquisition device 103 may be connected to the control device 101.

[0134] For example, the control device 101 can be used to control the first projection device 102 to project a pattern, image, or video. The control device 101 can also be used to control the image acquisition device 103 to capture an image or video. The image acquisition device 103 can then send the captured image or video to the control device 101. The control device 101 can process the received image or video. The image acquisition device 103 can be, for example, a device including an imaging system, such as a camera or a webcam.

[0135] For example, in one possible implementation, control device 101 controls first projection device 102 to project a calibration pattern. Then, control device 101 controls image acquisition device 103 to capture the projected calibration pattern and acquire an image. Image acquisition device 103 transmits the captured image to control device 101. Control device 101 can calibrate first projection device 102 based on the image.

[0136] By way of example, the first projection device 102 may include multiple pixel units for emitting light. Each pixel unit can be individually controlled to be turned on or off. A pixel unit being on indicates that the pixel unit is controlled to emit light, i.e., illuminated. A pixel unit being off indicates that the pixel unit is controlled not to emit light, i.e., extinguished. By controlling the illumination or extinguishing of each of the multiple pixel units, the light emitted by the first projection device 102 can be projected to form various patterns or shapes, or to form various images. By way of example, the first projection device 102 may be a light-emitting module implemented based on digital light processing (DLP). Alternatively, by way of example, the first projection device 102 may be a light-emitting module including multiple light-emitting diodes (LEDs). Alternatively, by way of example, the first projection device 102 may be a light-emitting module based on reflective liquid crystal projection technology. This light-emitting module based on reflective liquid crystal projection technology may include a liquid crystal on silicon (LCoS) chip. An LCoS chip may integrate millions of pixel electrodes (i.e., pixel units). The description here is only an example and does not constitute a limitation on the embodiments of the present application. The embodiments of the present application do not limit the implementation of the first projection device 102.

[0137] In one example, the above Figure 1 The projection processing system shown can be a system deployed in a projector. Then, the control device 101 can be a control module in the projector. The first projection device 102 can be a projection module in the projector. The image acquisition device 103 can be a camera in the projector.

[0138] In another example, the projection processing system may be a system deployed on a vehicle. Then, the control device 101 may be a controller in the vehicle. The first projection device 102 may be a pixel headlight in the vehicle or a projection device in the vehicle cabin. The image acquisition device 103 may be, for example, a camera in the vehicle. If the first projection device 102 is a pixel headlight in the vehicle, then the image acquisition device 103 may be a forward-looking camera in the vehicle, etc. If the first projection device 102 is a projection device in the vehicle cabin, then the image acquisition device 103 may be a camera in the vehicle cabin. Alternatively, the image acquisition device 103 may be a camera built into the projection device.

[0139] Exemplarily, the above-mentioned control device 101 can be, for example, a domain controller in a vehicle. Exemplarily, the control device 101 can be, for example, an intelligent driving computing module MDC in a vehicle. The MDC can be called a mobile data center (MDC), or can be called a motion domain controller (MDC), etc. Alternatively, the control device 101 can be, for example, a cockpit domain controller (CDC) in a vehicle. Alternatively, the control device 101 can be a combination of the MDC and the CDC. Alternatively, in other possible implementations, it can also be other domain controllers in the vehicle, such as a vehicle domain controller (VDC) or a vehicle control unit (VCU), etc. It can be understood that the name of the domain controller described here is only an example, and in other possible implementations it can be other names, but the corresponding functions can also be implemented.

[0140] For example, if the first projection device 102 is a pixel headlight in a vehicle, then the first projection device 102 can be a front pixel headlight on the left side of the vehicle. Alternatively, it can be a front pixel headlight on the right side of the vehicle. Alternatively, it can be a headlight of the vehicle, etc. This embodiment of the application is not limited to this.

[0141] In another possible implementation, for example, see Figure 2 The projection processing system may further include a second projection device 104. This second projection device 104 is also connected to the control device 101. The control device 101 can also be used to control the second projection device 104 to project a pattern, image, or video. For the remaining description of the second projection device 104, refer to the description of the first projection device 102 and will not be repeated here.

[0142] In one exemplary embodiment, if the above Figure 2The projection processing system shown is a system deployed on a vehicle. Therefore, the first projection device 102 and the second projection device 104 can be the headlights on the left and right sides of the vehicle. The remaining description is based on the aforementioned related introduction and is not repeated here.

[0143] It is understandable that the above Figure 1 and Figure 2 The projection processing system shown is only an example and does not constitute a limitation to the embodiments of the present application.

[0144] Projection devices generally need to be calibrated to ensure that the position and direction of the projected image in space are accurate to improve the user's viewing experience. Figure 2 The projection processing system shown in FIG2 is used to describe a scenario in which the two projection devices perform fused projection. In this scenario, accurate projection calibration of the two projection devices can make the projection positions of the two projection devices controllable. If the calibration accuracy is high, pixel-level fused projection can also be achieved, reducing the situation where the projection image fusion effect is poor. To facilitate understanding of the embodiments of the present application, the following is an illustrative introduction to the projection calibration scenarios that may be involved in the embodiments of the present application.

[0145] In a possible implementation scenario, the projection processing system is, for example, the above Figure 1 As shown, only one projection device, the first projection device 102, is included. Therefore, it is only necessary to perform projection calibration on the first projection device 102. Exemplarily, performing projection calibration on the first projection device 102 is to obtain a transformation relationship matrix from the first preset coordinate system to the second preset coordinate system. For the convenience of subsequent description, the transformation relationship matrix from the first preset coordinate system to the second preset coordinate system is referred to as the first transformation matrix. The first preset coordinate system is the coordinate system corresponding to the image acquisition device 103 in the projection processing system. For example, the first preset coordinate system can be an image coordinate system or a pixel coordinate system corresponding to the image acquisition device 103. The second preset coordinate system is the coordinate system corresponding to the first projection device 102. For example, the second preset coordinate system can be a pixel coordinate system corresponding to the first projection device 102. If the first projection device 102 is a pixel headlight of a vehicle, then the second preset coordinate system can be the pixel coordinate system of the pixel headlight.

[0146] For example, the transformation relationship matrix from the first preset coordinate system to the second preset coordinate system may be, for example, a perspective transformation relationship matrix from the first preset coordinate system to the second preset coordinate system. Figure 3 Take this as an example.

[0147] See for example Figure 3 , it is assumed that the content projected onto the ground by the first projection device 102 is a triangle pattern. Figure 3 exemplarily shows the position of the triangle in the plane of the pixel coordinate system (i.e., the second preset coordinate system) of the first projection device 102. Then, the image acquisition device 103 captures the projected triangle on the ground to obtain an image. The imaging plane of the image is the plane of the first preset coordinate system. Figure 3 The position of the triangle obtained by shooting is also shown as an example in the plane where the first preset coordinate system is located. Figure 3 It can be seen that there is a perspective relationship between the triangle in the second preset coordinate system and the triangle projected onto the ground. There is also a perspective relationship between the triangle in the first preset coordinate system and the triangle projected onto the ground. Then, there is also a perspective relationship from the first preset coordinate system to the second preset coordinate system. It is only necessary to know the coordinates of at least four points in the first preset coordinate system and the coordinates corresponding to the at least four points in the second preset coordinate system to solve the perspective transformation matrix from the first preset coordinate system to the second preset coordinate system. The specific algorithm for solving the perspective transformation matrix is ​​not limited in the embodiment of this application, and any feasible solution algorithm can be used, which is not described in detail in the embodiment of this application.

[0148] It is understandable that the above Figure 3 What is shown is merely an example and does not constitute a limitation to the embodiments of the present application.

[0149] For example, after the calibration of the first projection device 102 is completed, the first transformation matrix can be obtained. The first projection device 102 can then be controlled to project image content based on the first transformation matrix. For example, if the coordinates of the projected content in the first preset coordinate system are known, the coordinates of the projected content in the second preset coordinate system can be obtained using the calibrated first transformation matrix. This allows the first projection device 102 to be precisely controlled to project the content, thereby achieving controllable projection position.

[0150] In another possible implementation scenario, the projection processing system is, for example, the above Figure 2 As shown, it includes a first projection device 102 and a second projection device 104. If the first projection device 102 and the second projection device 104 are to achieve fusion projection, both the first projection device 102 and the second projection device 104 need to be calibrated. In one example, it can be exemplified by referring to Figure 4 .exist Figure 4As can be seen, the projection calibration process for the second projection device 104 is similar to that for the first projection device 102. In both cases, after the projection device projects a pattern, the image acquisition device captures an image of the projected pattern, and then calibrates the corresponding projection device based on the captured image. The difference is that the projection calibration for the second projection device 104 involves acquiring a transformation matrix from the first preset coordinate system to a third preset coordinate system. The third preset coordinate system is the coordinate system corresponding to the second projection device 104. For example, the third preset coordinate system can be the pixel coordinate system corresponding to the second projection device 104. If the second projection device 104 is a pixel headlight of a vehicle, then the third preset coordinate system can be the pixel coordinate system of the pixel headlight. The transformation matrix from the first preset coordinate system to the third preset coordinate system can, for example, be the perspective transformation matrix from the first preset coordinate system to the third preset coordinate system. For more information on projection device calibration, please refer to the aforementioned description of projection calibration for the first projection device 102 and will not be repeated here. After calibration is completed, the first and second projection devices can merge their projections.

[0151] In another example, see Figure 5 As shown, Figure 4 The difference is that the image acquisition device also captures an image when neither of the two projection devices projects a pattern. This image is the background image. After the projection device projects the pattern, the image captured by the image acquisition device can be subtracted from the background image to obtain a foreground image. The foreground image is an image including the projected pattern. Exemplarily, for example, a foreground extraction algorithm or a background removal algorithm can be used to obtain a foreground image based on the background image and the image captured after the pattern is projected. The specific implementation process of this embodiment of the present application is not described in detail. After obtaining the foreground image, the projection calibration of the projection device is performed based on the foreground image. In this implementation scheme, the image after the background is removed is used for projection calibration, which can reduce background interference and improve the accuracy of calibration.

[0152] By way of example, after completing the projection calibration of the first projection device 102, a transformation matrix from the first preset coordinate system to the second preset coordinate system can be obtained. This is referred to as the first transformation matrix. Similarly, after completing the projection calibration of the second projection device 104, a transformation matrix from the first preset coordinate system to the third preset coordinate system can be obtained. For ease of subsequent description, the transformation matrix from the first preset coordinate system to the third preset coordinate system will be referred to as the second transformation matrix.

[0153] For example, in one possible implementation, after obtaining the first and second transformation matrices, a third transformation matrix can be combined to achieve fused projection of the first projection device 102 and the second projection device 104. The third transformation matrix is ​​a matrix representing the change relationship from a fourth preset coordinate system to the first preset coordinate system. The fourth preset coordinate system can be, for example, the original coordinate system of the original content controlled by the control device 101 for projection. For example, the control device 101 controls the projection of content in the user interface of an application. In this case, the fourth preset coordinate system can be the coordinate system of the user interface of the application. The user interface of the application is the medium for interaction or information exchange between the user and the device. For example, the third transformation matrix can be a preset matrix, or a matrix determined based on the first and second transformation matrices. This embodiment of the present application does not limit the specific implementation method for determination. For example, the third transformation matrix can be different for different projection aspect ratios. For example, the third transformation matrix corresponding to a projection aspect ratio of 16:9 is different from the third transformation matrix corresponding to a projection aspect ratio of 4:3. It should be understood that the description herein is merely illustrative and does not constitute a limitation on the embodiments of the present application. In order to understand the process of fusion projection, you can refer to the example Figure 6 shown.

[0154] See for example Figure 6 As can be seen, the original content to be projected can be transformed into the first preset coordinate system using the third transformation matrix, obtaining the content represented in the first preset coordinate system. This step can be understood as first mapping the original content to the imaging plane of the image acquisition device. Then, the content represented in the first preset coordinate system is mapped to the second preset coordinate system using the first transformation matrix, obtaining the content represented in the second preset coordinate system. This step can be understood as mapping the content of the imaging plane of the image acquisition device to the pixel coordinate system of the first projection device. Furthermore, the content represented in the first preset coordinate system is mapped to the third preset coordinate system using the second transformation matrix, obtaining the content represented in the third preset coordinate system. This step can be understood as mapping the content of the imaging plane of the image acquisition device to the pixel coordinate system of the second projection device. The first projection device can then project based on the content represented in the second preset coordinate system, and the second projection device can project based on the content represented in the third preset coordinate system, achieving fused projection.

[0155] As mentioned above, projection calibration is crucial for projectors. However, existing projection calibration solutions use either a QR code or a combination of a checkerboard and a QR code as calibration patterns. However, these calibration solutions require significant computational power and time to extract the required information from the complex patterns.

[0156] In addition, for solutions that use QR codes as calibration patterns to implement projection calibration, rectangular corner features are obtained from the QR code pattern through contour extraction. However, these rectangular corner features are significantly affected by the environment, algorithm, and optical machine, resulting in low feature detection precision and accuracy. This affects the accuracy and precision of the calibration and cannot meet the requirements of pixel-level fusion scenarios. For solutions that use a combination of a checkerboard and a QR code as a calibration pattern to implement projection calibration, the checkerboard detection algorithm relies on the integrity of the pattern. If a corner of the checkerboard is blocked, the entire detection algorithm will fail. In complex outdoor environments, it is difficult to provide an ideal projection environment, so the calibration success rate is low.

[0157] Based on the above description, the embodiments of the present application provide a projection calibration processing method and related apparatus, which can reduce computing power and time costs while ensuring the robustness of the calibration algorithm. Furthermore, it can also improve the success rate and accuracy of projection calibration.

[0158] The following first introduces the projection calibration processing method provided by the embodiment of the present application. The method can be executed by a control device in the projection processing system. The control device can be, for example, Figure 1 or Figure 2 The control device 101 shown. Figure 7 , the method may include but is not limited to steps S701 to S702.

[0159] S701, controlling a first projection device to project one or more first geometric figures onto a first area, and controlling the first projection device to project a second geometric figure onto the first area; the first geometric figure includes at least four vertices; the second geometric figure is a geometric figure without vertices.

[0160] For example, the first projection device may be Figure 1 or Figure 2 The first projection device 102 in the projection processing system is shown.

[0161] For example, if the first projection device 102 is a projector or a projection module in an in-vehicle projection device, then the first area can be any area that can be projected, such as a projection screen area or a wall area. If the first projection device 102 is a pixel headlight in a vehicle, then the first area can be, for example, a ground area, a wall area, a slope area, or any other area that can be projected by the headlight, and this embodiment of the application does not limit this.

[0162] For example, the control device may control the first projection device to project the first geometric figure and the second geometric figure into the first area. The first geometric figure and the second geometric figure are projected into the projection area of ​​the first projection device. The projection area of ​​the first projection device is the area covered by the projection range of the first projection device in the first area. For easier understanding of the projection area, please refer to the following example: Figure 8 .exist Figure 8 , the first area is shown as an example. As can be seen, the first projection device projects onto the first area, and the area covered by the projection range is the projection area. For example, the first geometric figure and the second geometric figure can be projected at any position within the projection area. Figure 8 The projection area shown is illustrated by taking a rectangle as an example. For example, if the first area is the ground or a slope, the projection ratio will be different due to the different distances. For example, the farther the projection distance is, the larger the pattern will be. Because the part farther away from the first projection device is, the larger the projection will be. Based on this, the projection area actually projected onto the ground area or the slope area appears more like a trapezoid. The embodiments of this application will not be described one by one. For the convenience of subsequent introduction, the projection area below is illustrated by taking a rectangular shape as an example.

[0163] For example, the first geometric figure can be a polygon such as a right rectangle, a pentagon, or a hexagon. Alternatively, the first geometric figure can be a concave polygon such as a concave quadrilateral or a pentagram. It should be understood that the description of the first geometric figure herein is merely illustrative and does not constitute a limitation on the embodiments of the present application.

[0164] For example, if the first projection device projects multiple first geometric figures onto the first area, the multiple first geometric figures may be the same or different. For example, if two first geometric figures are projected, one may be a rectangle and the other may be a hexagon. Alternatively, both may be rectangles. It should be understood that the description herein is merely illustrative and does not constitute a limitation on the embodiments of the present application.

[0165] Exemplarily, the above-mentioned second geometric figure can be, for example, a circular spot shape or an elliptical shape. Alternatively, the second geometric figure can be a regular n-gon that is infinitely close to a circle. The n can be an integer greater than or equal to 8. Because the second geometric figure is required to be a full figure without vertices. The center point of this full circle or ellipse without vertices has a high probability of being detected in subsequent image detection, and the detection accuracy is high. This is due to the subsequent detection algorithm of the second geometric figure. This is conducive to improving the accuracy of subsequent projection calibration. If the second geometric figure is a regular n-gon, and the regular n-gon is infinitely close to a circle, then the regular n-gon can be approximated as a circle. Similarly, the probability of being detected in subsequent image detection is high, and the detection accuracy is high. For the detection of the second geometric figure, please refer to the subsequent introduction and will not be repeated here.

[0166] For example, in one possible implementation, the first projection device projects at least four second geometric figures onto the first area. And the at least four second geometric figures and the one or more first geometric figures are projected onto the first area at the same time. In this case, the at least four second geometric figures are surrounded by the outside of the first geometric figure. If the second geometric figures are less than four, subsequent precise calibration will not be completed. The more the number of second geometric figures, the better the anti-interference ability and robustness. For example, in one possible implementation, in order to improve the robustness of the algorithm and take into account the computing power cost, the number of second geometric figures can be set between 10 and 20. For ease of understanding, please refer to the example. Figure 9 . Figure 9 The projection of a first geometric figure is taken as an example.

[0167] Figure 9 Seven possible first patterns (a) to (g) are shown in FIG. The first pattern includes the first geometric figure and the second geometric figure. It can be seen that Figure 9 In (a) and (b), the case where the first pattern is located on the right side of the projection area is taken as an example. Figure 9 In (c), (e) and (f), the first pattern is located in the middle of the projection area as an example. Figure 9 In (d) and (g), the first pattern is located on the left side of the projection area. The top, bottom, left, right or center of the projection area is distinguished by the user standing in the projection area facing the projection area. It can be understood that Figure 9 Shown are examples only.

[0168] in, Figure 9 In the first pattern shown in (a), (b) and (c), the first geometric figure is a right-angled rectangle, which can be seen in FIG. Figure 9 The second geometric figure is a circular spot shape, which can be seen in (a), (b) and (c). Figure 9As shown in (a), (b) and (c) ②. It can be seen that there are multiple circular spot-shaped graphics surrounding the right-angled rectangle. For example, see Figure 9 (b), some of the circular spot-shaped graphics can also be framed. The frame can be exemplified by Figure 9 ③ in (b) can be a rounded rectangle. Exemplarily, the corners of the rounded rectangle outer frame are rounded to distinguish them from the right angles of the right-angled rectangle.

[0169] For example, see Figure 9 As shown in (d), (e) and (f), the first geometric figure can be a five-pointed star, a pentagon and a hexagon respectively. The second geometric figure is also in the shape of a circular spot. In addition, see Figure 9 As shown in (f), some of the circular spot-shaped graphics can also be framed. It can be seen that the frame is not limited to a rounded rectangle, but can also be a triangle or any other shape. For example, see Figure 9 As shown in ③ in (f).

[0170] For example, see Figure 9 As shown in (g), the first pattern includes not only the first geometric figure (shown as a rectangle) and the second geometric figure (shown as a circular spot shape), but also prompt words and other textures. For example, the prompt words can be exemplified by Figure 9 As shown in (g) ④. Other textures can be exemplified by Figure 9 As shown in ⑤ in (g). Figure 9 The prompt word shown in (g) is "Automatic calibration". It is understandable that this prompt word is only an example and does not constitute a limitation on the embodiments of the present application. In a specific implementation, the prompt word can be any other expression, and the embodiments of the present application do not limit this. In addition, Figure 9 The other textures shown in (g) ⑤ are also only examples and do not constitute a limitation on the embodiments of the present application. In a specific implementation, the first pattern may also include other arbitrary texture patterns, which are not limited in the embodiments of the present application.

[0171] The graphics projected by the first projection device include a first geometric figure with at least four vertices and a second geometric figure surrounding the first geometric figure, and the pattern design is simple. This allows at least four vertices of the first few figures in the first image subsequently captured to serve as the corner points required for calibration. This eliminates the need to detect corner points in complex patterns, reducing the computational effort required for calibration corner point detection and lowering computing power costs. Furthermore, at least four second geometric figures are provided outside the first geometric figure in the pattern. The second geometric figures are used to assist in achieving precise calibration. Because more precise calibration may not be achieved using only a few corner points of the first geometric figure, the second geometric figure can be used to further obtain corner points for calibration. Combining the corner points obtained from the first and second geometric figures allows for more precise calibration. Furthermore, the second geometric figure has no vertices to distinguish it from the first geometric figure, allowing the first geometric figure to be quickly identified from the pattern and its corner points to be detected, further reducing the computational effort required for calibration corner point detection.

[0172] In one possible implementation, the control device can also control the first projection device to project one or more third geometric figures onto the first area. One or more third geometric figures are embedded in the first geometric figure. The shape of the third geometric figure may be different from or the same as the shape of the first geometric figure. For example, the third geometric figure may be a circular spot shape, or may be a polygon such as a triangle or a quadrilateral, or may be any shape such as a concave quadrilateral or a pentagonal star. The embodiment of the present application does not limit the shape of the third geometric figure. In order to facilitate the understanding of the implementation of including the third geometric figure in the first pattern, you can refer to the example. Figure 10 shown. Figure 10 Combined with the above Figure 9 (a) is used as an example, and the others are similar and will not be described in detail. Figure 10 ⑥ in (a), (b) and (c). It can be seen that one or more third geometric figures are nested inside the first geometric figure. Figure 10 (a), (b) and (c) in FIG. 3 are respectively illustrated by taking the third geometric figure as a circular spot, a triangle and a right-angled rectangle as examples. It can be understood that Figure 10 What is shown is merely an example and does not constitute a limitation to the embodiments of the present application.

[0173] For example, if the first projection device projects two identical first geometric figures onto the first area, the third geometric figures nested within the two first geometric figures may be different, so as to distinguish the two identical first geometric figures.

[0174] For example, the embodiment of the present application does not limit the sizes of the first geometric figure, the second geometric figure, and the third geometric figure. The above description of the projected geometric figures is only an example and does not constitute a limitation on the embodiment of the present application.

[0175] In a possible implementation, since the graphics projected by the first projection device include multiple geometric graphics, the multiple geometric graphics can be split and projected sequentially. For example, the control device can control the first projection device to project the first geometric graphic onto the first area alone, and the control device can control the first projection device to project the second geometric graphic onto the first area alone. For ease of understanding, the following is combined with Figure 11 Take this as an example.

[0176] See for example Figure 11 , Figure 11 Is a combination of the above Figure 9 (a) is shown as an example. Figure 11 (a) is a schematic diagram showing, by way of example, a first projection device projecting a first geometric figure onto a first area. Figure 11 (b) shows a schematic diagram of the first projection device projecting the second geometric figure onto the first area. Figure 11 The position of the first geometric figure (see the figure indicated by ①) after projection in (a) in the projection area, and Figure 9 The first geometric figure shown in (a) has the same position in the projection area. Figure 11 The position of the second geometric figure (see the figure indicated by ②) after projection in the projection area, and Figure 9 The second geometric figure shown in (a) has the same position in the projection area. Figure 11 The first geometric figure in (a) and Figure 11 The second geometric figure in (b) can be superimposed to obtain Figure 9 The first pattern shown in (a) of FIG. That is, although the multiple geometric figures are projected separately, the relative positions of the geometric figures remain unchanged. Moreover, the positions of the geometric figures in the projection area and in the second preset coordinate system remain unchanged.

[0177] above Figure 11 In another possible implementation, the relative positions of the multiple geometric figures projected separately do not change. In another possible implementation, the relative positions of the multiple geometric figures projected separately can be changed. The position of the second geometric figure projected separately in the projection area can overlap with the position of the first geometric figure projected separately in the projection area. In addition, the number of the second geometric figures projected separately can be greater. For example, see Figure 11A As shown. It can be seen that the number of the second geometric figures projected separately is less than that of the above Figure 9The number of the second geometric figures shown in (a) is greater. The greater the number of second geometric figures, the better the anti-interference ability and the robustness of the algorithm for achieving projection calibration. In addition, because the first geometric figure and the second geometric figure are projected successively, Figure 11A The second geometry in the projection can cover Figure 9 The position of the first geometric figure shown in (a). It can be understood that Figure 11A What is shown is merely an example and does not constitute a limitation to the embodiments of the present application.

[0178] In a possible implementation, if the first projection device also projects the third geometric figure, then the third geometric figure can be projected together with the first geometric figure. That is, the control device controls the first projection device to simultaneously project the first geometric figure and the third geometric figure onto the first area. For example, see Figure 12 The projected schematic diagram is shown. Figure 12 Here, ① represents the first geometric figure, and ⑥ represents the third geometric figure. Figure 12 The third geometric figure is a circular spot shape as an example, which does not constitute a limitation to the embodiments of the present application.

[0179] In another possible implementation, if the first projection device also projects the third geometric figure, the first geometric figure, the second geometric figure, and the third geometric figure can be projected separately. For example, see Figure 13 shown. Figure 13 Is a combination of the above Figure 10 (a) is shown as an example. Figure 13 (a) exemplarily shows a schematic diagram in which the first geometric figure is projected alone. Figure 13 (b) exemplarily shows a schematic diagram in which the third geometric figure is projected alone. Figure 13 (c) exemplarily illustrates a schematic diagram in which the third geometric figure is projected separately. Similarly, although the first, second, and third geometric figures are projected separately, the relative positions of the geometric figures remain unchanged. Furthermore, the positions of the geometric figures in the projection area and in the second preset coordinate system remain unchanged. Figure 13 The first geometric figure in (a) Figure 13 The third geometric figure in (b) and Figure 13 The second geometric figure in (c) can be superimposed to obtain Figure 10 Alternatively, in another possible implementation, when the first geometric figure, the second geometric figure, and the third geometric figure are projected separately, they can be projected at any position in the projection area. The embodiment of the present application does not limit the position where the separate projection is the projection of the geometric figure.

[0180] It will be understood that the above description of the geometric figures projected by the first projection device is merely an example and does not constitute a limitation to the embodiments of the present application.

[0181] S702 : Perform projection calibration on the first projection device based on a first image, where the first image is an image obtained by an image acquisition device photographing a first geometric figure and a second geometric figure projected in the first area.

[0182] For example, after the first projection device projects the geometric figure in the first area, the control device can control the image acquisition device to capture the geometric figure projected on the first area to obtain a first image. Then, the image acquisition device can send the captured first image to the control device. For example, the image acquisition device can be, for example, the above-mentioned Figure 1 or Figure 2 The image acquisition device 103 in the projection processing system is shown.

[0183] For example, if the multiple geometric figures are simultaneously projected onto the first area, the image acquisition device can capture the multiple geometric figures simultaneously. That is, the captured first image includes the one or more first geometric figures and the at least four second geometric figures. Alternatively, if a third geometric figure is also simultaneously projected, the first image simultaneously includes the one or more first geometric figures, the at least four second geometric figures, and the one or more third geometric figures.

[0184] For example, if the first geometric figure and the second geometric figure are projected twice, an image can be obtained for each projection. For example, after the first projection device projects the first geometric figure or simultaneously projects the first geometric figure and the third geometric figure, the control device can control the image acquisition device to capture an image. The image includes the first geometric figure or the first geometric figure and the third geometric figure. For the convenience of subsequent description, the image is referred to as the second image. In addition, after the first projection device projects the second geometric figure, the control device can control the image acquisition device to capture an image. The image includes the second geometric figure. For the convenience of subsequent description, the image is referred to as the third image. Then, the first image obtained above includes the second image and the third image.

[0185] Exemplarily, if the first geometric figure, the second geometric figure, and the third geometric figure are projected three times, an image can be obtained for each projection. For example, after the first projection device projects the first geometric figure, the control device can control the image acquisition device to capture an image. The image includes the first geometric figure. For the convenience of subsequent description, the image is referred to as image a. After the first projection device projects the third geometric figure, the control device can control the image acquisition device to capture an image. The image includes the third geometric figure. For the convenience of subsequent description, the image is referred to as image b. After the first projection device projects the second geometric figure, the control device can control the image acquisition device to capture an image. The image includes the second geometric figure. For the convenience of subsequent description, the image is referred to as image c. Then, the first image obtained above includes image a, image b, and image c.

[0186] It is understood that the above description of projecting and capturing the first image is merely an example and does not limit the embodiments of the present application. In other possible implementations, the above-mentioned various geometric figures can be arbitrarily split and projected, and the embodiments of the present application do not limit this.

[0187] For example, after acquiring the first image, the control device may calibrate the first projection device based on the first image. This means that the first transformation matrix is ​​obtained. As previously described, the first transformation matrix is ​​the transformation matrix from the first preset coordinate system to the second preset coordinate system, obtained after the first projection device is calibrated.

[0188] For example, since the environment in which the first area of ​​the projection is located is complex and changeable, it is extremely difficult to calculate a first transformation matrix with high accuracy at one time. Therefore, the calibration implementation idea of ​​the embodiment of the present application is to first detect a first geometric figure that is simple, easy to distinguish, but has poor positioning accuracy in the first image obtained, and obtain a preliminary transformation relationship. Then use this preliminary transformation relationship to adjust and optimize the first image, so as to further detect a second geometric figure that is more complex, difficult to distinguish, but has high positioning accuracy in the optimized image, and obtain a more accurate transformation relationship. If iteration is performed again, the optimized transformation relationship is used to further adjust and optimize the image that was adjusted and optimized the previous time, so as to continue to detect a second geometric figure that is more complex, difficult to distinguish, but has high positioning accuracy in the further optimized image, and further obtain a more accurate transformation relationship. Based on this idea, the implementation process of obtaining the above-mentioned first transformation matrix is ​​exemplarily introduced below.

[0189] Exemplarily, the control device may first obtain a first matrix based on the first geometric figure in the first image. The first matrix is ​​an initial transformation relationship matrix from the first preset coordinate system to the second preset coordinate system. In one example, the control device may detect the first geometric figure in the first image and obtain the coordinates of multiple vertices of the first geometric figure in the first preset coordinate system. For the convenience of subsequent description, the coordinates of the multiple vertices of the first geometric figure in the first preset coordinate system are referred to as the first coordinate set. Exemplarily, a contour detection algorithm may be used to detect the first geometric figure in the first image. Then, the vertices of the first geometric figure are identified, and the coordinates of the vertices in the first preset coordinate system are obtained. Alternatively, in other possible implementations, any corner detection method or image segmentation algorithm may be used to realize the recognition of the vertices of the first geometric figure. For example, a machine vision method may be used, or a traditional solution based on OpenCV may be used, or an AI model for key point detection may be used to detect the vertices of the first geometric figure in the image. The embodiments of the present application are not limited to this.

[0190] Exemplarily, after obtaining the first coordinate set, the control device can determine the first matrix based on the first coordinate set and the second coordinate set. The second coordinate set includes the coordinates of multiple vertices of the first geometric figure in the second preset coordinate system. Since the first geometric figure is projected by the control device under the control of the first projection device, the control device already knows the second coordinate set. Alternatively, the first projection device already knows the second coordinate set. The first projection device then transmits the second coordinate set to the control device. Exemplarily, if the first coordinate set includes the coordinates of at least four vertices of the first geometric figure in the first preset coordinate system, then the second coordinate set includes the coordinates of at least four vertices of the first geometric figure in the second preset coordinate system. Based on this, a transformation matrix from the first preset coordinate system to the second preset coordinate system can be solved based on the first coordinate set and the second coordinate set. This transformation matrix is ​​the first matrix. Exemplarily, the first matrix can be a perspective transformation matrix from the first preset coordinate system to the second preset coordinate system. That is, the first matrix is ​​a perspective transformation matrix. The specific algorithm for solving the first matrix based on the first coordinate set and the second coordinate set is not limited in the embodiment of the present application. Any feasible solution algorithm can be used, and the embodiment of the present application will not go into details.

[0191] In one possible implementation, to reduce computational complexity, the control device may process only a first image region within the first image. This first image region includes the first geometric figure. For example, the first image region may be a preset image range. Specifically, the control device retrieves the first image region from the first image according to the preset image range. Then, based on the first geometric figure within the first image region, the control device retrieves the first coordinate set, thereby obtaining the first matrix. The specific implementation process is described above and will not be elaborated upon here.

[0192] In one possible implementation, the control device may pre-process the first image or the first image area before detecting the vertices of the first geometric figure. Exemplarily, the pre-processing may include one or more of the following operations: grayscale processing, denoising processing, and pixel stretching processing. The grayscale and denoising processing can reduce the noise captured by the image acquisition device. Pixel stretching can normalize the projection content under different lighting conditions and reduce the interference of strong light environments. This improves the accuracy of subsequent calibration. Exemplarily, the embodiments of the present application do not limit the specific processing algorithms of grayscale processing, denoising processing, and pixel stretching processing. Any feasible algorithm can be used to implement these pre-processings, and the embodiments of the present application will not elaborate on this.

[0193] In one possible implementation, although the first image or the first image area has been preprocessed to reduce noise or environmental interference, the detection of the vertices of the first geometric figure is still difficult due to variable conditions such as interference from other light sources or interference from the color and texture of the projection surface. In order to improve the accuracy of vertex coordinate acquisition, the contrast between the foreground and background of the first image or the first image area can be enhanced. This makes the first geometric figure in the image clearer. For example, the contrast between the foreground and background of the first image or the first image area can be enhanced by methods such as gamma transformation, histogram equalization, linear transformation, histogram normalization or Laplace operator. Based on this, the first coordinate set can be obtained based on the first image or the first image area after the contrast between the foreground and background is enhanced. Then, the first matrix can be obtained. The specific implementation process is described above and will not be repeated here. For the convenience of subsequent description, the contrast between the foreground and background in the image or image area will be referred to as the contrast of the image or image area in the subsequent introduction.

[0194] In one example, if the vertices of the first geometric figure cannot be successfully detected after the contrast between the foreground and background of the first image or the first image region is enhanced, the contrast of the first image or the first image region may be enhanced multiple times until the vertices of the first geometric figure are successfully detected. Figure 14 The contrast enhancement by gamma transformation is taken as an example.

[0195] See for example Figure 14 , exemplarily shows a flow chart of obtaining a first coordinate set by enhancing the contrast of a first image or a first image area through gamma transformation. It can be seen that the first image or the first image area is first subjected to gamma transformation to enhance the contrast. Then, the first image or the first image area with enhanced contrast is subjected to separation and contour detection operations to detect the first geometric figure. After the first geometric figure is detected, the vertices of the first geometric figure can be extracted. Then, it is determined whether the vertices of the first geometric figure are successfully extracted. For example, if the area of ​​the detected geometric figure meets the preset area range, the degree of concavity meets the preset range of concavity, and the number of corner points (i.e., vertices) of the first geometric figure meets the preset number of corner points. It is determined that the detected geometric figure is the first geometric figure. Then, the coordinates of the vertices of the first geometric figure are obtained, that is, the above-mentioned first coordinate set is obtained. That is, the vertices of the first geometric figure are successfully extracted. On the contrary, if the detected geometric figure does not meet the aforementioned conditions, the vertex extraction of the first geometric figure fails.

[0196] Exemplarily, if the vertices of the first geometric figure are successfully extracted, a first coordinate set is output. If extraction is unsuccessful, the control device may adjust the gamma value, for example, by increasing it, and process the first image or the first image region using the gamma transform algorithm with the adjusted gamma value to obtain a first image or the first image region with further enhanced contrast. Exemplarily, a larger gamma value suppresses background noise and interference, resulting in greater contrast. This makes the first geometric figure in the image clearer. Similarly, the first image or the first image region with further enhanced contrast is subjected to separation and contour detection again to detect the first geometric figure. After detecting the first geometric figure, the vertices of the first geometric figure can be extracted. Next, a determination is made as to whether the vertices of the first geometric figure are successfully extracted. If so, the first coordinate set is output. If not, the gamma value is adjusted again for the next iteration until the vertices of the first geometric figure are successfully extracted and the first coordinate set is output. This implementation ensures that the vertices of the first geometric figure are extracted while improving the accuracy of vertex detection.

[0197] In one possible implementation, the graphics projected by the first projection device further include one or more third geometric graphics nested inside the first geometric graphics. In this case, if the first image includes one or more graphics with the same or similar shapes as the first geometric graphics, the true first geometric graphics can be accurately identified through the third geometric graphics. The true first geometric graphics is the first geometric graphics projected by the first projection device, rather than a graphics similar to the first geometric graphics formed by being captured by the image acquisition device due to ground reflections or other interferences invading the first area. For ease of understanding, taking the ground reflection scene as an example, for example, you can refer to Figure 15 and Figure 16 shown.

[0198] See for example Figure 15 , assuming that the first projection device projects a pattern onto the wall area. Figure 9 Take the pattern shown in (c) as an example. As you can see, due to the reflection of the ground, the pattern projected onto the wall will be reflected on the ground. However, since the ground is not a smooth mirror, the reflected pattern has afterimages or some details are not reflected. For example, the third few figures ( Figure 15 (as shown in ⑥) is not clearly reflected. When the image acquisition device takes pictures, it will capture the pattern reflected on the ground. Then, in the above-mentioned first image, there will be a projected first geometric figure and a first geometric figure reflected on the ground. For example, see Figure 16 shown. Figure 16 The first image in FIG is only an example of a portion of the captured content, and other content such as the second geometric image is not shown. Figure 16 As can be seen in the figure, the third geometric figure is not clearly reflected in the ground pattern. Therefore, in the first image captured, the third geometric figure is not included in the first geometric figure reflected on the ground. However, the third geometric figure is embedded in the projected first geometric figure.

[0199] Based on the above description, the true first geometric figure can be screened out by detecting whether a third geometric figure exists within the first geometric figure. For example, the third geometric figure can be identified from the image using methods such as contour detection or image segmentation algorithms. This allows determination of whether the third geometric figure exists within the first geometric figure. The first geometric figure with the third geometric figure nested within it is the true first geometric figure.

[0200] In the above implementation, the target graphics can still be accurately identified in scenarios such as ground reflections or intrusion of interference objects, thereby effectively ensuring the accuracy of subsequent calibration.

[0201] In one possible implementation, if one or more third geometric figures are nested within the first geometric figure, the control device may detect some or all of the third geometric figures in the first image. The control device may also obtain the first coordinates of a preset point within the one or more detected third geometric figures. The first coordinates of the preset point are the coordinates within the first preset coordinate system. For ease of subsequent description, the first coordinates of the preset points within the one or more third geometric figures are referred to as a third coordinate set.

[0202] For example, the preset point of the third geometric figure can be, for example, the center point of the third geometric figure. Alternatively, in another possible implementation, if the third geometric figure is a geometric figure with vertices, the preset point sequence can be the center point or vertex of the third geometric figure. However, since the center point of the third geometric figure has higher detection accuracy, the preset point is preferably the center point of the third geometric figure.

[0203] Exemplarily, after obtaining the third coordinate set, the third coordinate set and the first coordinate set obtained based on the first geometric figure in the first image can be used together to solve the first matrix. Exemplarily, the first matrix can be solved based on the third coordinate set, the first coordinate set, the second coordinate of the preset point of the third geometric figure corresponding to the third coordinate set in the second preset coordinate system, and the second coordinate set. As above, since the third geometric figure is projected by the first projection device controlled by the control device, the control device knows the second coordinate of the preset point of the third geometric figure. Alternatively, the first projection device knows the second coordinate of the preset point of the third geometric figure. Then, the first projection device sends the second coordinate of the preset point of the third geometric figure to the control device. Based on this, the transformation relationship matrix of the perspective transformation from the first preset coordinate system to the second preset coordinate system can be solved. The transformation relationship matrix is ​​the first matrix. The specific algorithm for solving the first matrix is ​​not limited in the embodiment of this application. Any feasible solution algorithm can be used, and the embodiment of this application will not be described in detail.

[0204] It will be understood that the above implementation of obtaining the first matrix is ​​merely an example and does not constitute a limitation to the embodiments of the present application.

[0205] For example, in the above solution, since there is a third coordinate set corresponding to the third geometric figure, even when the vertices of the first geometric figure are blocked, the first matrix can still be calculated, thereby enhancing the robustness of the algorithm.

[0206] Based on the calibration implementation ideas of the embodiment of the present application introduced above, it can be known that the first matrix obtained above is the initial transformation relationship matrix. This is because the first geometric figure is a polygon with multiple vertices, and due to the limitations of the polygon detection algorithm, the detection accuracy of the edge position of the polygon is low. Therefore, the accuracy of the coordinates of the vertices of the detected first geometric figure is also low. Based on this, the accuracy of the solved first matrix is ​​also low. Therefore, in order to further improve the accuracy of the calibration, the embodiment of the present application uses the preliminary transformation relationship matrix and the first matrix to adjust and optimize the first image, and then further solves a more accurate transformation relationship matrix based on the adjusted and optimized first image. The implementation process is exemplified below.

[0207] For example, the control device can transform the first image into an image represented in the second preset coordinate system based on the first matrix. For ease of subsequent description, this transformed image is referred to as the first corrected image. A second matrix is ​​then obtained based on the first geometric figure in the first image and at least one second geometric figure in the first corrected image. This second matrix is ​​the optimized relationship matrix for transforming from the first preset coordinate system to the second preset coordinate system.

[0208] Exemplarily, since the above-mentioned first matrix is ​​a perspective transformation relationship matrix from the first preset coordinate system to the second preset coordinate system, the first image represented in the first preset coordinate system can be perspectively transformed into the first corrected image represented in the second preset coordinate system through the first matrix. This is equivalent to converting the image on the imaging plane of the image acquisition device to the projection plane of the first projection device. Since the pattern in the first image was originally captured after being projected by the first projection device, converting the first image to the projection plane of the first projection device is equivalent to de-perspectivizing the first image. That is, the first corrected image is the image after de-perspectivization of the first image. After de-perspectivization, the geometric shape of the first pattern in the first corrected image is also corrected. The distortion of the shape is improved. This can greatly improve the success rate of shape detection and also improve the detection accuracy.

[0209] For example, after obtaining the first corrected image, the first corrected image can be divided into regions according to the region where each second geometric figure is located in the second preset coordinate system during projection, to obtain one or more sub-regions. A sub-region includes the region where a second geometric figure is located in the second preset coordinate system during projection. For ease of understanding, the following is combined with Figure 17 Example introduction.

[0210] See for example Figure 17 , Figure 17 (a) exemplarily shows a schematic diagram of the projection in the second preset coordinate system. Figure 17(b) shows a schematic diagram of the pattern in the first correction image in the second preset coordinate system. Figure 17 (a) and Figure 17 (b) shows that the first geometric figure ( Figure 17 Indicated by ①), the second geometric figure ( Figure 17 Indicated by ②) and the third geometric figure ( Figure 17 In the first correction image, the position and size of the graphics are distorted to varying degrees. Assume that each second geometric figure in the second preset coordinate system is divided into a sub-region during projection, and each sub-region includes a second geometric figure, for example, see Figure 17 In order to clearly see the sub-region of the second geometric figure, Figure 17 The borders of sub-regions 1 and 2 are bolded in the lower right corner of (a). A sub-region can be clearly distinguished from the bolded borders. For example, Figure 17 24 second geometric figures are shown in (a) of FIG. , so a total of 24 sub-regions are divided. The first corrected image is divided according to the positions of the 24 sub-regions. Then, 24 sub-regions can also be obtained by dividing the first corrected image, for example, see Figure 17 As shown in (b) of FIG. Taking subregion 1 as an example, the coordinates of the center point of subregion 1 in the first corrected image are the same as the coordinates of the center point of subregion 1 in the second preset coordinate system during projection. Furthermore, the shape and size of subregion 1 in the first corrected image are the same as the shape and size of subregion 1 in the second preset coordinate system during projection.

[0211] For example, from the above Figure 17 As can be seen from (b), the position, shape and size of the figure in the first corrected image are distorted. Therefore, in order to ensure that the second geometric figure in the first corrected image is located as much as possible within the divided sub-region, the size of the divided sub-region is appropriately expanded.

[0212] It is understandable that the above Figure 17 What is shown is merely an example and does not constitute a limitation to the embodiments of the present application.

[0213] For example, after the first corrected image is divided into subregions, the second geometric figure can be detected in each subregion. The coordinates of the center point of the second geometric figure in the second preset coordinate system are obtained. For example, the second geometric figure can be identified from the image using contour detection or an image segmentation algorithm.

[0214] In one possible implementation, to reduce computational complexity, the control device may first extract a second image region from the first corrected image. The second image region is a region encompassing all second geometric figures. For example, the second image region may be extracted based on subregions. For example, the region comprised of all subregions constitutes the second image region.

[0215] In one possible implementation, the control device may pre-process the first corrected image or the second image area before detecting the second geometric figure in each sub-area. Exemplarily, the pre-processing may include one or more of the following operations: grayscale processing, denoising processing, and pixel stretching processing. The grayscale and denoising processing can reduce the noise captured by the image acquisition device. Pixel stretching can normalize the projection content under different lighting conditions and reduce the interference of strong light environments. This improves the accuracy of subsequent calibration. Exemplarily, the embodiments of the present application do not limit the specific processing algorithms of grayscale processing, denoising processing, and pixel stretching processing. Any feasible algorithm can be used to implement these pre-processings, and the embodiments of the present application will not elaborate on this.

[0216] A possible implementation is that although the first corrected image or the second image area has been pre-processed to reduce noise or environmental interference, the detection of the second geometric figure is still difficult due to variable situations such as interference from other light sources or interference from the color and texture of the projection surface. In order to improve the accuracy of obtaining the coordinates of the center point of the second geometric figure, the contrast between the foreground and background of the first corrected image or the second image area can be enhanced. This makes the second geometric figure in the image clearer. For example, the contrast between the foreground and background of the first corrected image or the second image area can be enhanced by methods such as gamma transformation, histogram equalization, linear transformation, histogram normalization or Laplace operator.

[0217] In one example, if the second geometric figure cannot be successfully detected after the contrast between the foreground and background of the first corrected image or the second image region is enhanced, multiple iterations may be performed to enhance the contrast of the first corrected image or the second image region. Figure 18 The contrast enhancement by gamma transformation is taken as an example.

[0218] See for example Figure 18, exemplarily shows a flow chart of obtaining the coordinates of the center point of the second geometric figure by enhancing the contrast of the first corrected image or the second image area through gamma transformation. It can be seen that the first corrected image or the second image area is first subjected to gamma transformation to enhance the contrast. Then, the first corrected image or the second image area with enhanced contrast is subjected to separation and contour detection operations to detect the second geometric figure. If only one second geometric figure detected in a sub-area meets the preset conditions, the second geometric figure in the sub-area is successfully detected. And the coordinates of the center point of the second geometric figure in the sub-area are output. If no second geometric figure that meets the preset conditions is detected in a sub-area, or if two or more second geometric figures that meet the preset conditions are detected, it is considered that the detection of the second geometric figure in the sub-area has failed. The coordinates of the center point of the detected second geometric figure may not be output if the detection fails.

[0219] Exemplarily, the second geometric figure satisfies the preset conditions including: the area of ​​the second geometric figure is within a preset area range, the roundness of the second geometric figure is within a preset roundness range, the length between the two farthest points in the second geometric figure is within a first preset length range, and the length between the two closest points is within a second preset length range.

[0220] For example, in one example, if the second geometric figure is successfully detected in no subregion, the control device may adjust the gamma value, for example, by increasing the gamma value, and process the first corrected image or the second image region using the gamma transform algorithm with the adjusted gamma value to obtain a first corrected image or the second image region with further enhanced contrast. This makes the second geometric figure in the image clearer. Then, the first corrected image or the second image region with further enhanced contrast is again separated and contour detected to detect the second geometric figure. For example, in one example, if the second geometric figure is successfully detected in a preset number of subregions, the iterative calculation may be stopped.

[0221] It will be understood that the above description of the implementation of the second geometric figure detection in the sub-area is merely an example and does not constitute a limitation to the embodiments of the present application.

[0222] Based on the above description, when a second geometric figure meeting a preset condition is detected in each sub-region of at least one sub-region, the second matrix can be obtained based on the first geometric figure in the aforementioned first image and the at least one detected second geometric figure meeting the preset condition. An exemplary description is given below.

[0223] By way of example, based on the above description, after successful detection of a second geometric figure in a sub-region, the coordinates of the center point of the second geometric figure in the second preset coordinate system can be output. The coordinates of the center point of the second geometric figure in the second preset coordinate system are then converted to coordinates in the first preset coordinate system. The following example describes the conversion process.

[0224] For example, in one possible implementation, the transformation matrix from the first preset coordinate system to the second preset coordinate system, i.e., the first matrix, has been obtained. Then, by solving the inverse matrix of the first matrix, the transformation matrix from the second preset coordinate system to the first preset coordinate system can be obtained. Then, using the inverse matrix of the first matrix, the coordinates of the center point of the second geometric figure obtained in the second preset coordinate system can be converted to coordinates in the first preset coordinate system.

[0225] Exemplarily, in another possible implementation, the above-mentioned first matrix can be solved by the first coordinate set and the second coordinate set, then the transformation matrix A can also be solved based on the first coordinate set and the second coordinate set. The transformation matrix A is a transformation relationship matrix for perspective transformation from the second preset coordinate system to the first preset coordinate system. In another implementation, the above-mentioned transformation matrix A can also be solved based on the above-mentioned third coordinate set, the first coordinate set, the second coordinate of the preset point of the third geometric figure corresponding to the third coordinate set in the second preset coordinate system, and the above-mentioned second coordinate set. The specific solution process is not limited in the embodiment of this application. After obtaining the transformation matrix A, the coordinates of the center point of the second geometric figure obtained above in the second preset coordinate system can be converted into coordinates in the first preset coordinate system through the transformation matrix A.

[0226] Based on the above description, at least one corrected coordinate of the center point of the second geometric figure in the first preset coordinate system can be obtained. For ease of description, the coordinates of the at least one corrected first preset coordinate system will be referred to as the fourth coordinate set. This fourth coordinate set is then used together with the first coordinate set obtained based on the first geometric figure in the first image to solve the second matrix.

[0227] Exemplarily, when the first preset projection device projects the second geometric figure, the coordinates of the center point of the second geometric figure in the at least one sub-area in the second preset coordinate system are known. For the convenience of subsequent description, the coordinates of the center point of the second geometric figure in the at least one sub-area in the second preset coordinate system during projection are referred to as the fifth coordinate set. Based on this, the second matrix can be solved according to the fourth coordinate set, the first coordinate set, the fifth coordinate set and the second coordinate set. The second matrix is ​​a transformation relationship matrix of the perspective transformation from the first preset coordinate system to the second preset coordinate system. The specific algorithm for solving the second matrix is ​​not limited in the embodiment of the present application, and any feasible solution algorithm can be used, which is not described in detail in the embodiment of the present application.

[0228] It will be understood that the above implementation of obtaining the second matrix is ​​merely an example and does not constitute a limitation to the embodiments of the present application.

[0229] In a possible implementation, the second matrix can be used as the final calibrated transformation matrix from the first preset coordinate system to the second preset coordinate system. That is, the second matrix is ​​the first transformation matrix.

[0230] In another possible implementation, in order to obtain a more accurate first transformation matrix, the first transformation matrix can be obtained according to the second matrix and the first corrected image.

[0231] For example, the control device may further de-perspective the first corrected image based on the second matrix to obtain a second corrected image. Then, a third matrix is ​​obtained based on the at least four second geometric figures in the second corrected image. The third matrix is ​​a further optimized relational matrix for transforming from the first preset coordinate system to the second preset coordinate system.

[0232] For example, the aforementioned de-perspective transformation of the first corrected image treats it as an image captured by the image acquisition device. Specifically, the first corrected image is treated as an image represented in a first preset coordinate system. Therefore, using the second matrix, the first corrected image can be perspective-transformed into a second corrected image represented in a second preset coordinate system. After further de-perspective transformation, the geometric figures in the resulting second corrected image are further corrected, and the distortion of the figures is further reduced. This can further improve the success rate of figure detection and enhance detection accuracy.

[0233] For example, after obtaining the second corrected image, the second corrected image can be divided into regions according to the region where each second geometric figure is located in the second preset coordinate system during projection, to obtain one or more sub-regions. A sub-region includes the region where a second geometric figure is located in the second preset coordinate system during projection. The implementation of the region division of the second corrected image can be exemplarily referred to the above Figure 17 The exemplary introduction of region division of the first corrected image in is omitted here.

[0234] For example, the situation after the second corrected image is divided into regions can be exemplified by referring to Figure 19 shown. Figure 19 The schematic diagram of the pattern in the second corrected image in the second preset coordinate system is shown as an example. And the schematic diagram of each sub-area is shown. Figure 17 (b) and Figure 19 It can be seen that in the second corrected image obtained after de-perspective again, most of the second geometric figures ( Figure 19 That is, the position and shape of the second geometric figure in the second corrected image are well improved.

[0235] For example, comparing the above Figure 17 (b) and Figure 19 As can be seen, the size of the subregion in the second corrected image is smaller than that in the first corrected image. The position and shape distortion of the second geometric figure in the second corrected image have been significantly improved. Even in the smaller region, the majority of the second geometric figure is located within the region. This lays the foundation for subsequent detection of the second geometric figure.

[0236] It is understandable that the above Figure 19 What is shown is merely an example and does not constitute a limitation to the embodiments of the present application.

[0237] For example, after the second corrected image is divided into subregions, the second geometric figure can be detected in each subregion. The coordinates of the center point of the second geometric figure in the second preset coordinate system are obtained. For example, the detection of the second geometric figure within the subregions of the second corrected image can be seen in the description of the implementation of the detection of the second geometric figure within the subregions of the first corrected image, and will not be repeated here.

[0238] For example, in one possible implementation, in the second corrected image, when a second geometric figure meeting a preset condition is detected in each of at least four subregions, the third matrix can be obtained based on the at least four detected second geometric figures meeting the preset condition. An example is provided below.

[0239] By way of example, based on the above description, after successful detection of a second geometric figure in a sub-region, the coordinates of the center point of the second geometric figure in the second preset coordinate system can be output. The coordinates of the center point of the second geometric figure in the second preset coordinate system are then converted to coordinates in the first preset coordinate system. The following describes an exemplary conversion process.

[0240] For example, in one possible implementation, the optimized transformation matrix from the first preset coordinate system to the second preset coordinate system, i.e., the second matrix, has been obtained. Then, by solving the inverse matrix of the second matrix, the optimized transformation matrix from the second preset coordinate system to the first preset coordinate system can be obtained. Then, using the inverse matrix of the second matrix, the coordinates of the center point of the second geometric figure obtained in the second preset coordinate system can be converted to coordinates in the first preset coordinate system.

[0241] For example, in another possible implementation, the second matrix can be solved based on the fourth coordinate set, the first coordinate set, the fifth coordinate set, and the second coordinate set. Then, the transformation matrix B can also be solved based on the fourth coordinate set, the first coordinate set, the fifth coordinate set, and the second coordinate set. The transformation matrix B is a transformation relationship matrix for perspective transformation from the second preset coordinate system to the first preset coordinate system. The specific solution process is not limited in the embodiment of this application. After obtaining the transformation matrix B, the coordinates of the center point of the second geometric figure obtained above in the second preset coordinate system can be converted into coordinates in the first preset coordinate system through the transformation matrix B.

[0242] Based on the above description, then, at least the coordinates of the center points of the four second geometric figures in the first preset coordinate system after correction can be obtained. For the convenience of description, the coordinates of the at least four corrected first preset coordinate systems will be referred to as the sixth coordinate set below. In addition, when the above-mentioned first preset projection device projects the above-mentioned first pattern, the coordinates of the center points of the second geometric figures in the at least four sub-areas in the second preset coordinate system are known. For the convenience of subsequent description, the coordinates of the center points of the second geometric figures in the at least four sub-areas in the second preset coordinate system during projection will be referred to as the seventh coordinate set. Based on this, the above-mentioned third matrix can be solved according to the sixth coordinate set and the seventh coordinate set. The third matrix is ​​the transformation relationship matrix of the perspective transformation of the first preset coordinate system to the second preset coordinate system after optimization again. The specific algorithm for solving the third matrix is ​​not limited in the embodiment of the present application, and any feasible solution algorithm can be used, which is not described in detail in the embodiment of the present application.

[0243] It will be understood that the above implementation of obtaining the third matrix is ​​merely an example and does not constitute a limitation to the embodiments of the present application.

[0244] In a possible implementation, the third matrix can be used as the final calibrated transformation matrix from the first preset coordinate system to the second preset coordinate system. That is, the third matrix is ​​the first transformation matrix.

[0245] In another possible implementation, although the first image is corrected by de-perspective correction, there are errors in the second geometric figure detected in the corrected image, which causes errors in the coordinates of the center point of the second geometric figure. In order to reduce this error, the accuracy of the calculated first change matrix is ​​further optimized. The sixth coordinate set and the seventh coordinate set for solving the third matrix can be processed in combination with the random sample consensus (RANSAC) algorithm. That is, the coordinate combination of the center point of the abnormal second geometric figure with large errors or that does not meet the spatial consistency in the sixth coordinate set and the seventh coordinate set is eliminated. The coordinate combination includes the coordinates of the center point of the abnormal second geometric figure in the first preset coordinate system and the coordinates in the second preset coordinate system. That is, the coordinate combination includes the coordinates of the abnormal geometric figure in the sixth coordinate set and the seventh coordinate set, respectively.

[0246] For example, after eliminating the coordinate combinations of the center points of the abnormal second geometric figures in the sixth and seventh coordinate sets, the remaining coordinate combinations can be used to re-solve the transformation relationship matrix from the first preset coordinate system to the second preset coordinate system. The resulting transformation relationship matrix is ​​the further optimized first transformation relationship matrix.

[0247] In summary, in the solution of the present application, the pattern used for projection calibration includes a first geometric figure with at least four vertices and a second geometric figure surrounding the first geometric figure, and the pattern design is simple. At least four vertices of the first few figures in the first image obtained by capture can be used as the corner points required for calibration. There is no need to detect corner points in a complex pattern, which reduces the amount of calculation for calibration corner point detection and reduces computing power costs. In addition, at least four second geometric figures are provided outside the first geometric figure in the pattern. The second geometric figure is used to assist in achieving precise calibration, and more precise calibration cannot be achieved using only a few corner points of the first geometric figure. Therefore, the second geometric figure can be used to further obtain points for calibration. The points obtained by combining the first geometric figure and the second geometric figure can achieve more precise calibration. In addition, the second geometric figure has no vertices to distinguish it from the first geometric figure, so that the first geometric figure can be quickly identified from the pattern and the corner points of the first geometric figure can be detected, further reducing the amount of calculation for calibration corner point detection.

[0248] Furthermore, based on the above description, the projection calibration processing method provided by the embodiments of the present application can still achieve relatively accurate projection calibration in one or more of the following complex environments: for example, when the first geometric figure is obstructed, when there are reflections on the ground, when the projection is made on a non-absolutely flat surface such as a lawn, a forest path, or a stone brick road, etc. It can be seen that the use of the embodiments of the present application can greatly enhance the robustness of the algorithm.

[0249] Furthermore, the patterns in the embodiments of the present application are simple and easy to implement. They can be customized to create aesthetically pleasing patterns, reducing the need for engineering. For example, in addition to the geometric shapes described above, other shapes or lines can be drawn within the pattern to further enhance the pattern, add some interest, and improve the user experience during the calibration process.

[0250] For example, the above exemplary embodiment describes a projection calibration processing method when the projection processing system includes one projection device, namely, the first projection device. In another possible implementation, the projection processing system may include two projection devices. For example, the first projection device and the second projection device are included. The two projection devices are used to implement fusion projection. In this case, both the first projection device and the second projection device need to be calibrated. For example, the projection calibration of the first projection device is described above. Figure 7 The description of the method and its possible implementation methods will not be repeated here.

[0251] Exemplarily, the implementation process of projection calibration of the second projection device is similar to the implementation process of projection calibration of the first projection device described above. For example, the control device can control the second projection device to project one or more fourth geometric figures onto the first area; and control the second projection device to project a fifth geometric figure onto the first area. The fourth geometric figure includes at least four vertices. The fifth geometric figure is a geometric figure without vertices. Then, projection calibration of the second projection device is performed based on the fourth image. The fourth image is an image obtained by the image acquisition device by capturing the fourth and fifth geometric figures projected into the first area. Optionally, the control device can also control the second projection device to project one or more sixth geometric figures onto the first area. Then, the captured fourth image can also include the sixth geometric figure. The one or more sixth geometric figures are nested within the aforementioned fourth geometric figure.

[0252] For example, the description of the fourth geometric figure can be made by referring to the description of the first geometric figure included in the first pattern. The fourth geometric figure projected by the second projection device and the first geometric figure projected by the first projection device can be the same or different. The description of the fifth geometric figure can be made by referring to the description of the second geometric figure included in the first pattern. The fifth geometric figure projected by the second projection device and the second geometric figure projected by the first projection device can be the same or different. The description of the sixth geometric figure can be made by referring to the description of the third geometric figure included in the first pattern. The sixth geometric figure projected by the second projection device and the third geometric figure projected by the first projection device can be the same or different.

[0253] Furthermore, the acquisition of the fourth image can refer to the exemplary implementation of the control device acquiring the first image in step S702, which is not further described here. The projection calibration of the second projection device based on the fourth image can refer to the implementation of the projection calibration of the first projection device based on the first image described in step S702 and its possible implementations, which is not further described here.

[0254] In one possible implementation, the calibration accuracy of the two projection devices when merging the projections can be optimized by designing the position of the pattern projected by the first projection device and the position of the pattern projected by the second projection device. For example, for ease of understanding, let the first projection device be the left front pixel headlight of the vehicle and the second projection device be the right front pixel headlight. Then, the pattern projected by the first projection device can be set to the right side of the projection area, and the pattern projected by the second projection device can be set to the left side of the projection area. For ease of understanding, you can refer to the example Figure 20 .

[0255] For example, see Figure 20 As shown, assuming Figure 20 (a) shows a pattern projected by the first projection device. The pattern projected by the first projection device includes, for example, a first geometric figure ①, a second geometric figure ② and a third geometric figure ⑥. Figure 20 (b) shows the pattern projected by the second projection device. The pattern projected by the second projection device includes, for example, the fourth geometric figure ⑦, the fifth geometric figure ⑧, and the sixth geometric figure ⑨. It can be seen that the pattern projected by the first projection device can be located on the right side of the projection area. The pattern projected by the second projection device can be located on the left side of the projection area. It is understandable that Figure 20 The patterns shown are merely examples and do not constitute a limitation to the embodiments of the present application.

[0256] In the above scheme, in scenarios where two projection devices are fused together, such as when fusion projection is performed using two pixel headlights on a vehicle, the two projection devices can be calibrated relative to the same image acquisition device, thereby achieving precise fusion projection of the two projection devices. Because the projection calibration processing method described above optimizes the relational transformation matrix used for the calibration of the projection devices, and thus the calibration accuracy of the projection devices, it is possible to achieve pixel-level fusion projection of the two projection devices, reducing the likelihood of poor fusion of the projection images.

[0257] The above mainly introduces the projection calibration processing method provided in the embodiment of the present application. It is understandable that, in order to realize the corresponding functions mentioned above, each controller or device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0258] The embodiment of the present application can divide the controller or device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0259] In the case of dividing each functional module according to each function, an embodiment of the present application also provides an apparatus for implementing any of the above methods, for example, providing an apparatus including units (or means) for implementing each step in any of the above methods.

[0260] For example, see Figure 21 , which is a virtual structural diagram of a projection calibration processing device 2100 provided in an embodiment of the present application. Figure 21 The projection calibration processing device 2100 shown may be a control device for implementing any embodiment of the projection calibration processing method described above. The projection calibration processing device 2100 may include a control unit 2101 and a processing unit 2102.

[0261] The control unit 2101 is used to control the first projection device to project one or more first geometric figures onto the first area; and to control the first projection device to project a second geometric figure onto the first area; the first geometric figure includes at least four vertices; the second geometric figure is a geometric figure without vertices. The control unit 2101 is used to implement the above Figure 7 Step S701 shown.

[0262] The processing unit 2102 is configured to perform projection calibration on the first projection device based on the first image; the first image is an image obtained by the image acquisition device shooting the first geometric figure and the second geometric figure projected in the first area. The processing unit 2102 is configured to implement the above Figure 7 Step S702 is shown.

[0263] In a possible implementation, one or more first geometric figures and at least four second geometric figures are simultaneously projected onto the first area, and the at least four second geometric figures surround the first geometric figures; the first image includes the first geometric figures and the at least four second geometric figures.

[0264] In one possible implementation, one or more first geometric figures and at least four second geometric figures are projected successively into the first area; the first image includes a second image and a third image, the second image is an image obtained by an image acquisition device photographing the one or more first geometric figures projected onto the first area, and the third image is an image obtained by an image acquisition device photographing the at least four second geometric figures projected onto the first area.

[0265] In a possible implementation, the control unit 2101 is further configured to control the first projection device to project one or more third geometric figures onto the first area, where the one or more third geometric figures are nested inside the first geometric figure.

[0266] In a possible implementation, the control unit 2101 is further configured to: control the second projection device to project one or more fourth geometric figures onto the first area; the fourth geometric figures include at least four vertices;

[0267] The control unit 2101 is further configured to: control the second projection device to project a fifth geometric figure onto the first area; the fifth geometric figure is a geometric figure without vertices;

[0268] The processing unit 2102 is further configured to perform projection calibration on the second projection device based on the fourth image; the fourth image is an image obtained by the image acquisition device capturing the fourth geometric figure and the fifth geometric figure projected in the first area.

[0269] In a possible implementation, the control unit 2101 is further configured to control the second projection device to project one or more sixth geometric figures onto the first area, where the one or more sixth geometric figures are nested inside the fourth geometric figure.

[0270] In one possible implementation, the processing unit 2102 is specifically configured to:

[0271] Obtaining a first matrix based on a first geometric figure in the first image; the first matrix is ​​a transformation relationship matrix from a first preset coordinate system to a second preset coordinate system; the first preset coordinate system is a coordinate system corresponding to the image acquisition device, and the second preset coordinate system is a coordinate system corresponding to the projection;

[0272] Obtaining a second matrix based on the first matrix and the first image; the second matrix is ​​an optimized relationship matrix transformed from the first preset coordinate system to the second preset coordinate system;

[0273] A first transformation matrix is ​​determined according to the second matrix; the first transformation matrix is ​​a transformation relationship matrix from the first preset coordinate system to the second preset coordinate system that is finally calibrated.

[0274] In one possible implementation, the processing unit 2102 is specifically configured to:

[0275] Acquire a first coordinate set according to a first geometric figure in the first image; the first coordinate set includes coordinates of a plurality of vertices of the first geometric figure in a first preset coordinate system;

[0276] A first matrix is ​​determined according to a first coordinate set and a second coordinate set; the second coordinate set includes coordinates of a plurality of vertices of the first geometric figure in a second preset coordinate system.

[0277] In one possible implementation, the processing unit 2102 is further configured to:

[0278] Acquire a first image region of a first image; the first image region includes a first geometric figure;

[0279] Preprocessing the first image region; the preprocessing includes one or more of the following operations: grayscale processing, denoising processing, and pixel stretching processing;

[0280] A first coordinate set is obtained according to the preprocessed first image region.

[0281] In one possible implementation, the processing unit 2102 is further configured to:

[0282] enhancing the contrast between the foreground and background of the preprocessed first image region;

[0283] A first coordinate set is obtained according to the first image region after the contrast between the foreground and the background is enhanced.

[0284] In one possible implementation, one or more third geometric figures are nested inside the first geometric figure; if the first image also includes one or more figures with the same or similar shapes as the first geometric figure, the processing unit 2102 is also used to determine the figure with one or more third geometric figures nested inside as the first geometric figure from the multiple figures with the same or similar shapes in the first image before obtaining the first matrix based on the first geometric figure in the first image.

[0285] In one possible implementation, the processing unit 2102 is further used to: obtain a third coordinate set based on one or more third geometric figures in the first image; the third coordinate set includes the coordinates of the center points of one or more third geometric figures in the first preset coordinate system; the processing unit 2102 is further specifically used to: obtain a first matrix based on the first geometric figure and the third coordinate set in the first image.

[0286] In one possible implementation, the processing unit 2102 is further specifically configured to:

[0287] transforming the first image into a first corrected image represented in a second preset coordinate system according to the first matrix;

[0288] A second matrix is ​​obtained based on the first geometric figure in the first image and at least one second geometric figure in the first corrected image.

[0289] In one possible implementation, the processing unit 2102 is further specifically configured to:

[0290] A first transformation matrix is ​​obtained according to the second matrix and the first corrected image.

[0291] In a possible implementation, the control unit 2101 is further configured to: control the first projection device to project an image onto the first area according to the first transformation matrix.

[0292] In a possible implementation, the processing unit 2102 is further specifically configured to:

[0293] Divide the first correction pattern into regions according to the region where each second geometric figure is located in the second preset coordinate system when the first pattern is projected, to obtain one or more sub-regions; a sub-region includes a region where a second geometric figure is located in the second preset coordinate system when the first pattern is projected;

[0294] In a case where a second geometric figure meeting a preset condition is detected in each of the at least one sub-area, a second matrix is ​​acquired according to the first geometric figure in the first image and the at least one detected second geometric figure meeting the preset condition.

[0295] Figure 21The specific operations and beneficial effects of each unit in the projection calibration processing device 2100 can be found in the above Figure 7 The corresponding descriptions in possible embodiments thereof will not be repeated here.

[0296] Figure 22 The figure shows a possible hardware structure diagram of a projection calibration processing device provided in this application. The projection calibration processing device 2200 can be a control device for implementing the method described in the above embodiment. The projection calibration processing device 2200 includes: a processor 2201, a memory 2202, and a communication interface 2203. The processor 2201, the communication interface 2203, and the memory 2202 can be interconnected or connected to each other via a bus 2204.

[0297] Exemplarily, the memory 2202 is used to store computer programs and data of the projection calibration processing device 2200. The memory 2202 may include but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or portable read-only memory (CD-ROM), etc.

[0298] The software or program codes required for the functions of all or part of the units implemented by the control device in the above method embodiment are stored in the memory 2202 .

[0299] In one possible implementation, if the software or program code required for the functions of some units is stored in the memory 2202, the processor 2201, in addition to calling the program code in the memory 2202 to implement some functions, can also cooperate with other components (such as the communication interface 2203) to jointly complete other functions described in the method embodiment (such as the function of receiving or sending data).

[0300] There may be multiple communication interfaces 2203 , which are used to support the projection calibration processing device 2200 to communicate, such as receiving or sending data or signals.

[0301] Exemplarily, the processor 2201 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The processor 2201 may be used to read the program stored in the memory 2202 and execute the above-mentioned Figure 2 The operations performed by the control device in the method described and possible embodiments thereof.

[0302] Figure 22 The specific operations and beneficial effects of each unit in the projection calibration processing device 2200 can be found in the corresponding description in the above method embodiment, and will not be repeated here.

[0303] The embodiment of the present application also provides a vehicle, which includes the above-mentioned Figure 21 or Figure 22 Alternatively, the vehicle includes the projection calibration processing device. Figure 1 or Figure 2 The projection processing system shown.

[0304] The embodiment of the present application provides a chip, which includes a logic circuit and an interface, wherein the logic circuit and the interface are coupled. The interface is used to input and / or output information, and the logic circuit is used to execute the above Figure 7 and the methods described in possible embodiments thereof.

[0305] The present invention also provides a computer-readable storage medium that stores a computer program or computer instructions, which are executed by a processor to implement the above Figure 7 And the method implemented by the control device in possible embodiments thereof.

[0306] For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. It should be understood that the description of the computer-readable storage medium herein is merely illustrative and does not constitute a limitation on the embodiments of the present application.

[0307] The present application also provides a computer program product. When the computer program product is read and executed by a computer, the above Figure 7 The method implemented by the control device in its possible embodiments will be executed.

[0308] Illustratively, the computer program product includes, but is not limited to, a computer program, code, or electronic (digital) signal for transmitting computer program instruction code that can implement the method when executed on a computer. It should be understood that the description of the computer program product herein is merely illustrative and does not constitute a limitation on the embodiments of the present application.

[0309] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0310] It will also be understood that the term “comprise” (also known as “includes,” “including,” “comprises,” and / or “comprising”) when used in this specification specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0311] It should also be understood that references throughout this specification to "one embodiment," "an embodiment," or "one possible implementation" mean that specific features, structures, or characteristics associated with that embodiment or implementation are included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment," "in an embodiment," or "one possible implementation" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0312] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A projection calibration processing method, characterized in that: The method comprises: Controlling a first projection device to project one or more first geometric figures onto a first area; the first geometric figures include at least four vertices; Controlling the first projection device to project a second geometric figure onto the first area; the second geometric figure is a geometric figure without vertices; The first projection device is projected and calibrated based on a first image; the first image is an image obtained by an image acquisition device photographing the first geometric figure and the second geometric figure projected in the first area.

2. The method according to claim 1, characterized in that The one or more first geometric figures and at least four second geometric figures are projected onto the first area simultaneously, and the at least four second geometric figures surround the first geometric figure; The first image includes the first geometric figure and the at least four second geometric figures.

3. The method according to claim 1 or 2, characterized in that The one or more first geometric figures and at least four second geometric figures are projected successively onto the first area; The first image includes a second image and a third image, the second image being an image obtained by the image acquisition device photographing the one or more first geometric figures projected on the first area, and the third image being an image obtained by the image acquisition device photographing the at least four second geometric figures projected on the first area.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The first projection device is controlled to project one or more third geometric figures onto the first area, wherein the one or more third geometric figures are nested inside the first geometric figure.

5. The method according to any one of claims 1 to 4, characterized in that The method comprises: Controlling the second projection device to project one or more fourth geometric figures onto the first area; the fourth geometric figures include at least four vertices; controlling the second projection device to project a fifth geometric figure onto the first area; the fifth geometric figure is a geometric figure without vertices; The second projection device is projected and calibrated based on the fourth image; the fourth image is an image obtained by the image acquisition device by photographing the fourth geometric figure and the fifth geometric figure projected in the first area.

6. The method according to claim 5, characterized in that The method further comprises: The second projection device is controlled to project one or more sixth geometric figures onto the first area, wherein the one or more sixth geometric figures are nested inside the fourth geometric figure.

7. The method according to any one of claims 1 to 6, characterized in that The performing projection calibration on the first projection device based on the first image includes: Obtaining a first matrix based on a first geometric figure in the first image; the first matrix is ​​a transformation relationship matrix from a first preset coordinate system to a second preset coordinate system; the first preset coordinate system is a coordinate system corresponding to the image acquisition device, and the second preset coordinate system is a coordinate system corresponding to the projection; Acquire a second matrix according to the first matrix and the first image; the second matrix is ​​an optimized relationship matrix transformed from the first preset coordinate system to the second preset coordinate system; A first transformation matrix is ​​determined based on the second matrix; the first transformation matrix is ​​a finally calibrated transformation relationship matrix from the first preset coordinate system to the second preset coordinate system.

8. The method according to claim 7, characterized in that The obtaining a first matrix according to a first geometric figure in the first image includes: Acquire a first coordinate set according to a first geometric figure in the first image; the first coordinate set includes coordinates of a plurality of vertices of the first geometric figure in the first preset coordinate system; The first matrix is ​​determined according to the first coordinate set and a second coordinate set; the second coordinate set includes coordinates of multiple vertices of the first geometric figure in the second preset coordinate system.

9. The method according to claim 8, characterized in that The obtaining a first coordinate set according to a first geometric figure in the first image includes: Acquire a first image region of the first image; the first image region includes the first geometric figure; Preprocessing the first image area; the preprocessing includes one or more of the following operations: grayscale processing, denoising processing, and pixel stretching processing; The first coordinate set is acquired according to the preprocessed first image area.

10. The method according to claim 9, characterized in that The acquiring a first coordinate set according to the preprocessed first image area includes: enhancing the contrast between the foreground and background of the preprocessed first image region; The first coordinate set is obtained according to the first image region after the contrast between the foreground and the background is enhanced.

11. The method according to any one of claims 7 to 10, characterized in that: One or more third geometric figures are nested inside the first geometric figure; If the first image also includes one or more figures with the same or similar shapes as the first geometric figure, before obtaining the first matrix according to the first geometric figure in the first image, the method further includes: From a plurality of graphics with the same or similar shapes in the first image, a graphic having one or more third geometric graphics nested therein is determined as the first geometric graphic.

12. The method according to claim 11, characterized in that The method further includes: acquiring a third coordinate set according to the one or more third geometric figures in the first image; the third coordinate set includes coordinates of center points of the one or more third geometric figures in the first preset coordinate system; The obtaining of the first matrix according to the first geometric figure in the first image includes: obtaining the first matrix according to the first geometric figure in the first image and the third coordinate set.

13. The method according to any one of claims 7 to 12, characterized in that: The acquiring a second matrix according to the first matrix and the first image includes: transforming the first image into a first corrected image represented in the second preset coordinate system according to the first matrix; The second matrix is ​​obtained based on a first geometric figure in the first image and at least one second geometric figure in the first corrected image.

14. The method according to claim 13, wherein: The determining of the first transformation matrix according to the second matrix includes: The first transformation matrix is ​​obtained according to the second matrix and the first corrected image.

15. The method according to any one of claims 7 to 14, characterized in that: The method further comprises: The first projection device is controlled to project an image onto a first area according to the first transformation matrix.

16. A projection calibration processing device, characterized in that: The device comprises a control unit and a processing unit, wherein: The control unit is configured to control the first projection device to project one or more first geometric figures onto the first area; the first geometric figures include at least four vertices; The control unit is further configured to control the first projection device to project a second geometric figure onto the first area; the second geometric figure is a geometric figure without vertices; The processing unit is used to perform projection calibration on the first projection device based on a first image; the first image is an image obtained by an image acquisition device photographing the first geometric figure and the second geometric figure projected in the first area.

17. The device according to claim 16, characterized in that The one or more first geometric figures and at least four second geometric figures are projected onto the first area simultaneously, and the at least four second geometric figures surround the first geometric figure; The first image includes the first geometric figure and the at least four second geometric figures.

18. The device according to claim 16 or 17, characterized in that The one or more first geometric figures and at least four second geometric figures are projected successively onto the first area; The first image includes a second image and a third image, the second image being an image obtained by the image acquisition device photographing the one or more first geometric figures projected on the first area, and the third image being an image obtained by the image acquisition device photographing the at least four second geometric figures projected on the first area.

19. The device according to any one of claims 16 to 18, characterized in that The control unit is further configured to: The first projection device is controlled to project one or more third geometric figures onto the first area, wherein the one or more third geometric figures are nested inside the first geometric figure.

20. The device according to any one of claims 16 to 19, characterized in that The control unit is further configured to: control the second projection device to project one or more fourth geometric figures onto the first area; the fourth geometric figures include at least four vertices; The control unit is further configured to: control the second projection device to project a fifth geometric figure onto the first area; the fifth geometric figure is a geometric figure without vertices; The processing unit is further configured to: perform projection calibration on the second projection device based on the fourth image; The fourth image is an image obtained by the image acquisition device photographing the fourth geometric figure and the fifth geometric figure projected in the first area.

21. The device according to claim 20, characterized in that The control unit is further configured to: The second projection device is controlled to project one or more sixth geometric figures onto the first area, wherein the one or more sixth geometric figures are nested inside the fourth geometric figure.

22. The device according to any one of claims 16 to 21, characterized in that The processing unit is specifically configured to: Obtaining a first matrix based on a first geometric figure in the first image; the first matrix is ​​a transformation relationship matrix from a first preset coordinate system to a second preset coordinate system; the first preset coordinate system is a coordinate system corresponding to the image acquisition device, and the second preset coordinate system is a coordinate system corresponding to the projection; Obtain a second matrix according to the first matrix and the first image; The second matrix is ​​an optimized relationship matrix transformed from the first preset coordinate system to the second preset coordinate system; determining a first transformation matrix according to the second matrix; The first transformation matrix is ​​a finally calibrated transformation relationship matrix from the first preset coordinate system to the second preset coordinate system.

23. The device according to claim 22, characterized in that The processing unit is specifically configured to: Acquire a first coordinate set according to a first geometric figure in the first image; the first coordinate set includes coordinates of a plurality of vertices of the first geometric figure in the first preset coordinate system; The first matrix is ​​determined according to the first coordinate set and a second coordinate set; the second coordinate set includes coordinates of multiple vertices of the first geometric figure in the second preset coordinate system.

24. The device according to any one of claims 22-23, characterized in that One or more third geometric figures are nested inside the first geometric figure; If the first image also includes one or more figures with the same or similar shapes as the first geometric figure, the processing unit is further configured to, before obtaining the first matrix according to the first geometric figure in the first image, From a plurality of graphics with the same or similar shapes in the first image, a graphic having one or more third geometric graphics nested therein is determined as the first geometric graphic.

25. The device according to claim 24, characterized in that The processing unit is further configured to: obtain a third coordinate set according to the one or more third geometric figures in the first image; the third coordinate set includes coordinates of center points of the one or more third geometric figures in the first preset coordinate system; The processing unit is further specifically configured to obtain the first matrix according to the first geometric figure in the first image and the third coordinate set.

26. The device according to any one of claims 22 to 25, characterized in that The processing unit is further specifically configured to: transforming the first image into a first corrected image represented in the second preset coordinate system according to the first matrix; The second matrix is ​​obtained based on a first geometric figure in the first image and at least one second geometric figure in the first corrected image.

27. The device according to claim 26, characterized in that The processing unit is further specifically configured to: The first transformation matrix is ​​obtained according to the second matrix and the first corrected image.

28. The device according to any one of claims 22 to 27, characterized in that The control unit is further configured to: The first projection device is controlled to project an image onto a first area according to the first transformation matrix.

29. A projection calibration processing device, characterized in that: The device includes a processor, wherein the processor is coupled to a memory, the memory is used to store computer programs or computer instructions, and the processor is used to execute the computer programs or computer instructions stored in the memory, so that the device performs the method according to any one of claims 1 to 15.

30. A projection system, characterized in that: The projection system includes a control device and a projection module, the projection module includes a first projection device, or the projection module includes a first projection device and a second projection device, and the control device is used to execute the method according to any one of claims 1 to 15.

31. A means of transport, characterized in that: The vehicle comprises the projection system of claim 30.

32. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program or computer instructions, and the computer program or computer instructions are executed by a processor to implement the method according to any one of claims 1 to 15.

33. A computer program product, characterized in that When the computer program product is executed by a processor, the method according to any one of claims 1 to 15 will be implemented.

Images