Projection processing method and related apparatus

Abstract

A projection processing method and an apparatus. The method comprises: controlling a first projection module of a vehicle to project a first pattern to a first area, wherein an alignment condition between the first pattern and a target object in the first area is not satisfied (S501); on the basis of a relative position between the first pattern and the target object in the first area, adjusting a projection position of the first pattern in the first area (S502); and after the projection position is adjusted, controlling the first projection module to project the first pattern to the first area, wherein the alignment condition between the first pattern after the projection position is adjusted and the target object is satisfied (S503). The method and apparatus can enable a pattern projected by a projection module of a vehicle to be aligned with a target object, thereby improving the user experience.

Description

Projection processing methods and related devices

[0001] This application claims priority to Chinese Patent Application No. 202510082124.7, filed on January 18, 2025, with the China National Intellectual Property Administration, entitled “Projection Processing Method and Related Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of projection technology, specifically to a projection processing method and related apparatus. Background Technology

[0003] With the continuous development of vehicle applications, in-vehicle projection has become a popular application in the market. For example, headlight projection often results in misalignment of the projection area due to errors in the installation of vehicle headlights or unevenness of the projected area, affecting the user experience. Summary of the Invention

[0004] This application provides a projection processing method and related apparatus, which can align the pattern projected by the vehicle projection module with the target object, thereby improving the user experience.

[0005] In a first aspect, this application provides a projection processing method, which includes: controlling a first projection module of a vehicle to project a first pattern onto a first region; where the first pattern and a target object in the first region do not meet alignment conditions. Then, adjusting the projection position of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region. After the projection position is adjusted, controlling the first projection module to project the first pattern onto the first region; where the first pattern with the adjusted projection position meets alignment conditions with the target object.

[0006] In this solution, the projection position of the first pattern in the first region can be adjusted according to the relative position between the first pattern and the target object, thereby ensuring that the projected first pattern and the target object meet the alignment requirements. This provides users with a better projection effect and improves the user experience.

[0007] In one possible implementation, the target object is a second pattern projected by the second projection module of the vehicle onto the first region; the alignment conditions include one or more constraints that align the pattern projected by the first projection module with the pattern projected by the second projection module.

[0008] The above solution can utilize two projection modules of the vehicle to achieve projection, and the patterns projected by the two projection modules can also satisfy the above alignment conditions.

[0009] In one possible implementation, the target object is a straight line within a first region, and the alignment conditions include one or more constraints that make the pattern projected by the first projection module parallel or perpendicular to the straight line.

[0010] Alternatively, the target object may be a temporary reference point within a first region, and the alignment conditions may include one or more constraints that cause the pattern projected by the first projection module to cover the location of the temporary reference point.

[0011] The above solution expands the application scenarios of this solution, enabling it to provide better projection effects in more scenarios and improve the user experience.

[0012] In one possible implementation, the first pattern and the second pattern are patterns in the user interface of the vehicle's first application;

[0013] Adjusting the projection position of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region includes:

[0014] The first target corner point in the first pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, which includes the first and second patterns.

[0015] Based on the first coordinate, the second coordinate, the third coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, and the fourth coordinate of the second target corner point in the second preset coordinate system, adjust the projection position of the first pattern in the first area.

[0016] For example, adjusting the projection position of the first pattern in the first region according to the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate includes:

[0017] A first mapping relationship is constructed based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate; the first mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system;

[0018] The target deviation is determined based on the first coordinate, the second coordinate, and the first mapping relationship; the target deviation includes the first deviation and / or the second deviation, the first deviation indicating the deviation of the center point of the first pattern and the center point of the second pattern in the vertical axis direction in the second preset coordinate system; the second deviation indicating the deviation between the width of the circumscribed rectangle of the first pattern and the width of the circumscribed rectangle of the second pattern in the second preset coordinate system.

[0019] Adjust the projection position of the first pattern in the first region according to the target deviation.

[0020] For example, adjusting the projection position of the first pattern in the first region according to the target deviation includes:

[0021] The first pattern and / or the second pattern are translated along the vertical axis of the second preset coordinate system according to the first deviation; and / or the width of the circumscribed rectangle of the first pattern and / or the second pattern is scaled along the horizontal axis of the second preset coordinate system according to the second deviation.

[0022] In the above scheme, a mapping relationship is established between the two coordinate systems to achieve the transformation from the first preset coordinate system to the second preset coordinate system. Then, the target deviation can be determined in the second preset coordinate system (i.e., within the application's user interface), and the deviation can be adjusted within the second preset coordinate system. This ensures that the reprojected pattern after adjustment meets the corresponding alignment conditions with the target object, improving the projection effect.

[0023] In one possible implementation, the first pattern is a pattern in the user interface of the vehicle's first application; after the projection position is adjusted and the first projection module is controlled to project the first pattern onto the first area, the implementation further includes:

[0024] The fifth coordinate of the first target corner point in the first pattern after the projection position adjustment is identified based on the second image; the second image is the first pattern after the projection position adjustment captured by the vehicle's camera.

[0025] Based on the fifth coordinate and the sixth coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, a second mapping relationship is constructed; the second mapping relationship is the coordinate transformation relationship between the first preset coordinate system and the second preset coordinate system.

[0026] In the above scheme, adjusting and reprojecting the pattern to construct a new mapping relationship can make the mapping relationship more accurate, thus making subsequent scene recognition or object recognition more accurate.

[0027] In one possible implementation, the above method further includes: storing a fifth coordinate; the stored fifth coordinate is used to retrieve the coordinates of the first target corner point if the next attempt to obtain the coordinates fails.

[0028] In the above scheme, previously obtained data (such as the fifth coordinate mentioned above) can be used to achieve subsequent projection position adjustments, so as to ensure that the deviation between the projection patterns can still be calibrated even if the real-time key data acquisition fails.

[0029] Secondly, this application provides a projection processing apparatus, the apparatus comprising:

[0030] The control unit controls the first projection module of the vehicle to project a first pattern onto a first area; the first pattern and the target object in the first area do not meet the alignment condition.

[0031] An adjustment unit is used to adjust the projection position of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region;

[0032] The control unit is also used to control the first projection module to project the first pattern onto the first area after the projection position is adjusted; the first pattern after the projection position is adjusted satisfies the alignment condition with the target object.

[0033] In one possible implementation, the target object is a second pattern projected by the second projection module of the vehicle onto the first region; the alignment conditions include one or more constraints that align the pattern projected by the first projection module with the pattern projected by the second projection module.

[0034] In one possible implementation, the target object is a straight line within a first region, and the alignment conditions include one or more constraints that make the pattern projected by the first projection module parallel or perpendicular to the straight line.

[0035] Alternatively, the target object may be a temporary reference point within a first region, and the alignment conditions may include one or more constraints that cause the pattern projected by the first projection module to cover the location of the temporary reference point.

[0036] In one possible implementation, the first pattern and the second pattern are patterns in the user interface of the vehicle's first application; the adjustment unit is specifically used for:

[0037] The first target corner point in the first pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, which includes the first and second patterns.

[0038] Based on the first coordinate, the second coordinate, the third coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, and the fourth coordinate of the second target corner point in the second preset coordinate system, adjust the projection position of the first pattern in the first area.

[0039] In one possible implementation, the adjustment unit is specifically used for:

[0040] A first mapping relationship is constructed based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate; the first mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system;

[0041] The target deviation is determined based on the first coordinate, the second coordinate, and the first mapping relationship; the target deviation includes the first deviation and / or the second deviation, the first deviation indicating the deviation of the center point of the first pattern and the center point of the second pattern in the vertical axis direction in the second preset coordinate system; the second deviation indicating the deviation between the width of the circumscribed rectangle of the first pattern and the width of the circumscribed rectangle of the second pattern in the second preset coordinate system.

[0042] Adjust the projection position of the first pattern in the first region according to the target deviation.

[0043] In one possible implementation, the adjustment unit is specifically used for:

[0044] The first pattern and / or the second pattern are translated along the vertical axis of the second preset coordinate system according to the first deviation; and / or the width of the circumscribed rectangle of the first pattern and / or the second pattern is scaled along the horizontal axis of the second preset coordinate system according to the second deviation.

[0045] In one possible implementation, the first pattern is a pattern in the user interface of the vehicle's first application; the control unit, after adjusting the projection position and controlling the first projection module to project the first pattern onto the first area, further includes:

[0046] The fifth coordinate of the first target corner point in the first pattern after the projection position adjustment is identified based on the second image; the second image is the first pattern after the projection position adjustment captured by the vehicle's camera.

[0047] Based on the fifth coordinate and the sixth coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, a second mapping relationship is constructed; the second mapping relationship is the coordinate transformation relationship between the first preset coordinate system and the second preset coordinate system.

[0048] In one possible implementation, the device further includes a storage unit for: storing a fifth coordinate; the stored fifth coordinate is to be retrieved and used in the event that the coordinates of the first target corner point fail to be acquired in the next attempt.

[0049] Thirdly, this application provides a projection processing apparatus, which includes a processor and a memory. The memory provides storage space for storing computer instructions, and the processor invokes the computer instructions stored in the memory to execute the method described in any of the first aspects above.

[0050] Fourthly, this application provides a projection processing system, which includes a projection module and a projection processing device, wherein the projection processing device is the projection processing device of any of the second aspects above, or the projection processing device of the third aspect above.

[0051] Fifthly, this application provides a vehicle that includes the projection processing apparatus of any of the second aspects described above, or the projection processing apparatus of the third aspect described above, or the projection processing system of the fourth aspect described above.

[0052] Sixthly, this application provides a computer-readable storage medium for storing computer instructions; when the instructions are executed by a processor, the method described in any of the first aspects above is performed.

[0053] In a seventh aspect, this application provides a computer program product that includes computer language code or computer instructions; when the computer program product is executed by a processor, it causes the method described in any of the first aspects above to be performed. Attached Figure Description

[0054] Figure 1 is a schematic diagram of the projection processing system provided in an embodiment of this application.

[0055] Figures 2 to 4 are schematic diagrams of projection scenes provided in the embodiments of this application.

[0056] Figure 5 is a schematic diagram of the method flow provided in the embodiment of this application.

[0057] Figures 6 to 8 are schematic diagrams of projection scenes provided in the embodiments of this application.

[0058] Figure 9 is a schematic diagram of a pattern provided in an embodiment of this application.

[0059] Figure 10 is a schematic diagram of the coordinate system provided in the embodiment of this application.

[0060] Figures 11 to 13 are schematic diagrams of the first image and the desired image provided in the embodiments of this application.

[0061] Figures 14 and 15 are schematic diagrams of the apparatus provided in the embodiments of this application. Detailed Implementation

[0062] In this application embodiment, "multiple" refers to two or more. In this application embodiment, "and / or" is used to describe the association relationship of related objects, indicating three relationships that can exist independently. For example, A and / or B can mean: A exists alone, B exists alone, or A and B exist simultaneously. The description methods used in this application embodiment, such as "at least one of a1, a2, ... and an (or at least one of them)," include the case where any one of a1, a2, ... and an exists alone, as well as the case where any combination of any multiple of a1, a2, ... and an exists alone. Each case can exist alone. For example, the description method of "at least one of a, b, and c" includes the cases where a, b, c, a and b combined, a and c combined, b and c combined, or a, b, and c combined.

[0063] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with substantially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another.

[0064] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0065] The following description, in conjunction with the accompanying drawings, provides an exemplary account.

[0066] First, let's introduce a projection processing system that may be applicable to the embodiments of this application. For example, please refer to Figure 1. Figure 1 illustrates a possible projection processing system, which may include a camera 101, a first controller 102, a second controller 103, and a projection module 104. The second controller 103 may house a first application 1031 and a projection processing module 1032.

[0067] For example, the camera 101 is connected to the first controller 102. The camera 101 can send the captured images and / or videos to the first controller 102. The first controller 102 can send the received images and / or videos to the projection processing module 1032 in the second controller 103. The projection processing module 1032 can process the received images and / or videos to obtain processing results, and send the processing results to the first application 1031. This allows the first application 1031 to adjust its user interface (UI) according to the processing results, and then control the projection module 104 to project the desired user interface pattern. For details on the operation of the projection processing module 1032 and the first application 1031, please refer to the following description; details will not be elaborated here.

[0068] For example, in one possible implementation, the projection processing system described above can be a system deployed on a vehicle. The camera 101 described above can be, for example, a camera in the vehicle, such as a forward-facing camera in the vehicle, etc., and this application embodiment does not limit this.

[0069] For example, the first controller 102 and the second controller 103 described above can be domain controllers in the vehicle. For instance, the first controller 102 can be an intelligent driving computing platform (MDC) in the vehicle. The MDC can be translated as a mobile data center (MDC) or a motion domain controller (MDC), etc. The second controller 103 can be, for example, a cockpit domain controller (CDC). In other possible implementations, the first controller 102 or the second controller 103 can also be other domain controllers in the vehicle, such as a vehicle domain controller (VDC), a vehicle control unit (VCU), or an intelligent driving controller, etc. Alternatively, in another possible implementation, the first controller 102 and the second controller 103 can belong to the same domain controller. The embodiments of this application do not limit the specific implementation of the first controller 102 and the second controller 103.

[0070] For example, the projection module 104 described above can be a pixel headlight in the vehicle described above. For example, it can include the left-side headlight pixel headlight of the vehicle. Or, it can include the right-side headlight pixel headlight of the vehicle. Or, it can include the headlight pixel headlights on both sides of the vehicle. Or, it can include the headlights of the vehicle, etc. This application embodiment does not limit this.

[0071] Exemplarily, the projection module 104 described above may include multiple pixel units for emitting light. Each pixel unit can be individually controlled to be turned on or off. Turning a pixel unit on indicates that the pixel unit is controlled to emit light, i.e., it is lit. Turning a pixel unit off indicates that the pixel unit is controlled to not emit light, i.e., it is turned off. By controlling the lighting or extinguishing of each pixel unit among the multiple pixel units, the light emitted by the projection module 104 can be projected to form various patterns and shapes. Exemplarily, the projection module 104 may be, for example, a light-emitting module implemented based on digital light processing (DLP). Alternatively, exemplarily, the projection module 104 may be, for example, a light-emitting module including multiple light-emitting diodes (LEDs). Alternatively, exemplarily, the projection module 104 may also be, for example, a light-emitting module based on reflective liquid crystal projection technology, etc. This application embodiment does not limit the implementation method of the projection module 104.

[0072] For example, the first application 1031 described above can be any application. For instance, it can be an application for various games including motion-sensing games, audio and video applications, or social applications, etc. This application embodiment does not limit the specific application type.

[0073] For example, in another possible implementation, the projection processing module 1032 may be a processing module included in the first application 1031. That is, the projection processing module 1032 and the first application 1031 may be set separately, or the projection processing module 1032 may be embedded in the first application 1031. The specific choice depends on the actual application requirements, and this application embodiment does not limit this.

[0074] It is understood that the projection processing system shown in Figure 1 above is only an example and does not constitute a limitation on the embodiments of this application.

[0075] For example, due to installation errors of the projection module in the vehicle or unevenness of the projected area, the pattern projected by the projection module may have some deviations. For instance, due to installation errors of the projection modules on the left and right sides of the vehicle, the projection areas of the left and right projection modules are often misaligned. For ease of understanding, please refer to Figure 2 for example. In Figure 2, it can be seen that the projection areas of the left and right projection modules of the vehicle are not aligned side by side, and there is a positional offset in the direction of the vehicle's front. Or, for example, in the scenario of projection by a single projection module on the vehicle, the projection area is not aligned with some specific target objects (such as straight lines or temporary reference points around the projection area). For ease of understanding, please refer to Figure 3 or Figure 4 for example. Figure 3 or Figure 4 shows the target object as a straight line. In Figures 3 and 4, it can be seen that the projection area of ​​the projection module is not parallel or perpendicular to the straight line that is the target object, that is, the boundary of the area close to the straight line has an angle α with the straight line. This misalignment will affect the user experience. It is understood that the misalignment shown in Figures 2 to 4 is only an example and does not constitute a limitation on the embodiments of this application. Other misalignment situations may occur in specific implementations.

[0076] Based on the above description, embodiments of this application provide a projection processing method and related apparatus, which can align the pattern projected by the vehicle projection module with the target object, thereby improving the user experience. Exemplary details are described below.

[0077] First, the projection processing method provided in the embodiments of this application is introduced. Exemplarily, this method can be executed by a target controller in a vehicle. This target controller can, for example, be the second controller 103 in the projection processing system shown in FIG1 above. Referring to FIG5, the method may include, but is not limited to, the following steps S501 to S503.

[0078] S501, The target controller controls the first projection module of the vehicle to project a first pattern onto a first area; the first pattern and the target object in the first area do not meet the alignment condition.

[0079] For example, the first projection module described above may be a single pixel headlight in the vehicle. For instance, it could be a left-side or right-side headlight. This first projection module may be one of the projection modules included in projection module 104 of the projection processing system shown in FIG1.

[0080] For example, the first region mentioned above can be any region that can be projected, such as a ground region or a wall region, and this application embodiment does not limit this.

[0081] For example, the target controller described above can control the first projection module to project a first pattern onto a first area via a first application deployed in the target controller. The first pattern is a pattern in the user interface of the first application. The user interface of the first application is a medium presented to the user to enable interaction or information exchange between the user and the device. That is, the first projection module projects a user interface (referred to as the first user interface) of the first application onto the first area, and the first user interface includes the first pattern. The first application may, for example, be the first application 1031 included in the projection processing system shown in FIG1.

[0082] For example, the first pattern projected by the first projection module onto the first area does not meet the alignment condition with the target object. The following describes the cases in illustrative ways.

[0083] In one possible implementation, the target object is the second pattern projected onto the first region by the second projection module of the aforementioned vehicle. The misalignment between the first pattern projected onto the first region by the first projection module and the target object means that the first pattern projected onto the first region by the first projection module and the second pattern projected onto the first region by the second projection module do not meet the alignment condition. This alignment condition may include one or more constraints that align the patterns projected by the first projection module and the patterns projected by the second projection module. See the subsequent example descriptions for details, which will not be elaborated here.

[0084] For example, the second projection module may be one of the projection modules included in projection module 104 of the projection processing system shown in FIG1. ​​This second projection module may be, for example, another pixel headlight in a vehicle. For instance, if the first projection module is the left-side headlight pixel headlight of the vehicle, then the second projection module is the right-side headlight pixel headlight of the vehicle. Or, if the first projection module is the right-side headlight pixel headlight of the vehicle, then the second projection module is the left-side headlight pixel headlight of the vehicle. Similarly, the second pattern is a pattern projected by the first application control of the target controller onto the first area by the second projection module. This second pattern is also a pattern in the first user interface of the first application. That is, the first projection module projects the first user interface of the first application onto the first area, and the first user interface also includes the second pattern.

[0085] For example, the first pattern indicated by the first application to be projected by the first projection module and the second pattern indicated by the second projection module to be projected can be patterns of the same shape and size. However, in some possible cases, due to installation errors of the first and second projection modules or uneven ground, the shape or size of the first and second patterns projected in the first area may deviate. See the following description for details.

[0086] For example, in one possible implementation, the first and second patterns can be arbitrary polygonal patterns. The vertices of the polygon are corner points. Corner points can also be feature points or points of interest. For example, the polygonal pattern can be a hollow pattern, which can reduce the problem of reflections from the ground or wall during camera shooting. Based on this, for example, the alignment condition of the first and second patterns can include, for example, the alignment of the two patterns projected in the first region side-by-side in the direction of the vehicle's front. For example, this side-by-side alignment means, for example, that the positional offset of the center points of the two patterns in the direction of the vehicle's front tends to be zero. It is understood that, ideally, this side-by-side alignment means that the positional offset of the center points of the two patterns in the direction of the vehicle's front is equal to zero. However, due to some limitations of the device itself or the unevenness of the first region, the positional offset of the center points of the two patterns in the direction of the vehicle's front cannot be completely equal to zero, but can only infinitely approach zero. Alternatively, for example, the alignment condition of the first and second patterns can include, for example, the width of the bounding rectangle of the two patterns tending to be zero. Similarly, due to limitations of the device itself or unevenness in the first region, the width of the outer rectangles of the two patterns cannot be exactly zero, but can only approach zero infinitely. Therefore, the first and second patterns not meeting the alignment condition may include: the center points of the two patterns are significantly offset in the direction of the vehicle's front, for example, exceeding threshold 1, making it obvious to the user in the first region that the two patterns are not aligned side-by-side. And / or, the first and second patterns not meeting the alignment condition may include a significant deviation in the width of the outer rectangles of the two patterns, for example, exceeding threshold 2, making it obvious to the user in the first region that the widths of the outer rectangles of the two patterns are different. The width of the outer rectangle of the pattern can be, for example, the length of the side of the pattern in the direction perpendicular to the direction of the vehicle's front (hereinafter referred to as the direction perpendicular to the vehicle's front). For ease of understanding, see Figures 6 and 7 for example. Figures 6 and 7 use the first and second patterns projected by the first application indicator as rectangles, and the first region as the ground region as an example. The same applies to other regions such as wall areas, and will not be elaborated further.

[0087] For example, the projection scale varies with distance. For instance, the farther the projection distance, the larger the pattern appears. Based on this, in the projection scenarios shown in Figures 6 and 7, although the first application indicates that the first and second patterns to be projected are rectangles, the actual projection of the first and second patterns onto the ground area appears more like trapezoids. This is because the portion farther from the front of the vehicle is projected larger. However, Figures 6 and 7 still illustrate the first and second patterns as rectangular shapes.

[0088] Figure 6 illustrates, for example, a schematic diagram of the positional offset of the center points of the two patterns in the direction of the vehicle's front. As shown in Figure 6, assume the first pattern is projected from the right-side projection module of the vehicle, and the second pattern is projected from the left-side projection module. The positional offset of the center point O1 of the first pattern and the center point O2 of the second pattern in the direction of the vehicle's front is represented by d1. This positional offset is large; for example, d1 is greater than the aforementioned threshold 1. This results in the first and second patterns not being aligned side-by-side in the direction of the vehicle's front, meaning the two projected patterns do not meet the alignment condition.

[0089] Figure 7 illustrates the width offset of the bounding rectangles of the two patterns. As seen in Figure 6, assume the first pattern is projected from the right side of the vehicle's projection module, and the second pattern is projected from the left side. Since both patterns are rectangles, their bounding rectangles are the outlines of the patterns themselves. The width of the bounding rectangle of the first pattern is denoted as d2, which represents the side of the bounding rectangle perpendicular to the vehicle's front direction. The width of the bounding rectangle of the second pattern is denoted as d3, similarly representing the side of the bounding rectangle perpendicular to the vehicle's front direction. Figure 7 shows that d2 and d3 are not equal; for example, d3 is smaller than d2. The absolute value of the difference between d2 and d3 represents the deviation Δd, which is greater than the aforementioned second threshold. This results in a noticeable difference in the width of the bounding rectangles of the two patterns.

[0090] For example, to better understand the concept of the circumscribed rectangle of the above patterns, please refer to Figure 8. Figure 8 illustrates the first and second patterns as rhombuses. As can be seen in Figure 8, compared to Figure 7, the right rectangle of the projected pattern in Figure 8 has become a rhombus, and the circumscribed rectangle of the pattern is the circumscribed rectangle of this rhombus. Other descriptions are similarly based on the relevant descriptions in Figure 7 above, and will not be repeated here.

[0091] It is understood that Figures 6 to 8 above are merely examples and do not constitute a limitation on the embodiments of this application.

[0092] In another possible implementation, the first and second patterns described above are not limited to arbitrary polygonal patterns. For example, they can be other patterns, such as cross-shaped patterns, multi-pointed star patterns, or patterns composed of multiple points, etc. The embodiments of this application do not limit the shape or size of the projected pattern. Specifically, the projected pattern can be selected according to the actual application requirements. For example, if it is a cross-shaped pattern, a multi-pointed star pattern, or a pattern composed of multiple points, the corner points of the pattern are the points that contact the circumscribed rectangle of the pattern. For example, see Figure 9. Figure 9 exemplarily shows a schematic diagram of the corner points of a cross-shaped pattern, a pentagram pattern, and a pattern composed of four points. For example, in a cross-shaped pattern, points P11, P12, P13, and P14 that contact the circumscribed rectangle of the cross-shaped pattern are the corner points of the cross-shaped pattern. In a pentagram pattern, points P21, P22, P23, P24, and P25 that contact the circumscribed rectangle of the pentagram pattern are the corner points of the pentagram pattern. In the pattern formed by the four points, points P31, P32, P33, and P34 that contact the outer rectangle of the pattern are the corner points of the cross-shaped pattern. It is understood that Figure 9 is merely an example and does not constitute a limitation on the embodiments of this application. The corner points of other patterns are similar and will not be described in detail here.

[0093] The above example, using the second pattern projected by the second projection module onto the first area as the target object (the vehicle), illustrates some possible examples where the alignment condition between the first pattern projected by the first projection module onto the first area and the target object is not met. This does not constitute a limitation on the embodiments of this application.

[0094] In another possible implementation, the target object can be a straight line within the first region. Therefore, the misalignment between the first pattern projected by the first projection module onto the first region and the target object means that the first pattern projected by the first projection module onto the first region and the straight line within that first region do not meet the alignment condition. A description of the first pattern can be found in the foregoing introduction, and will not be repeated here. This alignment condition may include one or more constraints that the pattern projected by the first projection module and the straight line are parallel or perpendicular. For example, it could be that a side of the circumscribed rectangle of the pattern projected by the first projection module is parallel or perpendicular to the straight line. Therefore, the misalignment between the first pattern projected by the first projection module onto the first region and the straight line means that a side of the circumscribed rectangle of the first pattern is not parallel or perpendicular to the straight line. For example, see Figures 3 or 4 above; this embodiment does not impose such limitations.

[0095] In another possible implementation, the target object can be a temporary reference point within the first area. For example, it could be a person or other static or dynamic objects. Therefore, the misalignment between the first pattern projected by the first projection module onto the first area and the target object refers to one or more constraints where the first pattern projected by the first projection module does not cover the position of the temporary reference point. A description of this first pattern can be found in the foregoing introduction and will not be repeated here. This alignment condition may include the first pattern being projected onto the location of the temporary reference point. For example, in an interactive welcoming scenario, the welcoming pattern can be projected directly onto the user's feet.

[0096] It is understood that the above-described implementation of the first pattern projected by the first projection module onto the first area not meeting the alignment condition with the target object is merely an example and does not constitute a limitation on the embodiments of this application.

[0097] For example, in one possible implementation, the misalignment between the first pattern and the target object in the first region can be determined by the target controller. The implementation of this controller determination can be exemplarily referred to in the corresponding description in S502 below, and will not be detailed here. Alternatively, in another possible implementation, the user can input a command indicating that the misalignment between the first pattern and the target object is not met. For example, after the first projection module projects the first pattern onto the first region, the user sees that the projected first pattern and the target object do not meet the corresponding alignment condition. Therefore, the user can input a command through the vehicle's human-machine interface to indicate that the misalignment between the first pattern and the target object is not met, triggering the target controller to perform subsequent adjustments to make the projected first pattern and the target object meet the corresponding alignment condition.

[0098] S502, The target controller adjusts the projection position of the first pattern in the first area according to the relative position between the first pattern and the target object in the first area.

[0099] For example, based on the above description, since the alignment condition between the first pattern projected by the first projection module onto the first area and the target object is not met, the user experience is affected. Therefore, the target controller can adjust the projection position of the first pattern in the first area according to the relative position between the first pattern projected in the first area and the target object. The following describes examples in different scenarios.

[0100] In one possible implementation, if the target object is a second pattern projected onto the first region by the second projection module of the vehicle, the projection position of the first pattern can be adjusted so that it satisfies the alignment condition with the projection position of the second pattern. For example, the two patterns projected into the first region can be aligned side-by-side in the direction of the vehicle's front, and / or the width of the bounding rectangle of the two patterns can be made to approach zero. The implementation process of the adjustment is described below as an example.

[0101] For example, after the first projection module projects the first pattern and the second projection module projects the second pattern, a first image including the first and second patterns can be captured by a vehicle camera. For example, the camera can be camera 101 in the projection processing system shown in FIG1. ​​The target controller can acquire the first image captured by the camera. For example, referring to FIG1, after camera 101 captures the first image, it can send the first image to the first controller 102. The first controller 102 then sends the first image to the second controller 103, i.e., to the target controller. The target controller can transmit the first image to a projection processing module (e.g., projection processing module 1032 shown in FIG1) for processing.

[0102] For example, the projection processing module can identify the first coordinates of the first target corner point in the first pattern in the first preset coordinate system based on the received first image, and identify the second coordinates of the second target corner point in the second pattern in the first preset coordinate system based on the first image. For example, any corner detection method or image segmentation algorithm can be used to identify corner points. For instance, machine vision methods can be used, or traditional OpenCV-based solutions can be used, or AI models for keypoint detection can be used to detect corner points in the image. This application does not limit this approach.

[0103] For example, the first coordinate is used to indicate the position of the first pattern in the first region, and the second coordinate is used to indicate the position of the second pattern in the first region. For example, the first preset coordinate system can be, for example, the camera coordinate system corresponding to the camera, or, for example, the image coordinate system of the image captured by the camera, or, for example, the pixel coordinate system corresponding to the image captured by the camera. The camera coordinate system, image coordinate system, and pixel coordinate system can be transformed between each other. The specific coordinate system is selected according to the actual application requirements, and this application embodiment does not impose any limitations on this.

[0104] For example, the first target corner point in the first pattern described above may include one or more corner points, and the first coordinate may include the coordinates of each of the one or more corner points. Similarly, the second target corner point in the second pattern described above may include one or more corner points, and the second coordinate may include the coordinates of each of the one or more corner points.

[0105] For example, after obtaining the first and second coordinates, the projection processing module can construct a first mapping relationship based on the first coordinates, the second coordinates, the third coordinate of the first target corner point in the second preset coordinate system corresponding to the first user interface, and the fourth coordinate of the second target corner point in the second preset coordinate system. This first mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system. For example, this first mapping relationship can be represented by a coordinate transformation matrix from the first preset coordinate system to the second preset coordinate system. For example, the second preset coordinate system can be, for example, the coordinate system of the user interface of the first application. For ease of understanding, please refer to Figure 10. A schematic diagram of the coordinate system of the user interface of the first application is shown. As can be seen in Figure 10, the origin of the coordinate system of the user interface of the first application is a vertex in the user interface, such as point O shown in Figure 10. Then, two perpendicular edges contacting point O are located, one on the horizontal axis of the coordinate system and the other on the vertical axis of the coordinate system, as shown in Figure 10. It is understood that Figure 10 is merely an example and does not constitute a limitation on the embodiments of this application. In other possible implementations, the origin of the coordinate system of the user interface of the first application may be, for example, the center point of the user interface, etc., and this application embodiment does not limit this.

[0106] For example, since the first and second patterns are projected by the corresponding projection module controlled by the first application, the first application knows the third coordinate of the first target corner point in the second preset coordinate system and the fourth coordinate of the second target corner point in the same second preset coordinate system. The projection processing module can obtain the third and fourth coordinates from the first application. The third coordinate includes the coordinates of each corner point among the one or more corner points included in the first target corner point; the fourth coordinate includes the coordinates of each corner point among the one or more corner points included in the second target corner point.

[0107] For example, before constructing the first mapping relationship based on the first, second, third, and fourth coordinates, the projection processing module first determines the desired first coordinates and the desired second coordinates based on the first and second coordinates. The desired first coordinate can be the coordinates of the first target corner point identified from the captured image in a first preset coordinate system, assuming the first and second patterns of the projection meet alignment conditions. Similarly, the desired second coordinate can be the coordinates of the second target corner point identified from the captured image in a first preset coordinate system, assuming the first and second patterns of the projection meet alignment conditions. For ease of understanding, an illustrative description is provided below with reference to Figures 11 and 12.

[0108] For example, Figure 11 is an example of a pattern projected from Figure 6. Figure 11 exemplarily shows a possible first image and a desired image. The desired image is the image captured when the first and second patterns of the assumed projection meet the alignment conditions. It also shows the coordinates of the first target corner point of the first pattern and the second target corner point of the second pattern in the first preset coordinate system. The coordinates of the first target corner point include (x... 11 ,y 11 ), (x 12 ,y 12 ), (x 13 ,y 13 ) and (x 14 ,y 14 The coordinates of the second target corner point include (x, y, y). 21 ,y 21 ), (x 22 ,y 22 ), (x 23 ,y 23 ) and (x 24 ,y 24 Part or all of the following. For example, assume a corner point (x) 11 ,y 11 ) and corner point (x 12 ,y 12 If the side containing the given information is parallel to the horizontal axis of the first preset coordinate system, then y 11 and y 12 They are equal. Similarly, suppose y 13 and y 14 Equal, y 21 and y 22 Equal, y 23 and y 24 They are equal. Therefore, the positional deviation Δy = y_0 along the vertical axis of the first preset coordinate system is equal to the positional deviation of the center point O1 of the first pattern and the center point O2 of the second pattern. 11 -y 21, or Δy = y 14 -y 24 For example, the projection scale may differ due to variations in distance, y 11 -y 21 y 14 -y 24 The y-axis coordinates of points O1 and O2 are not equal, indicating some deviation. However, it is still possible to choose y. 11 -y 21 and y 14 -y 24 The positional deviation Δy can be represented by any one of the values ​​in the coordinate system. This will not affect the subsequent calculation results. Based on this positional deviation Δy, the coordinates of the aligned first and second target corner points in the desired image can be determined, that is, the desired first and second coordinates mentioned above can be determined. For example, in one possible implementation, the second pattern can be translated upwards by Δy along the vertical axis of the first preset coordinate system to align the first and second patterns, as shown in the desired image in Figure 11. In this case, y 21 '=y 21 +Δy,y 22 '=y 22 +Δy,y 23 '=y 23 +Δy,y 24 '=y 24 +Δy. That is, the expected first coordinate includes (x... 11 ,y 11 ), (x 12 ,y 12 ), (x 13 ,y 13 ) and (x 14 ,y 14 Part or all of the following. The expected second coordinate includes (x... 21 ,y 21 '), (x 22 ,y 22 '), (x 23 ,y 23 ') and (x 24 ,y 24The first pattern can be aligned with the second pattern by translating the first pattern downwards by Δy along the vertical axis of the first preset coordinate system. Alternatively, the first pattern can be translated downwards by Δy1 along the vertical axis of the first preset coordinate system, and the second pattern can be translated upwards by Δy2 along the vertical axis of the first preset coordinate system. Where Δy1 + Δy2 = Δy. Regardless of the method, the desired first and second coordinates can be obtained. The embodiments in this application will not be described in detail.

[0109] For example, Figure 12 is an example of a pattern projected from Figure 7. Figure 12 exemplarily shows another possible first image and a desired image. The desired image is the image captured when the first and second patterns of the assumed projection meet the alignment conditions. It also shows the coordinates of the first target corner point of the first pattern and the second target corner point of the second pattern in the first preset coordinate system. The coordinates of the first target corner point include (x... 31 ,y 31 ), (x 32 ,y 32 ), (x 33 ,y 33 ) and (x 34 ,y 34 The coordinates of the second target corner point include (x, y, y). 41 ,y 41 ), (x 42 ,y 42 ), (x 43 ,y 43 ) and (x 44 ,y 44 Part or all of the following. For example, assume a corner point (x) 31 ,y 31 ) and corner point (x 34 ,y 34 If the side containing the x-axis is parallel to the vertical axis of the first preset coordinate system, then x... 31 and x 34 They are equal. Similarly, suppose x 32 and x 33 Equal, x 41 and x 44 Equal, x 42 and x 43 They are equal. Therefore, the deviation Δx between the width d2' of the bounding rectangle of the first pattern and the width d3' of the bounding rectangle of the second pattern along the horizontal axis of the first preset coordinate system is Δx = d2' - d3'. d2' = x 32 -x 31 =y 33 -y34 d3'=x 42 -x 41 =y 43 -y 44 Based on this positional deviation Δx, the coordinates of the aligned first and second target corner points in the desired image can be determined, i.e., the desired first and second coordinates can be determined. For example, in one possible implementation, the width of the circumscribed rectangle of the second pattern can be enlarged by Δx along the horizontal axis of the first preset coordinate system to align the widths of the circumscribed rectangles of the first and second patterns, as shown in the desired image in Figure 12. In the case shown in Figure 12, the horizontal coordinate x in the second pattern... 41 and x 44 x remains unchanged 42 '=x 42 +Δx, x 43 '=x 43 +Δx. That is, the expected first coordinate includes (x... 31 ,y 31 ), (x 32 ,y 32 ), (x 33 ,y 33 ) and (x 34 ,y 34 Part or all of the following. The expected second coordinate includes (x... 41 ,y 41 ), (x 42 ',y 42 ), (x 43 ',y 43 ) and (x 44 ,y 44 Part or all of the ) in ). Or, in another possible implementation, it could be adjusting the x-coordinate in the second pattern. 41 and x 44 x 42 and x 43 It remains unchanged. Alternatively, in another possible implementation, the x-coordinate in the second pattern can be adjusted simultaneously. 41 x 42、 x 43 and x 44Alternatively, in another possible implementation, the width of the outer rectangle of the first pattern can be reduced by Δx along the horizontal axis of the first preset coordinate system to align the widths of the outer rectangles of the first and second patterns. Alternatively, in another possible implementation, the widths of the outer rectangles of the first and second patterns can be simultaneously scaled and adjusted to align them. Regardless of the method, the desired first and second coordinates can be obtained. The embodiments in this application will not be described in detail.

[0110] For example, after obtaining the desired first coordinates and desired second coordinates, the projection processing module can construct the first mapping relationship based on the desired first coordinates, desired second coordinates, and the desired third and fourth coordinates. Since the desired first coordinates and desired second coordinates are multiple coordinates in the desired first preset coordinate system, and the desired third and fourth coordinates are multiple coordinates in the desired second preset coordinate system, an affine transformation relationship from the desired first coordinate system to the desired second preset coordinate system can be established based on the desired first, second, third, and fourth coordinates. Then, the affine transformation matrix from the desired first coordinate system to the desired second preset coordinate system can be solved, thereby obtaining the aforementioned first mapping relationship. The specific algorithm for solving the affine transformation matrix is ​​not limited in this application embodiment; any feasible solution algorithm can be used, and this application embodiment will not elaborate on it.

[0111] For example, in one possible implementation, after the projection processing module obtains the first and second coordinates based on the first image, before constructing the first mapping relationship, it can first determine whether the projected first and second patterns meet the corresponding alignment conditions. For example, referring to the situation shown in Figure 11, it can first determine whether the positional deviation Δy of the center point O1 of the first pattern and the center point O2 of the second pattern in the vertical direction of the first preset coordinate system is greater than a preset first deviation threshold. If it is greater, it indicates that the first and second patterns do not meet the corresponding alignment conditions. Then, subsequent deviation adjustment operations need to be performed. Alternatively, for example, referring to the situation shown in Figure 12, it can first determine whether the deviation Δx of the width d2' of the outer rectangle of the first pattern and the width d3' of the outer rectangle of the second pattern in the horizontal direction of the first preset coordinate system is greater than a preset second deviation threshold. If it is greater, it indicates that the first and second patterns do not meet the corresponding alignment conditions. Then, subsequent deviation adjustment operations need to be performed. It is understood that the implementation method for determining whether the projected first and second patterns meet the corresponding alignment conditions is only an example and does not constitute a limitation on the embodiments of this application.

[0112] For example, after obtaining the first mapping relationship, the projection processing module can further determine the target deviation by combining the first coordinates of the first target corner point of the first pattern obtained based on the first image recognition and the second coordinates of the second target corner point of the second pattern. This target deviation may include a first deviation and / or a second deviation. For example, the first deviation indicates the deviation of the center point of the first pattern and the center point of the second pattern in the vertical axis direction within the second preset coordinate system. The second deviation indicates the deviation between the width of the circumscribed rectangle of the first pattern and the width of the circumscribed rectangle of the second pattern in the second preset coordinate system.

[0113] For example, in one possible implementation, when the first mapping relationship is constructed by combining the desired first coordinate and the desired second coordinate obtained in the example situation shown in Figure 11, the first deviation can be determined based on the first mapping relationship and the corresponding first and second coordinates. For example, based on the first mapping relationship, the coordinates of the first target corner point in the second preset coordinate system corresponding to the first coordinate can be determined. That is, it is equivalent to mapping the first coordinate to the user interface of the first application, and the coordinates obtained by this mapping will be referred to as the first mapping coordinates. Similarly, based on the first mapping relationship, the coordinates of the second target corner point in the second preset coordinate system corresponding to the second coordinate can be determined. That is, it is equivalent to mapping the second coordinate to the user interface of the first application, and the coordinates obtained by this mapping will be referred to as the second mapping coordinates. Based on the first mapping coordinates and the second mapping coordinates, the deviation Δy' of the center point of the first pattern and the center point of the second pattern in the vertical direction in the second preset coordinate system can be determined. That is, the first deviation is determined. For details, please refer to the relevant implementation method for determining Δy in the first preset coordinate system in Figure 11 above, which will not be elaborated here. For example, after obtaining the deviation Δy', the projection processing module can send the deviation Δy' to the first application. The first application then translates the first pattern and / or the second pattern along the vertical axis in the second preset coordinate system based on Δy'. This ensures that the second pattern and the second pattern after reprojection satisfy the alignment conditions described above. For example, the implementation of translating the first pattern and / or the second pattern along the vertical axis in the second preset coordinate system can be exemplarily referred to the relevant implementation method of translating the first pattern and / or the second pattern along the vertical axis in the first preset coordinate system in Figure 11 above, and will not be repeated here.

[0114] For example, in another possible implementation, when the first mapping relationship is constructed by combining the expected first coordinate and the expected second coordinate obtained in the example shown in Figure 12, the second deviation can be determined based on the first mapping relationship and the corresponding first and second coordinates. For example, based on the first mapping relationship, the coordinates of the first target corner point in the second preset coordinate system corresponding to the first coordinate can be determined. That is, it is equivalent to mapping the first coordinate to the user interface of the first application, and the coordinates obtained by this mapping will be referred to as the third mapping coordinates. Similarly, based on the first mapping relationship, the coordinates of the second target corner point in the second preset coordinate system corresponding to the second coordinate can be determined. That is, it is equivalent to mapping the second coordinate to the user interface of the first application, and the coordinates obtained by this mapping will be referred to as the fourth mapping coordinates. Based on the third and fourth mapping coordinates, the deviation Δx' between the width of the outer rectangle of the first pattern and the width of the outer rectangle of the second pattern in the second preset coordinate system can be determined. That is, the second deviation is determined. For details, please refer to the relevant implementation method for determining Δx in the first preset coordinate system in Figure 12 above, which will not be repeated here. For example, after obtaining the deviation Δx', the projection processing module can send the deviation Δx' to the first application. The first application then scales the width of the circumscribed rectangle of the first pattern and / or the second pattern in the horizontal direction of the second preset coordinate system according to Δx'. This ensures that the second pattern and the second pattern after reprojection satisfy the alignment conditions described above. For example, the implementation of scaling the width of the circumscribed rectangle of the first pattern and / or the second pattern in the horizontal direction of the second preset coordinate system can be exemplarily referred to the relevant implementation method of scaling the width of the circumscribed rectangle of the first pattern and / or the second pattern in the horizontal direction of the first preset coordinate system in Figure 12, which will not be repeated here.

[0115] For example, in another possible implementation, if the first and second patterns of the above projection are not aligned side-by-side in the direction of the vehicle's front, and the widths of the bounding rectangles of the two patterns differ significantly, then the adjustment of the deviation Δy' in the example case shown in Figure 11 and the adjustment of the deviation Δx' in the example case shown in Figure 12 can be implemented simultaneously. For specific implementation details, please refer to the foregoing description, which will not be repeated here.

[0116] For example, the first application translates the first pattern and / or the second pattern along the vertical axis of the second preset coordinate system according to Δy', and / or the first application scales the width of the bounding rectangle of the first pattern and / or the second pattern along the horizontal axis of the second preset coordinate system according to Δx'. This can change the positions of the first and second patterns in the user interface of the first application. These adjustments can compensate for projection deviations caused by errors in the installation of the projection module or uneven ground. This results in a corresponding change in the projection positions of the first and / or second patterns in the first area, so that the adjusted first and second patterns reprojected into the first area can meet the above alignment conditions.

[0117] The above example, using the second pattern projected by the second projection module onto the first area as the target object (the vehicle), illustrates some possible examples of adjusting the projection position of the first pattern and / or the second pattern in the first area, and does not constitute a limitation on the embodiments of this application.

[0118] In another possible implementation, the target object can be a straight line within the first region. The projection position of the first pattern can be adjusted so that the first pattern and the straight line satisfy the alignment condition described above. For example, one side of the outer rectangle of the first pattern projected by the first projection module can be parallel or perpendicular to the straight line. The implementation process of the adjustment is illustrated below.

[0119] For example, after the first projection module projects the first pattern, a first image including the first pattern and the straight line can be captured by the camera. For example, after obtaining the first image, the projection processing module can identify the first coordinates of the first target corner point in the first pattern in a first preset coordinate system, and identify the coordinates of the third target corner point on the straight line in the first preset coordinate system. For a description of the first target corner point, the first coordinates, and the first preset coordinate system, please refer to the foregoing description, which will not be repeated here. The third target corner point may include any number of points on the straight line; this embodiment does not limit this.

[0120] For example, after obtaining the first coordinates, the projection processing module can construct a first mapping relationship based on the first coordinates and the third coordinates of the first target corner point in the second preset coordinate system corresponding to the first user interface. For a description of the first mapping relationship and the second preset coordinate system, please refer to the foregoing introduction, which will not be repeated here.

[0121] For example, since the first pattern is projected by the projection module corresponding to the first application, the first application knows the third coordinates of the first target corner point in the second preset coordinate system. The projection processing module can obtain these third coordinates from the first application.

[0122] For example, before constructing the first mapping relationship based on the first coordinate and the third coordinate, the projection processing module first determines the desired first coordinate based on the first coordinate and the coordinate of the third target corner point. This desired first coordinate can be the coordinate of the first target corner point identified from the captured image in a first preset coordinate system, assuming that the first pattern of the projection and the straight line satisfy the corresponding alignment conditions. For ease of understanding, this will be illustrated below with reference to Figure 13.

[0123] For example, Figure 13 is an example of a pattern projected from Figure 3. Figure 13 exemplarily shows a possible first image and a desired image. The desired image is the image captured when the first pattern of the assumed projection and the straight line satisfy the corresponding alignment conditions described above. It also shows the coordinates of the first target corner point of the first pattern and the third target corner point on the straight line in the first preset coordinate system. The coordinates of the first target corner point include (x... 51 ,y 51 ), (x 52 ,y 52 ), (x 53 ,y 53 ) and (x 54 ,y 54 This may be part or all of the following. Taking a third target corner point as an example, which includes two corner points, the coordinates of these two corner points are (x...). 61 ,y 61 ) and (x 62 ,y 62 As can be seen in the first image of Figure 13, the corner point (x) in the first pattern 53 ,y 53 ) and corner point (x 54 ,y 54 The edge containing the first target corner point (x) is not parallel to the straight line, and there is an angle α between them. For example, based on the coordinates of the first and third target corner points, the coordinates of the aligned first target corner point in the desired image can be determined, i.e., the aforementioned desired first coordinates can be determined. For example, the first pattern can be rotated according to the coordinates of the first and third target corner points, such that a certain edge of the first pattern (e.g., a corner point (x)) is aligned with the first target corner point. 53 ,y 53 ) and corner point (x 54 ,y 54 The edge containing the first target corner point (x) is parallel to the straight line. Alternatively, the first pattern can be rotated based on the coordinates of the first and third target corner points so that a certain edge of the first pattern (e.g., a corner point (x)) is parallel to the straight line. 51 ,y 51 ) and corner point (x 54 ,y 54The edge containing the first pattern is perpendicular to the straight line. The specific process of rotating to obtain the desired first coordinates is not detailed here. For example, in one possible implementation, after rotation, a certain edge of the first pattern is made parallel or perpendicular to the straight line to obtain the desired image as shown in Figure 13. Then the desired first coordinates include (x...) as shown in Figure 13. 51 ',y 51 '), (x 52 ',y 52 '), (x 53 ',y 53 ') and (x 54 ',y 54 Figure 13 is merely an example and does not constitute a limitation on the embodiments of this application.

[0124] For example, after obtaining the desired first coordinates, the projection processing module can construct the first mapping relationship based on the desired first coordinates and the third coordinates. A detailed description of the construction process is provided above and will not be repeated here.

[0125] For example, in one possible implementation, after the projection processing module obtains the coordinates of the first coordinate and the third target corner point based on the first image, before constructing the first mapping relationship, it can first determine whether the projected first pattern and the line satisfy the corresponding alignment conditions. For example, referring to the situation shown in Figure 13, it can first determine whether the included angle α is greater than a preset included angle threshold. If it is greater, it indicates that the first pattern and the line do not satisfy the corresponding alignment conditions. Then, subsequent adjustment of the deviation needs to be performed. It is understood that the implementation method for determining whether the projected first pattern and the line satisfy the corresponding alignment conditions is merely an example and does not constitute a limitation on the embodiments of this application.

[0126] For example, after obtaining the first mapping relationship, the projection processing module can combine the first coordinates of the first target corner point of the first pattern obtained based on the first image recognition and the coordinates of the third target corner point of the straight line to determine the coordinates of the first target corner point in the second preset coordinate system corresponding to the first coordinates obtained based on the first image recognition. This is equivalent to mapping the first coordinates to the user interface of the first application, and these mapped coordinates will be referred to as the fifth mapping coordinates. Similarly, based on the first mapping relationship, the coordinates in the second preset coordinate system corresponding to the coordinates of the third target corner point on the straight line can be determined. This is equivalent to mapping the coordinates of the third target corner point to the user interface of the first application, and these mapped coordinates will be referred to as the sixth mapping coordinates. Then, the projection processing module can send the fifth and sixth mapping coordinates to the first application. The first application then rotates the first pattern in the second preset coordinate system based on the fifth and sixth mapping coordinates. This is equivalent to rotating the first pattern in the user interface of the first application, so that one edge of the rotated first pattern in the user interface is parallel or perpendicular to the straight line indicated by the sixth mapping coordinate. The specific implementation process of the rotation is not described in this embodiment. After the rotation is completed, the reprojected first pattern and the aforementioned straight line in the first region can satisfy the corresponding alignment conditions.

[0127] For example, the first application rotates the first pattern in a second preset coordinate system according to the fifth and sixth mapping coordinates. This can change the position and / or orientation of the first pattern in the user interface of the first application, and these adjustments can compensate for projection deviations. This causes a corresponding change in the projected position and / or orientation of the first pattern in the first region, so that the adjusted first pattern reprojected into the first region and the aforementioned straight line can satisfy the corresponding alignment conditions.

[0128] The above example, which uses a straight line within the first region as the target object, illustrates some possible examples of adjusting the projection position and / or orientation of the first pattern within the first region. These examples do not constitute a limitation on the embodiments of this application.

[0129] In another possible implementation, the target object can be a temporary reference point within the first region. The projection position of the first pattern can be adjusted so that the projection of the first pattern covers the location of the temporary reference point. The implementation process of the adjustment is illustrated below.

[0130] For example, after the first projection module projects the first pattern, it can capture a first image including the first pattern and the temporary reference point using the camera. For example, after obtaining the first image, the projection processing module can identify the first coordinates of the first target corner point in the first pattern in a first preset coordinate system, and identify the coordinates of the fourth target corner point on the temporary reference point in the first preset coordinate system. For a description of the first target corner point, the first coordinates, and the first preset coordinate system, please refer to the foregoing description; it will not be repeated here. The fourth target corner point may include any one or more points on the temporary reference point; this embodiment does not limit this.

[0131] For example, the projection processing module can determine the desired first coordinates based on the first coordinates and the coordinates of the fourth target corner point. These desired first coordinates can be the coordinates of the first target corner point identified from the captured image in a first preset coordinate system, assuming the projection position of the first pattern covers the location of the temporary reference point. For example, the first target corner point can be moved within the first preset coordinate system so that the fourth target corner point is within the enclosure of the first target corner point. The fourth target corner point being within the enclosure of the first target corner point corresponds to the first pattern projection covering the location of the temporary reference point. Therefore, the moved first target corner point is the aforementioned desired first coordinate. Then, according to the currently used mapping relationship from the first preset coordinate system to the second preset coordinate system, the desired first coordinates are mapped to the second preset coordinate system to obtain the mapped coordinates. These mapped coordinates are the coordinates on the user interface of the first application. The projection processing module can then send these mapped coordinates to the first application, causing the first application to control the first projection module to project the first pattern based on these mapped coordinates. This ensures that the reprojected first pattern covers the location of the temporary reference point.

[0132] For example, in one possible implementation, before the projection processing module determines the desired first coordinates based on the first coordinates and the coordinates of the fourth target corner point, it can first determine whether the projection position of the first pattern covers the position of the temporary reference point. For example, the coordinates of the fourth target corner point can be compared with the aforementioned first coordinates. If the fourth target corner point is not within the enclosure of the first target corner point, it indicates that the projection position of the first pattern does not cover the position of the temporary reference point. Subsequent adjustment operations are then required. It is understood that the description herein is merely an example and does not constitute a limitation on the embodiments of this application.

[0133] The above examples, which use the target object as a temporary reference point in the first region, illustrate some possible examples of adjusting the projection position of the first pattern in the first region, and do not constitute a limitation on the embodiments of this application.

[0134] It is understood that the above-described implementation process of the target controller adjusting the projection position of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region is merely an example and does not constitute a limitation on the embodiments of this application. Exemplarily, some of the above implementation steps can be adaptively adjusted according to different patterns and target objects, and these adjustments are also within the protection scope of this application. The embodiments of this application will not be described in detail.

[0135] S503. After the projection position is adjusted, the target controller controls the first projection module to project the first pattern onto the first area; the first pattern after the projection position is adjusted satisfies the alignment condition with the target object.

[0136] Based on the above description, after the target controller adjusts the projection position and / or pose of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region, the reprojected first pattern and the target object can meet the alignment conditions. For details, please refer to the aforementioned description, which will not be repeated here.

[0137] In one possible implementation, the corner points identified by the target controller based on the first image, such as the first, second, third, or fourth target corner points, need to meet preset requirements. For example, these preset requirements might include: the first, second, third, or fourth target corner point must fall on a surface, such as a ground area or a wall area; it cannot fall partly on the ground area and partly on the wall area. Alternatively, the preset requirements might include that the number of corner points in the identified pattern meets a certain quantity requirement, etc. This application embodiment does not limit the specific content of these preset requirements. Detecting whether the identified corner points meet these preset requirements beforehand can optimize subsequent processing results.

[0138] In one possible implementation, after the projection position is adjusted in S503 above, the target controller controls the first projection module to project the first pattern onto the first area, and then the camera can re-capture a second image. This second image includes the first pattern after the projection position adjustment. Then, based on the second image, the fifth coordinate of the first target corner point in the first pattern after the projection position adjustment is identified in the first preset coordinate system. Then, based on this fifth coordinate and the sixth coordinate of the first target corner point in the adjusted first pattern in the second preset coordinate system, a second mapping relationship is constructed and stored. This second mapping relationship is the coordinate transformation relationship between the adjusted first preset coordinate system and the second preset coordinate system. For a detailed explanation of the process of constructing this second mapping relationship, please refer to the foregoing description of constructing the first mapping relationship based on the desired first coordinate, the desired second coordinate, and the aforementioned third and fourth coordinates. Further details are omitted here.

[0139] For example, if the second image further includes the second pattern, i.e., the target object is the second pattern, the target controller can also identify the seventh coordinate of the second target corner point in the second pattern within the first preset coordinate system based on the second image. Then, the second mapping relationship can be constructed based on the fifth, seventh, and sixth coordinates. For the specific implementation process of constructing this second mapping relationship, please refer to the foregoing description of constructing the first mapping relationship based on the desired first and second coordinates, as well as the third and fourth coordinates. Further details are omitted here.

[0140] For example, after the second mapping relationship is constructed, it can be used for subsequent projection recognition, such as recognizing objects in images captured during motion-sensing games. Alternatively, the first application can use the second mapping relationship to project indicator patterns in the first area to guide the game player to hit or stand in position. This application embodiment does not limit this.

[0141] For example, by reconstructing the mapping relationship between the first preset coordinate system and the second preset coordinate system based on the corner coordinates of the image obtained by reprojection after adjustment, a more accurate mapping can be obtained, thereby improving the accuracy and precision of subsequent recognition.

[0142] In one possible implementation, the target controller can store the coordinates of the corner points identified from the second image. For example, these stored corner point coordinates can be retrieved if the acquisition of these corner point coordinates fails in the next attempt. For instance, consider the coordinates of the first and second target corner points. If, in the next attempt to adjust the projection position of the first pattern in the first region to ensure alignment between the first pattern and the target object, the coordinates of the first and second target corner points identified from the real-time captured image do not meet the preset requirements, then the acquisition of the first and second target corner point coordinates can be considered a failure. In this case, the previously stored coordinates of the first and second target corner points can be retrieved and used. This allows the process of adjusting the projection position of the first pattern in the first region to ensure alignment between the first pattern and the target object can be completed.

[0143] In one possible implementation, for scenarios where only one projection module, such as the first projection module described above, projects the first application, it may not be necessary to execute all the steps in S502 and S503. Instead, after the target controller completes the steps in S501, the camera captures an image including the first pattern. The target controller identifies the coordinates of the first target corner point of the first pattern in the image in a first preset coordinate system. Then, based on the identified coordinates and the coordinates of the first target corner point in the first pattern in the second preset coordinate system, a third mapping relationship is constructed and stored. Similarly, this third mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system. For the specific implementation process of constructing this third mapping relationship, please refer to the foregoing description of constructing the first mapping relationship based on the desired first coordinates, the desired second coordinates, and the third and fourth coordinates. Further details are omitted here.

[0144] For example, once the third mapping relationship is constructed, it can be used for subsequent projection recognition, such as the recognition of objects in images captured during motion-sensing games, etc. This application embodiment does not limit this.

[0145] In one possible implementation, the following describes the implementation process in a game projection scenario, using the first application mentioned above as a game application as an example. For instance, a user launches a game application in a vehicle that projects onto the ground or a wall. This could be a whack-a-mole, skiing, or a motion-sensing game requiring user interaction such as jumping or slapping. The game application then projects the first and second patterns respectively through projection modules on the left and right sides of the vehicle, such as the projection pixel headlights on the left and right sides. The first and second patterns do not meet the corresponding alignment conditions. Then, the projection processing module executes the operation described in S502, where the target object is the second pattern. This obtains the target deviation. The projection processing module then sends the target deviation to the game application. The game application adjusts the first and / or second patterns by translation and / or scaling in the second preset coordinate system based on the target deviation, as detailed in the preceding description. After this adjustment, the game application can control the projection modules on the left and right sides of the vehicle to reproject the adjusted first and second patterns. Then, the camera re-captures a new image. The projection processing module can reconstruct and save the mapping relationship between the first and second coordinate systems based on the new image. It then notifies the game application that the new mapping is complete. The game application can then project the game screen using the projection modules on the left and right sides of the vehicle. Players can move within the projection area, and the projection processing module continuously monitors the player's position. Using the newly established mapping relationship, it calculates the player's coordinates within the game screen and transmits them to the game application. The game application then adjusts accordingly based on the player's position. For example, the implementation details of the steps performed by the projection processing module and the game application in the aforementioned process can be found in the corresponding descriptions above, and will not be repeated here.

[0146] Alternatively, in another possible implementation, after the new mapping is constructed, the game application projects the game screen through projection modules on the left and right sides of the vehicle. Furthermore, the projection processing module continuously monitors the player's position and calculates the player's coordinates within the game screen using the established new mapping relationship, transmitting this information to the game application. The game application can then project any type of pattern, such as a human-shaped frame, onto the player's location. If the game application is replaced with a welcoming application, it can also project welcoming patterns or similar designs onto the player's location. There are no restrictions on the type of pattern.

[0147] The methods of the embodiments of this application have been described above. The apparatus of the embodiments of this application is provided below.

[0148] It should be understood that the division of units in the apparatus provided in this application embodiment is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the apparatus can be implemented by a processor calling software. For example, the apparatus includes a processor connected to a memory, which stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit of the apparatus. The processor is, for example, a general-purpose processor, such as a CPU or MPU, and the memory is either internal or external to the apparatus.

[0149] Alternatively, the units in the device can be implemented as hardware circuits. The functionality of some or all of the units can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functionality of some or all of the above units is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD (Programmable Logic Controller). Taking an FPGA as an example, it can include a large number of logic gates, and the connection relationships between these logic gates are configured through configuration files to achieve the functionality of some or all of the above units.

[0150] In the embodiments of this application, each unit in the device may be one or more processors (or processing circuits) configured to implement the above methods, such as: ECU, CPU, GPU, NPU, TPU, DPU, MPU, digital signal processor (DSP), ASIC, FPGA, or a combination of at least two of these processor forms.

[0151] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-a-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as including a CPU and an FPGA, or including a CPU and an MCU, or including a CPU and a GPU, etc. Several possible devices are listed below.

[0152] Please refer to Figure 14, which is a schematic diagram of a projection processing apparatus provided in an embodiment of this application. Optionally, the projection processing apparatus 140 can be a standalone device, such as a vehicle or a computing device (e.g., an ECU). Alternatively, the projection processing apparatus 140 can also be a component within a standalone device (such as a vehicle or an ECU), such as a chip or an integrated circuit. Alternatively, the projection processing apparatus 140 can be, for example, a target controller in the above-described method embodiments. The projection processing apparatus 140 is used to implement the aforementioned projection processing method, such as the projection processing method and its possible implementations shown in Figure 5.

[0153] For example, the projection processing device 140 includes a control unit 1401 and an adjustment unit 1402.

[0154] The control unit 1401 is used to control the first projection module of the vehicle to project a first pattern onto a first area; the first pattern and the target object in the first area do not meet the alignment conditions.

[0155] Adjustment unit 1402 is used to adjust the projection position of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region;

[0156] The control unit 1401 is also used to control the first projection module to project the first pattern onto the first area after the projection position is adjusted; the first pattern after the projection position is adjusted satisfies the alignment condition with the target object.

[0157] In one possible implementation, the target object is a second pattern projected by the second projection module of the vehicle onto the first region; the alignment conditions include one or more constraints that align the pattern projected by the first projection module with the pattern projected by the second projection module.

[0158] In one possible implementation, the target object is a straight line within a first region, and the alignment conditions include one or more constraints that make the pattern projected by the first projection module parallel or perpendicular to the straight line.

[0159] Alternatively, the target object may be a temporary reference point within a first region, and the alignment conditions may include one or more constraints that cause the pattern projected by the first projection module to cover the location of the temporary reference point.

[0160] In one possible implementation, the first pattern and the second pattern are patterns in the user interface of the vehicle's first application; the adjustment unit 1402 is specifically used for:

[0161] The first target corner point in the first pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, and the second target corner point in the second pattern is identified in the first image, which includes the first and second patterns.

[0162] Based on the first coordinate, the second coordinate, the third coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, and the fourth coordinate of the second target corner point in the second preset coordinate system, adjust the projection position of the first pattern in the first area.

[0163] In one possible implementation, the adjustment unit 1402 is specifically used for:

[0164] A first mapping relationship is constructed based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate; the first mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system;

[0165] The target deviation is determined based on the first coordinate, the second coordinate, and the first mapping relationship; the target deviation includes the first deviation and / or the second deviation, the first deviation indicating the deviation of the center point of the first pattern and the center point of the second pattern in the vertical axis direction in the second preset coordinate system; the second deviation indicating the deviation between the width of the circumscribed rectangle of the first pattern and the width of the circumscribed rectangle of the second pattern in the second preset coordinate system.

[0166] Adjust the projection position of the first pattern in the first region according to the target deviation.

[0167] In one possible implementation, the adjustment unit 1402 is specifically used for:

[0168] The first pattern and / or the second pattern are translated along the vertical axis of the second preset coordinate system according to the first deviation; and / or the width of the circumscribed rectangle of the first pattern and / or the second pattern is scaled along the horizontal axis of the second preset coordinate system according to the second deviation.

[0169] In one possible implementation, the first pattern is a pattern in the user interface of the vehicle's first application; the control unit 1401 is further configured to, after adjusting the projection position and controlling the first projection module to project the first pattern onto the first area, also include:

[0170] The fifth coordinate of the first target corner point in the first pattern after the projection position adjustment is identified based on the second image; the second image is the first pattern after the projection position adjustment captured by the vehicle's camera.

[0171] Based on the fifth coordinate and the sixth coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, a second mapping relationship is constructed; the second mapping relationship is the coordinate transformation relationship between the first preset coordinate system and the second preset coordinate system.

[0172] In one possible implementation, the device further includes a storage unit for: storing a fifth coordinate; the stored fifth coordinate is to be retrieved and used in the event that the coordinates of the first target corner point fail to be acquired in the next attempt.

[0173] Please refer to Figure 15, which is a schematic diagram of another projection processing device provided in an embodiment of this application. The projection processing device 150 shown in Figure 15 can be a standalone device, such as a vehicle or a computing device (e.g., an ECU). Alternatively, the projection processing device 150 can also be a component within a standalone device (such as a vehicle or an ECU), such as a chip or integrated circuit. Alternatively, the projection processing device 140 can be, for example, a target controller as described in the above method embodiments. The projection processing device 150 is used to implement the aforementioned projection processing method, such as the projection processing method shown in Figure 5 and its possible implementations.

[0174] The projection processing device 150 may include at least one processor 1501 and a memory 1503. Optionally, it may also include a communication interface 1502. Further optionally, it may also include a connection line 1504, wherein the processor 1501, the communication interface 1502 and / or the memory 1503 are connected via the connection line 1504, and / or communicate with each other via the connection line 1504 to transmit control signals and / or data signals.

[0175] Processor 1501 is a module that performs arithmetic and / or logical operations, and may specifically include one or more of the following modules: CPU, application processor (AP), MCU, ECU, GPU, MPU, ASIC, image signal processor (ISP), DSP, FPGA, complex programmable logic device (CPLD), or coprocessor, etc.

[0176] The communication interface 1502 can be used to provide information input or output to at least one processor, or to receive and / or transmit signals to externally transmitted signals. For example, the communication interface 1502 may include interface circuitry. For instance, the communication interface 1502 may include a wired link interface such as an Ethernet cable, or a wireless link interface (Wi-Fi, Bluetooth, general wireless transmission, vehicular short-range communication technology, and other short-range wireless communication technologies, etc.). Optionally, the communication interface 1502 may also include a radio frequency transmitter, an antenna, etc. If the communication interface 1502 includes an antenna, the number of antennas can be one or more.

[0177] As one possible design, if the projection processing device 150 is a standalone device, the communication interface 1502 may include a receiver and a transmitter. The receiver and transmitter may be the same component or different components. When the receiver and transmitter are the same component, this component may be referred to as a transceiver.

[0178] As another possible design, if the projection processing device 150 is a chip or circuit, the communication interface 1502 may include an input interface and an output interface, which may be the same interface or different interfaces.

[0179] Alternatively, the functions of the communication interface 1502 can be implemented by a transceiver circuit or a dedicated transceiver chip.

[0180] The memory 1503 provides storage space, in which data such as the operating system and computer programs can be stored. The memory 1503 can be one or a combination of several of the following: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0181] The functions and operations of each module or unit in the projection processing device 150 listed above are merely illustrative examples.

[0182] Each functional unit in the projection processing apparatus 150 can be used to implement the aforementioned projection processing method, such as the projection processing method and its possible implementation shown in FIG5.

[0183] Optionally, the processor 1501 may be a processor specifically designed to perform the aforementioned methods (for ease of distinction, referred to as a dedicated processor), or a processor that performs the aforementioned methods by calling a computer program (for ease of distinction, referred to as a dedicated processor). Optionally, at least one processor may include both dedicated processors and general-purpose processors.

[0184] Optionally, if the projection processing apparatus 150 includes at least one memory 1503, and the processor 1501 implements the aforementioned projection processing method by calling a computer program, the computer program can be stored in the memory 1503.

[0185] This application also provides a chip including logic circuitry and a communication interface. The communication interface is used to receive and / or send information, or to input and / or output information. The logic circuitry is used to process the information. This chip is used to implement the aforementioned projection processing method, such as the projection processing method shown in FIG5 and its possible implementations.

[0186] This application provides a vehicle that includes the aforementioned target controller, or the projection processing device shown in FIG14 or FIG15, or the projection processing system shown in FIG1.

[0187] This application also provides a computer-readable storage medium storing instructions that, when executed on at least one processor (or projection processing device), implement the aforementioned projection processing method, such as the projection processing method and its possible implementations shown in Figure 5 and other embodiments.

[0188] This application also provides a computer program product, which includes computer instructions for implementing the aforementioned projection processing method, such as the projection processing method and its possible implementations shown in FIG5.

[0189] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0190] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0191] It should also be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A projection processing method, characterized in that, The method includes: The first projection module of the vehicle projects a first pattern onto a first area; the first pattern and the target object in the first area do not meet the alignment condition. Adjust the projection position of the first pattern in the first region according to the relative position of the first pattern and the target object in the first region; After the projection position is adjusted, the first projection module is controlled to project the first pattern onto the first area; the first pattern after the projection position is adjusted satisfies the alignment condition with the target object.

2. The method according to claim 1, characterized in that, The target object is the second pattern projected by the second projection module of the vehicle onto the first area; The alignment conditions include one or more constraints that align the pattern projected by the first projection module with the pattern projected by the second projection module.

3. The method according to claim 1, characterized in that, The target object is a straight line within the first region, and the alignment condition includes one or more constraints that make the pattern projected by the first projection module parallel or perpendicular to the straight line. Alternatively, the target object may be a temporary reference point within the first region, and the alignment condition may include one or more constraints that cause the pattern projected by the first projection module to cover the position of the temporary reference point.

4. The method according to claim 2, characterized in that, The first pattern and the second pattern are patterns in the user interface of the first application of the vehicle; The step of adjusting the projection position of the first pattern in the first region according to the relative position between the first pattern in the first region and the target object includes: The first target corner point in the first pattern is identified in the first preset coordinate system based on the first image, and the second target corner point in the second pattern is identified in the first preset coordinate system based on the first image; the first image is an image including the first pattern and the second pattern captured by the vehicle's camera, the first coordinate is used to indicate the position of the first pattern in the first area, and the second coordinate is used to indicate the position of the second pattern in the first area; Based on the first coordinates, the second coordinates, the third coordinates of the first target corner point in the second preset coordinate system corresponding to the user interface, and the fourth coordinates of the second target corner point in the second preset coordinate system, adjust the projection position of the first pattern in the first area.

5. The method according to claim 4, characterized in that, Adjusting the projection position of the first pattern in the first region according to the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate includes: A first mapping relationship is constructed based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate; the first mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system. The target deviation is determined based on the first coordinate, the second coordinate, and the first mapping relationship; the target deviation includes a first deviation and / or a second deviation, wherein the first deviation indicates the deviation between the center point of the first pattern and the center point of the second pattern in the second preset coordinate system in the vertical direction; and the second deviation indicates the deviation between the width of the circumscribed rectangle of the first pattern and the width of the circumscribed rectangle of the second pattern in the second preset coordinate system. Adjust the projection position of the first pattern in the first region according to the target deviation.

6. The method according to claim 5, characterized in that, The step of adjusting the projection position of the first pattern in the first region according to the target deviation includes: The first pattern and / or the second pattern are translated along the vertical axis of the second preset coordinate system according to the first deviation; and / or the width of the circumscribed rectangle of the first pattern and / or the second pattern is scaled along the horizontal axis of the second preset coordinate system according to the second deviation.

7. The method according to any one of claims 1-6, characterized in that, The first pattern is a pattern in the user interface of the first application of the vehicle; After the projection position is adjusted, and the first projection module is controlled to project the first pattern onto the first area, the method further includes: The fifth coordinate of the first target corner point in the first pattern after the projection position adjustment is identified in the first preset coordinate system based on the second image; the second image is the first pattern after the projection position adjustment captured by the vehicle's camera. Based on the fifth coordinate and the sixth coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, a second mapping relationship is constructed; the second mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system.

8. The method according to claim 7, characterized in that, The method further includes: storing the fifth coordinate; the stored fifth coordinate is used to retrieve and use the coordinates of the first target corner point in the next failure to obtain the coordinates.

9. A projection processing device, characterized in that, The device includes: A control unit is used to control the first projection module of the vehicle to project a first pattern onto a first area; the first pattern and the target object in the first area do not meet the alignment condition. The adjustment unit is used to adjust the projection position of the first pattern in the first region according to the relative position between the first pattern and the target object in the first region; The control unit is further configured to, after the projection position is adjusted, control the first projection module to project the first pattern onto the first area; the first pattern after the projection position is adjusted satisfies the alignment condition with the target object.

10. The apparatus according to claim 9, characterized in that, The target object is the second pattern projected by the second projection module of the vehicle onto the first area; The alignment conditions include one or more constraints that align the pattern projected by the first projection module with the pattern projected by the second projection module.

11. The apparatus according to claim 9, characterized in that, The target object is a straight line within the first region, and the alignment condition includes one or more constraints that make the pattern projected by the first projection module parallel or perpendicular to the straight line. Alternatively, the target object may be a temporary reference point within the first region, and the alignment condition may include one or more constraints that cause the pattern projected by the first projection module to cover the position of the temporary reference point.

12. The apparatus according to claim 10, characterized in that, The first pattern and the second pattern are patterns in the user interface of the first application of the vehicle; the adjustment unit is specifically used for: The first target corner point in the first pattern is identified in the first preset coordinate system based on the first image, and the second target corner point in the second pattern is identified in the first preset coordinate system based on the first image; the first image is an image including the first pattern and the second pattern captured by the vehicle's camera, the first coordinate is used to indicate the position of the first pattern in the first area, and the second coordinate is used to indicate the position of the second pattern in the first area; Based on the first coordinates, the second coordinates, the third coordinates of the first target corner point in the second preset coordinate system corresponding to the user interface, and the fourth coordinates of the second target corner point in the second preset coordinate system, adjust the projection position of the first pattern in the first area.

13. The apparatus according to claim 12, characterized in that, The adjustment unit is specifically used for: A first mapping relationship is constructed based on the first coordinate, the second coordinate, the third coordinate, and the fourth coordinate; the first mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system. The target deviation is determined based on the first coordinate, the second coordinate, and the first mapping relationship; the target deviation includes a first deviation and / or a second deviation, wherein the first deviation indicates the deviation between the center point of the first pattern and the center point of the second pattern in the second preset coordinate system in the vertical direction; and the second deviation indicates the deviation between the width of the circumscribed rectangle of the first pattern and the width of the circumscribed rectangle of the second pattern in the second preset coordinate system. Adjust the projection position of the first pattern in the first region according to the target deviation.

14. The apparatus according to claim 13, characterized in that, The adjustment unit is specifically used for: The first pattern and / or the second pattern are translated along the vertical axis of the second preset coordinate system according to the first deviation; and / or the width of the circumscribed rectangle of the first pattern and / or the second pattern is scaled along the horizontal axis of the second preset coordinate system according to the second deviation.

15. The apparatus according to any one of claims 9-14, characterized in that, The first pattern is a pattern in the user interface of the first application of the vehicle; The control unit is further configured to, after adjusting the projection position and controlling the first projection module to project the first pattern onto the first area, also include: The fifth coordinate of the first target corner point in the first pattern after the projection position adjustment is identified in the first preset coordinate system based on the second image; the second image is the first pattern after the projection position adjustment captured by the vehicle's camera. Based on the fifth coordinate and the sixth coordinate of the first target corner point in the second preset coordinate system corresponding to the user interface, a second mapping relationship is constructed; the second mapping relationship is a coordinate transformation relationship from the first preset coordinate system to the second preset coordinate system.

16. The apparatus according to claim 15, characterized in that, The device further includes a storage unit for: storing the fifth coordinate; the stored fifth coordinate is used to retrieve and use in the event that the coordinates of the first target corner point fail to be obtained in the next attempt.

17. A projection processing apparatus, characterized in that, The projection processing device includes a processor and a memory. The memory provides storage space for storing computer instructions. The processor is used to invoke computer instructions stored in the memory to execute the method as described in any one of claims 1-8.

18. A projection processing system, characterized in that, The projection system includes a projection module and a projection processing device, wherein the projection processing device is the projection processing device according to any one of claims 9-16, or the projection processing device according to claim 17.

19. A vehicle, characterized in that, The vehicle includes the projection processing device according to any one of claims 9-16, or the projection processing device according to claim 17, or the projection processing system according to claim 18.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer instructions; When the instruction is executed by the processor, the method described in any one of claims 1-8 is performed.

21. A computer program product, characterized in that, The computer program product includes computer language code or computer instructions; When the computer program product is executed by a processor, the method described in any one of claims 1-8 is performed.

Images