Projection method and device, vehicle and AR-HUD

Abstract

This application relates to the field of intelligent vehicles, specifically providing a projection method and device, a vehicle, and an AR-HUD. The projection method includes: acquiring image information and position information of a calibration object; projecting the calibration object based on the image information, position information, and an imaging model; and adjusting the parameters of the imaging model when the overlap between the calibration object and its projection surface is less than a first threshold. This application enables the projected image to align with the real world, improving the projection display effect.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicles, and in particular to a projection method and device, a vehicle, and an AR-HUD. Background Technology

[0002] A Head-Up Display (HUD) is a display device that projects images onto the driver's field of vision. It primarily utilizes the principle of optical reflection to project important, relevant information as a two-dimensional image onto the windshield of a car, at approximately eye level. When the driver looks forward through the windshield, they see the HUD projected image onto a virtual image surface in front of the windshield. Compared to traditional instrument panels and center console screens, drivers do not need to look down when viewing the HUD image, avoiding the need to switch between the image and the road surface, reducing reaction time in emergencies, and improving driving safety. Augmented Reality (AR) HUDs, which have been proposed in recent years, can integrate the AR effects projected by the HUD with real road information, enhancing the driver's understanding of road conditions and enabling functions such as AR navigation and AR warnings.

[0003] To achieve the road navigation and warning functions of AR-HUD, the 3D perception data obtained by sensors needs to be sent into a virtual 3D space for augmented reality rendering. After rendering, it is mapped onto the 2D virtual image surface displayed on the HUD, and finally mapped back into 3D space by the human eye. During this process, it is crucial to ensure that the "human eye - HUD display - real object" remain in a straight line, guaranteeing that the size and position of the HUD display observed by the human eye are consistent with the real object. Figure 1 As shown, the virtual image displayed on the HUD, as perceived by the human eye, can be perfectly integrated with the corresponding real object, achieving a match between the AR effect and the display scene. Furthermore, for the same driver in different driving positions, or for different drivers, changes in the position of the driver's eyes require corresponding adjustments to the HUD display to ensure that the HUD display observed by the human eye is always integrated with the real road information.

[0004] Therefore, ensuring that the HUD display remains integrated with the real world under different driver postures or different drivers has become a key research direction for improving the display effect of AR-HUD. Summary of the Invention

[0005] In view of this, this application provides a projection method and apparatus, a vehicle and an AR-HUD, which can ensure that the projected image is always aligned with the real world, thereby improving the projection display effect.

[0006] It should be understood that in the solution provided in this application, the projection method can be performed by a projection device or some components of the projection device, wherein the projection device has projection functionality, such as an AR-HUD, HUD, or other device with projection functionality. Some components of the projection device can be processing chips, processing circuits, processors, etc.

[0007] A first aspect of this application provides a projection method, comprising: acquiring image information and position information of a calibration object; projecting the calibration object based on the image information, position information, and an imaging model; and adjusting the parameters of the imaging model when the overlap between the calibration object and its projection surface is less than a first threshold.

[0008] As described above, this method acquires the image and position information of a real-world calibration object, and projects the object onto the screen based on this information and the imaging model. Furthermore, it adjusts the parameters of the imaging model according to the overlap between the calibration object and its projection surface, ensuring maximum overlap and achieving alignment to enhance the user's immersive experience. This method can be applied to AR-HUDs, HUDs, or other devices with projection capabilities to calibrate and standardize these devices, thereby improving the projection display effect.

[0009] In one possible implementation of the first aspect, adjusting the parameters of the imaging model includes adjusting one or more parameters of the field of view and the position of the imaging plane.

[0010] Based on the obtained image and position information of the calibration object, a two-dimensional image corresponding to the calibration object can be generated on the imaging surface of the imaging model. During projection, the imaging surface of the imaging model is projected as a complete image. For example, the imaging model can be in the form of an imaging cone, an imaging cylinder, or an imaging cube. The field-of-view parameter of the imaging model determines the area of ​​the imaging surface and the scale of the two-dimensional image of the calibration object relative to the imaging surface. The position parameter of the imaging surface determines the position of the two-dimensional image of the calibration object relative to the imaging surface. Therefore, when the overlap between the calibration object and its projection surface is lower than a preset first threshold, the field-of-view angle or imaging surface position of the imaging model can be adjusted accordingly based on area offset, position offset, or size offset.

[0011] In one possible implementation of the first aspect, adjusting the parameters of the imaging model when the overlap between the calibration object and the projection surface of the calibration object is less than a first threshold specifically includes adjusting the field of view of the imaging model when the area difference between the calibration object and the projection surface of the calibration object is greater than a second threshold.

[0012] Therefore, when the area difference between the calibration object and its projection surface is greater than the preset second threshold, the area of ​​the imaging surface can be adjusted by adjusting the field of view of the imaging model. When the area of ​​the calibration object's projection surface is greater than the area of ​​the calibration object, the field of view of the imaging model can be enlarged, and the imaging surface will be enlarged proportionally. The proportion of the generated two-dimensional image of the calibration object in the imaging surface will be reduced proportionally. At this time, the area of ​​the projection surface of the calibration object displayed relative to the area of ​​the calibration object will also be reduced proportionally, so that the area difference between the projection surface of the calibration object and the area of ​​the calibration object is less than the preset second threshold. Similarly, when the area of ​​the calibration object's projection surface is less than the area of ​​the calibration object, the field of view of the imaging model can be reduced, and the imaging surface will be reduced proportionally. The proportion of the generated two-dimensional image of the calibration object in the imaging surface will be enlarged proportionally. At this time, the area of ​​the projection surface of the calibration object displayed relative to the area of ​​the calibration object will also be enlarged proportionally, so that the area difference between the projection surface of the calibration object and the area of ​​the calibration object is less than the preset second threshold.

[0013] In one possible implementation of the first aspect, adjusting the parameters of the imaging model when the overlap between the calibration object and the projection surface of the calibration object is less than a first threshold specifically includes: adjusting the two-dimensional position of the imaging surface of the imaging model when the offset between the calibration object and the projection surface of the calibration object is greater than a third threshold.

[0014] Therefore, when the offset between the calibration object and the projection surface of the calibration object is greater than the preset third threshold, since the position of the calibration object is fixed, the two-dimensional position of the imaging surface of the imaging model can be adjusted. Specifically, the two-dimensional position refers to the vertical position and the horizontal position of the imaging surface. This can be used to adjust the relative position of the generated two-dimensional image of the calibration object in the imaging surface, thereby making the offset between the calibration object and the projection surface of the calibration object less than the preset third threshold.

[0015] In one possible implementation of the first aspect, the degree of overlap between the calibrator and the projection surface of the calibrator is determined by the pixel offset between the calibrator and the projection surface of the calibrator; the pixel offset is determined by an image captured by a camera that includes the calibrator and the projection surface of the calibrator.

[0016] Therefore, when calibrating or standardizing the calibration object and its projection surface, this method can place a camera at the user's eye level to simulate the effect of human eye observation. The camera captures images of the calibration object and its projection surface, generating one or more images. Based on the generated images, the pixel offset between the calibration object and its projection surface is determined, thereby determining the degree of overlap between the two projection surfaces. Using a camera to capture images can improve the accuracy of detecting the degree of overlap between the calibration objects and their projection surfaces, and the results are presented intuitively in the form of data, avoiding errors caused by human eye observation.

[0017] In one possible implementation of the first aspect, the imaging model is trained on a training set comprising multiple training samples, wherein the training samples include human eye position information parameters, image information and position information parameters of the calibration object, and the overlap parameter between the calibration object and the projection surface of the calibration object.

[0018] Therefore, to improve the accuracy of the imaging model, neural networks or deep learning can be used. The imaging model can be trained using a training set consisting of multiple training samples. The input can be the human eye position information parameters, the image information and position information parameters of the calibration object, and the output can be the overlap parameter between the calibration object and its projection surface. This forms a training sample. Through multiple training sessions, the overlap between the calibration object and its projection surface can be improved, making the imaging model more applicable and giving it the characteristics of deep learning and optimization, thus meeting the user experience needs of different users.

[0019] One possible implementation of the first aspect also includes:

[0020] The system obtains the user's calibration requirements and sends a calibration start notification message. It also obtains the user's eye position and calibrates the imaging model's parameters based on that position. Upon completion of calibration, it sends a calibration completion notification message to the user.

[0021] As described above, this method can automatically calibrate the parameters of the imaging model based on the user's eye position without the user's awareness. It can also guide the user to make calibration requests through human-computer interaction and achieve the calibration of the imaging model parameters through voice prompts, display prompts, etc. After calibration is completed, a calibration completion prompt message is sent to the user to improve the user experience.

[0022] One possible implementation of the first aspect also includes:

[0023] Determine by eye whether the calibration object and its projection surface coincide;

[0024] When the calibration object does not coincide with the projection plane of the calibration object, the parameters of the calibrated imaging model are adjusted according to the user's adjustment instructions.

[0025] Therefore, this method can calibrate the parameters of the imaging model according to the user's eye position so that the overlap between the calibration object and the projection surface of the calibration object reaches a preset threshold. At the same time, when the user is not satisfied with the overlap between the current calibration object and the projection surface of the calibration object, the parameters of the imaging model can be adjusted according to subjective experience to achieve customization of the projection display and meet the user's target needs.

[0026] A second aspect of this application provides a projection device, comprising:

[0027] The acquisition module is used to acquire image and location information of the calibration object;

[0028] The projection module is used to project the calibration object based on its image information, position information, and imaging model.

[0029] The adjustment module is used to adjust the parameters of the imaging model when the overlap between the calibration object and the projection surface of the calibration object is less than a first threshold.

[0030] In one possible implementation of the second aspect, when the adjustment module is used to adjust the parameters of the imaging model, it is specifically used for:

[0031] Adjust one or more parameters of the imaging model, including the field of view and the position of the imaging plane.

[0032] In one possible implementation of the second aspect, the adjustment module is specifically used for:

[0033] When the area difference between the calibration object and the projection surface of the calibration object is greater than the second threshold, the field of view of the imaging model is adjusted.

[0034] In one possible implementation of the second aspect, the adjustment module is specifically used for:

[0035] When the offset between the calibration object and the projection surface of the calibration object is greater than the third threshold, the two-dimensional position of the imaging surface of the imaging model is adjusted.

[0036] In one possible implementation of the second aspect, the degree of overlap between the calibrator and the projection surface of the calibrator is determined by the pixel offset between the calibrator and the projection surface of the calibrator; the pixel offset is determined by an image captured by a camera that includes the calibrator and the projection surface of the calibrator.

[0037] In one possible implementation of the second aspect, the imaging model is trained on a training set comprising multiple training samples, wherein the training samples include human eye position information parameters, image information and position information parameters of the calibration object, and the overlap parameter between the calibration object and the projection surface of the calibration object.

[0038] One possible implementation of the second aspect also includes:

[0039] The prompt module is used to send a prompt message to the user indicating that calibration has started when the user's calibration request is received;

[0040] The adjustment module is also used to calibrate the parameters of the imaging model based on the acquired position of the user's eye.

[0041] This notification module is also used to send a notification message to the user indicating that calibration is complete after calibration.

[0042] In one possible implementation of the second aspect,

[0043] The prompt module is also used to prompt the user to determine by eye whether the calibration object and the projection surface of the calibration object coincide;

[0044] The adjustment module is also used to adjust the parameters of the calibrated imaging model according to the user's adjustment instructions when the calibration object does not coincide with the projection surface of the calibration object.

[0045] To achieve the above objectives, a third aspect of this application provides a system comprising:

[0046] Projection devices and vehicle-mounted systems are provided in the second aspect and the various alternative implementations described above.

[0047] In one possible implementation, the system further includes: a storage device for storing the imaging model and a training set of the imaging model; and a communication device for enabling communication and interaction between the storage device and the cloud.

[0048] In one possible implementation, the system is a vehicle.

[0049] A fourth aspect of this application provides a computing device, including: a processor and a memory storing program instructions thereon, which, when executed by the processor, cause the processor to perform a projection method as provided in the first aspect and various alternative implementations described above.

[0050] In one possible implementation, the computing device is either an AR-HUD or a HUD.

[0051] In one possible implementation, the computing device is a vehicle.

[0052] In one possible implementation, the computing device is one of the vehicle's infotainment system or onboard computer.

[0053] The fifth aspect of this application provides a computer-readable storage medium storing program code that, when executed by a computer or processor, causes the computer or processor to perform the projection method provided in the first aspect and the various alternative implementations described above.

[0054] The sixth aspect of this application provides a computer program product containing program code that, when executed by a computer or processor, causes the computer or processor to perform the projection method provided in the first aspect and the various alternative implementations described above.

[0055] It should be understood that the above-mentioned technical solutions also provide multiple thresholds related to projection adjustment, including: a first threshold, a second threshold, and a third threshold. It should be understood that these thresholds are not mutually exclusive and can be used in combination. They can be decimals or relative proportions, such as percentages. For any of these thresholds, when the projected area, overlap, area difference, or offset equals one of the above-mentioned preset thresholds, it can be considered a critical state. For a critical state, it can be considered that the threshold judgment condition is met, and corresponding subsequent operations are performed; or it can be considered that the threshold judgment condition is not met, and corresponding subsequent operations are not performed.

[0056] In summary, the projection method, apparatus, vehicle, and AR-HUD provided in this application acquire image and position information of a calibration object, project the calibration object according to an imaging model, and improve the projection display effect by adjusting the parameters of the imaging model to increase the overlap between the calibration object and its projection surface. In this application, the imaging model can generate a two-dimensional image of the calibration object on its imaging surface based on the acquired user's eye position information, the image and position information of the calibration object, and project it through a projection device. The overlap between the calibration object and its projection surface can be used to evaluate the accuracy and stability of the imaging model. In some embodiments of this application, the imaging model can also be trained using neural networks or deep learning to continuously optimize its accuracy and stability, making it adaptable to changes in the user's eye position. Furthermore, with the rapid development of 5G technology and smart cars, the imaging model can also be optimized and trained through cloud interaction to be suitable for different in-vehicle projection devices, and automatically adjust one or more parameters of the imaging model according to the hardware parameters of different in-vehicle projection devices to meet the customized needs of different users. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating the imaging capabilities of existing AR-HUDs in various usage scenarios.

[0058] Figure 2 A schematic diagram illustrating an application scenario of the projection method provided in this application embodiment;

[0059] Figure 3 A schematic diagram illustrating another application scenario of the projection method provided in the embodiments of this application;

[0060] Figure 4 A flowchart illustrating a projection method provided in an embodiment of this application;

[0061] Figure 5 A flowchart illustrating a calibration method provided in an embodiment of this application;

[0062] Figure 6 This is a schematic diagram of the system architecture of the AR-HUD provided in the embodiments of this application;

[0063] Figure 7 A flowchart illustrating a projection method for an AR-HUD provided in this application embodiment;

[0064] Figure 8A A schematic diagram of the imaging cone provided in the embodiments of this application;

[0065] Figure 8B This is a schematic diagram illustrating the spatial transformation from the imaging cone to the AR-HUD provided in an embodiment of this application.

[0066] Figure 9A This is a head-up view of the virtual human eye and the imaging cone in a virtual coordinate system provided in an embodiment of this application;

[0067] Figure 9B A top-view schematic diagram of the human eye and the virtual image plane of the AR-HUD in the real coordinate system provided for the embodiments of this application;

[0068] Figure 10A A schematic diagram showing the vertical offset between the target frame of the AR-HUD virtual image surface display and the calibration plate provided in the embodiments of this application;

[0069] Figure 10B A schematic diagram showing the horizontal offset between the target frame and the calibration plate in the virtual image display of the AR-HUD provided in this embodiment of the application;

[0070] Figure 11 An architectural diagram of a projection device provided in an embodiment of this application;

[0071] Figure 12A This is a schematic diagram of a human-computer interaction interface according to an embodiment of this application;

[0072] Figure 12B This is a schematic diagram of another human-computer interaction interface according to an embodiment of this application;

[0073] Figure 13 This is an architectural diagram of a computing device according to an embodiment of this application.

[0074] It should be understood that the dimensions and shapes of the blocks in the above structural diagrams are for reference only and should not constitute an exclusive interpretation of the embodiments of the present invention. The relative positions and inclusion relationships between the blocks presented in the structural diagrams are only schematic representations of the structural relationships between the blocks, and are not intended to limit the physical connection methods of the embodiments of the present invention. Detailed Implementation

[0075] The technical solutions provided in this application will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the system architecture and business scenarios provided in the embodiments of this application are mainly for illustrating possible implementations of the technical solutions of this application and should not be construed as the sole limitation on the technical solutions of this application. Those skilled in the art will recognize that the technical solutions provided in this application are equally applicable to similar technical problems as system architectures evolve and new business scenarios emerge.

[0076] It should be understood that the memory management solutions provided in this application include projection methods, devices, vehicles, and AR-HUDs. Since these technical solutions solve problems based on the same or similar principles, some repetitive details may not be repeated in the following descriptions of specific embodiments. However, it should be considered that these specific embodiments have mutual references and can be combined with each other.

[0077] Head-up displays (HUDs) are typically installed inside a car cabin. They project information onto the windshield, where it is reflected and then enters the user's eyes, appearing in front of the vehicle. This blends the displayed information with the real-world environment, creating an augmented reality effect. For example, by establishing a camera coordinate system and a human eye coordinate system, and determining the correspondence between them, an augmented reality image is determined based on image information captured by the vehicle's cameras and this correspondence. The image is then projected based on the mapping between the augmented reality image and the HUD image. However, this implementation requires real-time calibration of the transformation relationship between the human eye coordinate system and the camera coordinate system during driving, resulting in a large computational load and high task complexity.

[0078] To achieve better projection display effects, embodiments of this application provide a projection method and device, a vehicle, and an AR-HUD. This allows for real-time adjustment of the projection display effect based on changes in the user's eye position, ensuring the projected AR image is always aligned with the real world, thus improving the projection display effect. The user is typically the driver. However, the user could also be a front passenger or rear passenger, for example, multiple HUD devices can be installed in the vehicle cabin, each targeting a different user. During adjustment, the HUD for the driver in the driver's seat can be adjusted according to the driver's eye position, ensuring the AR image seen by the driver is aligned with the real world ahead. This AR image can display navigation information, vehicle speed information, or other roadside information. Similarly, the HUD for the front passenger in the front passenger seat can be adjusted according to the passenger's eye position, ensuring the AR image seen by the passenger is also aligned with the real world ahead.

[0079] Figures 2-3The diagram illustrates an application scenario of the projection method provided in this application embodiment, with reference to... Figures 2-3 The application scenario of this embodiment specifically involves a vehicle, which has a data acquisition device 10, a projection device 20, and a display device 30.

[0080] The data acquisition device 10 may include an external data acquisition device and an internal data acquisition device. The external data acquisition device may specifically employ a lidar, an in-vehicle camera, or other devices with image acquisition or optical scanning capabilities, or a combination of multiple such devices. It can be installed on the top, head, or side of the rearview mirror facing outwards in the vehicle 1, either inside or outside the vehicle. Its main function is to detect and acquire image and positional information of the environment in front of the vehicle, including information related to vehicles ahead, obstacles, and road signs. The internal data acquisition device may specifically employ an in-vehicle camera, an eye detector, or other similar devices. In its implementation, the internal data acquisition device can be positioned according to requirements, for example, on the A-pillar, B-pillar, or side of the rearview mirror facing the user in the vehicle cabin. It can also be positioned near the steering wheel, center console, or above the display screen behind the seat. Its main function is to detect and acquire the eye position information of the driver or passengers in the vehicle cabin. There may be one or multiple internal data acquisition devices; this application does not limit their location or number.

[0081] The projection device 20 can be a HUD, AR-HUD, or other device with projection capabilities. It can be installed above or inside the center console of the vehicle cabin. It typically includes a projector, a reflector, a projection mirror, an adjustment motor, and a control unit. The control unit is an electronic device, specifically a conventional chip processor such as a central processing unit (CPU) or a microprocessor (MCU), or terminal hardware such as a mobile phone or tablet. This control unit is communicatively connected to both the acquisition device 10 and the display device 30. The control unit can have a preset imaging model or acquire a preset imaging model from other vehicle components. The parameters of this imaging model are correlated with the eye position information collected by the in-vehicle acquisition device. It can perform parameter calibration based on the eye position information, and then generate a projected image based on the environmental information collected by the external acquisition device, which is then output to the projector. Figure 3 As shown, the projected image may include augmented reality display images generated based on environmental information, as well as images such as vehicle speed and navigation.

[0082] The display device 30 can be the windshield of a vehicle or a transparent screen with independent display, used to reflect the image light emitted by the projection device and enter the user's eyes, so that when the driver looks out of the vehicle through the display device 30, he can see a virtual image with a depth effect, which overlaps with the real world environment, presenting an augmented reality display effect to the user.

[0083] The acquisition device 10, projection device 20, and other devices can communicate data via wired or wireless communication (such as Bluetooth or Wi-Fi). For example, after acquiring image information, the acquisition device 10 can transmit the image information to the projection device 20 via Bluetooth. Similarly, the projection device 20 can send control signals to the acquisition device 10 via Bluetooth and adjust the acquisition parameters of the acquisition device 10, such as the shooting angle. It should be understood that data processing can be completed in the projection device 20, the acquisition device 10, or other processing devices, such as vehicle infotainment systems or in-vehicle computers.

[0084] Through the above structure, the vehicle can achieve augmented reality display effects based on real-world environmental information, and can adjust the generated projected image according to the user's eye position information so that the projected augmented reality display image always overlaps with the real-world environmental information, thereby improving the user's immersive viewing experience.

[0085] Figure 4 A flowchart of a projection method provided in an embodiment of this application is shown. This projection method can be executed by a projection device or some components within the projection device, such as an AR-HUD, HUD, vehicle, or processor. Specifically, it can realize functions such as calibration, standardization, and projection display of the aforementioned projection device or some components within the projection device. Its application can occur when the vehicle is stationary and starting, or while the vehicle is in motion. Figure 4 As shown, the projection method includes:

[0086] S401: Acquire image and location information of the calibration object;

[0087] The calibration object can be a static object located outside the vehicle, such as a stationary vehicle, tree, traffic sign, or a calibration board with a geometric shape, or a dynamic object located outside the vehicle, such as a moving vehicle or a pedestrian. The processor can obtain image and position information of the calibration object collected by the acquisition device through an interface circuit. The image information can be an image captured by a camera, point cloud data collected by a LiDAR, or other forms of information, including resolution, size, dimensions, and color. The position information can be coordinate data, direction information, or other forms of information. The processor can be a processor for a projection device, or a processor for an in-vehicle processing device such as a vehicle infotainment system or vehicle computer.

[0088] S402: Project the calibration object based on its image information, position information, and imaging model;

[0089] Based on the image and position information of the calibration object obtained in step S401, the processor can generate a calibration image corresponding to the calibration object in the imaging model and project it out through the interface circuit. This imaging model can be constructed based on parameters such as the human eye position, the HUD position, the HUD's field of view (FOV), the HUD's projection surface (virtual image surface), the HUD's display resolution, and the downward viewing angle from the human eye to the HUD. The constructed imaging model includes parameters such as the origin, field of view, near plane (imaging surface), and far plane. For example, the imaging model can be in the form of an imaging cone, an imaging cylinder, or an imaging cube. For instance, when the imaging model is an imaging cone, the origin can be determined based on the human eye position, the field of view can be determined based on the HUD's field of view to determine the field of view range of the imaging cone, the near plane serves as the imaging surface during imaging, and the far plane can be determined based on the farthest viewing distance of the human eye. Based on the acquired image and position information of the calibration object, the processor can generate a two-dimensional image corresponding to the calibration object on the imaging surface of the imaging model, and during projection, the imaging surface of the imaging model is projected and displayed as a complete projected image.

[0090] S403: When the overlap between the calibration object and the projection surface of the calibration object is less than a first threshold, adjust the parameters of the imaging model.

[0091] In some embodiments, the degree of overlap between the calibration object and its projection surface can be determined by the user's visual observation. In this case, the first threshold may no longer be a specific numerical value, but rather a subjective experience of the user, such as whether they overlap. Subsequent adjustments are made based on user feedback. In other embodiments, the degree of overlap between the calibration object and its projection surface can be determined using information obtained by a data acquisition device. For example, it can be determined based on the pixel offset between the calibration object and its projection surface. For instance, a camera is placed at a position simulating the user's eye, and images containing the calibration object and its projection surface are acquired using this camera. By capturing one or more images, the pixel offset between the calibration object and its projection surface can be determined based on the image resolution. The degree of overlap between the calibration object and its projection surface can be calculated based on this pixel offset. The calculated degree of overlap can specifically be a percentage value. In this case, the first threshold is also a specific percentage value. By comparing the degree of overlap and the first threshold, it is determined whether the parameters of the imaging model need to be adjusted. It should be understood that the degree of overlap can also be a decimal or other form, and this application does not limit this.

[0092] When the overlap between the calibration object and its projection surface is lower than a preset first threshold, the processor of the projection device can improve the overlap by adjusting the parameters of the imaging model. The adjustable parameters of the imaging model include one or more parameters such as the field of view and the imaging surface position. For example, the field of view parameter determines the area of ​​the imaging surface of the imaging model and the scale of the two-dimensional image of the calibration object relative to the imaging surface; the imaging surface position parameter determines the position of the two-dimensional image of the calibration object relative to the imaging surface. Therefore, when the overlap between the calibration object and its projection surface is lower than the preset first threshold, the field of view or the imaging surface position of the imaging model can be adjusted accordingly based on the area offset or position offset. Specifically, step S403 is implemented as follows:

[0093] When the area difference between the calibration object and the projection surface of the calibration object is greater than the second threshold, the field of view of the imaging model is adjusted;

[0094] When the offset between the calibration object and the projection surface of the calibration object is greater than a third threshold, the two-dimensional position of the imaging surface of the imaging model is adjusted.

[0095] In this embodiment, the first threshold, second threshold, and third threshold can all be preset and adjusted according to user needs or industry standards. When the area difference between the calibration object and its projection surface is greater than the preset second threshold, the area of ​​the imaging surface can be adjusted by adjusting the field of view of the imaging model. When the area of ​​the calibration object's projection surface is greater than the area of ​​the calibration object, the field of view of the imaging model can be enlarged, and the imaging surface will be enlarged proportionally. The proportion of the generated two-dimensional image of the calibration object in the imaging surface will be reduced proportionally. At this time, the area of ​​the projection surface of the calibration object displayed relative to the area of ​​the calibration object will also be reduced proportionally, so that the area difference between the projection surface of the calibration object and the area of ​​the calibration object is less than the preset second threshold. Similarly, when the area of ​​the calibration object's projection surface is less than the area of ​​the calibration object, the field of view of the imaging model can be reduced, and the imaging surface will be reduced proportionally. The proportion of the generated two-dimensional image of the calibration object in the imaging surface will be enlarged proportionally. At this time, the area of ​​the projection surface of the calibration object displayed relative to the area of ​​the calibration object will also be enlarged proportionally, so that the area difference between the projection surface of the calibration object and the area of ​​the calibration object is less than the preset second threshold. When the offset between the calibration object and its projection plane exceeds a preset third threshold, since the position of the calibration object is fixed, the relative position of the generated two-dimensional image of the calibration object in the imaging plane can be adjusted by adjusting the two-dimensional position of the imaging plane of the imaging model. Specifically, the two-dimensional position refers to the vertical and horizontal position of the imaging plane on the two-dimensional plane of the imaging model. For example, when the two-dimensional position of the imaging plane of the imaging model is moved upward, the position of the two-dimensional image of the calibration object in the imaging plane will move downward accordingly. Similarly, when the two-dimensional position of the imaging plane of the imaging model is moved to the left, the position of the two-dimensional image of the calibration object in the imaging plane will move to the right accordingly. By adjusting the two-dimensional position of the imaging plane of the imaging model, the offset between the calibration object and its projection plane is made less than the preset third threshold.

[0096] It should be understood that the area difference, overlap, etc., mentioned above are just some exemplary comparison parameters that can be used in combination or substituted for each other. Other similar comparison parameters, such as size difference, can also be used instead. The main purpose is to determine the magnitude of the difference between the image of the calibration object acquired by the current acquisition device and the projected image of the calibration object, so as to adjust the imaging parameters or imaging model.

[0097] Furthermore, to improve processing efficiency, the imaging model constructed in this embodiment can also be implemented using a neural network model or a deep learning model. Specifically, the imaging model can be trained using a training set consisting of multiple training samples. The training sample can be composed primarily of human eye position information parameters, image information of the calibration object, and position information parameters, with the overlap parameter between the calibration object and its projection surface as the output. Using a predetermined overlap threshold as the target (label), the imaging model is trained multiple times by introducing multiple training samples to obtain a result close to the target, thus obtaining the corresponding imaging model. Based on the trained imaging model, when projecting the calibration object, the overlap between the calibration object and its projection surface can meet the requirements. Moreover, with the use of this imaging model, it has the characteristic of continuous deep learning and optimization, enabling the projection effect of the imaging model to improve and its applicability to a wider range, thus meeting the user experience needs of different users.

[0098] The projection method provided in this embodiment can automatically calibrate the parameters of the imaging model based on the user's eye position, thereby adjusting the projection display effect. With the development of intelligent driving technology, this projection method can be applied not only to projection at the driver's position, but also to projection at the front passenger position or rear passenger position, such as projection of audio-visual entertainment content. In some extended embodiments, the projection method of this application can also guide the user to calibrate the projection display. For example, when the user has a calibration need, a calibration request or calibration start prompt message can be sent to the user, and the user's eye position can be obtained through the in-vehicle camera or eye detector. Based on the user's eye position, the parameters of the imaging model are calibrated, and a calibration completion prompt message is sent to the user when the calibration is complete. This calibration process can be guided by the vehicle's Human Machine Interface (HMI) or by the Driver Monitor System (DMS). The prompt message can be a voice prompt, a graphic prompt on the vehicle's central control screen, etc., so that the user can intuitively experience the calibration process. Simultaneously, during this calibration process, users can also send adjustment commands based on their personal subjective experience to adjust the parameters of the imaging model, thereby meeting their customized needs. For example... Figures 12A-12B In the schematic diagram of a human-machine interface shown, when the calibration process is performed through the vehicle's human-machine interface, graphic and text prompts can be displayed to the user via the vehicle's central control screen to guide the user through the calibration and adjustment process of the projection device. For example, when a user is detected entering the vehicle, the calibration function of the projection device can be automatically activated. Figure 12AThe central control screen displays a message stating, "The vehicle's projection device calibration has been activated. Please maintain a correct seating posture." It then calibrates the parameters of the projection device's imaging model by acquiring the user's eye position. After calibration, the system then... Figure 12B The central control screen displays a message stating "Vehicle has completed the calibration of the projection device." In some variations, during this calibration process, the user can also adjust the parameters of the imaging model on the vehicle's central control screen based on their personal subjective experience. In other variations, the calibration process can also be achieved through voice interaction; the vehicle can send voice prompts to the user through the audio system and obtain the user's voice feedback through the microphone, thereby completing the calibration process.

[0099] As described above, the projection method provided in this application embodiment can realize the functions of calibration, standardization, and projection display of the above-mentioned projection device. Its application can be performed when the vehicle is stationary or while it is in motion. For example, Figure 5 A flowchart of a calibration method provided in an embodiment of this application is shown. This calibration method can be implemented when the vehicle is stationary and in operation. Specifically, it involves the construction and adjustment of an imaging model. The adjusted imaging model can automatically calibrate parameters for different users' eye positions, ensuring that the projected image is always integrated with the real-world environmental information. In this embodiment, the projection device can be an AR-HUD, the imaging model can be an imaging cone, the user can be the vehicle driver, and the calibration method can be verified using the driver's eyes. Figure 5 The calibration method shown includes:

[0100] S501: Construct a virtual imaging cone with the driver's eye as the origin;

[0101] For example, an AR-HUD inside the vehicle or another fixed point inside the vehicle can be selected as the origin to construct a real coordinate system and a virtual coordinate system, and the correspondence between the virtual coordinate system and the real coordinate system can be determined. The real coordinate system is a coordinate system in real three-dimensional space, used to determine the real positions of the human eye, the virtual image surface of the AR-HUD, and the calibration objects in the real world. The virtual coordinate system is a coordinate system in virtual three-dimensional space, used to determine the virtual positions of the human eye, the virtual image surface of the AR-HUD, and the calibration objects in the real world, in order to facilitate the rendering of 3D AR effects.

[0102] In this embodiment, since the position of the driver's eyes will change constantly, the human eye is generally not chosen as the origin for constructing the real coordinate system and the virtual coordinate system.

[0103] In this embodiment, based on the constructed real-world coordinate system, the detected information such as the human eye, the calibration object, and the installation position and projection angle of the AR-HUD are incorporated into this real-world coordinate system. This allows the acquisition of the positions of the human eye, the virtual image plane of the AR-HUD, and the calibration object within this real-world coordinate system. Specifically, these positions can be three-dimensional coordinates within the real-world coordinate system. The virtual image plane of the AR-HUD is the virtual image plane visible to the human eye through the car's windshield. Through human observation, the two-dimensional image displayed on this virtual image plane can be mapped onto the three-dimensional real world. For ease of calibration, when selecting a calibration object, it is necessary to choose an observation range formed by the human eye and the virtual image surface of the AR-HUD. The selected calibration object can be an object with a regular geometric shape, for example, a quadrilateral calibration board. The calibration image generated based on the calibration board can specifically be a quadrilateral virtual frame. When the virtual frame is projected onto the virtual image surface of the AR-HUD for display, the human eye observes whether the virtual frame and the calibration board completely overlap to verify whether the virtual frame and the calibration board are aligned and displayed on the virtual image surface of the AR-HUD.

[0104] Based on the position of the human eye in the real coordinate system and the correspondence between the virtual coordinate system and the real coordinate system, the position of the human eye in the virtual coordinate system is obtained. Using the position of the human eye in the virtual coordinate system as the origin, and according to the set field of view angle, the imaging frustum is constructed, with the calibration object located within the frustum range of the imaging frustum. In this embodiment, the constructed imaging frustum can specifically be a horizontal imaging frustum, where the origin of the imaging frustum and the center points of the near and far planes of the imaging frustum are on the same horizontal line; the imaging frustum can also be a downward-looking imaging frustum, where the origin of the imaging frustum is higher than the center points of the near and far planes of the imaging frustum, so that the origin, at a downward angle, forms a frustum with the near and far planes.

[0105] By constructing a virtual coordinate system with the same origin as the real-world coordinate system, a correspondence between virtual and real space can be achieved. When generating calibration images, it is only necessary to transform the positions of the calibration objects and the viewer's eyes in the real-world coordinate system to the virtual coordinate system. Furthermore, since the virtual and real coordinate systems share the same origin, the transformation calculation process is relatively simple. Based on the viewer's eye position in the virtual coordinate system, a suitable field of view can be selected to construct an imaging frustum with the viewer's eye position as its origin. This allows for the rendering of all objects within the frustum's range using augmented reality (AR) effects, such as lane lines and traffic signs.

[0106] S502: Based on the position of the calibration object located outside the vehicle in the imaging cone, generate a calibration image of the calibration object on the imaging surface of the imaging cone;

[0107] Based on the correspondence between the constructed virtual coordinate system and the real coordinate system, the calibration object in the real coordinate system is transformed to the virtual coordinate system, and the position of the calibration object in the virtual coordinate system is obtained. The calibration object is located within the viewing cone range of the imaging cone in the virtual coordinate system. Based on the position of the calibration object in the imaging cone and the origin of the imaging cone, and based on the imaging principle of forward mapping of the imaging cone, a near plane between the calibration object and the origin of the imaging cone is selected as the imaging surface. Based on the distance relationship between the calibration object and the imaging surface, a cone mapping is performed on the imaging surface to generate a calibration image of the calibration object, wherein the calibration image is a one- or two-dimensional image.

[0108] S503: Project the imaging surface containing the calibration image onto the virtual image surface of the augmented reality head-up display (AR-HUD) for display;

[0109] When the imaging surface of the imaging cone is used as the input image of the AR-HUD and projected onto the virtual image surface of the AR-HUD, the calibration image will also be displayed at the corresponding position on the virtual image surface of the AR-HUD according to its position on the imaging surface. This allows the generated calibration image to be projected onto the calibration object in the real world and mapped into the three-dimensional world through the human eye's viewing angle, thus achieving enhanced display.

[0110] In some embodiments, when the imaging surface of the imaging cone is input as an input image into the AR-HUD, the AR-HUD will crop the received input image according to the limitations of the screen it can display, and display the screen of an appropriate size on its virtual image surface.

[0111] S504: Adjust the parameters of the imaging cone so that the calibration image observed by the human eye on the virtual image plane is aligned with the calibration object.

[0112] In this embodiment, the alignment effect of the calibration image and the calibration object on the virtual image plane of the AR-HUD is directly verified by the human eye. The alignment effect can specifically include scale alignment and position alignment.

[0113] When the calibration image and the calibration object are not aligned in scale on the virtual image plane of the AR-HUD as observed by the human eye, since the imaging plane is a near plane of the imaging cone, this embodiment can adjust the scale of the imaging plane by adjusting the field of view of the imaging cone. Because the relative distance between the imaging plane and the origin of the imaging cone remains unchanged, the scale of the calibration image generated by the imaging plane does not change, but its proportion relative to the imaging plane does. When the scale-adjusted imaging plane is re-inputted into the AR-HUD as an input image and projected onto the virtual image plane of the AR-HUD for display, the scale of the calibration image on the virtual image plane of the AR-HUD will change accordingly. Therefore, by adaptively adjusting the field of view parameters of the imaging cone based on the display effect observed by the human eye on the virtual image plane of the AR-HUD, the calibration image and the calibration object can be aligned in scale on the virtual image plane of the AR-HUD for display.

[0114] Similarly, when the calibration image and the calibration object are not aligned on the virtual image plane of the AR-HUD as observed by the human eye, since the imaging plane is a near plane of the imaging cone, this embodiment can adjust the position of the imaging plane of the imaging cone in the two-dimensional plane of the virtual coordinate system. Because the position of the target object in the virtual coordinate system remains unchanged, when the two-dimensional position of the imaging plane changes, the relative position of the calibration image generated by the imaging plane will adapt accordingly. When the adjusted imaging plane is re-inputted into the AR-HUD as the input image and projected onto the virtual image plane of the AR-HUD for display, the relative position of the calibration image on the virtual image plane of the AR-HUD will also change accordingly. Therefore, by adaptively adjusting the two-dimensional offset of the imaging plane of the imaging cone in the virtual coordinate system based on the display effect observed by the human eye on the virtual image plane of the AR-HUD, the calibration image and the calibration object can be aligned and displayed on the virtual image plane of the AR-HUD.

[0115] In some embodiments of this application, since the position of the human eye affects the initial position of the origin of the constructed imaging cone, the adjusted correspondence between the imaging cone and the human eye position can be obtained based on the aligned calibration image and calibration object. In this correspondence, when the position of the human eye changes, the origin of the imaging cone also changes, and the position of the imaging surface of the imaging cone is adjusted accordingly based on the aforementioned two-dimensional offset. This ensures that the parameters of the imaging cone change accordingly when the driver's eye position changes or when different drivers are present, thereby guaranteeing that the calibration image displayed on the virtual image surface of the AR-HUD observed by the human eye is always aligned with the real world, reducing projection display jitter and preventing dizziness. Simultaneously, the adjusted imaging cone can also be used to generate calibration images of real-world objects detected during driving in real time and display them on the virtual image surface of the AR-HUD in real time, thereby enhancing the driver's acquisition of road information and achieving an immersive experience.

[0116] This application also provides a projection method for an AR-HUD. The goal of this method is to align the AR effect projected onto the AR-HUD as observed by the human eye with the real world. To achieve this goal, this embodiment uses the human eye as a direct verification method. By constructing a virtual imaging model corresponding to the real-world human eye imaging model, the scale and position alignment of the AR-HUD display are calibrated to match the real world. This embodiment also uses a human eye detection module to acquire the position information of the human eye in real time, enabling the AR-HUD display to adapt to changes in the human eye's position in real time. This ensures that the AR-HUD display is always aligned with the real world, guaranteeing the display effect and immersive experience of the AR-HUD.

[0117] like Figure 6 As shown, the system architecture of the projection method described in this embodiment will be introduced first. The system architecture of this embodiment includes a road detection module 601, an AR module 602, a HUD module 603, and a human eye detection module 604; wherein, the HUD module 603 specifically includes an alignment module 6031 and a display module 6032.

[0118] The road detection module 601 can be... Figure 2 The external image acquisition device shown, such as a lidar, vehicle-mounted camera, or other device or combination of devices with image acquisition or optical scanning functions, can be installed on the roof, head, or side of the rearview mirror facing outwards in the vehicle. Its main purpose is to detect and acquire image and positional information of the environment in front of the vehicle. The environment in front of the vehicle can include related information such as vehicles ahead, obstacles, and road signs. The human eye detection module 604 can be... Figure 2The in-vehicle data acquisition devices shown, such as in-vehicle cameras and eye detectors, can be installed on the A-pillars, B-pillars, or the side of the rearview mirror facing the user in the vehicle cabin. Their primary function is to detect and collect the eye position information of the driver or passengers in the vehicle cabin. The AR module 602 and HUD module 603 can be integrated into... Figure 2 The projection device 20 shown is implemented as a complete AR-HUD terminal product.

[0119] During driving, environmental information such as the three-dimensional coordinates of pedestrians and lanes, and lane line positions are obtained through the road detection module 601. This detected environmental information is then transmitted to the AR module 602, where a three-dimensional virtual coordinate system is constructed. A three-dimensional AR effect is then rendered at the corresponding location of the environmental information and mapped into a two-dimensional image. After mapping, the human eye position is detected in real-time by the human eye detection module 604, and the alignment module 6031 in the HUD module 603 performs scale and position alignment between the two-dimensional image and the environmental information. Finally, the aligned two-dimensional image is projected onto the display module 6032 for display. Within the effective projection range of the AR-HUD, regardless of changes in the human eye position, the two-dimensional image projected by the AR-HUD is always perfectly aligned with the environmental information on the road.

[0120] based on Figure 6 The system architecture shown is as follows, referencing Figure 7 The flowchart shown provides a detailed description of the AR-HUD projection method provided in this embodiment. The alignment effect between the AR-HUD and the real world achieved by this method will be maintained throughout the entire driving process. Before driving begins, the alignment calibration between the AR-HUD and the real world can be performed in advance. This alignment calibration process specifically includes:

[0121] S701: Construct a real coordinate system and a virtual coordinate system with a point in space as the origin;

[0122] This embodiment can construct both a real-world coordinate system and a virtual coordinate system using a specific point inside the vehicle as the origin. The origins of the real-world and virtual coordinate systems are the same and have a corresponding relationship. Specifically, this specific point inside the vehicle can be a camera inside the vehicle or an AR-HUD inside the vehicle. The real-world coordinate system is used to determine the three-dimensional coordinates of environmental information in the real world, and its unit can be meters. The unit of the virtual coordinate system can be pixels, where 1 meter in the real-world coordinate system corresponds proportionally to 1 unit in the virtual coordinate system. Based on the obtained three-dimensional coordinates of the environmental information in the real-world coordinate system and the correspondence between the real-world and virtual coordinate systems, a corresponding three-dimensional AR effect can be drawn in the virtual coordinate system, and this three-dimensional AR effect can be mapped into a two-dimensional image. The alignment and calibration process in this embodiment is the alignment and calibration process between the two-dimensional image and the environmental information.

[0123] S702: Set up a calibration plate at the location of the virtual image plane of the AR-HUD;

[0124] Based on the driver's eye detected by the human eye detection module, the position of the human eye in the real coordinate system can be obtained. Based on the installation position and projection angle of the AR-HUD, the position of the virtual image surface of the AR-HUD in the real coordinate system can be obtained. The virtual image surface of the AR-HUD is the virtual image display plane of the AR-HUD as observed by the driver's human eye. Generally, the virtual image surface of the AR-HUD is located 7-10 meters in front of the driver's eyes facing the vehicle. By observing the two-dimensional image on the virtual image surface through the driver's human eye, the two-dimensional image can be mapped into the real world to achieve a three-dimensional display effect.

[0125] By setting a calibration plate on the virtual image surface of the AR-HUD, the calibration plate serves as a calibration reference during the alignment calibration process in this embodiment. In this embodiment, the calibration plate can specifically be a substrate with a regular geometric shape.

[0126] S703: Generates a target bounding box on the imaging surface of the virtual coordinate system and projects it onto the virtual image surface of the AR-HUD for display;

[0127] Based on step S702, the position of the human eye in the real coordinate system, the position of the virtual image surface of the AR-HUD, and the correspondence between the real coordinate system and the virtual coordinate system, the corresponding virtual human eye is determined in the virtual coordinate system. Since the real coordinate system and the virtual coordinate system have the same origin, the position of the virtual human eye in the virtual coordinate system corresponds to the position of the human eye in the real coordinate system, and the position of the imaging surface in the virtual coordinate system corresponds to the position of the virtual image surface of the AR-HUD in the real coordinate system. Furthermore, the imaging surface and the virtual image surface have the same correspondence as the real coordinate system and the virtual coordinate system.

[0128] like Figure 8AAs shown, with the virtual human eye as the origin and a field of view (FOV) set, a cone-shaped perspective projection model is constructed in the virtual coordinate system. This perspective projection model is specifically an imaging cone, used to render AR effects of real-world environmental information and to perform a two-dimensional mapping of these AR effects. The virtual human eye is the origin of the imaging cone, and the field of view determines the cone's range. By selecting a near plane of the imaging cone as the imaging surface, this embodiment can select the corresponding near plane of the imaging cone in the virtual coordinate system as the imaging surface, based on the position of the AR-HUD's virtual image surface in the real coordinate system. This ensures that the position of the imaging surface in the virtual coordinate system corresponds to the position of the AR-HUD's virtual image surface in the real coordinate system.

[0129] like Figure 8A As shown, at an infinite distance from the imaging cone, there is also a far plane. According to the imaging principle of the imaging cone, AR effects drawn within the field of view (FOV) of the imaging cone and between the imaging plane and the far plane will be proportionally mapped onto the imaging plane in a cone-shaped manner according to their distance, that is, a two-dimensional image of the AR effect will be generated on the imaging plane. Figure 8B As shown, the imaging surface mapped with the 2D image is sent as the input image to the AR-HUD. This imaging surface has a corresponding projection relationship with the virtual image surface of the AR-HUD. Based on this projection relationship, the 2D image on the imaging surface can be projected and displayed on the virtual image surface of the AR-HUD. Specifically, the drawing process in the imaging frustum and the projection process of the 2D image involve performing a matrix transformation on the 3D coordinates of the AR effect in the virtual coordinate system to convert them to coordinates in the real coordinate system. The formula for this matrix transformation is...

[0130] S = P * V * O

[0131] Where O represents the 3D coordinates of the AR effect drawn in the virtual coordinate system, V represents the observation matrix of the virtual human eye in the virtual coordinate system, P represents the mapping matrix of the imaging surface of the imaging cone, and S represents the coordinates of the virtual image surface of the HUD in the real coordinate system. The AR effect drawn in the virtual coordinate system is mapped as a 2D image onto the imaging surface of the imaging cone, and this imaging surface is used as the input image for the AR-HUD, which is then projected onto the virtual image surface of the AR-HUD.

[0132] In this embodiment, a corresponding target box can be generated on the imaging surface of the imaging cone based on the calibration plate of the virtual image surface of the AR-HUD. The target box has the same geometry as the calibration plate. Then, the imaging surface is used as the input image and projected onto the virtual image surface of the AR-HUD. The alignment calibration process in this embodiment is specifically the process of aligning the target box displayed on the virtual image surface of the AR-HUD with the calibration plate.

[0133] S704: Observe whether the target box and the calibration plate are aligned in scale;

[0134] In this embodiment, whether the scales are aligned can be specifically determined by whether the size of the target box on the virtual image plane of the AR-HUD is aligned with the size of the calibration board. If they are aligned, proceed to step S706; if they are not aligned, proceed to step S705.

[0135] S705: Scale alignment adjustment;

[0136] When the target bounding box is not aligned with the calibration board, it means that the target bounding box generated on the imaging surface, after projection, is not aligned with the scale of the calibration board on the virtual image surface of the AR-HUD. Since the imaging surface is used as the input image for the AR-HUD, the AR-HUD will crop the input image according to its display pixels, that is, it will crop the input imaging surface image to a scale that matches the display pixels for display. Given that the units of the virtual coordinate system, the imaging frustum, and the display pixels of the AR-HUD are all determined, when the target bounding box is not aligned with the calibration board, it is necessary to proportionally adjust the scale of the imaging surface of the imaging frustum to proportionally adjust the scale of the cropped image of the AR-HUD, thereby proportionally adjusting the relative size of the target bounding box in the cropped image to align it with the scale of the calibration board.

[0137] In this embodiment, adjusting the size of the imaging surface of the imaging cone can be achieved by adjusting the field of view of the imaging cone. Specifically, when the target box size is larger than the calibration plate, the field of view of the imaging cone can be increased to proportionally enlarge the size of the imaging surface, thereby achieving proportional enlargement of the imaging surface input to the AR-HUD. Similarly, when the target box size is smaller than the calibration plate, the field of view of the imaging cone can be decreased to proportionally shrink the size of the imaging surface, thereby achieving proportional shrinkage of the imaging surface input to the AR-HUD. Thus, with the generated target box size and imaging surface position remaining unchanged, the size adjustment of the target box displayed on the AR-HUD's virtual image surface can be achieved by adjusting the field of view of the imaging cone to achieve scale alignment with the calibration plate, that is, to achieve scale alignment between the imaging surface of the imaging cone and the virtual image surface of the AR-HUD.

[0138] S706: Observe whether the target box and the calibration plate are aligned;

[0139] After the adjustment in step S705, although the imaging surface of the imaging frustum in the virtual coordinate system is scale-aligned with the virtual image surface of the AR-HUD in the real coordinate system, the positions of the target box and calibration plate displayed on the virtual image surface of the AR-HUD are still offset. There are generally two reasons for this offset: one is that in the imaging frustum constructed in the virtual coordinate system, the virtual human eye corresponds to the center point of the near and far planes, such as... Figure 9A As shown; however, in the real coordinate system, the virtual image plane of an AR-HUD is usually located below the human eye level, meaning the center point of the virtual image plane is lower than the human eye level, such as... Figure 9B As shown. Therefore, when the imaging surface is projected as the input image onto the virtual image surface of the AR-HUD for display, the actual displayed 2D image will be lower than the environmental information in the real world, causing the position of the displayed target box to be lower than the position of the calibration board. Secondly, during human observation, the position of the human eye is not fixed, while the position of the virtual image surface of the installed AR-HUD is fixed. Therefore, when the position of the human eye moves, the relative position between the human eye and the center point of the virtual image surface of the AR-HUD will shift accordingly, causing the position of the displayed target box to be inconsistent with the position of the calibration board.

[0140] In this embodiment, whether the positions are aligned can be specifically determined by whether the position of the target box on the virtual image plane of the AR-HUD is aligned with the position of the calibration board. If they are aligned, proceed to step S708; if they are not aligned, proceed to step S707.

[0141] S707: Position alignment adjustment;

[0142] When the target bounding box is not aligned with the calibration board, it means that the target bounding box generated on the imaging surface, after projection, is not aligned with the calibration board on the virtual image surface of the AR-HUD. Since the imaging surface is used as the input image for the AR-HUD, the HUD will crop the input image according to its display pixels; that is, it will crop the input imaging surface image to a scale that matches the display pixels for display. Given that the units of the virtual coordinate system, the imaging frustum, the display pixels of the AR-HUD, and the cropping position are all determined, when the target bounding box is not aligned with the calibration board, it is necessary to adjust the position of the imaging surface of the imaging frustum in its own plane to adjust the position input to the AR-HUD imaging surface, and thus adjust the relative position of the target bounding box in the cropped image to align it with the calibration board.

[0143] In this embodiment, the relative position of the target box on the imaging surface can be adjusted by adjusting the two-dimensional offset of the imaging surface of the imaging cone in the virtual coordinate system. It should be noted that adjusting the two-dimensional offset of the imaging surface in the virtual coordinate system is essentially adjusting the horizontal or vertical position of the imaging surface in its plane.

[0144] Specifically, such as Figure 10A As shown, when the vertical position of the displayed target box is lower than the vertical position of the calibration board, the position of the imaging surface of the imaging cone in the virtual coordinate system can be moved downwards vertically. This moves the relative position of the target box and the imaging surface upwards vertically, making the relative position of the target box in the AR-HUD cropped image higher than its original position. This aligns the adjusted displayed target box with the vertical position of the calibration board. Similarly, as... Figure 10B As shown, when the horizontal position of the target box is relatively to the right compared to the horizontal position of the calibration board, the position of the imaging surface of the imaging frustum in the virtual coordinate system can be moved horizontally to the right. This, in turn, moves the relative position of the target box and the imaging surface horizontally to the left, causing the target box to be positioned to the left of its original position in the image cropped by the AR-HUD. This aligns the adjusted target box with the horizontal position of the calibration board. Therefore, without changing the size of the generated target box or the scale of the imaging surface, the position adjustment of the target box displayed on the AR-HUD's virtual image surface can be achieved by adjusting the position of the imaging surface of the imaging frustum, thus aligning it with the calibration board—that is, aligning the imaging surface of the imaging frustum with the virtual image surface of the AR-HUD.

[0145] Among them, based on the compensation principle, the horizontal and vertical offsets (X) of the imaging surface of the imaging cone can be calculated. offset Y offset Perform the following calculations.

[0146]

[0147]

[0148] In this system, the unit of the virtual coordinate system is pixels, and the unit of the real coordinate system is meters. Therefore, 1 pixel = m meters. hud Y hud (X) represents the x and y coordinates of the center point of the virtual image plane of the AR-HUD in the real coordinate system. eye Y eye Let X be the horizontal and vertical coordinates of the human eye in the real coordinate system. Based on the above calculation formula, the required horizontal offset X of the imaging surface of the imaging cone in the virtual coordinate system can be calculated. offset and vertical offset Y offset According to the horizontal offset X offset and vertical offset Yoffset The imaging surface of the imaging cone is adjusted in two dimensions, pixel by pixel, so that the target box displayed on the virtual image surface of the AR-HUD is aligned with the position of the calibration board.

[0149] S708: Move the calibration board behind the virtual image plane of the AR-HUD;

[0150] After completing the scale alignment and position alignment steps S704-S706, the effect of scale alignment and position alignment can be verified by moving the calibration board in the real coordinate system. By moving the calibration board behind the virtual image surface of the AR-HUD, that is, moving the calibration board to a greater distance from the human eye, we can observe whether the target box displayed on the virtual image surface is aligned with the calibration board.

[0151] S709: Observe whether the target box and the calibration plate are completely aligned;

[0152] When the calibration board is moved to a greater distance from the human eye, its position in the virtual coordinate system remains between the imaging plane and the far plane of the imaging cone. According to the imaging principle, the scale of the target box generated on the imaging plane will scale down proportionally as the distance to the calibration board increases. The calibration effect of this method is verified by observing whether the regenerated target box is perfectly aligned with the calibration board at the greater distance in the AR-HUD's virtual image plane. If perfectly aligned, proceed to step S710; if not perfectly aligned, proceed to step S704 to repeat the scale and position alignment adjustments.

[0153] S710: Move the calibration board in front of the virtual image plane of the AR-HUD;

[0154] By moving the calibration board in front of the virtual image surface of the AR-HUD, that is, moving the calibration board closer to the human eye, it is observed whether the virtual image surface can display the target box corresponding to the calibration board, and whether the target box is perfectly aligned with the calibration board.

[0155] S711: The virtual image surface of the AR-HUD can display the target bounding box;

[0156] Since the imaging surface in the constructed imaging frustum is selected based on the position of the virtual image surface of the AR-HUD in the real coordinate system, when the calibration plate is moved in front of the virtual image surface of the AR-HUD, the corresponding position of the calibration plate in the virtual coordinate system is also moved in front of the imaging surface. According to the imaging principle of the imaging frustum, the calibration plate located in front of the imaging surface at this time cannot be mapped onto the imaging surface.

[0157] S712: Close-up display adjustment;

[0158] Based on the imaging principle of the imaging frustum, this embodiment adjusts the position of the imaging surface within the imaging frustum according to the corresponding position of the calibration plate in the virtual coordinate system. Specifically, it reselects a near plane within the imaging frustum located between the corresponding position of the calibration plate in the virtual coordinate system and the origin of the imaging frustum as a new imaging surface. Then, according to the imaging principle, a new target bounding box corresponding to the calibration plate is generated within this new imaging surface. In this embodiment, changes in the relative distance between the imaging surface and the origin do not alter the scale of the imaging surface; the scale is determined solely by the field of view of the imaging frustum. By adjusting the distance of the imaging surface relative to the origin of the imaging frustum, selective two-dimensional mapping of environmental information within the frustum's range is achieved, thereby changing the number of two-dimensional images that the imaging surface can generate.

[0159] S713: Observe whether the target box and the calibration plate are completely aligned;

[0160] The calibration effect of this method is verified by observing whether the regenerated target box is perfectly aligned with the virtual image surface of the AR-HUD and the calibration board moved to a close distance. If perfectly aligned, proceed to step S714; if not perfectly aligned, proceed to step S704 to repeat the scale alignment and position alignment adjustments.

[0161] S714: Achieve alignment of AR-HUD with the real world;

[0162] By changing the position of the calibration board in the real coordinate system and aligning the target box generated by the repositioned calibration board with the display effect of the calibration board at the virtual image plane of the AR-HUD, the imaging surface of the imaging cone constructed based on the human eye position is aligned with the virtual image plane of the AR-HUD. After this alignment and calibration is completed, when the driver's eye position changes or different drivers drive, the constructed imaging cone will be adjusted accordingly to ensure that the display effect of the virtual image plane of the AR-HUD observed by the human eye is always completely aligned with the real world, improving the driver's observation experience and achieving a better driving assistance effect.

[0163] like Figure 11 As shown, this application provides a projection device that can be used to implement the projection method, calibration method, AR-HUD projection method, and display method described in the above embodiments, such as... Figure 11 As shown, the projection device 1100 has an acquisition module 1101, a projection module 1102, and an adjustment module 1103.

[0164] The acquisition module 1101 is used to execute step S401 of the projection method described above, and an example thereof. The projection module 1102 is used to execute any one of the steps S402 of the projection method described above, S501 to S503 of the calibration method described above, S701 to S703 of the AR-HUD projection method described above, and any optional example thereof. The adjustment module 1103 is used to execute any one of the steps S403 of the projection method described above, S504 of the calibration method described above, S704 to S714 of the AR-HUD projection method described above, and any optional example thereof. For details, please refer to the detailed description in the method embodiments, which will not be repeated here.

[0165] In some embodiments, the projection device 1100 may also include a prompting module 1104, which can implement the human-computer interaction parts of the above-described projection method, calibration method, and AR-HUD projection method. By sending prompt messages to the user, the prompting module 1104 can guide the user to participate in the calibration or adjustment process in the above-described projection method, calibration method, and AR-HUD projection method. For example, the prompting module 1104 can prompt the user to determine whether the calibration object and the projection surface of the calibration object coincide through the human eye. It can also send a prompt message to the user to start calibration and a prompt message to complete calibration when the user's calibration needs are obtained.

[0166] It should be understood that the projection device in this application embodiment can be implemented by software, for example, by a computer program or instruction with the above-mentioned functions. The corresponding computer program or instruction can be stored in the memory inside the terminal, and the above functions can be implemented by the processor reading the corresponding computer program or instruction in the memory. Alternatively, the projection device in this application embodiment can also be implemented by hardware. For example, the acquisition module 1101 can be implemented by a vehicle-mounted acquisition device, such as a vehicle-mounted camera or LiDAR, or the acquisition module 1101 can also be implemented by an interface circuit between the processor and the vehicle-mounted camera or LiDAR. The prompt module 1104 can be implemented by a vehicle-mounted central control screen or a device such as an audio system or microphone. The projection module 1102 can be implemented by a vehicle-mounted HUD or AR-HUD, or the projection module 1102 can also be implemented by a processor of a HUD or AR-HUD, or the projection module can also be implemented by a terminal such as a mobile phone or tablet. The adjustment module 1103 can be implemented by a processor of a HUD or AR-HUD, or the adjustment module 1103 can also be implemented by a processor of a vehicle-mounted processing device such as a vehicle infotainment system or vehicle-mounted computer. Alternatively, the projection device in the embodiments of this application can also be implemented by a combination of a processor and a software module.

[0167] It should be understood that the processing details of the apparatus or module in the embodiments of this application can be found by referring to... Figure 4, Figure 5 , Figure 7 The descriptions of the embodiments and related extended embodiments shown will not be repeated in this application.

[0168] Furthermore, this application embodiment also provides a vehicle equipped with the aforementioned projection device. This vehicle can be a passenger car, a truck, or a special vehicle such as an ambulance, fire truck, police car, or emergency rescue vehicle. The vehicle can use local storage to store the imaging model and related training set from the above embodiments. When the projection method and calibration method need to be implemented, the imaging model can be loaded more quickly, enabling rapid calibration or adjustment of the projection display based on the user's eye position, offering advantages such as low latency and a good user experience. In addition, the vehicle can also interact with the cloud, downloading the cloud-stored imaging model to its local machine to achieve calibration or adjustment of the projection display based on the user's eye position. Cloud interaction offers advantages such as abundant data, timely model updates, and higher accuracy.

[0169] Figure 13 This is a schematic structural diagram of a computing device 1500 provided in an embodiment of this application. This computing device can function as a projection device, executing various optional embodiments of the projection method, calibration method, or AR-HUD projection method described above. The computing device can be a terminal, or a chip or chip system within the terminal. Figure 13 As shown, the computing device 1500 includes: a processor 1510, a memory 1520, a communication interface 1530, and a bus 1540.

[0170] It should be understood that Figure 13 The communication interface 1530 in the computing device 1500 shown can be used to communicate with other devices, and may specifically include one or more transceiver circuits or interface circuits.

[0171] The processor 1510 can be connected to the memory 1520. The memory 1520 can be used to store the program code and data. Therefore, the memory 1520 can be a storage unit inside the processor 1510, an external storage unit independent of the processor 1510, or a component that includes both the storage unit inside the processor 1510 and the external storage unit independent of the processor 1510.

[0172] Optionally, the computing device 1500 may also include a bus 1540. The memory 1520 and communication interface 1530 can be connected to the processor 1510 via the bus 1540. The bus 1540 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1540 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0173] It should be understood that in the embodiments of this application, the processor 1510 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Alternatively, the processor 1510 may employ one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0174] The memory 1520 may include read-only memory and random access memory, and provides instructions and data to the processor 1510. A portion of the processor 1510 may also include non-volatile random access memory. For example, the processor 1510 may also store device type information.

[0175] When the computing device 1500 is running, the processor 1510 executes computer execution instructions stored in the memory 1520 to perform any of the operation steps of the projection method, calibration method, or AR-HUD projection method described above, as well as any of the optional embodiments thereof.

[0176] It should be understood that the computing device 1500 according to the embodiments of this application can correspond to the corresponding subject in executing the methods according to the various embodiments of this application, and the above and other operations and / or functions of each module in the computing device 1500 are respectively for implementing the corresponding processes of the methods of this embodiment. For the sake of brevity, they will not be described in detail here.

[0177] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0179] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0182] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0183] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs a diversified problem generation method, including at least one of the schemes described in the above embodiments.

[0184] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0185] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0186] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0187] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] It should be noted that the embodiments described in this application are merely some embodiments, not all embodiments. The components of the embodiments of this application typically described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the above detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0189] The terms "first, second, third, etc." or similar terms such as module A, module B, module C, etc., used in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that a specific order or sequence may be interchanged where permitted so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0190] In the above description, the step numbers involved, such as S110, S120, etc., do not mean that this step will necessarily be executed. It may also include intermediate steps or be replaced by other steps. Where permissible, the order of the preceding and following steps may be interchanged or executed simultaneously.

[0191] The term "comprising" as used in the specification and claims should not be construed as limiting itself to what follows; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the mentioned feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components, or groups thereof. Thus, the statement "device comprising means A and B" should not be limited to a device consisting solely of components A and B.

[0192] The terms "an embodiment" or "an embodiment" as used in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in at least one embodiment of this application. Therefore, the terms "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. Furthermore, in the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between different embodiments are consistent and can be mutually referenced. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0193] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present application has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A projection method, characterized by, include: The system acquires the user's eye position information, image information and position information of the calibration object, wherein the calibration object includes static objects outside the vehicle or dynamic objects outside the vehicle. Based on the human eye position information, the image information and position information of the calibration object, and the imaging model, a calibration image corresponding to the calibration object is projected. The imaging model includes a virtual imaging cone with the user's human eye as the origin. The calibration image is generated on the imaging surface of the imaging cone based on the position of the calibration object in the imaging cone, and the imaging surface including the calibration image is projected onto the virtual image surface of the projection device. When the overlap between the calibration object and the calibration image is less than a first threshold, one or more parameters of the field of view and the position of the imaging plane of the imaging model are adjusted to align the calibration image with the calibration object, including: adjusting the parameters of the imaging cone to align the calibration image located on the virtual image plane observed by the human eye with the calibration object.

2. The method of claim 1, wherein, When the overlap between the calibration object and the projection plane of the calibration object is less than a first threshold, adjusting one or more parameters of the field of view angle and the position of the imaging plane of the imaging model includes: When the area difference between the calibration object and the calibration image is greater than a second threshold, the field of view of the imaging model is adjusted.

3. The method of claim 1, wherein, When the overlap between the calibration object and the projection plane of the calibration object is less than a first threshold, adjusting one or more parameters of the field of view angle and the position of the imaging plane of the imaging model includes: When the offset between the calibration object and the calibration image is greater than a third threshold, the two-dimensional position of the imaging surface of the imaging model is adjusted.

4. The method according to claim 1, characterized in that, The degree of overlap between the calibration object and the calibration image is determined by the pixel offset between the calibration object and the calibration image; the pixel offset is determined by an image containing the calibration object and the calibration image captured by a camera.

5. The method according to claim 1, characterized in that, The imaging model is trained on a training set that includes multiple training samples, wherein the training samples include human eye position information parameters, image information and position information parameters of the calibration object, and the overlap parameter between the calibration object and the projection surface of the calibration object.

6. The method of claim 1, wherein, Also includes: Obtain the user's calibration requirements and send a prompt message to the user indicating that calibration has begun; Obtain the user's eye position, and calibrate the parameters of the imaging model based on the user's eye position; After calibration is complete, a notification message will be sent to the user indicating that calibration is complete.

7. The method of claim 6, wherein, Also includes: Whether the calibration object and the calibration image overlap are determined by the human eye; When the calibration object and the calibration image do not coincide, the parameters of the calibrated imaging model are adjusted according to the user's adjustment instructions.

8. A projection apparatus, characterized by comprising: include: The acquisition module is used to acquire the user's eye position information, the image information and position information of the calibration object, wherein the calibration object includes static objects outside the vehicle or dynamic objects outside the vehicle; The projection module is used to project a calibration image corresponding to the calibration object based on the human eye position information, the image information and position information of the calibration object, and the imaging model. The imaging model includes a virtual imaging cone with the user's human eye as the origin. The calibration image is generated on the imaging surface of the imaging cone based on the position of the calibration object in the imaging cone, and the imaging surface including the calibration image is projected onto the virtual image surface of the projection device. An adjustment module is used to adjust one or more parameters of the field of view and imaging plane position of the imaging model when the overlap between the calibration object and the calibration image is less than a first threshold, so as to align the calibration image with the calibration object, including: adjusting the parameters of the imaging cone so that the calibration image observed by the human eye located on the virtual image plane is aligned with the calibration object.

9. The apparatus of claim 8, wherein, The adjustment module is specifically used for: When the area difference between the calibration object and the calibration image is greater than a second threshold, the field of view of the imaging model is adjusted.

10. The apparatus of claim 8, wherein, The adjustment module is specifically used for: When the offset between the calibration object and the calibration image is greater than a third threshold, the two-dimensional position of the imaging surface of the imaging model is adjusted.

11. The apparatus according to claim 8, characterized in that, The degree of overlap between the calibration object and the calibration image is determined by the pixel offset between the calibration object and the calibration image; the pixel offset is determined by an image containing the calibration object and the calibration image captured by a camera.

12. The apparatus according to claim 8, characterized in that, The imaging model is trained on a training set that includes multiple training samples, wherein the training samples include human eye position information parameters, image information and position information parameters of the calibration object, and the overlap parameter between the calibration object and the calibration image.

13. The apparatus of claim 8, wherein, Also includes: The prompt module is used to send a prompt message to the user indicating that calibration has started when the user's calibration request is received; The adjustment module is also used to calibrate the parameters of the imaging model based on the acquired position of the user's eye. The notification module is also used to send a notification message to the user indicating that calibration is complete after calibration is finished.

14. The apparatus according to claim 13, characterized in that, The prompting module is also used to prompt the user to determine by visual inspection whether the calibration object and the calibration image overlap. The adjustment module is also used to adjust the parameters of the calibrated imaging model according to the user's adjustment instructions when the calibration object and the calibration image do not overlap.

15. A computing device, comprising: include: processor, and A memory having stored program instructions that, when executed by the processor, cause the processor to perform the projection method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The computer-readable medium stores program code that, when executed by a computer or processor, causes the computer or processor to perform the projection method according to any one of claims 1 to 7.

17. A computer program product, characterised in that, The program code contained in the computer program product, when executed by a computer or processor, causes the computer or processor to perform the projection method according to any one of claims 1 to 7.

Images