Method and device for calibrating AR HUD system

Through the on-board display screen and controller, combined with voice, buttons, touch and corner point detection algorithms, the calibration complexity of the AR HUD system is realized without professional equipment, solving the user's eye position changes and camera position deflection, and improving the calibration complexity of the Calibration accuracy and user experience.

CN119088275APending Publication Date: 2024-12-06SHANGHAI QINGGAN INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411047165.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When the user's eye position changes or camera position deflection, professional equipment is required to perform position calibration, resulting in high cost, poor user experience, and complex operation.

Method used

Through the vehicle's on-board display, AR HUD and controller, voice, keys, touch commands and corner point detection algorithms are used to realize position calibration without professional equipment. Users can complete calibration by adjusting calibration points and corner point coordinate groups. Simplify operations.

Benefits of technology

It realizes calibration of AR HUD system without professional equipment, reducing costs, improving user experience, simplifying operations, and improving calibration accuracy and sensitivity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088275A_ABST
    Figure CN119088275A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a method and device for calibrating an AR HUD system. The specific implementation mode of the method comprises the steps that in response to a received calibration request, a calibration interface is displayed on a vehicle-mounted display screen, and the calibration interface comprises an image of an area in front of a vehicle; displaying at least one angular point in the image on the calibration interface, and displaying at least one calibration point corresponding to the at least one angular point on a projection screen of the AR HUD system; in response to a received adjustment instruction for adjusting the position of the at least one calibration point, the at least one calibration point is moved to a target position indicated by the adjustment instruction, and the target position is the position, observed by human eyes, of the at least one calibration point on the projection screen consistent with at least one angular point displayed in the image; and in response to the fact that a coordinate group set meets a preset condition, the AR HUD system is calibrated based on the coordinate group set, and the coordinate group set comprises at least one coordinate group composed of the coordinates of the at least one calibration point on the projection screen and the coordinates of the at least one angular point on the vehicle-mounted display screen. According to the embodiment, the AR HUD system can be quickly and accurately calibrated without special hardware equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method and device for calibrating an AR HUD system. Background Art

[0002] AR HUD is a combination of AR (Augmented Reality) technology and HUD (Head Up Display) technology. AR HUD is a new way to achieve in-vehicle display by using reflection on the vehicle windshield. Specifically, the display light is emitted by the optical machine of the HUD display device, and is projected onto the windshield through the corresponding optical lens to produce a corresponding virtual image, which forms an enhanced display effect with the real world outside the windshield.

[0003] However, since each user change will cause the user's eye position to change, or the camera position to deflect unexpectedly, AR HUD needs to be calibrated to ensure that the virtual information is displayed in the correct position in the user's field of view to avoid visual errors. Summary of the invention

[0004] One embodiment of the present application provides a method and device for calibrating an AR HUD system, wherein no professional calibration equipment is required, and the user can complete the position calibration of the AR HUD system only through the vehicle itself, which is low-cost and easy to promote.

[0005] Another embodiment of the present application provides a method and device for calibrating an AR HUD system, wherein there is no need to learn professional instructions, and the AR HUD system calibration can be completed through conventional voice, button, touch and other operations. The operation is simple and the user experience is good.

[0006] Another embodiment of the present application provides a method and device for calibrating an AR HUD system, wherein touch instructions are obtained by a touch panel composed of piezoelectric vibrators, so that the position of the calibration point can be adjusted more sensitively and finely, thereby improving the calibration effect of the ARHUD system.

[0007] Another embodiment of the present application provides a method and device for calibrating an AR HUD system, wherein a corner point detection algorithm can be used to identify corner points that meet predetermined quality requirements in the image of the area in front of the vehicle as a reference for calibrating the AR HUD system, thereby improving the calibration effect of the AR HUD system.

[0008] Another embodiment of the present application provides a method and device for calibrating an AR HUD system, wherein a location more suitable for calibration can be found by changing the orientation or position of the vehicle to obtain sufficient new corner point coordinates, thereby reducing the situation where calibration cannot be performed due to too few corner points at the current position.

[0009] Another embodiment of the present application provides a method and device for calibrating an AR HUD system, wherein a location more suitable for calibration can be found by changing the orientation or position of the vehicle to obtain new corner point coordinates with better quality, thereby improving the accuracy of calibration of the AR HUD system.

[0010] To achieve one or more of the above-mentioned purposes, in a first aspect of the implementation of the present application, an embodiment of the present disclosure provides a method for calibrating an AR HUD system, comprising: in response to receiving a calibration request, displaying a calibration interface on a vehicle display screen, wherein the calibration interface includes an image of the area in front of the vehicle; displaying at least one corner point in the image on the calibration interface, and displaying at least one calibration point corresponding to the at least one corner point on the projection screen of the AR HUD system; in response to receiving an adjustment instruction to adjust the position of the at least one calibration point, moving the at least one calibration point to the target position indicated by the adjustment instruction, wherein the target position is a position where the at least one calibration point on the projection screen is consistent with the at least one corner point displayed in the image as observed by the human eye; in response to a coordinate group set satisfying a preset condition, calibrating the AR HUD system based on the coordinate group set, wherein the coordinate group set includes at least one coordinate group consisting of the coordinates of the at least one calibration point on the projection screen and the coordinates of the at least one corner point on the vehicle display screen.

[0011] In some embodiments, the adjustment instruction includes at least one of the following: a voice instruction, a key instruction, and a touch instruction.

[0012] In some embodiments, the touch control instruction is obtained via a touch screen or a touch pad, and the touch pad is composed of a plurality of piezoelectric vibrators.

[0013] In some embodiments, the preset condition includes: the number of the coordinate groups is greater than or equal to a first predetermined value and the adjustment instruction is executed.

[0014] In some embodiments, calibrating the AR HUD system based on the coordinate group set includes: calculating a homography matrix based on the coordinate group set to obtain a coordinate transformation matrix; and calibrating the ARHUD system based on the coordinate transformation matrix.

[0015] In some embodiments, displaying at least one corner point in the image on the calibration interface includes: receiving a corner point marking operation; and displaying the point in the image added by the corner point marking operation as a corner point.

[0016] In some embodiments, displaying at least one corner point in the image on the calibration interface includes: extracting the at least one corner point from the image by a corner point detection algorithm and displaying the corner point in the image.

[0017] In some embodiments, extracting corner points from the image by a corner detection algorithm and displaying them in the image includes: extracting a group of candidate corner points from the image by a corner detection algorithm; selecting at least one corner point that meets predetermined quality requirements from the group of candidate corner points, and displaying it in the image.

[0018] In some embodiments, selecting at least one corner point that meets the predetermined quality requirements from the group of candidate corner points includes at least one of the following methods: selecting at least one corner point whose preset quality index value is higher than the corresponding second predetermined value; dividing the image into multiple areas and selecting the corner point with the highest preset quality index value in each area.

[0019] In some embodiments, the method further includes: in response to the coordinate set not satisfying a preset condition within a predetermined time, instructing the vehicle to adjust its orientation to a target orientation, or to move to a recommended position.

[0020] In some embodiments, the target orientation is determined based on the following steps: acquiring a peripheral image of the vehicle to detect corner points, the peripheral image including images of areas other than the area in front of the vehicle; determining an image area in which the number of corner points in the peripheral image is greater than or equal to the first predetermined value and is evenly distributed; and determining the target orientation based on a shooting angle corresponding to the image area.

[0021] In some embodiments, the recommended location is determined based on the following steps: selecting an address closest to the current location of the vehicle or an address of the vehicle's expected route in future travel from a pre-stored recommended address library as the recommended location, and the recommended address library includes at least one address suitable for calibrating the AR HUD system.

[0022] To achieve one or more of the above-mentioned purposes, in a second aspect of the implementation of the present application, an embodiment of the present disclosure provides an AR HUD system, comprising: an on-board display screen, an AR HUD, and a controller that establishes communication connections with the on-board display screen and the AR HUD respectively; wherein the on-board display screen is configured to display a calibration interface in response to receiving a calibration request, wherein the calibration interface includes an image of the area in front of the vehicle; at least one corner point in the image is displayed on the calibration interface; the AR HUD is configured to display at least one calibration point corresponding to the at least one corner point; the controller is configured to move the at least one calibration point to a target position indicated by the adjustment instruction in response to receiving an adjustment instruction for adjusting the position of the at least one calibration point, wherein the target position is a position where the at least one calibration point on the projection screen is consistent with the at least one corner point displayed in the image as observed by the human eye; in response to a coordinate group set satisfying a preset condition, the AR HUD system is calibrated based on the coordinate group set, and the coordinate group set includes at least one coordinate group consisting of the coordinates of the at least one calibration point on the projection screen and the coordinates of the at least one corner point on the on-board display screen.

[0023] To achieve one or more of the above-mentioned purposes, in a third aspect of the implementation of the present application, an embodiment of the present disclosure provides an electronic device for calibrating an AR HUD system, comprising: one or more processors; a storage device having one or more computer programs stored thereon, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement a method as described in any one of the first aspects.

[0024] To achieve one or more of the above-mentioned purposes, in a fourth aspect of the implementation of the present application, an embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored, wherein when the computer program is executed by a processor, the method as described in any one of the first aspects is implemented.

[0025] To achieve one or more of the above-mentioned purposes, in the fifth aspect of the implementation of the present application, an embodiment of the present disclosure provides a vehicle, including an on-board display screen, an AR HUD, and a controller, an on-board camera, a touch screen or a touch pad that respectively establish communication connections with the on-board display screen and the AR HUD.

[0026] The method and device for calibrating the AR HUD system provided by the embodiments of the present disclosure can be calibrated by the driver only through the vehicle itself without relying on any equipment. The function is simple and easy to use.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0029] Figure 1 is a system architecture diagram of an embodiment of an AR HUD system according to the present disclosure;

[0030] Figure 2 is a flow chart of an embodiment of a method for calibrating an AR HUD system according to the present disclosure;

[0031] Figure 3 is a schematic diagram of an application scenario of the method for calibrating an AR HUD system according to the present disclosure;

[0032] Figure 4 is a flowchart of another embodiment of a method for calibrating an AR HUD system according to the present disclosure;

[0033] Figure 5 It is a structural diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] As mentioned earlier, since each user's sitting height is different, resulting in different eye positions, it is necessary to re-determine the projection matrix according to the user's eye position when using the AR HUD system. For example, in an AR HUD system with a single-eye box, the driver may change; for another example, in an AR HUD system that supports a binocular box, the main driver or co-driver changes. In addition, due to unexpected deflections in the camera installation position or installation orientation, the AR HUD needs to be calibrated to ensure that the virtual information is displayed in the correct position in the user's field of view to avoid visual errors. Among them, position calibration refers to the process of accurately aligning the virtual image with the actual road environment.

[0037] Figure 1 An exemplary system architecture of an AR HUD system (Augmented Reality Head-Up Display) to which the present disclosure can be applied is shown.

[0038] like Figure 1 As shown, the system architecture may include an on-board display screen 130, an AR HUD (not shown in the drawings), and a controller (not shown in the drawings) that establishes communication connections with the on-board display screen and the AR HUD respectively; the system architecture may also include an on-board camera 120, a touch panel 140 ( Figure 1 The example shown is only installed on the steering wheel, and it can actually be installed on other vehicle components such as the dashboard, armrest box, etc.). The front windshield of the vehicle can be used as a projection screen for the AR HUD, that is, the projection screen 110, for displaying augmented reality information.

[0039] The vehicle-mounted camera 120 can be a panoramic camera on the roof, or it can be a camera installed on each side of the vehicle body. In some embodiments, the position and angle of the vehicle-mounted camera 120 can be adjusted as needed, and the position and angle of the vehicle-mounted camera 120 can be adjusted by voice commands, button commands, touch commands, etc. For example, a user sends a voice command "adjust the angle of the front camera upward 10 degrees" or "adjust the position of the front camera downward 1 centimeter." In other embodiments, such as a panoramic camera on the roof, a panoramic image around the vehicle body can be collected, and an image within a certain angle range in the panoramic image can be intercepted according to instructions for image display or corner point recognition.

[0040] The touch panel 140 can be installed on a vehicle component such as a dashboard or a steering wheel, so that the user can input touch commands to adjust the position of the calibration point. Exemplarily, the touch panel is composed of multiple piezoelectric vibrators, and multiple piezoelectric vibrators can be mounted on the back of the touch panel. When the surface of the touch panel is subjected to pressure, elastic waves will be generated on the surface of the touch panel. At this time, the elastic waves will be transmitted to different piezoelectric vibrators, and corresponding elastic waveforms will be picked up on different piezoelectric vibrators. The elastic wave shapes of these elastic waveforms are basically the same, the difference is that the arrival time points are different and the arrival waveform amplitudes are different.

[0041] In some embodiments, the touch position can be identified based on the TOF (Time of Flight) principle to obtain the touch command received by the touch panel 140: receiving voltage signals respectively collected by multiple piezoelectric vibrators; determining a characteristic time point corresponding to each of the voltage signals based on at least one voltage point with similarity in the multiple voltage signals; the characteristic time point is determined based on at least one time point corresponding to the at least one voltage point; determining at least three characteristic time point pairs from the multiple characteristic time points; determining the touch position information based on the relative time differences corresponding to the at least three characteristic time point pairs, and the position information of a pair of piezoelectric vibrators corresponding to each relative time difference.

[0042] In other embodiments, the touch position can be identified based on the relationship between the inverse of the detected voltage value and the distance through geometric calculation to obtain the touch command received by the touch panel 140: by receiving electrical signals respectively collected by multiple piezoelectric vibrators, and based on at least one electrical signal point with similarity in the multiple electrical signals, determining the first electrical signal point corresponding to each of the electrical signals, the first electrical signal point thus obtained can represent the signal value of the touch point collected by the piezoelectric vibrator at the first time point; at the same time, by determining the proportional relationship between multiple first touch distances based on the first electrical signal point corresponding to each of the multiple electrical signals, the first touch distance is the distance between the piezoelectric vibrator and the touch point at the first time point. Based on the position information of the plurality of piezoelectric vibrators, the position information of the touch point at the first time point is determined. In this way, the principle that the farther away from the touch point, the greater the attenuation of the mechanical elastic wave, the smaller the pressure induction of the mechanical elastic wave to the piezoelectric vibrator, and the smaller the corresponding electrical signal output can be taken into consideration. Combined with the position information of each piezoelectric vibrator, the positioning of the touch point can be achieved through geometric relationships. At the same time, since the piezoelectric vibrator can be perfectly integrated with the surface material of the vehicle and the piezoelectric vibrator is resistant to exposure, it has stable performance in the face of complex vehicle usage scenarios. The proportional relationship between the first touch distances is determined by the first electrical signal point, and then combined with the position information of the piezoelectric vibrator, the position information of the touch point can be accurately determined.

[0043] The vehicle display screen 130 is configured to display a calibration interface in response to receiving a calibration request, wherein the calibration interface includes an image of the area in front of the vehicle, such as Figure 1 As shown in 130; at least one corner point in the image is displayed on the calibration interface, such as Figure 1As shown by the solid dots in 131. The vehicle-mounted display screen can be various types of display screens, for example, it can be a display screen of a DVR (Digital Video Recorder), or it can be a central control screen, an instrument screen, or a co-pilot screen. It can also be an electronic device display screen that establishes a communication connection with the vehicle. The vehicle-mounted camera 120 and the vehicle-mounted display screen 130 can be connected by wired communication or wireless communication. For example, the image captured by the vehicle-mounted camera 120 is transmitted to the vehicle-mounted display screen 130 for display via WiFi, Bluetooth or Star Flash technology. In some embodiments, the vehicle-mounted display screen 130 can be a touch screen for receiving touch events and sending them to the controller to generate an adjustment instruction for adjusting the position of the calibration point, and at the same time setting the calibration interface to shield or not respond to the touch event.

[0044] The AR HUD is configured to display at least one calibration point corresponding to the at least one corner point, as shown in 131' in the figure, corresponding to the corner point 131. The correspondence between the corner points and the calibration points can be one-to-one, that is, one corner point corresponds to a calibration point at a corresponding position. For example, according to the image coordinates of one of the corner points displayed on the vehicle display screen, the display coordinates of the calibration point are obtained based on the current projection matrix conversion, and then the calibration point corresponding to the corner point is displayed at the AR HUD. It can be understood that the current projection matrix is ​​not suitable for the current scene due to user changes or camera offsets, and thus the displayed calibration points do not match or align with the corner points of the scene in front of the vehicle in the eyes of the current user.

[0045] Optionally, the correspondence between the corner points and the calibration points may also be a correspondence in number, that is, a number of corner points may correspond to the same number of calibration points. For example, if 5 corner points are identified, 5 calibration points are displayed. It can be understood that, as mentioned above, since the projection matrix itself has errors that need to be calibrated, directly displaying the corresponding number of calibration points can save the step of calculating the display coordinates of the calibration points and reduce the amount of calculation.

[0046] The controller is configured to receive a signal sent by a user through a vehicle display screen, a touch panel, a microphone or a button, and match the received signal with a preset calibration request signal. If the matching degree is greater than a predetermined threshold, it is considered that the user has sent a calibration request.

[0047] The controller is also configured to, in response to detecting that the user has initiated a calibration request, send an instruction to the vehicle camera 120 to capture an image, and after receiving the image captured by the vehicle camera 120, send the image to the vehicle display screen for display on the calibration interface. The controller is also configured to identify the corner point in the image, and instruct the vehicle display screen to display the corner point 131, and instruct the AR HUD to display the calibration point 131' corresponding to the corner point on the projection screen 110.

[0048] The user can also send a signal to adjust the position of the calibration point through the vehicle display screen 130, the touch pad 140 or a button, match the received signal with the preset adjustment instruction, and if the matching degree is greater than a predetermined threshold, adjust the position of the calibration point according to the adjustment instruction that successfully matches. The controller can also receive voice data from the microphone, convert the voice data into text, and match it with the preset adjustment instruction text. If the matching degree is greater than a predetermined threshold, adjust the position of the calibration point according to the adjustment instruction that successfully matches. Or match the voice data with the voice data of the stored adjustment instruction, and if the matching degree is greater than a predetermined threshold, adjust the position of the calibration point according to the adjustment instruction that successfully matches. For example, the user sitting in the driver's seat can see both the real scene in front of the car and the image displayed on the projection screen 110 that is the same as the interface displayed on the vehicle display screen 130. If the user sees that the calibration point 131' is inconsistent with the corner point position in the real scene in front of the car, a signal to adjust the position of the calibration point is sent through the vehicle display screen 130, the touch pad 140 or a button to control the calibration point 131' to move to a position consistent with the corner point position in the real scene in front of the car. For example, if the user sees with naked eyes that the calibration point 131 ′ is on the right side of the real corner point, the calibration point 131 ′ may be controlled to move to the left until the two points overlap. Figure 1 Only one corner point is drawn for demonstration. In actual applications, multiple corner points need to be detected. Only when the number of corner points that meet the preset conditions reaches a predetermined number can the homography matrix be solved and the calibration of the AR HUD system be completed.

[0049] The controller moves the at least one calibration point to the target position indicated by the adjustment instruction in accordance with the adjustment instruction, wherein the target position is a position where the at least one calibration point on the projection screen observed by the human eye is consistent with the at least one corner point displayed in the image; in response to the coordinate group set satisfying a preset condition, the AR HUD system is calibrated based on the coordinate group set, and the coordinate group set includes at least one coordinate group consisting of the coordinates of the at least one calibration point on the projection screen and the coordinates of the at least one corner point on the vehicle-mounted display screen.

[0050] The controller is also configured to calibrate the AR HUD system based on the coordinate set in response to the coordinate set satisfying a preset condition.

[0051] It should be noted that the method for calibrating the AR HUD system provided in the embodiments of the present disclosure is generally executed by a controller.

[0052] Continue to refer Figure 2 , a process 200 of an embodiment of a method for calibrating an AR HUD system according to the present disclosure is shown. The method for calibrating an AR HUD system comprises the following steps:

[0053] Step 201, in response to receiving a calibration request, displaying a calibration interface on a vehicle-mounted display screen.

[0054] In this embodiment, the vehicle is usually parked in places with complex scenes such as parking lots and communities, and various corners or inflection points can be found. When the user starts the AR HUD and finds that the virtual image displayed on the projection screen 110 cannot be accurately aligned with the surrounding actual environment or there is a deviation, a calibration request can be issued. The calibration request can be triggered by voice, touch, gesture, button, etc., and this application is not limited to this. For example, a calibration request button can be displayed on the vehicle display screen, and the vehicle display screen can display the calibration interface after the user clicks the calibration request button. For another example, the user can also send a calibration request by voice. Among them, the calibration interface may include an image of the area in front of the vehicle. Exemplarily, after receiving the calibration request, the controller sends an instruction to capture the image to the vehicle camera. After receiving the instruction, the vehicle camera captures the image of the area in front of the vehicle and returns the image to the controller, and the controller instructs the vehicle display screen to display the image containing the area in front of the vehicle on the calibration interface.

[0055] Step 202: display at least one corner point in the image on the calibration interface, and display at least one calibration point corresponding to the at least one corner point on the projection screen of the AR HUD system.

[0056] In this embodiment, the corner points may be automatically detected and identified from the image displayed on the calibration interface by a corner point detection algorithm, or may be manually marked on the calibration interface. Among them, the corner points and the calibration points may correspond one to one, and the corner points and the calibration points with a mapping relationship may be regarded as a group. In some embodiments, different groups of corner points and calibration points may be distinguished by different display modes, and the corner points and calibration points in the same group may be displayed in the same display mode. Specifically, different colors, icons, serial numbers and other different identifiers may be used to represent different groups of corner points and calibration points. For example, if corner point 1 is displayed in red, the corresponding calibration point 1 is also displayed in red. If corner point 2 is displayed with a triangular icon, the corresponding calibration point 2 is also displayed with a triangular icon. The display mode is not limited to one, and may be a combination of multiple modes. For example, corner point 1 and corner point 2 not only have different colors, but also different icons. Similarly, calibration point 1 and calibration point 2 not only have different colors, but also different icons. Therefore, when the corner points are very dense, different corner points and corresponding calibration points can be distinguished more clearly, and it is convenient for users to select calibration points.

[0057] In some embodiments, the coordinate system and the scale (for example, a small grid on the scale represents 1mm, and a large grid represents 1cm) can also be displayed on the projection screen of the ARHUD system; the calibration point and the coordinates of the calibration point are displayed on the projection screen of the ARHUD system. The coordinate system is used to indicate the direction of adjustment of the calibration point. For example, if the user sends an adjustment instruction to move 1mm to the left, the calibration point is moved 1mm to the left along the x-axis. It corresponds to moving a small grid on the scale. Optionally, the grid lines can also be displayed on the projection screen of the ARHUD system, which can more accurately determine the position of the calibration point, thereby improving the accuracy of the position adjustment of the calibration point. For example, when moving by voice command, key command, or touch command, if the moving distance is not indicated, the default step length is moved, that is, the distance of one grid in the grid. For example, if the user says "left", the default is to move one grid in the grid line, or the user presses the left shift key once, and the default is to move one grid in the grid line. The granularity of the grid can be adjusted as needed to control the step length of each movement.

[0058] In some optional implementations of this embodiment, displaying at least one corner point in the image on the calibration interface includes: receiving a corner point marking operation; and displaying the point in the image added by the corner point marking operation as a corner point. Receiving the corner point marking operation may include two ways: one is that the user manually adds a corner point in the image, and the other is that the controller automatically detects candidate corner points in the image for the user to select, and the candidate corner points selected by the user are used as the corner points displayed on the calibration interface.

[0059] The first way to receive corner point marking operations is that the user can directly select corner points from the image captured by the vehicle camera. For example, click on the corner point in the image and choose to set it as a corner point. Different numbers, colors, and icons can also be set for the corner points to distinguish them. The calibration interface can provide multiple corner point display options for users to choose.

[0060] In the second method of receiving the corner point marking operation, a corner point detection algorithm first detects multiple candidate corner points, and then the user selects a corner point from the multiple candidate corner points for marking.

[0061] Users can also manually modify the corner mark format, for example, changing the original red corner mark to black.

[0062] In some optional implementations of this embodiment, the at least one corner point is extracted from the image by a corner detection algorithm and displayed in the image. Exemplarily, the at least one corner point can be extracted from the image by a corner detection algorithm such as Harris corner detection algorithm, SIFT corner detection algorithm, FAST corner detection algorithm, LBP corner detection algorithm, KLT corner detection algorithm, and SUSAN corner detection algorithm and displayed in the image.

[0063] In practical applications, the selection of a suitable corner detection method is based on specific needs and image characteristics. Different corner detection methods have different computational complexity, robustness, and accuracy, and need to be selected based on actual conditions. In addition, corner detection methods usually need to be combined with other image processing techniques, such as image filtering, threshold processing, and post-processing, to improve detection results.

[0064] Optionally, the category of the object in the image can be determined by a pre-trained target detection model, and a corresponding corner detection algorithm can be selected according to the category of the object. For example, if lane lines are detected in the image, the Harris corner detection algorithm is used to detect the corners in the lane lines, and if vehicles are detected in the image, the SIFT corner detection algorithm is used to detect the corners in the vehicles.

[0065] The current vehicle location can also be obtained through positioning, and then the corner detection algorithm is selected according to the current vehicle location. For example, if the current vehicle is in a parking lot, the Harris corner detection algorithm is used to detect the corners in the parking lot. If the current vehicle is parked on the side of the road, the SIFT corner detection algorithm is used to detect the corners on the side of the road.

[0066] Optionally, the corner detection algorithm can be selected according to user preference. For example, if the user likes to calibrate the AR HUD system quickly, the FAST or Harris corner detection algorithm can be selected for corner detection. If the user likes to calibrate the AR HUD system with the highest accuracy, the SIFT corner detection algorithm can be selected for corner detection.

[0067] Optionally, the user can select the mode of automatic corner detection + manual filtering, and use detection algorithms that are not as accurate but fast, such as the FAST corner detection algorithm and the LBP corner detection algorithm. The user can also select the fully automatic corner determination mode, where the controller directly selects accurate corners without manual filtering by the user, and can select the Harris corner detection algorithm for corner detection.

[0068] The user can confirm the corner point marks one by one, or batch confirm all automatically detected corner point marks. The user can also delete the incorrectly detected corner points individually, and then confirm all the remaining corner points.

[0069] In some optional implementations of this embodiment, extracting the at least one corner point from the image by a corner point detection algorithm and displaying it in the image includes: extracting a group of candidate corner points from the image by a corner point detection algorithm; selecting at least one corner point that meets a predetermined quality requirement from the group of candidate corner points and displaying it in the image. The extracted corner points are screened, and corner points with higher quality are selected as much as possible. In some embodiments, the quality requirements may include strong significance, regular shape, uniform distribution, non-collinearity, etc. Significance refers to the recognition of the corner point in the image, that is, the property or ability to be easily distinguished, and the significance can be determined by calculating the contrast between the corner point pixel and other point pixels. Irregularly shaped corner points may be corner points caused by noise and can be filtered. For corner points that are too densely distributed, only a part of the corner points are selected, and the remaining corner points are filtered out, so that the screened corner points are distributed more evenly, thereby avoiding excessive concentration of corner points, resulting in poor calibration effect.

[0070] In some optional implementations of this embodiment, selecting at least one corner point that meets the predetermined quality requirements from the group of candidate corner points includes at least one of the following methods: selecting at least one corner point whose preset quality index value is higher than the corresponding second predetermined value; dividing the image into multiple regions and selecting the corner point with the highest preset quality index value in each region.

[0071] Exemplarily, the preset quality indicator values ​​may include response intensity value, edge contrast, shape regularity, uniformity, etc., which are not limited in the present application.

[0072] The method of selecting at least one corner point that meets the predetermined quality requirement from the set of candidate corner points may include the following four methods:

[0073] 1. Select at least one corner point whose preset quality index value is higher than the corresponding second predetermined value.

[0074] Corner points may be selected based on single quality index values ​​such as significance and shape regularity. In some embodiments, corner points with strong significance may be screened out based on criteria such as high response intensity value and high edge contrast. For example, the response intensity value of the corner point may be obtained by a corner point detection algorithm, and the corner points with response intensity values ​​higher than a second predetermined value may be selected, and the corner points with response intensity values ​​lower than or equal to the second predetermined value may be filtered. In other embodiments, the value may be 1 for corner points with regular shapes, and 0 for corner points with irregular shapes. Corner points with shape regularity less than 1 may be removed, i.e., the corner points with irregular shapes may be filtered.

[0075] In some other embodiments, the corner points may be selected based on quality index values ​​such as significance, shape regularity, and uniformity. For example, the weighted sum of the quality index values ​​may be calculated, and the corner points whose weighted sum is greater than a predetermined second value may be selected.

[0076] 2. Divide the image into multiple regions and select the corner point with the highest preset quality index value in each region.

[0077] For example, the image can be divided into 9 grids, and the corner point with the highest preset quality index value is selected in each grid, so that the selected corner points are distributed in 9 areas and will not be concentrated in a certain area. Specifically, through non-maximum suppression, the corner points that are not the most prominent in the local area can be removed, thereby retaining the most prominent corner points, making the distribution of the selected corner points more uniform.

[0078] 3. First, select at least one corner point whose preset quality index value is higher than the corresponding second predetermined value as a candidate corner point, and then divide the image into multiple regions and select the candidate corner point with the highest preset quality index value in each region.

[0079] At least one corner point with a preset quality index value higher than the corresponding second preset value may be selected as a candidate corner point, and then the image may be divided into multiple regions and the corner point with the highest preset quality index value among the candidate corner points in each region may be selected. This can ensure the quality of the selected corner points and avoid excessive concentration of corner points. For example, a group of corner points with significance greater than the preset second value may be selected first, and then the image may be divided into 9 grids, and the corner point with the highest significance may be selected from each grid.

[0080] Different quality indicators can be used in combination in the two screening processes. For example, a group of corner points with uniformity greater than a predetermined second value are first selected, and then the image is divided into 9 grids, and the corner point with the highest significance is selected from each grid.

[0081] By combining two different screening methods, the accuracy of corner points can be improved, thereby improving the accuracy of AR HUD calibration.

[0082] 4. First, divide the image into multiple regions and select a predetermined number of corner points with the highest preset quality index values ​​in each region as candidate corner points, and then select at least one candidate corner point whose preset quality index value is higher than the corresponding second predetermined value.

[0083] For example, the image is divided into 9 grids, and the 10 most significant corner points are selected from each grid as candidate corner points. Then, at least one candidate corner point is selected whose significance is higher than the corresponding second predetermined value.

[0084] Different quality indicators can be used in combination in the two screening processes. For example, the image is first divided into 9 grids, and the 10 most significant corner points are selected from each grid. Then, the corner points whose uniformity is greater than a predetermined second value are selected from these 90 corner points.

[0085] By combining two different screening methods, the accuracy of corner points can be improved, thereby improving the accuracy of AR HUD calibration.

[0086] Step 203: In response to receiving an adjustment instruction for adjusting the position of at least one calibration point, move the at least one calibration point to a target position indicated by the adjustment instruction.

[0087] In this embodiment, the adjustment instruction may include at least one of the following: voice instructions, button instructions, and touch instructions. For example, the calibration points are all marked with serial numbers, and the coordinate system and / or scale values ​​are displayed on the projection screen of the AR HUD system. Then, you can voice input "move calibration point No. 1 1mm to the left". You can also select the calibration point through button instructions or touch instructions, and control the calibration point to move up, down, left and right. Among them, the target position is the position where the human eye observes that at least one calibration point on the projection screen is consistent with at least one corner point displayed in the image. For example, the human eye can include the field of view of the main driver and the field of view of the co-driver, thereby adapting to the calibration requirements of the AR HUD system with a binocular box design.

[0088] Step 204 , in response to the coordinate group set satisfying a preset condition, calibrate the AR HUD system based on the coordinate group set.

[0089] In this embodiment, the preset conditions include: the number of coordinate groups is greater than or equal to the first predetermined value and the adjustment instruction is executed. Among them, the coordinate group set includes at least one coordinate group consisting of the coordinates of at least one calibration point on the projection screen and the coordinates of at least one corner point on the vehicle display screen. Based on the coordinate group set, the coordinate conversion matrix from the vehicle display screen to the projection screen is obtained to calibrate the AR HUD system. For example, the corner points displayed on the vehicle display screen and the corresponding calibration points on the projection screen are marked with the same color, and the coordinates of a pair of points of the same color can be marked as a coordinate group. Similarly, the corner points displayed on the vehicle display screen and the corresponding calibration points on the projection screen are marked with the same serial number, and the coordinates of a pair of points with the same serial number can be marked as a coordinate group. The corner points displayed on the vehicle display screen and the corresponding calibration points on the projection screen are marked with the same icon, and the coordinates of a pair of points of the same icon can be marked as a coordinate group. If the number of coordinate groups reaches the predetermined requirement (for example, 8), the coordinate conversion matrix from the vehicle display screen to the projection screen can be obtained to calibrate the AR HUD system.

[0090] In some optional implementations of the present embodiment, the AR HUD system is calibrated based on the coordinate group set, including: calculating the homography matrix based on the coordinate group set to obtain the coordinate transformation matrix; calibrating the AR HUD system based on the coordinate transformation matrix. For example, the homography matrix can be solved by the epipolar geometry method. Epipolar geometry describes the intrinsic projective relationship between two views, which is independent of the external scene and only depends on the camera's internal parameters and the relative posture between the two views. Epipolar geometry is actually the "epipolar geometry between two images". It is the geometry of the intersection of the image plane and the plane bundle with the baseline as the axis (the baseline here refers to the straight line connecting the center of the camera). Homography, also called projective transformation, is an important knowledge point in computer vision. Projective transformation is a reversible mapping between two 2D projective planes that preserves linearity.

[0091] The method provided by the above embodiment of the present disclosure does not require professional calibration equipment. The driver can complete the calibration only through the vehicle itself, and the function is simple and easy to use.

[0092] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the method for calibrating an AR HUD system according to this embodiment. Figure 3 In the application scenario, the four corner points of the object in the image in front of the vehicle are displayed on the car display screen, and they are numbered respectively. The four calibration points of the object are also displayed on the AR HUD projection screen, represented by solid circles. The objects seen by the human eye on the AR HUD projection screen are represented by solid lines. It can be seen that there is an offset between the calibration points and the corner points of the object observed by the human eye. The user can move the calibration points to a position consistent with the corner points of the object observed by the human eye. After finding multiple sets of corresponding coordinate points, the homography matrix is ​​obtained by the epipolar geometry method, and the coordinate conversion matrix from the car display screen to the AR HUD screen can be obtained.

[0093] Further references Figure 4 , which shows a process 400 of another embodiment of a method for calibrating an AR HUD system. The process 400 of the method for calibrating an AR HUD system includes the following steps:

[0094] Step 401, in response to receiving a calibration request, displaying a calibration interface on the vehicle display screen.

[0095] Step 402: display at least one corner point in the image on the calibration interface, and display at least one calibration point corresponding to the at least one corner point on the projection screen of the AR HUD system.

[0096] Step 403: In response to receiving an adjustment instruction for adjusting the position of at least one calibration point, move the at least one calibration point to a target position indicated by the adjustment instruction.

[0097] Steps 401-403 are substantially the same as steps 201-203, and thus are not described in detail herein.

[0098] Step 404 , in response to the coordinate set not satisfying a preset condition within a predetermined time, instructing the vehicle to adjust its orientation to a target orientation, or to move to a recommended position.

[0099] In this embodiment, if the number of corner points detected within the predetermined time is too small or the number of corner points is sufficient but the number of corner points that meet the quality requirements is too small, and the homography matrix cannot be solved, new corner points can be found in other places, for example, by changing the direction of the vehicle head or moving to other locations.

[0100] The target orientation is determined based on the following steps: acquiring a peripheral image of the vehicle to detect corner points, the peripheral image including images of areas other than the area in front of the vehicle; determining an image area in which the number of corner points in the peripheral image is greater than or equal to the first predetermined value and is evenly distributed; and determining the target orientation based on a shooting angle corresponding to the image area.

[0101] In some embodiments, for example, the vehicle-mounted camera that captures images is a roof-mounted panoramic camera, and the image angle in front of the vehicle body is 0°, the target orientation can be determined according to the direction indicated by the angular bisector of the angle range corresponding to the image area that meets the quality requirements in the captured panoramic image. Specifically, the corner points can be detected as a whole in the panoramic image, and the image area with the largest number of corner points that meet the quality requirements can be selected in the panoramic image within a preset angle range (for example, 90°). The image angle in front of the vehicle body is 0°, and the direction (for example, 75°) of the angular bisector of the angle range (for example, 30° to 120°) corresponding to the image area is determined as the target orientation (for example, indicating that the front direction of the vehicle turns 75° clockwise).

[0102] In other embodiments, for example, if the vehicle-mounted camera that captures images is a camera installed on each side of the vehicle body, the target orientation can be determined based on the shooting angle of the camera that captures the image that meets the quality requirements. Specifically, corner points are detected from multiple images captured by each camera, and the image with the largest number of corner points that meets the quality requirements is selected, and the target orientation is determined based on the shooting angle of the camera that captured the image.

[0103] After the vehicle changes direction, the user can re-adjust the calibration points on the projection screen according to the real scene in front.

[0104] The recommended location is determined based on the following steps: the address closest to the current location of the vehicle or the address that the vehicle is expected to pass through in the future trip is selected from a pre-stored recommended address library as the recommended location, and the recommended address library includes at least one address suitable for calibrating the AR HUD system. The addresses in the recommended address library can be addresses that the user has successfully calibrated before, addresses that other users have calibrated, or calibration addresses recommended by the manufacturer.

[0105] If the number of corner points detected by the images collected around the vehicle is too small or the number of corner points is sufficient but the number of corner points that meet the quality requirements is too small, and the homography matrix cannot be solved, the vehicle can be moved to another location to re-collect images for corner point detection. In some embodiments, some addresses where corner points are easily detected can be pre-selected and stored in a recommended address library, such as a parking lot. The user can be recommended to drive to the parking lot closest to the current location. In other embodiments, the addresses in the recommended address library that the vehicle is expected to pass through in the future journey can be used as the recommended location. After arriving at the recommended location, the vehicle direction can still be re-determined or changed.

[0106] The method of changing the vehicle orientation and the method of recommending the position can be combined alternately until enough corner points that meet the quality requirements are detected.

[0107] Step 405 , in response to the coordinate group set satisfying a preset condition, calibrate the AR HUD system based on the coordinate group set.

[0108] Step 405 is substantially the same as step 204 and thus will not be described in detail.

[0109] The method provided by the above-mentioned embodiments of the present disclosure can find a location more suitable for calibration by changing the orientation or position of the vehicle to obtain sufficient or better quality corner point coordinates, which can reduce the situation where calibration cannot be performed due to too few corner points at the current position, or because better quality corner points can be selected, the accuracy of calibration of the AR HUD system can be improved.

[0110] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0111] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0112] An electronic device comprises: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200 or 400.

[0113] A computer-readable medium stores a computer program, wherein the computer program implements the method described in process 200 or 400 when executed by a processor.

[0114] A vehicle includes an on-board display screen, an AR HUD, and a controller, an on-board camera, a touch screen or a touch pad that respectively establish communication connections with the on-board display screen and the AR HUD. The controller is used to implement the method described in process 200 or 400.

[0115] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0116] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0117] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the road area planning method. For example, in some embodiments, the road area planning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the road area planning method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the road area planning method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for calibrating an AR HUD system, comprising: In response to receiving the calibration request, displaying a calibration interface on the vehicle display screen, wherein the calibration interface includes an image of the area in front of the vehicle; Displaying at least one corner point in the image on the calibration interface, and displaying at least one calibration point corresponding to the at least one corner point on the projection screen of the AR HUD system; In response to receiving an adjustment instruction for adjusting the position of the at least one calibration point, moving the at least one calibration point to a target position indicated by the adjustment instruction, wherein the target position is a position where the at least one calibration point on the projection screen is consistent with at least one corner point displayed in the image as observed by human eyes; In response to the coordinate group set satisfying a preset condition, the AR HUD system is calibrated based on the coordinate group set, wherein the coordinate group set includes at least one coordinate group consisting of the coordinates of the at least one calibration point on the projection screen and the coordinates of the at least one corner point on the vehicle display screen.

2. The method according to claim 1, wherein: The adjustment instruction includes at least one of the following: a voice instruction, a key instruction, and a touch instruction.

3. The method according to claim 2, wherein: The touch control instruction is obtained via a touch screen or a touch panel, and the touch panel is composed of a plurality of piezoelectric vibrators.

4. The method according to claim 1, wherein: The preset condition includes: the number of the coordinate groups is greater than or equal to a first predetermined value and the adjustment instruction is executed.

5. The method according to claim 1, wherein: The calibrating the AR HUD system based on the coordinate group set includes: Calculate the homography matrix based on the coordinate group set to obtain the coordinate transformation matrix; The AR HUD system is calibrated based on the coordinate transformation matrix.

6. The method according to claim 1, wherein: The displaying at least one corner point in the image on the calibration interface includes: Receive corner point marking operation; Points in the image added by the corner point marking operation are displayed as corner points.

7. The method according to claim 1, wherein: The displaying at least one corner point in the image on the calibration interface includes: The at least one corner point is extracted from the image by a corner point detection algorithm and displayed in the image.

8. The method according to claim 7, wherein: The step of extracting the at least one corner point from the image by using a corner point detection algorithm and displaying the corner point in the image comprises: Extracting a set of candidate corner points from the image by a corner detection algorithm; At least one corner point that meets a predetermined quality requirement is selected from the group of candidate corner points and displayed in the image.

9. The method according to claim 8, wherein: The selecting at least one corner point satisfying a predetermined quality requirement from the set of candidate corner points comprises at least one of the following methods: selecting at least one corner point whose preset quality index value is higher than a corresponding second predetermined value; The image is divided into a plurality of regions and a corner point having the highest value of the preset quality index in each region is selected.

10. The method according to any one of claims 1 to 9, wherein: The method further comprises: In response to the coordinate set not satisfying a preset condition within a predetermined time, the vehicle is instructed to adjust its orientation to a target orientation, or to move to a recommended position.

11. The method according to claim 10, wherein: The target orientation is determined based on the following steps: Acquire a peripheral image of the vehicle to detect corner points, wherein the peripheral image includes an image of an area other than an area in front of the vehicle; Determine an image region in which the number of corner points in the peripheral image is greater than or equal to the first predetermined value and is evenly distributed; The target orientation is determined according to a shooting angle corresponding to the image area.

12. The method according to claim 10, wherein: The recommended position is determined based on the following steps: The address closest to the current position of the vehicle or the address that the vehicle is expected to pass through in its future journey is selected from a pre-stored recommended address library as the recommended location, and the recommended address library includes at least one address suitable for calibrating the AR HUD system.

13. An AR HUD system, comprising: An on-board display screen, an AR HUD, and a controller that establishes communication connections with the on-board display screen and the AR HUD respectively; wherein, The vehicle-mounted display screen is configured to display a calibration interface in response to receiving a calibration request, wherein the calibration interface includes an image of the area in front of the vehicle; and display at least one corner point in the image on the calibration interface; The AR HUD is configured to display at least one calibration point corresponding to the at least one corner point; A controller is configured to, in response to receiving an adjustment instruction for adjusting the position of the at least one calibration point, move the at least one calibration point to a target position indicated by the adjustment instruction, wherein the target position is a position where the at least one calibration point on the projection screen is consistent with the at least one corner point displayed in the image as observed by the human eye; and in response to a coordinate group set satisfying a preset condition, calibrate the AR HUD system based on the coordinate group set, wherein the coordinate group set includes at least one coordinate group consisting of the coordinates of the at least one calibration point on the projection screen and the coordinates of the at least one corner point on the vehicle-mounted display screen.

14. An electronic device for calibrating an AR HUD system, comprising: one or more processors; a storage device having one or more computer programs stored thereon, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

15. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

16. A vehicle comprising the electronic device according to claim 14.