Augmented Reality-Based Information Display Method, System, Device and Projection Equipment

Abstract

An embodiment of the present application discloses an information display method, system, device, projection device, and storage medium based on augmented reality. The method includes: obtaining real-scene information collected by a scene perception device; obtaining a target display area determined based on the first spatial pose of a user; obtaining a coordinate transformation rule corresponding to mapping the real-scene information to the target display area; generating driving guidance information based on the real-scene information; and displaying the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule. This method displays the driving guidance information generated based on the real-scene information at a corresponding position in the target display area determined based on the first spatial pose of the user, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving, reducing the fatigue caused by frequent line-of-sight conversion when viewing road conditions and driving guidance information such as navigation, and improving driving safety and comfort.

Description

Technical Field

[0001] This application relates to the technical field of coordinate transformation, and more specifically, to an information display method, system, device, projection device, and storage medium based on augmented reality. Background Art

[0002] HUD (head up display) is a head-up display, which can display important information on a transparent glass in front of the line of sight. It was first applied to fighter jets, and its main purpose is to enable pilots not to need to frequently concentrate on looking down at the data on the instrument panel, thereby avoiding the situation where pilots cannot observe the environmental information in the flight front area when viewing the data on the instrument panel. In order to reduce accidents caused by users looking down at the instrument panel or the center console, HUD has been introduced from aircraft to the automotive field.

[0003] However, the existing HUD display information method is relatively single. Taking driving a car as an example, with the addition of more assisted driving information such as road conditions, navigation, and danger warnings, when a user with a relatively tall height drives a vehicle customized for a user with an average height (lower than the relatively tall height), it may cause a difference between the position where the user with a relatively tall height sees the environmental information displayed by the vehicle's HUD and the actual position of the environmental information, bringing many inconveniences to the user's driving and reducing the user experience. Summary of the Invention

[0004] In view of the above problems, this application proposes an information display method, system, device, projection device, and storage medium based on augmented reality to improve the above problems.

[0005] In a first aspect, an embodiment of this application provides an information display method based on augmented reality. The method includes: obtaining real scene information collected by a scene perception device; obtaining a target display area determined based on the first spatial pose of a user; obtaining a coordinate transformation rule corresponding to mapping the real scene information to the target display area; generating driving guidance information based on the real scene information; and displaying the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule.

[0006] In a second aspect, an embodiment of the present application provides an augmented reality information display device, which includes an image sensing module, a coordinate transformation module, and a display module: The image sensing module is configured to obtain real-world information collected by an image sensing device; The coordinate transformation module is configured to obtain a target display area determined based on the first spatial pose of the user; The coordinate transformation module is further configured to obtain a coordinate transformation rule corresponding to mapping the real-world information to the target display area; The display module is configured to generate driving guidance information based on the real-world information; The display module is further configured to display the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule.

[0007] In a third aspect, an embodiment of the present application provides an in-vehicle information display system based on augmented reality, the system includes: A scene sensing device, configured to collect real-world information of the external environment of the vehicle; An image processing device, configured to obtain the real-world information collected by the scene sensing device, obtain a target display area determined based on the first spatial pose of the user, obtain a coordinate transformation rule corresponding to mapping the real-world information to the target display area, generate driving guidance information based on the real-world information, and generate target position coordinates for displaying the driving guidance information in the target display area based on the coordinate transformation rule; A HUD display device, configured to display the driving guidance information at the target position coordinates in the target display area.

[0008] In a fourth aspect, an embodiment of the present application provides a projection device, including a data collection module, a projection module, one or more processors, and a memory; One or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs are configured to execute the method described in the first aspect above.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program code is stored, and when the program code runs, it executes the method described in the first aspect above.

[0010] An information display method, system, device, projection device, and storage medium based on augmented reality provided by this application obtain the real-scene information collected by an image sensing device, then obtain the target display area determined based on the first spatial pose of the user, then obtain the coordinate transformation rule corresponding to mapping the real-scene information to the target display area, then generate driving guidance information based on the real-scene information, and then display the driving guidance information at the corresponding position in the target display area based on the coordinate transformation rule. Thus, through the above method, the driving guidance information generated based on the real-scene information is displayed at the corresponding position in the target display area determined based on the first spatial pose of the user through the coordinate transformation rule, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving, without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue caused by frequent line-of-sight conversion when viewing road conditions and navigation and other driving guidance information, and improving the safety and comfort of driving. Brief Description of the Drawings

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of this application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0012] Figure 1 Fig. shows the flowchart of a method for an information display method based on augmented reality proposed in an embodiment of this application.

[0013] Figure 2 Fig. shows the structural example diagram of an in-vehicle information display system based on augmented reality applicable to the information display method based on augmented reality provided in this embodiment.

[0014] Figure 3 Fig. shows an example diagram of displaying driving guidance information through the in-vehicle information display system based on augmented reality proposed in this application in a dangerous scenario.

[0015] Figure 4 Fig. shows the flowchart of a method for an information display method based on augmented reality proposed in another embodiment of this application.

[0016] Figure 5 Fig. shows an example diagram of the display effect of the in-vehicle information display system based on augmented reality provided in this embodiment.

[0017] Figure 6 Fig. shows the flowchart of a method for an information display method based on augmented reality proposed in another embodiment of this application.

[0018] Figure 7Shows an example diagram of the target display area determined based on the user's first spatial pose provided in this embodiment.

[0019] Figure 8 Shows another example diagram of the target display area determined based on the user's first spatial pose provided in this embodiment.

[0020] Figure 9 Shows an example diagram of the target display area determined based on the user's spatial pose provided in this embodiment.

[0021] Figure 10 Shows a flowchart of a method for information display based on augmented reality proposed in another embodiment of this application.

[0022] Figure 11 Shows an example diagram of the processing process of the information display method based on augmented display proposed in this embodiment.

[0023] Figure 12 Shows a structural block diagram of an information display device based on augmented reality proposed in an embodiment of this application.

[0024] Figure 13 Shows a structural block diagram of a projection device for executing an information display method based on augmented reality according to an embodiment of this application.

[0025] Figure 14 Shows a storage unit for storing or carrying program codes for implementing an information display method based on augmented reality according to an embodiment of this application. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. It is obvious that the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in this application belong to the scope of protection of this application.

[0027] HUD (head up display) is a head-up display, which can display important information on a transparent glass in front of the line of sight. It was first applied to fighter planes, and its main purpose is to enable pilots not to frequently concentrate on looking down at the data on the instrument panel, thus avoiding the situation where pilots cannot observe the environmental information in the flight front area when viewing the data on the instrument panel. In order to reduce accidents caused by users looking down at the instrument panel or the center console, HUD has been introduced from airplanes to the automotive field.

[0028] HUDs are mainly divided into rear-mounted (also known as Combine HUD, C-type HUD) and front-mounted (also known as Windshield HUD, W-type HUD). Among them, the front-mounted HUD uses the windshield as a combiner, projecting the content required by the driver onto the front windshield through the optical system. The human eye can observe the HUD virtual image and the external scene at the same time through the windshield within the eye level range, thereby improving driving safety and driving comfort. However, some existing HUD devices only display virtual information in front of the driver's line of sight and are not integrated with the real environment. With the addition of more auxiliary driving information such as road conditions, navigation, and hazard warnings, this mismatch between virtual content and real scenes will instead distract the driver's attention.

[0029] Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world.

[0030] As a way, with the development of autonomous driving and augmented reality and mixed reality technologies, AR technology can be introduced into the HUD field. AR-HUD, through the combination of AR technology and front-mounted HUD, can solve the problem of separation and mismatch between virtual information and actual scenes in traditional HUD, and improve driving safety and comfort while enriching the HUD display content. However, the existing HUD display information method is relatively simple. Taking car driving as an example, with the addition of more auxiliary driving information such as road conditions, navigation and hazard warnings, when a taller user drives a vehicle customized for users of average height (lower than taller height), the position of the environmental information displayed by the taller user on the vehicle's HUD may be different from the actual position of the environmental information, causing many inconveniences to the user's driving and reducing the user experience.

[0031] Therefore, in order to improve the above problems, the inventor proposed the method provided in the present application, which can obtain real-scene information collected by an image perception device, and then obtain a target display area determined based on the user's first spatial posture, and then obtain a coordinate transformation rule corresponding to the target display area to map the real-scene information, and then generate driving guidance information based on the real-scene information, and then display the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule. This realizes that the driving guidance information generated based on the real-scene information is displayed at a corresponding position in the target display area determined based on the user's first spatial posture through a coordinate transformation rule, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue of frequent switching of vision caused by checking road conditions and driving guidance information such as navigation, and improving driving safety and comfort.

[0032] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0033] Please refer to Figure 1 , which is a flowchart of a method for information display based on augmented reality provided by an embodiment of the present application. The method of this embodiment can be executed by a device for processing real-scene information based on augmented reality, and the device can be implemented in a hardware and / or software manner. The method includes:

[0034] Step S110: Obtain real-scene information collected by a scene perception device.

[0035] Among them, the real-scene information in the embodiment of the present application can be real-scene information corresponding to multiple scenarios. Optionally, the multiple scenarios can include, but are not limited to, driving scenarios, tourism scenarios, and outdoor activity scenarios, etc. For example, if it is a driving scenario, the real-scene information can include lanes, signs, dangerous pedestrians (such as vulnerable groups like blind people, elderly people walking alone, pregnant women or children, etc.) and vehicles, etc.; if it is a tourism scenario, the real-scene information can include tourist destination signs, tourist routes, tourist attraction information, and tourist attraction weather information, etc.; if it is an outdoor activity scenario, the real-scene information can include the current location information and nearby convenience store information, etc.

[0036] Optionally, the scene perception device can include sensing devices such as lasers and lidars, and can also include image acquisition devices such as cameras (including monocular cameras, binocular cameras, and RGB-D cameras, etc.). As a way, the real-scene information corresponding to the current scene can be obtained through the scene perception device. For example, assuming that the current scene is a driving scene and the scene perception device is a camera, the camera can be installed on the vehicle (optionally, the installation position can be adjusted according to the vehicle model structure or actual needs), so that the camera can obtain real-scene information related to driving in real time. Among them, the acquisition principle and implementation of the real-scene information collected by the scene perception device (including lasers, lidars or cameras) can refer to related technologies and will not be elaborated here.

[0037] Step S120: Obtain a target display area determined based on the first spatial pose of the user.

[0038] Optionally, the first spatial pose of the user can be the sitting posture of the user in the driving state, or the sitting posture after adjusting the seat (here it can be the first time the current user adjusts the seat). It can be understood that different sitting postures of the user correspond to different spatial poses. As a way, the sitting posture state of the user after adjusting the seat can be used as the first spatial pose of the user.

[0039] In this embodiment, the target display area is an area for displaying virtual image information corresponding to real-scene information. Taking a driving scenario as an example, the target display area may be an area on the windshield of a vehicle for displaying projected virtual image information corresponding to real-scene information. Optionally, the target display areas corresponding to different spatial poses of the same user may be different, and the target display areas corresponding to the spatial poses of different users may be different.

[0040] To eliminate the display difference between the position of the virtual image information displayed corresponding to the real-scene information and the actual position of the real-scene information, as a method, the target display area determined based on the first spatial pose of the user can be obtained, so that the virtual image information corresponding to the real-scene information can be displayed in the target display area, reducing the aforementioned display difference, and further improving the accuracy of the display position of the virtual image information corresponding to the real-scene information.

[0041] Step S130: Obtain the coordinate transformation rule corresponding to the mapping of the real-scene information to the target display area.

[0042] Among them, the coordinate transformation rule can be used to map the coordinates of the real-scene information to the corresponding coordinates of the target display area. As a method, when the real-scene information and the target display area are obtained, the coordinate transformation rule corresponding to the mapping of the real-scene information to the target display area can be obtained, so that the driving guidance information corresponding to the real-scene information can be accurately displayed at the corresponding position in the target display area based on the coordinate transformation rule in the subsequent process.

[0043] Optionally, the coordinate transformation rule may include a first transformation matrix and a second transformation matrix. Among them, the first transformation matrix can be used to determine the reference world coordinates corresponding to the coordinates of the real-scene information collected by the scene perception device, and the second transformation matrix can be used to convert the reference world coordinates into the view coordinates within the target display area. Among them, the reference world coordinates can be understood as the relative position coordinates of the real-scene information in the established coordinate system corresponding to the scene perception device. Optionally, the reference world coordinates in this embodiment can be understood as the world coordinates relatively stationary with respect to the vehicle. The view coordinates can be understood as the relative position coordinates of the reference world coordinates in the coordinate system corresponding to the target display area.

[0044] As a realization method, the first transformation matrix and the second transformation matrix can be obtained, and then the product of the parameters represented by the first transformation matrix and the parameters represented by the second transformation matrix is used as the coordinate transformation rule corresponding to the mapping of the real-scene information to the target display area.

[0045] Optionally, the first transformation matrix may include a first rotation matrix and a first translation vector. The first rotation matrix may be used to rotate the coordinates of the real-world information collected by the scene perception device, and the first translation vector may be used to translate the coordinates. As a way, the reference world coordinates corresponding to the coordinates of the real-world information collected by the scene perception device may be determined based on the first rotation matrix and the first translation vector.

[0046] Optionally, the second transformation matrix may include a view matrix and a projection matrix. The projection matrix may be used to determine the mapping range for mapping the real-world information to the target display area, and the view matrix may be used to determine the relative position of the driving guidance information (which can be understood as the aforementioned virtual image information corresponding to the real-world information) displayed within this mapping range. As a way, the reference world coordinates may be converted into view coordinates within the target display area based on this mapping range and this relative position. The view matrix may include a second rotation matrix and a second translation vector. The second rotation matrix may be used to rotate the reference world coordinates, and the second translation vector may be used to translate the reference world coordinates; the projection matrix may include a field-of-view angle parameter, and the field-of-view angle may include a horizontal field-of-view angle and a vertical field-of-view angle.

[0047] The following takes the driving scenario as an example to give an exemplary description of this embodiment:

[0048] Please refer to Figure 2 , which is a structural example diagram of an in-vehicle information display system based on augmented reality applicable to the augmented reality-based information display method provided in this embodiment. As shown in Figure 2 , the in-vehicle information display system based on augmented reality may include a scene perception device, an image processing device, and a HUD display device. The scene perception device may be used to collect real-world information about the external environment of the vehicle. The image processing device may be used to obtain the real-world information collected by the scene perception device, obtain the target display area determined based on the user's first spatial pose, obtain the coordinate transformation rule for mapping the real-world information to the target display area, generate driving guidance information based on the real-world information, and generate the target position coordinates for displaying the driving guidance information in the target display area based on the coordinate transformation rule. The HUD display device may be used to display the driving guidance information at the target position coordinates in the target display area.

[0049] The image processing device may be a processor chip of the vehicle-mounted system, or a processing chip of an independent in-vehicle computer system, or a processor chip integrated in the scene perception device (such as a lidar), etc., which is not limited herein.

[0050] In one implementation, the in-vehicle information display system may include a vehicle, a driver, a scene perception device, an image processing device, and a HUD display device with AR-HUD function. As an implementation, the scene perception device may be installed on the vehicle and can obtain driving-related scene information (which can also be understood as the aforementioned real scene information). The driver sits in the driver's seat of the vehicle, and the HUD display device is installed on the front windshield of the vehicle. The position of the HUD display device can be adjusted so that the driver's eyes can see the entire virtual image corresponding to the driving scene information. The image processing device can convert the real scene information collected by the scene perception device into an image fused with the real scene information of the real scene and send it to the HUD display device for display.

[0051] As a way, after the scene perception device acquires the real scene information, it can obtain the position coordinates of the real scene information in the world coordinate system (such as Figure 2 the O-xyz shown) based on position acquisition methods such as GPS positioning. Subsequently, it can select the world coordinate origin and the coordinate axis directions based on the traveling direction of the vehicle, and determine the reference world coordinate system relatively static with the vehicle according to the world coordinate origin and the coordinate axis directions. By determining the reference world coordinate system relatively static with the vehicle, the reference world coordinates corresponding to the coordinates of the real scene information in the reference world coordinate system can be obtained. Among them, the selection method of the world coordinate origin and the coordinate axis directions can refer to related technologies and will not be elaborated here. It should be noted that the reference world coordinate system can be understood as the coordinate system obtained after rotating and / or translating the world coordinate system.

[0052] For example, as an implementation, when the reference world coordinate system relatively static with the vehicle is determined, the spatial poses of the scene perception device and the driver's eyes in the reference world coordinate system can be obtained. In this way, the perception module transformation matrix M (i.e., the aforementioned first transformation matrix) can be calculated according to the spatial pose of the scene perception device in the reference world coordinate system. Exemplarily, it can be assumed that the process of changing the world coordinate system to the reference world coordinate system includes the first rotation matrix (which can also be understood as the total rotation matrix of the scene perception device) R M and the first translation vector T M , optionally, the relationship between the perception module transformation matrix M and the first rotation matrix R M and the first translation vector T M can be expressed as:

[0053]

[0054] Among them,

[0055]

[0056]

[0057] Among them, R Mx and R My and R Mz are respectively the rotation matrices of the perception module transformation matrix M around the x-axis, y-axis, and z-axis of the world coordinate system, and the Euler angles of rotation are α M , β M , γ M . (T Mx , T My , T Mz ) is the coordinate of the real scene information in the reference world coordinate system.

[0058] Optionally, the HUD display device in this embodiment can be an inverse virtual camera model. Assuming that the pose of the virtual camera is the same as the pose of the driver's eyes, in this way, the aforementioned second transformation matrix can be calculated based on the relevant parameters of the HUD display device and the pose of the virtual camera. The second transformation matrix (which can also be understood as the imaging matrix of the virtual camera here) C can include the view matrix V and the projection matrix P, and the relationship among the three can be expressed as:

[0059] C = PV.

[0060] Among them, the view matrix V can include the second rotation matrix R H T and the second translation vector T H . The second rotation matrix R H T can be understood as the total rotation matrix that converts the coordinate of the real scene information in the reference world coordinate system into the coordinate in the coordinate system where the HUD display device is located. Optionally, the second translation vector T H can be used to translate the reference world coordinate or the coordinate in the coordinate system where the HUD display device is located. Optionally, the relationship between the view matrix V and the second rotation matrix R H T and the second translation vector T H can be expressed as:

[0061]

[0062] Among them,

[0063]

[0064]

[0065] Among them, RH x , RH y , RH zIt can be understood as rotation matrices respectively around the x-axis, y-axis, and z-axis of the reference world coordinate system, and the Euler angles of rotation are α H , β H , γ H , (T Hx , T Hy , T Hz ) are the coordinates of the pose of the virtual camera in the reference world coordinate system.

[0066] Optionally, the relational expression satisfied by the projection matrix P can be:

[0067]

[0068] Optionally, the projection matrix includes a field of view angle parameter. The field of view angle can include a horizontal field of view angle and a vertical field of view angle. hFOV and vFOV can respectively represent the horizontal field of view angle and the vertical field of view angle. n and f can be understood as the distances of the assumed near and far clipping planes. For example, as Figure 2 shown, the distance from the plane where the virtual image O h is located to the center of the vehicle front windshield can be understood as the distance of the assumed near clipping plane, and the distance from the plane where the virtual image Ow is located to the center of the vehicle front windshield can be understood as the distance of the assumed far clipping plane. It can be understood that this is only an example for illustration here, and the distances of the near and far clipping planes can be adjusted according to actual needs during actual implementation.

[0069] As a way, when the first transformation matrix and the second transformation matrix are obtained, the product of the parameters represented by the first transformation matrix and the parameters represented by the second transformation matrix can be obtained as the coordinate transformation rule corresponding to mapping the real scene information to the target display area. That is, in this way, the coordinate transformation rule can be expressed as:

[0070] F = CM = PVM.

[0071] Step S140: Generate driving guidance information based on the real scene information.

[0072] The driving guidance information in this embodiment can include navigation indication information corresponding to the road conditions, pedestrian warning information, tourist attraction reminder information, etc. The types and specific contents of the driving guidance information can be not limited. For example, as Figure 3 shown, it shows an example diagram of displaying driving guidance information through the information display system based on augmented reality proposed in this application in a dangerous scenario. As Figure 3 shown, the image processing device can convert the real scene information collected by the scene perception device into a HUD virtual image for display on the HUD display device. The specific content displayed is as shown in the right image of Figure 3 . In this case, the scene seen by the driver's eyes can include lane guidance information (that isFigure 3 the "navigation indication in the virtual image" shown in Figure 3 and the pedestrian warning information (i.e., the "pedestrian prompt box in the virtual image" shown in

[0073] As a way, when the real-scene information is obtained, the driving guidance information can be generated based on the real-scene information. Optionally, the prompting method of the driving guidance information in this embodiment may not be limited. For example, it can be prompted in the form of icons (such as arrows), pictures, animations, voices or videos, etc. Then, for the driving guidance information with different prompting methods, it can be generated in the corresponding way. Optionally, for the generation principle of the driving guidance information based on the real-scene information and corresponding to each prompting method, reference can be made to the related technologies, which will not be elaborated here. Optionally, the driving guidance information in this embodiment may include at least one prompting method. For example, on the basis of displaying the navigation indication icon corresponding to the road, the user can be prompted by voice, so as to more accurately prompt the user for driving guidance, ensure driving safety and improve the user experience.

[0074] Step S150: Display the driving guidance information at the corresponding position in the target display area based on the coordinate transformation rule.

[0075] Optionally, by displaying the driving guidance information at the corresponding position in the target display area based on the coordinate transformation rule, it is possible to avoid the difference between the position where the HUD displays the real-scene information and the actual position of the real-scene information, and improve the accuracy and reliability of the display.

[0076] An information display method based on augmented reality provided by this application obtains the real-scene information collected by the image sensing device, then obtains the target display area determined based on the first spatial pose of the user, then obtains the coordinate transformation rule corresponding to the mapping of the real-scene information to the target display area, then generates the driving guidance information based on the real-scene information, and then displays the driving guidance information at the corresponding position in the target display area based on the coordinate transformation rule. It realizes displaying the driving guidance information generated based on the real-scene information at the corresponding position in the target display area determined based on the first spatial pose of the user, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving, without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue caused by frequent line-of-sight conversion when viewing road conditions and navigation and other driving guidance information, and improving the safety and comfort of driving.

[0077] Please refer to Figure 4, which is a flowchart of a method for information display based on augmented reality provided by another embodiment of the present application. The method of this embodiment can be executed by a device for processing real-scene information based on augmented reality, and the device can be implemented in a hardware and / or software manner. The method includes:

[0078] Step S210: Obtain the real-scene information collected by the scene perception device.

[0079] Step S220: Obtain the target display area determined based on the first spatial pose of the user.

[0080] Step S230: Obtain the coordinate transformation rule corresponding to mapping the real-scene information to the target display area.

[0081] Step S240: Generate driving guidance information based on the real-scene information.

[0082] Step S250: Input the position coordinates of the real-scene information in the coordinate system corresponding to the scene perception device into the first transformation matrix to obtain a to-be-processed coordinate transformation matrix.

[0083] As a way, the position coordinates of the real-scene information in the coordinate system corresponding to the scene perception device can be input into the first transformation matrix, and the output result can be used as the to-be-processed coordinate transformation matrix. For example, in a specific application scenario, assume that the position coordinates of the real-scene information in the coordinate system corresponding to the scene perception device are O w (x, y, z). Optionally, after inputting the position coordinates O w (x, y, z) into the aforementioned first transformation matrix, the following can be obtained:

[0084]

[0085]

[0086] Among them, O' can be used as the to-be-processed coordinate transformation matrix, and O W uses homogeneous coordinates.

[0087] Step S260: Perform coordinate transformation on the to-be-processed coordinate transformation matrix according to the second transformation matrix to obtain the relative position coordinates of the real-scene information within the target display area.

[0088] As a way, the to-be-processed coordinate transformation matrix can be subjected to coordinate transformation according to the aforementioned second transformation matrix to obtain the relative position coordinates of the real-scene information within the target display area. Optionally, the specific implementation process of the coordinate transformation can refer to related technologies and will not be elaborated here.

[0089] For example, taking the above example, assume that in the HUD image of the target display area of the HUD display device, the position coordinates corresponding to the position coordinates O w (x, y, z) are represented as O h (u, v). Then, after performing coordinate transformation on the coordinate transformation matrix O' to be processed according to the second transformation matrix, the following can be obtained:

[0090]

[0091] where width and height are the width and height of the HUD image, and the unit can be pixels. In this way, O h (u, v) can be used as the relative position coordinates of the real scene information within the target display area.

[0092] Step S270: Display the driving guidance information at the position characterized by the relative position coordinates.

[0093] Optionally, in this way, the driving guidance information corresponding to the real scene information can be displayed at the position characterized by the relative position coordinates.

[0094] The following uses a specific example to illustrate this embodiment:

[0095] Please refer to Figure 5 , which shows an example diagram of the display effect of the information display system based on augmented reality provided by this embodiment. As Figure 5 shown, a virtual-real scene fusion system (which can be understood as the information display system based on augmented reality in this application) can be built in relevant modeling software (such as Unity3D, etc.). The virtual-real scene fusion system can include a car, a camera 1 (for simulating the driver's eyes), a camera 2, and a HUD imaging module (i.e., the aforementioned HUD display device) simulated by a plane together, the spatial scene information (which can be the spatial scene information under different scenarios) obtained by an image perception module simulated by a checkerboard, and the information transformation and image drawing and rendering of an image processing device completed by a program script.

[0096] Optionally, in the simulated virtual-real scene fusion system, the center position at the bottom of the car can be selected as the coordinate origin, the forward direction of the car is the positive direction of the Z axis, and a right-handed coordinate system is adopted. Assume that the driver is sitting in the driver's seat, the eyes of the driver simulated by camera 1 face forward, and the pose of the HUD virtual camera simulated by camera 2 is the same as the pose of the driver's eyes. In this way, the scene perception device can obtain the spatial corner point information of the checkerboard on the car, and the image processing device can draw the corner points into the HUD image space (as Figure 5The bottom left corner shown is the corner image drawn by the image processing device into the HUD image space), and then the drawn image is sent to the HUD module for display (as Figure 5 shown in delivering the corner image to the HUD for virtual image display). In this way, a driver's perspective scene as shown in Figure 5 can be obtained. Through the enlarged view of the virtual-real fusion result shown in Figure 5 , it can be seen that the virtual and real scenes can be accurately fused.

[0097] An information display method based on augmented reality provided by this application displays the driving guidance information generated based on real-scene information at the corresponding position of the target display area determined based on the user's first spatial pose after coordinate transformation through the first transformation matrix and the second transformation matrix respectively, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue caused by frequent line-of-sight conversion due to viewing driving guidance information such as road conditions and navigation, and improving the safety and comfort of driving.

[0098] Please refer to Figure 6 , which is a flowchart of an information display method based on augmented reality provided by another embodiment of this application. The method of this embodiment can be executed by a device for processing real-scene information based on augmented reality, and the device can be implemented in a hardware and / or software manner. The method includes:

[0099] Step S310: Obtain the real-scene information collected by the scene perception device.

[0100] Step S320: Obtain the target display area re-determined based on the changed spatial pose.

[0101] It can be understood that during the user's driving, the user's posture will change. For example, the user's sitting posture will change (including tilting the body left and right, adjusting the height of the seat up and down, or adjusting the tilt degree of the seat back forward and backward), or the user's head will shake as the driving road conditions change. In this case, the user's first spatial pose can change. If the original HUD display method is still used to display the driving guidance information corresponding to the real-scene information, potential safety hazards may be caused due to position display errors.

[0102] As a way to improve this problem, this embodiment takes real-time detection of the user's spatial pose. So, if it is detected that the first spatial pose changes, the target display area is re-determined based on the changed spatial pose, thereby ensuring the accuracy of the display position of the driving guidance information corresponding to the real-scene information. Without the need for the user to repeatedly confirm the accuracy of the driving guidance information, the flexibility of displaying the driving guidance information is improved, and thus the user experience is enhanced.

[0103] For example, as an implementation manner, please refer to Figure 7 , which shows an example diagram of the target display area determined based on the user's first spatial pose provided in this embodiment. As Figure 7 shown, if the spatial pose corresponding to the current sitting posture of user 22 is the first spatial pose, in this case, on the screen 21 of the front windshield of the car, the target display area 23 as shown in Figure 7 can be displayed. Optionally, if the spatial pose of this user changes from 22 to the pose 22' as shown in Figure 8 , on the screen 21, the target display area 23' as shown in Figure 8 can be displayed. The target display area 23' is the target display area re-determined based on the changed spatial pose 22'.

[0104] Optionally, in this embodiment, the corresponding relationship between the change range of the user's spatial pose and the change range of the target display area can be preset. For example, it can be set that the user's spatial pose includes change ranges A, B, C, D, and E, and the change ranges of the target display area corresponding to the change ranges A, B, C, D, and E can be set to 1, 2, 3, 4, 5 (assuming that the larger the value, the larger the corresponding change range, and the change range corresponding to a unit value is 5°). Optionally, assuming that change range A > B > C > D > E, the larger the change range, the larger the corresponding change range. Then in this case, if it is detected that the first spatial pose of the user changes, the change range of the spatial pose can be determined based on the changed parameter, and then the change range of the corresponding target display area can be determined according to the change range.

[0105] It can be understood that assuming that the change range of the user's first spatial pose is very small, that is, it does not reach any corresponding change range, in this case, the display position of the target display area can not be adjusted.

[0106] Step S330: Obtain the coordinate transformation rule corresponding to the mapping of the real-scene information to the target display area.

[0107] Optionally, if the first spatial pose of the user changes, in the above manner, the second coordinate transformation rule corresponding to the real-scene information mapped to the re-determined target display area can be obtained. The specific determination process of the second coordinate transformation rule can refer to the determination principle and process of the foregoing coordinate transformation rule, which will not be elaborated here.

[0108] Step S340: Generate driving guidance information based on the real-scene information.

[0109] Step S350: Display the driving guidance information at the corresponding position of the re-determined target display area based on the coordinate transformation rule.

[0110] Optionally, on the basis of re-determining the target display area based on the changed spatial pose, the driving guidance information can be displayed at the corresponding position of the re-determined target display area based on the second coordinate change rule.

[0111] As an implementation manner, the target display area in this embodiment can be adjusted according to the change of the first spatial pose of the user. For example, if it is detected that the user has a posture such as looking down, the target display area can be displayed at the corresponding position on the central control display screen; if it is detected that the user looks at the mobile phone frequently during driving, the target display area can be displayed on the display screen of the mobile phone; or other screens that can be used as the target display area in the driving scenario, such as the windows on the left and right sides of the driver's seat.

[0112] As another implementation manner, at least one target display area can be set simultaneously in this embodiment, so that the driving user can be assisted by other users to drive safely in a fatigued state or a state with poor vision. For example, as Figure 9 shown, the front windshield 21 can be divided into two areas, including a first display area 211 and a second display area 212. In this way, assuming that the target display area corresponding to the spatial pose of the driver 221 is 231, then the target display area 232 is the target display area corresponding to the spatial pose of the co-pilot user 222. Among them, the content displayed in the target display area 231 and the content displayed in the target display area 232 can be the same. Optionally, the display state of the target display area 232 can be turned off or on according to actual needs. For example, when the main driver (i.e., the driver 221) is in a relatively fatigued mental state, the display function of the target display area 232 can be selected to be turned on. The display position of the target display area 232 in the second display area 212 can change with the change of the spatial pose of the user 222. The specific change principle can refer to the foregoing corresponding description, which will not be elaborated here.

[0113] Optionally, if the display status of the target display area 232 is in the on state, during driving, if dangerous driving information is detected, the co-pilot user 222 can timely remind the driver user 221, so as to achieve the way of assisting the driver through other users, realizing multiple improvements in driving safety and comfort. At the same time, it reduces the fatigue caused by frequent line-of-sight conversion due to viewing road conditions and navigation and other driving guidance information, enhances the interaction between users during driving, and improves the user-friendly experience. Optionally, the implementation principle of the display function of the target display area 232 can refer to the description in the foregoing embodiments and will not be elaborated herein.

[0114] An information display method based on augmented reality provided by the present application realizes displaying the driving guidance information generated based on real-scene information at the corresponding position of the target display area determined based on the first spatial pose of the user through coordinate transformation rules, or displaying it at the corresponding position of the target display area re-determined based on the changed spatial pose of the user, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving, without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue caused by frequent line-of-sight conversion due to viewing road conditions and navigation and other driving guidance information, and enhancing the safety and comfort of driving.

[0115] Please refer to Figure 10 , which is a flowchart of an information display method based on augmented reality provided by another embodiment of the present application. The method of this embodiment can be executed by a device for processing real-scene information based on augmented reality, and the device can be implemented in a hardware and / or software manner. The method includes:

[0116] Step S410: Obtain the real-scene information collected by the scene perception device.

[0117] Step S420: Detect the change of the first spatial pose by obtaining the sitting posture adjustment parameter of the electric seat.

[0118] Optionally, in this embodiment, the seat of the vehicle can be an electric seat. In this way, if the user adjusts the position of the electric seat, the electric seat can automatically generate an adjustment parameter, and this parameter can be used as the sitting posture adjustment parameter of the user. Then, as a way, the change of the user's first spatial pose can be detected by obtaining the sitting posture adjustment parameter of the electric seat.

[0119] Step S430: Obtain the sitting posture adjustment parameter of the electric seat.

[0120] Optionally, the sitting posture adjustment parameter of the electric seat can be obtained by reading the data automatically generated by the electric seat, or a camera can be installed to collect the sitting posture adjustment parameter of the electric seat. Optionally, the specific obtaining method can be not limited.

[0121] Step S440: Obtain the change vector corresponding to the first spatial pose based on the sitting posture adjustment parameter.

[0122] After obtaining the sitting posture adjustment parameter of the electric seat, the change vector corresponding to the first spatial pose can be obtained based on the sitting posture adjustment parameter. Optionally, the specific calculation process can be implemented by referring to related technologies and will not be elaborated here.

[0123] Step S450: Adjust the target display area based on the change vector to obtain the re-determined target display area.

[0124] As a way, the display position of the target display area can be adjusted based on the change vector corresponding to the first spatial pose to obtain the re-determined target display area. Optionally, the specific adjustment principle can be referred to the description in the foregoing embodiments and will not be elaborated here.

[0125] Step S460: Obtain the coordinate transformation rule corresponding to the real scene information mapped to the target display area.

[0126] Step S470: Generate driving guidance information based on the real scene information.

[0127] Optionally, in this embodiment, the sequence order between each step can be not limited. For example, step S470 can be implemented after step S410.

[0128] Exemplarily, a specific implementation process is shown below:

[0129] As Figure 11 shown, a processing process example diagram of the information display method based on augmented display proposed in this embodiment is shown. In Figure 11 , the process pointed by the hollow arrow can be the initial process, and the process pointed by the solid arrow can be the real-time continuous process. As an implementation way, the coordinate system can be established first, then the spatial poses of the scene perception device and the driver's eyes are measured, then the scene perception device matrix M and the HUD imaging matrix C are calculated respectively, and then the total transformation matrix (i.e., the foregoing coordinate transformation rule) F = CM is obtained. Optionally, the scene perception device can obtain the real scene information in real time, use the real scene information as the information to be displayed and send it to the image processing device. The image processing device performs coordinate transformation processing on the coordinates corresponding to the real scene information and draws the finally obtained image, and projects the image onto the HUD display screen (i.e., the foregoing target real area) for display, so as to improve the accuracy of the display position of the driving guidance information, reduce user operations, and thus improve the user experience.

[0130] Step S480: Display the driving guidance information at the corresponding position of the re-determined target display area based on the coordinate transformation rule.

[0131] An information display method based on augmented reality provided by this application detects the change of the user's first spatial pose by obtaining the sitting posture adjustment parameters of the electric seat, and realizes that the driving guidance information generated based on the real scene information is displayed at the corresponding position of the target display area determined based on the user's first spatial pose through the coordinate transformation rule, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving, without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue caused by frequent line-of-sight conversion due to viewing road conditions and navigation and other driving guidance information, and improving the safety and comfort of driving.

[0132] Please refer to Figure 12 , an information display device 500 based on augmented reality provided by an embodiment of this application can run on a projection device, and the device 500 includes:

[0133] An image perception module 510, configured to obtain real scene information collected by an image perception device.

[0134] A coordinate transformation module 520, configured to obtain a target display area determined based on the user's first spatial pose.

[0135] Optionally, if it is detected that the first spatial pose changes, the coordinate transformation module 520 can be configured to obtain a target display area re-determined based on the changed spatial pose.

[0136] As a method, the change of the first spatial pose can be detected by obtaining the sitting posture adjustment parameters of the electric seat. In this method, the coordinate transformation module 520 can specifically be configured to obtain the sitting posture adjustment parameters of the electric seat; obtain a change vector corresponding to the first spatial pose based on the sitting posture adjustment parameters; and adjust the target display area based on the change vector to obtain a re-determined target display area.

[0137] As a method, the coordinate transformation module 520 can also be configured to obtain a coordinate transformation rule corresponding to mapping the real scene information to the target display area.

[0138] Optionally, the coordinate transformation rule may include a first transformation matrix and a second transformation matrix. The first transformation matrix is used to determine a reference world coordinate corresponding to the coordinate of the real scene information collected by the scene perception device, and the second transformation matrix is used to convert the reference world coordinate into a view coordinate within the target display area.

[0139] The first transformation matrix may include a first rotation matrix and a first translation vector. The first rotation matrix is used to rotate the coordinates of the real-scene information collected by the scene perception device, and the first translation vector is used to translate the coordinates. The first transformation matrix determines the reference world coordinates corresponding to the coordinates of the real-scene information collected by the scene perception device based on the first rotation matrix and the first translation vector.

[0140] The second transformation matrix may include a view matrix and a projection matrix. The projection matrix is used to determine the mapping range for mapping the real-scene information to the target display area, and the view matrix is used to determine the relative position for displaying the driving guidance information within the mapping range. The second transformation matrix converts the reference world coordinates into view coordinates within the target display area based on the mapping range and the relative position.

[0141] The view matrix may include a second rotation matrix and a second translation vector. The second rotation matrix is used to rotate the reference world coordinates, and the second translation vector is used to translate the reference world coordinates; the projection matrix includes a field-of-view angle parameter, and the field-of-view angle includes a horizontal field-of-view angle and a vertical field-of-view angle.

[0142] As an implementation, the product of the parameters represented by the first transformation matrix and the parameters represented by the second transformation matrix can be obtained as the coordinate transformation rule for mapping the real-scene information to the target display area.

[0143] The display module 530 is used to generate driving guidance information based on the real-scene information.

[0144] As a way, the display module 530 can also be used to display the driving guidance information at the corresponding position in the target display area based on the coordinate transformation rule.

[0145] Optionally, the display module 530 can specifically be used to input the position coordinates of the real-scene information in the coordinate system corresponding to the scene perception device into the first transformation matrix to obtain a to-be-processed coordinate transformation matrix; perform coordinate transformation on the to-be-processed coordinate transformation matrix according to the second transformation matrix to obtain the relative position coordinates of the real-scene information within the target display area; and display the driving guidance information at the position represented by the relative position coordinates.

[0146] Optionally, if it is detected that the first spatial pose changes, the driving guidance information can be displayed at the corresponding position in the re-determined target display area based on the coordinate transformation rule corresponding to the changed spatial pose.

[0147] It should be noted that the device embodiments in this application correspond to the foregoing method embodiments. For the specific principles in the device embodiments, reference may be made to the content in the foregoing method embodiments, which will not be elaborated herein.

[0148] Next, a projection device provided by this application will be described in conjunction with Figure 13 a projection device provided by this application will be described.

[0149] Please refer to Figure 13 , based on the above information display method, system, and device based on augmented reality, another projection device 100 that can execute the foregoing information display method based on augmented reality is further provided in an embodiment of this application. The projection device 100 includes one or more (only one is shown in the figure) processors 102, a memory 104, an image perception module 11, a coordinate transformation module 12, and a display module 13 that are coupled to each other. Among them, a program that can execute the content in the foregoing embodiments is stored in the memory 104, and the processor 102 can execute the program stored in the memory 104. The memory 104 includes the device 500 described in the foregoing embodiments.

[0150] Among them, the processor 102 may include one or more processing cores. The processor 102 connects various parts within the entire projection device 100 through various interfaces and lines, and executes various functions of the projection device 100 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and by calling data stored in the memory 104. Optionally, the processor 102 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 102 may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 102 and may be implemented separately through a communication chip.

[0151] The memory 104 may include a Random Access Memory (RAM), or may also include a Read-Only Memory. The memory 104 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, a video image playback function, etc.), instructions for implementing the above various method embodiments, etc. The data storage area can also store data created during the use of the projection device 100 (such as audio-visual data), etc.

[0152] The image perception module 11 is configured to obtain the real scene information collected by the image perception device; the coordinate transformation module 12 is configured to obtain a target display area determined based on the first spatial pose of the user; the coordinate transformation module 12 is further configured to obtain a coordinate transformation rule corresponding to the mapping of the real scene information to the target display area; the display module 13 is configured to generate driving guidance information based on the real scene information; the display module 13 is further configured to display the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule.

[0153] Please refer to Figure 14 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable medium 600, and the program code can be called by a processor to execute the method described in the above method embodiments.

[0154] The computer-readable storage medium 600 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk or a ROM. Optionally, the computer-readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 600 has a storage space for the program code 610 for executing any method step in the above method. These program codes can be read out from or written into one or more computer program products. The program code 610 can be compressed in an appropriate form, for example.

[0155] In summary, a method, system, device, projection device, and storage medium for information display based on augmented reality provided by this application obtain the real-world information collected by an image sensing device, then obtain a target display area determined based on the first spatial pose of the user, then obtain the coordinate transformation rule corresponding to mapping the real-world information to the target display area, then generate driving guidance information based on the real-world information, and then display the driving guidance information at the corresponding position in the target display area based on the coordinate transformation rule. Thus, through the above method, the driving guidance information generated based on the real-world information is displayed at the corresponding position in the target display area determined based on the first spatial pose of the user through the coordinate transformation rule, so that the user can accurately and conveniently view the virtual driving guidance information corresponding to the driving scene during driving, without repeatedly confirming the accuracy of the driving guidance information, reducing the fatigue caused by frequent line-of-sight conversion when viewing road conditions and navigation and other driving guidance information, and improving the safety and comfort of driving.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An information display method based on augmented reality, characterized in that: The method comprises: Acquiring real scene information collected by the scene perception device; Acquire a target display area determined based on a first spatial posture of the user; Obtaining a coordinate transformation rule corresponding to mapping the real scene information to the target display area; the coordinate transformation rule includes a first transformation matrix and a second transformation matrix, the first transformation matrix is used to determine a reference world coordinate corresponding to the coordinates of the real scene information collected by the scene perception device; the second transformation matrix includes a view matrix and a projection matrix, the projection matrix is used to determine a mapping range for mapping the real scene information to the target display area, the view matrix is used to determine the relative position of displaying driving guidance information within the mapping range, the projection matrix includes a field of view angle parameter, the field of view angle includes a horizontal field of view angle and a vertical field of view angle; the second transformation matrix converts the reference world coordinate into a view coordinate within the target display area based on the mapping range and the relative position; generating driving guidance information based on the real scene information; The driving guidance information is displayed at a corresponding position in the target display area based on the coordinate transformation rule.

2. The method according to claim 1, characterized in that The first transformation matrix includes a first rotation matrix and a first translation vector. The first rotation matrix is used to rotate the coordinates of the real scene information collected by the scene perception device, and the first translation vector is used to translate the coordinates. The first transformation matrix determines the reference world coordinates corresponding to the coordinates of the real scene information collected by the scene perception device based on the first rotation matrix and the first translation vector.

3. The method according to claim 1, characterized in that The view matrix includes a second rotation matrix and a second translation vector, the second rotation matrix is used to rotate the reference world coordinates, and the second translation vector is used to translate the reference world coordinates.

4. The method according to any one of claims 1 to 3, characterized in that The step of displaying the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule includes: Inputting the position coordinates of the real scene information in the coordinate system corresponding to the scene perception device into the first transformation matrix to obtain a coordinate transformation matrix to be processed; Performing coordinate transformation on the coordinate transformation matrix to be processed according to the second transformation matrix to obtain relative position coordinates of the real scene information in the target display area; The driving guidance information is displayed at the position represented by the relative position coordinates.

5. The method according to claim 4, characterized in that The obtaining of a coordinate transformation rule corresponding to mapping the real scene information to the target display area includes: The product of the parameters represented by the first transformation matrix and the parameters represented by the second transformation matrix is obtained as a coordinate transformation rule corresponding to mapping the real scene information to the target display area.

6. The method according to claim 1, characterized in that If a change in the first spatial posture is detected, obtaining a target display area determined based on the first spatial posture of the user includes: Obtaining a target display area that is re-determined based on the changed spatial posture; The step of displaying the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule includes: The driving guidance information is displayed at a corresponding position of the re-determined target display area based on the coordinate transformation rule.

7. The method according to claim 6, characterized in that The method further comprises: Detecting a change in the first spatial posture by acquiring a sitting posture adjustment parameter of the electric seat; The obtaining of the target display area re-determined based on the changed spatial posture includes: Obtaining a sitting posture adjustment parameter of the electric seat; Obtaining a change vector corresponding to the first spatial posture based on the sitting posture adjustment parameter; The target display area is adjusted based on the change vector to obtain a re-determined target display area.

8. An information display device based on augmented reality, characterized in that: The information display device includes an image perception module, a coordinate transformation module and a display module: The image perception module is used to obtain real scene information collected by the scene perception device; The coordinate transformation module is used to obtain a target display area determined based on a first spatial posture of the user; The coordinate transformation module is further used to obtain a coordinate transformation rule corresponding to mapping the real scene information to the target display area; the coordinate transformation rule includes a first transformation matrix and a second transformation matrix, the first transformation matrix is used to determine the reference world coordinates corresponding to the coordinates of the real scene information collected by the scene perception device, the second transformation matrix includes a view matrix and a projection matrix, the projection matrix is used to determine the mapping range for mapping the real scene information to the target display area, the view matrix is used to determine the relative position of the driving guidance information displayed within the mapping range, and the projection matrix includes a field of view angle parameter, and the field of view angle includes a horizontal field of view angle and a vertical field of view angle; The second transformation matrix converts the reference world coordinates into view coordinates within the target display area based on the mapping range and the relative position; The display module is used to generate driving guidance information based on the real scene information; The display module is further configured to display the driving guidance information at a corresponding position in the target display area based on the coordinate transformation rule.

9. An augmented reality-based vehicle information display system, characterized in that: The system comprises: A scene perception device for collecting real-scene information of the vehicle's external environment; An image processing device is configured to obtain real-scene information collected by the scene perception device, obtain a target display area determined based on a first spatial posture of a user, obtain a coordinate transformation rule corresponding to mapping the real-scene information to the target display area, generate driving guidance information based on the real-scene information, and generate target position coordinates of the driving guidance information displayed in the target display area based on the coordinate transformation rule; the coordinate transformation rule includes a first transformation matrix and a second transformation matrix, the first transformation matrix is used to determine reference world coordinates corresponding to the coordinates of the real-scene information collected by the scene perception device, the second transformation matrix includes a view matrix and a projection matrix, the projection matrix is used to determine a mapping range for mapping the real-scene information to the target display area, the view matrix is used to determine a relative position of displaying the driving guidance information within the mapping range, the projection matrix includes a field of view angle parameter, the field of view angle includes a horizontal field of view angle and a vertical field of view angle; the second transformation matrix converts the reference world coordinates into view coordinates within the target display area based on the mapping range and the relative position; The HUD display device is used to display the driving guidance information at the target position coordinates of the target display area.

10. A projection device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images