Image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of auxiliary driving, and the method comprises the steps: obtaining the rotation angle of a curved mirror in a head-up display in real time, and the viewpoint position coordinates of a target user; acquiring matched distortion correction parameters according to the rotation angle and the viewpoint position coordinates; and sending the distortion correction parameter to an image generation unit in the head-up display to enable the image generation unit to perform distortion adjustment according to the distortion correction parameter, and displaying the virtual image after distortion adjustment through the windshield. According to the technical scheme provided by the embodiment of the invention, the dynamic distortion of the image generated by the head-up display is corrected in real time, the phenomenon of graph distortion caused by the change of the rotating shaft of the curved mirror and the viewpoint position is avoided, the imaging quality of the head-up display is improved, and the image display effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of intelligent assisted driving technology, head-up display (HUD) is widely used in various vehicles. It can project information such as vehicle speed, navigation and warning in the form of virtual images in front of the driver with the help of optical components.

[0003] As a complex optical system, the head-up display itself often has distortion problems during imaging. In the prior art, the distortion correction of the image generated by the head-up display is usually to pre-image the virtual image seen at the center of the eye box, and then correct the generated image according to the expected imaging result, so that the center of the eye box sees a distortion-free image with a standard shape.

[0004] However, in the above correction method, when the eye box is moved to the high eye box, the low eye box or other positions of the middle eye box through the reflector, a certain dynamic distortion will occur. The distortion correction made for the center position of the eye box will not match the optical path well, causing the image to be distorted, thereby resulting in poor imaging effect. Summary of the invention

[0005] The present invention provides an image processing method, an apparatus, a head-up display and a storage medium to solve the image distortion problem of the head-up display.

[0006] According to one aspect of the present invention, there is provided an image processing method, which is applied to a head-up display, comprising:

[0007] Obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time;

[0008] Acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates;

[0009] The distortion correction parameters are sent to an image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after the distortion adjustment through a windshield.

[0010] The method of obtaining matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates includes: obtaining a real-time eye box according to the rotation angle of the curved mirror; obtaining a first position coordinate of the viewpoint position coordinate relative to the central axis of the real-time eye box, and obtaining matching distortion correction parameters according to the first position coordinates.

[0011] The real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: acquiring the rotation angle of the curved mirror in the head-up display according to the position of a limit switch configured in the head-up display and the number of rotation steps of the stepping motor in the head-up display.

[0012] The real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: acquiring the virtual image displayed on the windshield by positioning the camera; and acquiring the rotation angle of the curved mirror in the head-up display according to the second position coordinates of the marker in the current virtual image relative to the fixed reference object.

[0013] The real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: determining the grid area where the viewpoint of the target user is located according to the viewpoint position coordinates of the target user, and using the viewpoint area coordinates of the grid area where the viewpoint is located as the viewpoint position coordinates of the target user.

[0014] The method of obtaining matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates includes: if matching distortion correction parameters are not obtained through the correction rule mapping table according to the current viewpoint position coordinates, obtaining the first viewpoint position coordinates in the correction rule mapping table that are closest to the current viewpoint position coordinates; and using the distortion correction parameters corresponding to the first viewpoint position coordinates as the distortion correction parameters that match the current viewpoint position coordinates.

[0015] According to another aspect of the present invention, there is provided an image processing apparatus, comprising:

[0016] Angle coordinate acquisition module, used to obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time;

[0017] A correction parameter acquisition module, used to acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates;

[0018] The distortion adjustment execution module is used to send the distortion correction parameters to the image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after the distortion adjustment through the windshield.

[0019] The correction parameter acquisition module is specifically used to obtain a real-time eye box according to the rotation angle of the curved mirror; obtain the first position coordinates of the viewpoint position coordinates relative to the central axis of the real-time eye box, and obtain matching distortion correction parameters according to the first position coordinates.

[0020] The angle coordinate acquisition module is specifically used to acquire the rotation angle of the curved mirror in the head-up display according to the position of the limit switch configured in the head-up display and the number of rotation steps of the stepping motor in the head-up display.

[0021] The angle coordinate acquisition module is also used to acquire the virtual image displayed on the windshield by positioning the camera; and acquire the rotation angle of the curved mirror in the head-up display according to the second position coordinates of the marker in the current virtual image relative to the fixed reference object.

[0022] The angle coordinate acquisition module is further used to determine the grid area where the viewpoint of the target user is located according to the viewpoint position coordinates of the target user, and use the viewpoint area coordinates of the grid area where the viewpoint is located as the viewpoint position coordinates of the target user.

[0023] The correction parameter acquisition module is also used to obtain the first viewpoint position coordinates in the correction rule mapping table that are closest to the current viewpoint position coordinates if no matching distortion correction parameters are obtained through the correction rule mapping table based on the current viewpoint position coordinates; and use the distortion correction parameters corresponding to the first viewpoint position coordinates as the distortion correction parameters that match the current viewpoint position coordinates.

[0024] According to another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method described in any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method described in any embodiment of the present invention when executed.

[0026] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method according to any embodiment of the present invention is implemented.

[0027] The technical solution of the embodiment of the present invention is to obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time, and then obtain the matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates; then send the distortion correction parameters to the image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after distortion adjustment through the windshield. In this way, the real-time correction of the dynamic distortion of the image generated by the head-up display is realized, the change of the rotation axis of the curved mirror and the viewpoint position, which causes the occurrence of image distortion, improves the imaging quality of the head-up display, and enhances the image display effect.

[0028] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 is a flowchart of an image processing method provided according to Embodiment 1 of the present invention;

[0031] Figure 2 is a schematic diagram of an application scenario of a head-up display provided according to the first embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of an application scenario of a camera calibration provided according to the first embodiment of the present invention;

[0033] Figure 4 is a flowchart of an image processing method provided according to Embodiment 2 of the present invention;

[0034] Figure 5 is a flowchart of an image processing method provided according to Embodiment 3 of the present invention;

[0035] Figure 6 is a structural schematic diagram of an image processing device provided according to a fourth embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the image processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] Embodiment 1

[0040] Figure 1 This is a flowchart of an image processing method provided in the first embodiment of the present invention. The method is suitable for adjusting the distortion of a virtual image generated by an image generation unit according to the rotation angle of a curved mirror and the coordinates of the user's viewpoint position. The method can be executed by an image processing device in any embodiment of the present invention. The image processing device can be implemented in the form of hardware and / or software. The image processing device can be configured in an electronic device, for example, in a head-up display, or in a vehicle-mounted terminal device that is connected to the head-up display. Figure 1 As shown, the method includes:

[0041] S101, obtaining in real time the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user.

[0042] like Figure 2 As shown, after generating a virtual image to be displayed, the picture generation unit (PGU) in the head-up display transmits the image to one or more plane mirrors; the plane mirrors then reflect the acquired image to one or more curved mirrors; finally, the curved mirrors project the virtual image onto the windshield; accordingly, the user in the driving position can obtain relevant driving assistance information through the windshield while driving; Figure 2In the example, a head-up display including a flat mirror and a curved mirror is taken.

[0043] The vehicle-mounted terminal equipment can obtain the viewpoint position of the driver (i.e., the observer) in real time through a viewpoint detection device installed inside the cab; wherein the viewpoint detection device may include an eye tracker or a driver monitoring system (DMS), etc.; the viewpoint position actually refers to the driver's pupil position, and the viewpoint detection device can obtain the driver's viewpoint position information by detecting the driver's eye movement direction and line of sight direction; since the viewpoint detection device is installed at a fixed position, after it establishes a body coordinate system based on its own position, the viewpoint position coordinates are actually the coordinates of the driver's pupil position relative to a fixed position in the vehicle.

[0044] In particular, if the virtual image generated by the image generation unit is a 2D virtual image, the left eye pupil position or the right eye pupil position, or the center point position between the left and right eye pupils can be used as the viewpoint position of the target user, and a distortion correction parameter is obtained based on the viewpoint position to correct the image; if the virtual image generated by the image generation unit is a 3D virtual image, that is, a naked-eye 3D image, since it is actually synthesized from the left eye image and the right eye image, that is, a 3D synthesized image, the left eye pupil position and the right eye pupil position need to be used as different viewpoint position coordinates, respectively, so as to obtain the distortion correction parameters of the left and right eye images through the viewpoint position coordinates of the left and right eyes, so as to perform distortion correction on the left and right eye images respectively.

[0045] In particular, for some driving scenarios, the virtual image provided by the head-up display is used to provide auxiliary information for the co-pilot or the rear seat. At this time, the actual user of the head-up display is the co-pilot or the rear seat passenger. Correspondingly, the target user at this time specifically refers to the co-pilot or the rear seat passenger. Obviously, the viewpoint position coordinates obtained by them are also the viewpoint position coordinates of the co-pilot or the rear seat passenger. In addition, the rotation angle of the curved mirror in the head-up display can be obtained by an angle sensor configured at the curved mirror position, so as to realize real-time monitoring of the rotation angle of the curved mirror.

[0046] S102: Acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates.

[0047] The continuous rotation of the curved mirror in the head-up display will change the imaging position of the virtual image generated by the image generation unit on the windshield, and thereby change the virtual image viewing angle of the target user; the change of the user's viewpoint position coordinates will also change the target user's virtual image viewing angle, and the distortion phenomenon seen by the user at different viewing angles is also different. Therefore, the rotation angle of the curved mirror and the user's viewpoint position coordinates will affect the distortion shape of the virtual image; among them, the distortion correction parameters may include radial distortion parameters and tangential distortion parameters and other forms. In the embodiment of the present invention, the parameter type and parameter quantity of the distortion correction parameters are not specifically limited.

[0048] In addition, in an embodiment of the present invention, the distortion correction parameters can be calculated based on the rotation angle of the curved mirror and the user's viewpoint position coordinates through a pre-configured distortion correction calculation rule or a distortion correction parameter change curve, or the matching distortion correction parameters can be obtained by looking up a table based on the rotation angle of the curved mirror and the user's viewpoint position coordinates through a pre-configured correction rule mapping table; wherein the correction rule mapping table records the distortion correction parameters corresponding to different viewpoint position coordinates at each rotation angle.

[0049] like Figure 3 As shown, after the rotation angle of the curved mirror is fixed, an image with a human eye can be attached to the front end of the calibration camera, or a portrait sculpture, a real person calibration or a robot or other things that can be used to describe the position of the human eye can be installed at the front end of the calibration camera; the calibration camera is exposed from the human eye position of the image to simulate the position of the user's eye. At this time, the shooting position of the camera is equivalent to the pupil position of the user's eye. The actual virtual image (i.e., the distorted virtual image) at the current viewpoint position is obtained by shooting with the calibration camera, and compared with the expected virtual image (i.e., the expected display image without distortion under ideal conditions) to obtain the parameter difference between the two (i.e., the distortion correction parameter), and the distortion correction parameter at the current viewpoint position can be obtained based on this; by continuously changing the rotation angle and the human eye pupil position, the distortion correction parameters corresponding to the coordinates of different viewpoint positions at each rotation angle can be obtained, thereby completing the construction of the correction rule mapping table.

[0050] In particular, after obtaining the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time, under the premise that the rotation angle of the curved mirror has not changed, if the viewpoint position coordinates between the two previous sampling moments are different, it means that the viewpoint position of the target user has moved, that is, the viewpoint position at the next sampling moment is offset compared to the viewpoint position at the previous sampling moment; if the viewpoint position coordinates between the two previous sampling moments are the same, it means that the viewpoint position of the target user has not moved, that is, the viewpoint position at the next sampling moment is not offset compared to the viewpoint position at the previous sampling moment.

[0051] If the moving distance of the viewpoint position is small, it means that the user has only made a slight posture adjustment, and the user is actually still at the same viewpoint position. In this case, there is no need to perform distortion adjustment on the generated image. The pre-set viewpoint position moving distance threshold can be used to determine whether the user's viewpoint position has shifted. In other words, if the moving distance between the previous and next sampling moments is less than the moving distance threshold, it indicates that the viewpoint position of the target user has not shifted. Similarly, if the moving distance of the viewpoint position is large, it means that the user has changed the viewpoint position and the user is at a different viewpoint position. In this case, distortion adjustment on the generated image is required. In other words, if the moving distance between the previous and next sampling moments is greater than or equal to the moving distance threshold, it indicates that the viewpoint position of the target user has shifted.

[0052] Optionally, in an embodiment of the present invention, the real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: determining the grid area where the viewpoint of the target user is located according to the viewpoint position coordinates of the target user, and using the viewpoint area coordinates of the grid area where the viewpoint is located as the viewpoint position coordinates of the target user.

[0053] Specifically, the space inside the vehicle can be pre-divided into multiple different grid areas, and the grid area where the current viewpoint is located can be determined based on the viewpoint position coordinates of the target user; if the viewpoint position coordinates obtained at two adjacent sampling moments are located in the same grid area, it means that the grid area where the viewpoint of the target user is located has not changed. At this time, it is determined that the viewpoint position of the target user has not shifted or the shift range is extremely small. If the rotation angle of the curved mirror has not changed, then the image distortion correction is not required at this time.

[0054] If the viewpoint position coordinates obtained at two adjacent sampling moments are located in different grid areas, it means that the grid area where the target user's viewpoint is located has changed. At this time, it is determined that the viewpoint position of the target user has shifted. Regardless of whether the rotation angle of the curved mirror changes, the image needs to be distorted. Therefore, by pre-dividing the interior space into multiple different grid areas, and performing image distortion correction according to the grid area where the target user's viewpoint is located, the coordinate point comparison is converted into the point area comparison, which improves the comparison efficiency of the viewpoint position, and at the same time reduces the amount of mapping information that needs to be configured in the correction rule mapping table, and reduces the construction complexity and management complexity of the correction rule mapping table.

[0055] Optionally, in an embodiment of the present invention, obtaining matching distortion correction parameters based on the rotation angle and the viewpoint position coordinates includes: if matching distortion correction parameters are not obtained through the correction rule mapping table based on the current viewpoint position coordinates, obtaining the first viewpoint position coordinates in the correction rule mapping table that are closest to the current viewpoint position coordinates; and using the distortion correction parameters corresponding to the first viewpoint position coordinates as the distortion correction parameters that match the current viewpoint position coordinates.

[0056] Specifically, when constructing the correction rule mapping table, it may not be possible to plan the distortion correction parameters for all viewpoint positions, or the user's actual viewpoint position exceeds the viewpoint range recorded in the correction rule mapping table. At this time, under the premise that the rotation angle of the curved mirror is the same, the current viewpoint position coordinates are compared with the viewpoint positions recorded in the correction rule mapping table to obtain the first viewpoint position coordinates that are closest, and the distortion correction parameters corresponding to the first viewpoint position coordinates are used as the distortion correction parameters that match the current viewpoint position coordinates. When the accurate distortion correction parameters cannot be obtained through the correction rule mapping table, similar distortion correction parameters can be obtained by obtaining them nearby, thereby minimizing the distortion effect of the generated image and improving the image display effect.

[0057] The technical solution of the embodiment of the present invention is to obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time, and then obtain the matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates; then send the distortion correction parameters to the image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after distortion adjustment through the windshield. In this way, the real-time correction of the dynamic distortion of the image generated by the head-up display is realized, the change of the rotation axis of the curved mirror and the viewpoint position, which causes the occurrence of image distortion, improves the imaging quality of the head-up display, and enhances the image display effect.

[0058] Embodiment 2

[0059] Figure 4 This is a flow chart of an image processing method provided by Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiment is that the eye box area is first adjusted according to the rotation angle of the curved mirror, and then the position of the viewpoint position coordinates in the adjusted eye box area is obtained. Figure 4 As shown, the method includes:

[0060] S201, obtaining in real time the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user.

[0061] S202: Acquire a real-time eye box according to the rotation angle of the curved mirror.

[0062] The eyebox represents the eye distribution range of the target user that can see the complete image, and is usually represented by a virtual box as a rectangular area. As the rotation angle of the curved mirror changes, the imaging position on the windshield changes, and the corresponding eyebox area of ​​the target user also changes synchronously. Based on this, the current eyebox area can be obtained in real time through the head-up display according to the rotation angle of the curved mirror.

[0063] S203, obtaining first position coordinates of the viewpoint position coordinates relative to the central axis of the real-time eye box, and obtaining matching distortion correction parameters according to the first position coordinates.

[0064] After obtaining the real-time eye box, the central axis of the current eye box is used as the standard view position, and the eye box coordinate system is constructed accordingly. Then, the viewpoint position coordinates in the current eye box are obtained relative to the position coordinates of the central axis of the real-time eye box area, that is, the first position coordinates. Then, the distortion correction parameters matching the current first position coordinates can be obtained by looking up the distortion rule mapping table; wherein, the distortion rule mapping table records the distortion correction parameters corresponding to different position coordinates (that is, the first position coordinates).

[0065] Different from the correction rule mapping table in the above technical solution, the distortion rule mapping table does not need to consider the rotation angle of the curved mirror, but only needs to record the mapping relationship between different position coordinates and distortion correction parameters, which greatly reduces the construction complexity and management complexity of the mapping table, improves the efficiency of obtaining distortion correction parameters, and ensures the timeliness of the display of the generated image.

[0066] S204: Send the distortion correction parameters to an image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after the distortion adjustment through a windshield.

[0067] The technical solution of the embodiment of the present invention, after obtaining the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time, first obtains the real-time eye box according to the rotation angle of the curved mirror, and then obtains the first position coordinates of the viewpoint position coordinates relative to the central axis of the real-time eye box, so as to obtain the matching distortion correction parameters according to the first position coordinates. This greatly reduces the complexity of mapping table construction and management, improves the efficiency of obtaining distortion correction parameters, and ensures the timeliness of display of generated images.

[0068] Embodiment 3

[0069] Figure 5 This is a flow chart of an image processing method provided by Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that the rotation angle of the curved mirror can be obtained in a variety of ways. Figure 5 As shown, the method includes:

[0070] S301, obtaining a rotation angle of a curved mirror in the head-up display according to a position of a limit switch configured in the head-up display and a number of rotation steps of a stepping motor in the head-up display.

[0071] By configuring a limit switch on the rotating shaft of the curved mirror or on the shell of the head-up display, any vertex position of the curved mirror can be stuck by the limit switch as the origin position, and the curved mirror can only rotate in one direction at this time; the head-up display controls the rotation of the curved mirror through a stepper motor, and thus by recording the number of rotation steps of the stepper motor, the rotation angle of the curved mirror relative to the origin position can be obtained; thereafter, when the curved mirror needs to be adjusted to other positions, it is only necessary to control the curved mirror to return to the origin position, and then read the number of rotation steps of the stepper motor from the origin position again, so that the curved mirror can be adjusted to the desired position, and thus the head-up display can obtain the rotation angle of the curved mirror through the internally configured limit switch, and the vehicle-mounted terminal device can obtain the angle value through the head-up display.

[0072] S302: Obtain the viewpoint position coordinates of the target user.

[0073] S303: Acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates.

[0074] S304: Send the distortion correction parameters to an image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after the distortion adjustment through a windshield.

[0075] Optionally, in an embodiment of the present invention, the real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: acquiring the virtual image displayed on the windshield by positioning the camera; and acquiring the rotation angle of the curved mirror in the head-up display based on the second position coordinates of the marker in the current virtual image relative to the fixed reference object.

[0076] Specifically, the image generating unit can be guided to pre-configure a marker in the generated virtual image, and then the relative position error between the marker and a fixed reference object located on the vehicle body in the image (for example, the protruding point of the hood) can be obtained through image recognition technology. In the expected image, the marker and the fixed reference object have a fixed position error. By comparing the above two position errors, the rotation angle of the curved mirror can be determined.

[0077] In addition, it is also possible not to guide the image generation unit to pre-configure markers in the generated virtual image, but to use the image elements existing in the virtual image as markers. For example, the center point of the lower edge of the virtual image is used as the marker. Based on this, the positioning camera configured inside the vehicle is used to obtain the rotation angle of the curved mirror in the head-up display without changing the internal structure of the head-up display, thereby simplifying the structural complexity of the head-up display and avoiding the hardware modification process of the head-up display.

[0078] The technical solution of the embodiment of the present invention is to obtain the rotation angle of the curved mirror in the head-up display according to the second position coordinates of the marker in the current virtual image relative to the fixed reference object after acquiring the virtual image displayed on the windshield through the positioning camera. Based on this, the rotation angle of the curved mirror in the head-up display is obtained through the positioning camera configured inside the vehicle without changing the internal structure of the head-up display, thereby simplifying the structural complexity of the head-up display and avoiding the hardware modification process of the head-up display.

[0079] Embodiment 4

[0080] Figure 6 : is a structural block diagram of an image processing device provided by Embodiment 4 of the present invention, and the image processing device specifically includes:

[0081] Angle coordinate acquisition module 601, used to acquire the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time;

[0082] A correction parameter acquisition module 602, used to acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates;

[0083] The distortion adjustment execution module 603 is used to send the distortion correction parameters to the image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after distortion adjustment through the windshield.

[0084] The technical solution of the embodiment of the present invention is to obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time, and then obtain the matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates; then send the distortion correction parameters to the image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after distortion adjustment through the windshield. In this way, the real-time correction of the dynamic distortion of the image generated by the head-up display is realized, the change of the rotation axis of the curved mirror and the viewpoint position, which causes the occurrence of image distortion, improves the imaging quality of the head-up display, and enhances the image display effect.

[0085] The correction parameter acquisition module 602 is specifically used to obtain a real-time eye box according to the rotation angle of the curved mirror; obtain the first position coordinates of the viewpoint position coordinates relative to the central axis of the real-time eye box, and obtain matching distortion correction parameters according to the first position coordinates.

[0086] The angle coordinate acquisition module 601 is specifically used to acquire the rotation angle of the curved mirror in the head-up display according to the position of the limit switch configured in the head-up display and the number of rotation steps of the stepping motor in the head-up display.

[0087] The angle coordinate acquisition module 601 is also used to acquire the virtual image displayed on the windshield by positioning the camera; and acquire the rotation angle of the curved mirror in the head-up display according to the second position coordinates of the marker in the current virtual image relative to the fixed reference object.

[0088] The angle coordinate acquisition module 601 is further used to determine the grid area where the viewpoint of the target user is located according to the viewpoint position coordinates of the target user, and use the viewpoint area coordinates of the grid area where the viewpoint is located as the viewpoint position coordinates of the target user.

[0089] The correction parameter acquisition module 602 is also used to obtain the first viewpoint position coordinates in the correction rule mapping table that are closest to the current viewpoint position coordinates if no matching distortion correction parameters are obtained through the correction rule mapping table based on the current viewpoint position coordinates; and use the distortion correction parameters corresponding to the first viewpoint position coordinates as the distortion correction parameters that match the current viewpoint position coordinates.

[0090] The image processing device provided by the present invention can execute the image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the image processing method provided by any embodiment of the present invention.

[0091] Embodiment 5

[0092] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0093] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0094] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0095] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an image processing method.

[0096] In some embodiments, the image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0097] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable image processing device, so that when the computer programs are executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0099] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein may be implemented on a heterogeneous hardware accelerator having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the heterogeneous hardware accelerator. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0101] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0102] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0103] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0104] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time; Acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates; The distortion correction parameters are sent to an image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after the distortion adjustment through a windshield.

2. The method according to claim 1, characterized in that The step of obtaining matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates includes: Acquire a real-time eye box according to the rotation angle of the curved mirror; A first position coordinate of the viewpoint position coordinate relative to the central axis of the real-time eye box is obtained, and a matching distortion correction parameter is obtained according to the first position coordinate.

3. The method according to claim 1, characterized in that The real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: The rotation angle of the curved mirror in the head-up display is obtained according to the position of the limit switch configured in the head-up display and the number of rotation steps of the stepping motor in the head-up display.

4. The method according to claim 1, characterized in that: The real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: Acquire a virtual image displayed by the windshield by positioning a camera; The rotation angle of the curved mirror in the head-up display is obtained according to the second position coordinates of the marker in the current virtual image relative to the fixed reference object.

5. The method according to claim 1, characterized in that The real-time acquisition of the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user includes: According to the viewpoint position coordinates of the target user, the grid area where the viewpoint of the target user is located is determined, and the viewpoint area coordinates of the grid area where the viewpoint is located are used as the viewpoint position coordinates of the target user.

6. The method according to claim 1, characterized in that The step of obtaining matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates includes: If no matching distortion correction parameters are obtained through the correction rule mapping table according to the current viewpoint position coordinates, obtaining the first viewpoint position coordinates in the correction rule mapping table that are closest to the current viewpoint position coordinates; The distortion correction parameter corresponding to the first viewpoint position coordinate is used as the distortion correction parameter matching the current viewpoint position coordinate.

7. An image processing device, characterized in that: include: Angle coordinate acquisition module, used to obtain the rotation angle of the curved mirror in the head-up display and the viewpoint position coordinates of the target user in real time; A correction parameter acquisition module, used to acquire matching distortion correction parameters according to the rotation angle and the viewpoint position coordinates; The distortion adjustment execution module is used to send the distortion correction parameters to the image generation unit in the head-up display, so that the image generation unit performs distortion adjustment according to the distortion correction parameters, and displays the virtual image after the distortion adjustment through the windshield.

8. The device according to claim 7, characterized in that The correction parameter acquisition module is specifically used to obtain a real-time eye box according to the rotation angle of the curved mirror; A first position coordinate of the viewpoint position coordinate relative to the central axis of the real-time eye box is obtained, and a matching distortion correction parameter is obtained according to the first position coordinate.

9. The device according to claim 7, characterized in that The angle coordinate acquisition module is specifically used to acquire the rotation angle of the curved mirror in the head-up display according to the position of the limit switch configured in the head-up display and the number of rotation steps of the stepping motor in the head-up display.

10. The device according to claim 7, characterized in that The angle coordinate acquisition module is also used to acquire the virtual image displayed on the windshield by positioning the camera; and acquire the rotation angle of the curved mirror in the head-up display according to the second position coordinates of the marker in the current virtual image relative to the fixed reference object.

11. The device according to claim 7, characterized in that The angle coordinate acquisition module is further used to determine the grid area where the viewpoint of the target user is located according to the viewpoint position coordinates of the target user, and use the viewpoint area coordinates of the grid area where the viewpoint is located as the viewpoint position coordinates of the target user.

12. The device according to claim 7, characterized in that The correction parameter acquisition module is also used to obtain the first viewpoint position coordinates in the correction rule mapping table that are closest to the current viewpoint position coordinates if no matching distortion correction parameters are obtained through the correction rule mapping table based on the current viewpoint position coordinates; and use the distortion correction parameters corresponding to the first viewpoint position coordinates as the distortion correction parameters that match the current viewpoint position coordinates.

13. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image processing method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method according to any one of claims 1 to 6 when executed.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Images