Image partitioned blurring method and apparatus, device, and medium

Abstract

An image partitioned blurring method and apparatus, a device, and a medium. The method comprises: acquiring left and right eye position data of a user in real time, and on the basis of the left and right eye position data, determining a viewpoint position of the user in an imaging plane (S110); on the basis of the viewpoint position and a human eye observation range, determining a virtual image blurred region in the imaging plane (S120); acquiring an original projection image to be displayed that matches the imaging plane, and blurring an image region of the original projection image that matches the virtual image blurred region, so as to obtain a target projection image (S130); and projecting the target projection image onto a display screen for display (S140).

Description

Image partition blurring method, device, equipment and medium

[0001] The present application claims priority to the Chinese patent application No. 202411031553.3, filed on July 30, 2024, to the Chinese Patent Office, the whole content of the above application being incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of 3D head-up display technology, for example to an image partition blurring method, device, equipment and medium. BACKGROUND

[0003] Naked-eye 3D display technology refers to a kind of 3D display technology that users can directly watch three-dimensional images with naked eyes without wearing special 3D glasses and other auxiliary equipment, and presents 3D effect. Naked-eye 3D display technology has a wide range of applications, such as movie playing, interactive projection, stage performance and head-up display, etc.

[0004] In the head-up display application scenario of a vehicle, naked-eye 3D display technology can provide more rich and intuitive information display for the driver, increase driving safety and convenience, especially the naked-eye 3D display combined with augmented reality effect, which can provide 3D holographic images in head-up display, helping the driver to obtain road information in a lifelike way.

[0005] In the related art, an image can contain both 2D interface and 3D interface. For the head-up display picture of a vehicle head-up display device (HUD, Head-Up display), or when the audience needs to focus their attention on a certain target on the picture, if the complete picture is projected, the 2D interface in the picture can interfere with the user's viewing of important parts, affecting the visual experience of the audience. SUMMARY

[0006] The present application provides an image partition blurring method, device, equipment and medium, which can blur the picture area outside the virtual image observable by the human eye, and dynamically adjust the blurred area with the movement of the human eye, improve the picture display effect in the observable area of the human eye, and thus provide better visual experience for the user.

[0007] According to an aspect of the present application, an image partition blurring method is provided, comprising:

[0008] real-time acquiring left and right eye position data of a user, and determining a viewpoint position of the user in an imaging plane according to the left and right eye position data;

[0009] determining a virtual image blurring area in the imaging plane according to the viewpoint position and the human eye observation range;

[0010] acquire a raw projection image to be displayed matching the imaging plane, and perform blur processing on an image region in the raw projection image matching the virtual image blur region to obtain a target projection image;

[0011] project the target projection image into a display screen for display.

[0012] Optionally, left and right eye position data of the user are acquired in real time, and a viewpoint position of the user in the imaging plane is determined according to the left and right eye position data, including:

[0013] determining a raw viewpoint position according to the left and right eye position data acquired by the DMS in real time;

[0014] determining the viewpoint position of the user in the imaging plane according to a calibration result of the viewpoint position determined by the DMS in the imaging plane and the raw viewpoint position.

[0015] The advantage of this arrangement is that the left and right eye position data are acquired in real time by the DMS, dynamic detection of human eye data is achieved, the viewing state of the user is tracked in real time, and the blur region can be dynamically adjusted in subsequent methods. The viewpoint position of the user in the imaging plane is determined according to the calibration result of the viewpoint position determined by the DMS in the imaging plane and the raw viewpoint position, the position of the viewpoint in the imaging plane can be accurately determined, and the calculation accuracy of the blur region is improved.

[0016] Optionally, the viewpoint position of the user in the imaging plane is determined according to the calibration result of the viewpoint position determined by the DMS in the imaging plane and the raw viewpoint position, including:

[0017] acquiring a horizontal and vertical coordinate offset of the viewpoint position determined by the DMS in the imaging plane;

[0018] wherein, when observing a calibration point in the imaging plane by the user, the horizontal and vertical coordinate offset is calibrated by acquiring the calibration viewpoint position determined by the DMS;

[0019] determining the viewpoint position of the user in the imaging plane according to the horizontal and vertical coordinate offset, the raw viewpoint position, size information of the imaging plane, and a viewpoint position range determined by the DMS.

[0020] The advantage of this arrangement is that the viewpoint position of the user in the imaging plane is determined, the left and right eye position is replaced by the raw viewpoint position for calculation of the blur region, the observation range of the human eye is finally determined, the calculation complexity is effectively reduced, and the calculation accuracy is ensured.

[0021] Optionally, based on the horizontal and vertical coordinate offsets, the original viewpoint position, the size information of the imaging plane, and the viewpoint position range determined by DMS acquisition, the user's viewpoint position is determined in the imaging plane, including:

[0022] Based on the size information of the imaging plane and the range of viewpoint position determined by DMS acquisition, a first ratio value and a second ratio value are calculated respectively; wherein, the first ratio value is the ratio between the width of the imaging plane and the range of the horizontal coordinate of the viewpoint, and the second ratio value is the ratio between the height of the imaging plane and the range of the vertical coordinate of the viewpoint.

[0023] The x-coordinate of the user's viewpoint position in the imaging plane is calculated using the formula (viewpointX+OffsetX)*DeltaX-ScreenWidth / 2, and the y-coordinate of the user's viewpoint position in the imaging plane is calculated using the formula (viewpointY-OffsetY)*DeltaY-ScreenHeight / 2.

[0024] Where viewpointX and viewpointY are the horizontal and vertical coordinates of the original viewpoint position, respectively; OffsetX and OffsetY are the horizontal and vertical coordinate offsets, respectively; DeltaX and DeltaY are the first and second scale values, respectively; ScreenWidth is the width of the imaging plane; and ScreenHeight is the height of the imaging plane.

[0025] The advantage of this setting is that by pre-calibrating the horizontal and vertical coordinate offsets and then calculating the viewpoint position based on the horizontal and vertical coordinate offsets, the error generated by the DMS data acquisition can be effectively reduced. By calculating the viewpoint position based on the first and second scale values, the influence of scaling of different-sized imaging planes on the viewpoint position can be taken into account, thereby improving the accuracy of the viewpoint position calculation.

[0026] Optionally, based on the viewpoint position and the human eye's observation range, the blurred region of the virtual image is determined in the imaging plane, including:

[0027] Based on the range of human eye observation, the visible width is determined on the imaging plane;

[0028] Based on the horizontal coordinate value of the user's viewpoint in the viewpoint position and the visible width, the horizontal coordinate values ​​of the left and right boundaries to be blurred in the imaging plane are determined respectively.

[0029] Based on the boundaries of the imaging plane, the abscissa values ​​of the left and right virtual image boundaries to be blurred, the left and right virtual image blurred regions are determined in the imaging plane.

[0030] The advantage of the arrangement is that: by determining the visible width on the imaging plane according to the human eye observation range, and then determining the horizontal coordinate value of the to-be-blurred boundary according to the visible width, and then determining the left virtual image blurred area and the right virtual image blurred area in the imaging plane according to the boundary of the imaging plane, the horizontal coordinate value of the left to-be-blurred boundary and the horizontal coordinate value of the right to-be-blurred boundary, the to-be-blurred area in the imaging plane can be quickly divided, and the calculation efficiency is improved while the blurred area is accurately divided.

[0031] Optionally, the horizontal coordinate value of the left to-be-blurred boundary and the horizontal coordinate value of the right to-be-blurred boundary in the imaging plane are respectively determined according to the horizontal coordinate value of the user's eye point in the eye point position and the visible width, including:

[0032] The horizontal coordinate value vagueXL of the left to-be-blurred boundary and the horizontal coordinate value vagueXR of the right to-be-blurred boundary in the imaging plane are calculated according to the formula vagueXL=EyeRect.anchoredPosition.x-△Width / 2 and the formula vagueXR=EyeRect.anchoredPosition.x+△Width / 2.

[0033] Wherein, EyeRect.anchoredPosition.x is the horizontal coordinate value of the user's eye point in the eye point position, and △Width is the visible width.

[0034] The advantage of the arrangement is that: the calculation is simple, the calculation result is accurate, the field of view area and the blurred area can be accurately divided, and then the to-be-blurred area in the imaging plane can be quickly divided, and the calculation efficiency is improved while the blurred area is accurately divided.

[0035] Optionally, the image area matched with the virtual image blurred area in the original projection image is blurred to obtain a target projection image, including:

[0036] A blurred projection image covering the virtual image blurred area is generated by calling a calculation shader in an image processing engine.

[0037] The covering center point position of the blurred projection image in the original projection image is calculated, and the blurred projection image is covered on the original projection image according to the covering center point position to obtain a target projection image.

[0038] The advantage of the arrangement is that: the image can be quickly blurred by using the image processing engine, the blurred image generated by using the calculation shader has good blurred effect, not only can the blurred effect be generated, but also the user can roughly understand the content of the blurred area, and the user's visual experience is improved.

[0039] Optionally, the blurred projection image covering the virtual image blur area is generated by calling a compute shader in the image processing engine, including:

[0040] The covering image with the same size as the virtual image blur area is generated by a compute shader in the image processing engine.

[0041] The material of the covering image is replaced by a Gaussian blur material ball by a compute shader in the image processing engine, to generate the blurred projection image covering the virtual image blur area.

[0042] The advantage of this setting is that the blurred projection image covering the virtual image blur area can be quickly generated by using the characteristics of the image processing engine, and the replacement of the material can make the blurred image part both recognizable to the user and have a certain blur effect, highlighting the content of the unblurred area.

[0043] Optionally, the virtual image blur area is determined in the imaging plane according to the viewpoint position and the human eye observation range, including:

[0044] The left and right boundaries of the visible area are determined on the imaging plane according to the viewpoint position and the visible radius.

[0045] The left and right virtual image blur areas are determined in the imaging plane according to the boundaries of the imaging plane, the left and right boundaries of the visible area.

[0046] The advantage of this setting is that when setting a visible area of different shapes, the virtual image blur area can be accurately determined for different shapes of the visible area, so that the blurred image can be accurately divided according to the designer's requirements.

[0047] Optionally, the image area in the original projection image matching the virtual image blur area is blurred to obtain a target projection image, including:

[0048] The covering image with the same size as the virtual image blur area is generated by a compute shader in the image processing engine, and the material of the covering image is replaced by a Gaussian blur material ball, and the shape of the covering image is converted to a target shape matching the left and right boundaries of the visible area.

[0049] The covering center point position of the blurred projection image in the original projection image is calculated, and the blurred projection image is overlaid on the original projection image according to the covering center point position to obtain a target projection image.

[0050] The advantage of this setting is that the image processing engine can be used to blur the blurred area with a curved boundary, meeting the setting requirements of the developer for different shapes of the visible area.

[0051] Optionally, the target projection image is projected into a display screen for display, including: projecting the target projection image into the display screen for display by the three-dimensional head-up display device.

[0052] The advantage of such an arrangement is that the image after partition blurring can be displayed by 3D projection, so that the 3D elements in the image are clearly displayed, and the user can read the content of the blurred 2D elements in the image, and the blurred 2D elements can highlight the 3D elements to some extent, thereby providing a better 3D head-up display experience for the driver.

[0053] Optionally, the target projection image includes a three-dimensional head-up display image.

[0054] The advantage of such an arrangement is that by blurring the image in the partition of the three-dimensional head-up display image, the user can be shown clear 3D projection elements when the driver uses the three-dimensional head-up display, thereby improving the driving experience of the user.

[0055] According to another aspect of the present application, an image partition blurring device is provided, including:

[0056] A viewpoint position determination module is configured to acquire left and right eye position data of a user in real time, and determine a viewpoint position of the user in an imaging plane according to the left and right eye position data.

[0057] A virtual image blurring area determination module is configured to determine a virtual image blurring area in the imaging plane according to the viewpoint position and a human eye observation range.

[0058] A projection image generation module is configured to acquire an original projection image to be displayed matched with the imaging plane, and blur an image area in the original projection image matched with the virtual image blurring area to obtain a target projection image.

[0059] An image projection module is configured to project the target projection image into a display screen for display.

[0060] According to another aspect of the present application, a projection device is provided, including:

[0061] At least one processor; and

[0062] A memory in communication connection with the at least one processor; wherein,

[0063] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image partition blurring method according to any one of the embodiments of the present application.

[0064] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the image partition blur method according to any of the embodiments of the present application when executed.

[0065] The technical solution of the embodiments of the present application determines the viewpoint position of the user in the imaging plane according to the real-time acquired left and right eye position data, determines the virtual image blur area in the imaging plane according to the viewpoint position and the human eye observation range, performs blur processing on the image area in the original projection image to be displayed which matches the virtual image blur area in the imaging plane, obtains the target projection image, and projects the target projection image to the display screen for display. In this way, the image area to be blurred can be determined according to the real-time position of the human eye, and then the projection image is projected and displayed after blur processing, so that the image area outside the field of view in the user's visual angle is blurred, the user can better focus on the image in the field of view, the display effect of the 3D content in the field of view is effectively improved, and the blur area can be dynamically adjusted, so that the user can achieve better viewing effect at any viewing position.

[0066] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0067] Fig. 1 is a flowchart of an image partition blur method according to an embodiment of the present application;

[0068] Fig. 2 is a schematic diagram of a head-up display imaging process according to an embodiment of the present application;

[0069] Fig. 3 is a schematic diagram of the relationship between the viewpoint and the virtual image blur area according to an embodiment of the present application;

[0070] Fig. 4 is another schematic diagram of the relationship between the viewpoint and the virtual image blur area according to an embodiment of the present application;

[0071] Fig. 5 is an image partition blur effect diagram of a head-up display image according to an embodiment of the present application;

[0072] Fig. 6 is a schematic diagram of the principle of naked-eye 3D imaging according to an embodiment of the present application;

[0073] Fig. 7 is a flowchart of another image partition blur method according to an embodiment of the present application;

[0074] Fig. 8 is a flowchart of another image partition blur method according to an embodiment of the present application;

[0075] Fig. 9 is a structural schematic diagram of an image partition blurring device according to an embodiment of the present application;

[0076] Fig. 10 is a structural schematic diagram of a projection device implementing an image partition blurring method according to an embodiment of the present application. DETAILED DESCRIPTION

[0077] For those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0078] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0079] The present application can be applied to the head-up display function of a vehicle, and the application of the image partition blurring method in the head-up display function of a vehicle is specifically described in the embodiments of the present application, wherein the vehicle is intended to represent various forms of vehicles. For example, the vehicles can include but are not limited to land vehicles such as cars, aerial vehicles (or called aircraft) such as airplanes, or water or underwater vehicles, etc.

[0080] Optionally, in the application of the head-up display of the vehicle, the head-up display device of the vehicle can implement the processing and projection display of the projection image. In the head-up display device, at least one display can be included as a display pixel unit, at least one optical grating device can be included as a pixel imaging unit, at least one optical mirror device can be included as an imaging unit, and at least one optical sensor device can be included as an image capturing unit. The front display part of the naked-eye 3D display technology needs to calculate and generate a projection image through the pixel imaging unit, so that the left eye and the right eye of a person see different pictures and fuse into a 3D effect picture in the brain.

[0081] Optionally, with the development of head-up display, the head-up display in the related art can provide augmented reality (AR, Augmented Reality) content, and AR-HUD (Augmented Reality-Head Up Display) refers to the augmented reality head-up display function. Compared with the traditional HUD, the AR-HUD has a larger imaging area, a longer projection distance, and a more vivid and intuitive image. The AR-HUD can reasonably superimpose some driving information in the driver's visual field, fuse navigation and ADAS (Advanced Driving Assistance System, Advanced Driving Assistance System) information with the front road, and directly display steering instructions, obstacle warnings, lane deviations, front vehicle warnings, blind area monitoring, and other contents. In the actual use of the AR-HUD, the virtual image displayed can be composed of naked-eye 3D content and 2D UI (User Interface, User Interface) content. When the user gazes at the naked-eye 3D content, if the 2D UI interface is too clear, it will affect the user's 3D experience, thereby affecting the product perception.

[0082] Embodiment one

[0083] FIG. 1 is a flowchart of an image partition blurring method provided by an embodiment of the present application. The embodiment can be applicable to the case of blurring the image outside the user's field of view in a 3D projection image. The method can be executed by an image partition blurring device, which can be realized in the form of hardware and / or software and can generally be configured in a projection device with image processing function. As shown in FIG. 1, the method comprises:

[0084] S110, real-time acquisition of left and right eye position data of the user, and determination of the user's viewpoint position in the imaging plane according to the left and right eye position data.

[0085] FIG. 2 is a schematic diagram of an optional head-up display imaging process. As shown in FIG. 2, the image source in the HUD emits light imaging and is projected onto the windshield of the vehicle via a plane mirror and a curved mirror. The windshield is an optional imaging element. If the vehicle is configured with other imaging elements, the projection can also be performed on other imaging elements. The eyebox refers to the range of imaging light that can be accepted by the driver's eye height. The actual head-up display content observed in the driver's visual angle is actually a virtual image on a virtual image plane in space. Through the modification of the projection image, the virtual image on the virtual image plane can be adjusted, and finally the image change effect of the driver's visual angle can be realized.

[0086] Optionally, the left and right eye position data of the user can be acquired by a DMS (Driver Monitoring System) configured in the vehicle. The eye tracking device in the DMS is a device that acquires the eye state of the driver in real time by installing an optical camera and an infrared camera on the vehicle. The eye tracking device can analyze the acquired information through a deep learning algorithm to determine the state of the driver, thereby achieving driver identification, driver fatigue monitoring, driver attention monitoring, and dangerous driving behavior monitoring, and providing different levels of warnings. In the case of ensuring numerical accuracy and refresh rate, the data acquired by the DMS can be sent to the projection device through the UDP (User Datagram Protocol) protocol.

[0087] Optionally, the left and right eye position data acquired by the DMS can be the position coordinates of the left and right eyes of the driver in the real world, and the left and right eye position coordinates can be determined based on the DMS coordinate system in the real world.

[0088] Optionally, after acquiring the left and right eye position coordinates, the original viewpoint position in the real world can be determined according to the left and right eye position coordinates. The original viewpoint position can be the center point position of the left and right eye positions. The original viewpoint position is used instead of the left and right eye positions for calculation to finally determine the eye observation range, which can effectively reduce the calculation complexity while ensuring the calculation accuracy.

[0089] Optionally, the imaging plane can refer to the plane on which the projected image observed by the user is located. If applied to the head-up display of the vehicle, the imaging plane can refer to the virtual image plane on which the target observed by the user is located. If applied to screen projection, the imaging plane can refer to the plane on which the screen is located.

[0090] It can be understood that the original viewpoint and the imaging plane generally have a certain distance, and therefore, the position of the user's viewpoint on the imaging plane cannot be determined according to the original viewpoint position. Therefore, the position of the viewpoint in the imaging plane needs to be determined through calculation to calculate the field of view of the user.

[0091] S120, determining a virtual image blur area in the imaging plane according to the viewpoint position and the eye observation range.

[0092] Fig. 3 is a schematic view of a relationship between an optional viewpoint and a virtual image blur area. As shown in Fig. 3, the outermost rectangle can refer to a complete range of a projected image displayed in an imaging plane. The viewpoint is a viewpoint on the imaging plane. After the position of the viewpoint is determined, a field of view area of the viewpoint can be determined. Generally, the content in the field of view area can be focused on by a user. The content outside the field of view area is a blur area. In a user's perspective, the content in the blur area is blurred relative to the field of view area. However, if the blur area is not further blurred, the visual residual caused by the blur area will affect the user's viewing of the content in the field of view area. Therefore, in order to improve the visual experience of the image in the field of view area, the present application proposes to blur the part of the projected image that will be projected in the blur area, so that the projected image falling into the blur area is the image that has been blurred. It should be noted that the division of the field of view area and the blur area in Fig. 3 is only an optional example. The field of view area can also be processed as a rectangle instead of a circle. This is only an exemplary illustration and does not limit the specific division method of the field of view area and the blur area.

[0093] Optionally, the human eye observation range can be a preset fixed value, or can be dynamically set. For example, in a 3D head-up display application scenario, if the projection element is a 3D navigation element, the prompt to the user generally depends on the road condition. If the distance of the road to be prompted is far, the human eye observation range can be dynamically adjusted to a smaller range. If the distance of the road to be prompted is close, the human eye observation range can be dynamically adjusted to a larger range. However, this is only an exemplary illustration and does not limit the setting method of the human eye observation range.

[0094] Fig. 4 is another schematic view of a relationship between an optional viewpoint and a virtual image blur area. As shown in Fig. 4, after the position of the viewpoint is determined, the human eye observation range can be determined according to a preset visible width. The black area is the blur area and the white area is the field of view area.

[0095] Optionally, the human eye observation range can be the radius of the circular field of view area as shown in Fig. 3, or can be the visible width of the rectangular field of view area as shown in Fig. 4. The shape of the field of view area is not limited, and therefore the human eye observation range is not limited. The human eye observation range can be determined according to the calculation requirement.

[0096] S130, acquiring an original projected image to be displayed matched with the imaging plane, and blurring the image area in the original projected image matched with the virtual image blur area to obtain a target projected image.

[0097] Optionally, the original projection image is a complete image to be displayed without blur processing, and according to the determined virtual image blur area, an image area corresponding to the virtual image blur area can be determined in the original projection image, and then the area is subjected to blur processing to obtain the target projection image.

[0098] Optionally, an image processing engine capable of creating and rendering a 3D model can be selected to perform blur processing on the original projection image. A compute shader in the image processing engine can be used to define the appearance of a material, including color, texture, lighting, and other attributes. By writing code for the compute shader, the surface of the 3D model can be controlled to reflect light and display color.

[0099] Optionally, the area of the imaging plane is proportionally mapped to the original projection image. After determining the virtual image blur area in the imaging plane, the image area to be blurred in the original projection image can be determined.

[0100] Optionally, an image processing engine can be used to generate an image with the same size as the image area to be blurred. Then, material replacement can be performed on the generated image. Material replacement can be performed by a compute shader in the image processing engine, replacing the material with a Gaussian blur material ball. Then, the image after material replacement is overlaid on the corresponding position in the original projection image to achieve blur processing on the image area matching the virtual image blur area.

[0101] Optionally, Gaussian blur is a blur algorithm based on a normal distribution function. The basic principle is to weight each pixel point in the image according to the normal distribution function, thereby obtaining a more blurred image. In practical applications, Gaussian blur uses a convolution kernel to calculate each pixel point. The size of the convolution kernel can be defined by the user. Generally, the larger the size of the convolution kernel, the more obvious the blur effect of the image. Specifically, the convolution kernel is generally a two-dimensional matrix. In the calculation process of Gaussian blur, each pixel point is covered by the convolution kernel together with the surrounding pixel points. The values in the convolution kernel represent the weight of each pixel point. A higher weight represents a greater contribution of the pixel point to the entire pixel point. The convolution process adds the weights of all pixel points to obtain a new pixel point, thereby achieving image blur processing.

[0102] S140, projecting the target projection image into the display screen for display.

[0103] Fig. 5 is an optional partition blur effect diagram of a head-up display image, as shown in Fig. 5, the left side is the projection effect of the original projection image without image blur, and the right side is the projection effect of the target projection image after image partition blur, the arrow projected in the middle of the image indicates 3D projection, which can be a dynamic element of navigation indication, the ghosting effect of the arrow in the image is actually the superposition effect of the left and right eye images, the arrow in the driver's eye view is 3D dynamic, and the blurred image content on both sides is 2D display content, for example, vehicle speed, vehicle status identifier, etc., such content does not need to be dynamically displayed, and can be displayed through 2D display, after image blur processing in the right image, the driver can pay more attention to the navigation arrow indication in the middle of the image, in the driver's view, the navigation arrow indication is a clear 3D dynamic effect, and the 2D elements outside the field of view are blurred, but Fig. 5 is only illustrative and does not limit the specific application scenarios.

[0104] Optionally, for the head-up display function of the vehicle, the display screen can be the front windshield of the vehicle, or other imaging elements configured in the vehicle, which are not listed here and are only illustrative.

[0105] Fig. 6 is an optional naked eye 3D imaging principle diagram, in Fig. 6, the HUD is optically designed so that the user's left eye sees image P1 through the imaging structure, such as the windshield, and the right eye sees image P2. Due to the binocular disparity formed by image P1 and image P2, the user sees the object with depth and spatial sense, and the right eye sees the right eye image, which is synthesized into a stereoscopic picture with depth in the user's brain. Naked eye 3D imaging can change the position between the two images to adjust the binocular disparity, so that the user subjectively feels that the virtual image distance has changed (the actual distance of the virtual image does not change), the closer the two images, the closer the user subjectively feels the virtual image distance; on the contrary, the farther the two images, the farther the user subjectively feels the virtual image distance.

[0106] The technical scheme of the embodiment of the present application determines the viewpoint position of the user in the imaging plane according to the real-time acquired left and right eye position data, determines the virtual image blur area in the imaging plane according to the viewpoint position and the human eye observation range, performs blur processing on the image area matched with the virtual image blur area in the original projection image to be displayed matched with the imaging plane to obtain a target projection image, and projects and displays the target projection image in the display screen, which can determine the image area to be blurred according to the real-time position of the human eye, and then project and display the projection image after blur processing, so that the image area outside the field of view in the user's view is blurred, the user can better focus on the image in the field of view, effectively improve the 3D content display effect in the field of view, and dynamically adjust the blur area, so that the user can achieve better viewing effect at any viewing position.

[0107] Embodiment Two

[0108] FIG. 7 is a flow chart of another image partition blurring method provided by Embodiment Two of the present application. This embodiment further illustrates the image partition blurring method based on the above-mentioned embodiments. As shown in FIG. 7, the method comprises the following steps:

[0109] S210. Determine the original viewpoint position according to the left and right eye position data collected by the DMS in real time.

[0110] Optionally, the left and right eye position data can be the left eye coordinate and the right eye coordinate in the DMS coordinate system. After obtaining the left eye coordinate and the right eye coordinate, the central point coordinate between the left eye coordinate and the right eye coordinate can be determined as the coordinate of the original viewpoint position. This coordinate can only represent the central point position of the viewpoint. Referring to the human eye, a certain range around the central point position also belongs to the viewpoint, but the central point of the selected viewpoint can be calculated when calculating.

[0111] In an optional example, when the left eye coordinate is (x1, y1) and the right eye coordinate is (x2, y2), the original viewpoint position coordinate can be ((x1+x2) / 2,(y1+y2) / 2). This is only an exemplary illustration and does not limit the original viewpoint position.

[0112] S220. Determine the viewpoint position of the user in the imaging plane according to the calibration result of the viewpoint position determined by the DMS collection in the imaging plane and the original viewpoint position.

[0113] Optionally, determining the viewpoint position of the user in the imaging plane according to the calibration result of the viewpoint position determined by the DMS collection in the imaging plane and the original viewpoint position comprises:

[0114] Obtain the horizontal and vertical coordinate offset of the viewpoint position determined by the DMS collection in the imaging plane;

[0115] Wherein, when observing the user fixating on the calibration point in the imaging plane, the horizontal and vertical coordinate offset is calibrated by obtaining the calibration viewpoint position determined by the DMS collection;

[0116] Determine the viewpoint position of the user in the imaging plane according to the horizontal and vertical coordinate offset, the original viewpoint position, the size information of the imaging plane, and the range of the viewpoint position determined by the DMS collection.

[0117] Optionally, the calibration point in the imaging plane can be the center point of the imaging plane, or other predetermined points, which are not limited here. Taking the center point of the imaging plane as an example, the coordinates of the center point can be set as (0, 0). Then, when the user gazes at the calibration point, the viewpoint position (x3, y3) collected by the DMS when the user gazes at the calibration point is collected, and then the horizontal coordinate difference and the vertical coordinate difference are calculated respectively according to the coordinates of the calibration point and the coordinates of the viewpoint position, so as to obtain the horizontal coordinate offset and the vertical coordinate offset. The horizontal and vertical coordinate offsets can be calibrated only once in a certain period of time.

[0118] Optionally, according to the horizontal and vertical coordinate offsets, the original viewpoint position, the size information of the imaging plane, and the viewpoint position range determined by the DMS collection, the viewpoint position of the user in the imaging plane is determined, comprising:

[0119] According to the size information of the imaging plane and the viewpoint position range determined by the DMS collection, a first proportion value and a second proportion value are calculated respectively; wherein the first proportion value is the ratio between the width of the imaging plane and the horizontal coordinate range of the viewpoint, and the second proportion value is the ratio between the height of the imaging plane and the vertical coordinate range of the viewpoint.

[0120] According to the formula (viewpointX+OffsetX)*DeltaX-ScreenWidth / 2, the horizontal coordinate of the viewpoint position of the user in the imaging plane is calculated, and according to the formula (viewpointY-OffsetY)*DeltaY-ScreenHeight / 2, the vertical coordinate of the viewpoint position of the user in the imaging plane is calculated.

[0121] Wherein, viewpointX and viewpointY are the horizontal and vertical coordinate values in the original viewpoint position, OffsetX and OffsetY are the horizontal and vertical coordinate offsets, DeltaX and DeltaY are the first proportion value and the second proportion value respectively, ScreenWidth is the width of the imaging plane, and ScreenHeight is the height of the imaging plane.

[0122] Optionally, the size information of the imaging plane can include the height of the imaging plane and the width of the imaging plane.

[0123] In an optional example, the first proportion value DeltaX can be calculated according to the formula: DeltaX=ScreenWidth / VPXL, and the second proportion value DeltaY can be calculated according to the formula: DeltaY=ScreenHeight / VPYL, wherein VPXL is the horizontal coordinate range of the viewpoint, and VPYL is the vertical coordinate range of the viewpoint.

[0124] It can be understood that the human eye is not a point, but a small area, therefore, the view point is actually a small coordinate range, the horizontal coordinate range of the view point can be calculated according to the minimum horizontal coordinate and the maximum horizontal coordinate of the view point position range, and the vertical coordinate range of the view point can be calculated according to the minimum vertical coordinate and the maximum vertical coordinate of the view point position range.

[0125] S230, determining a visible width on the imaging plane according to the human eye observation range.

[0126] Optionally, taking FIG. 4 as an example, the △Width can be the human eye observation range, which represents the visible width of the field of view area that the user can focus on on the imaging plane, if taking FIG. 3 as an example, the human eye observation range can also be the radius of the field of view area, after the field of view area is determined according to the radius, the visible width can be determined according to the leftmost horizontal coordinate value and the rightmost horizontal coordinate value of the field of view area, the human eye observation range value is different in different application scenarios, which is not limited here.

[0127] S240, respectively determining the horizontal coordinate value of the left side to-be-blurred boundary and the horizontal coordinate value of the right side to-be-blurred boundary in the imaging plane according to the horizontal coordinate value of the user's view point in the view point position and the visible width.

[0128] Optionally, respectively determining the horizontal coordinate value of the left side to-be-blurred boundary and the horizontal coordinate value of the right side to-be-blurred boundary in the imaging plane according to the horizontal coordinate value of the user's view point in the view point position and the visible width, including:

[0129] calculating the horizontal coordinate value vagueXL of the left side to-be-blurred boundary and the horizontal coordinate value vagueXR of the right side to-be-blurred boundary in the imaging plane according to the formula vagueXL=EyeRect.anchoredPosition.x-△Width / 2 and the formula vagueXR=EyeRect.anchoredPosition.x+△Width / 2;

[0130] Wherein, EyeRect.anchoredPosition.x is the horizontal coordinate value of the user's view point in the view point position, and △Width is the visible width.

[0131] Optionally, the boundary between the field of view area and the blurred area can be used as the to-be-blurred boundary, taking FIG. 4 as an example, the positions of the horizontal coordinate vagueXL of the left side to-be-blurred boundary and the horizontal coordinate vagueXR of the right side to-be-blurred boundary are shown, which is convenient for understanding.

[0132] S250, determining the left side virtual image blurred area and the right side virtual image blurred area in the imaging plane according to the boundary of the imaging plane, the horizontal coordinate value of the left side to-be-blurred boundary and the horizontal coordinate value of the right side to-be-blurred boundary.

[0133] Optionally, according to the left and right boundaries of the imaging plane, the horizontal coordinate value of the left side to be blurred boundary, and the horizontal coordinate value of the right side to be blurred boundary, the left and right side rectangular virtual image blur areas shown in FIG. 4 can be determined.

[0134] In another optional example shown in FIG. 3, after the horizontal coordinate value of the left side to be blurred boundary and the horizontal coordinate value of the right side to be blurred boundary are determined, the virtual image blur areas with arc-shaped boundaries on the left and right sides can also be determined according to the radius of the field of view in the human eye observation range and the boundary of the imaging plane.

[0135] S260, generate a blur projection image covering the virtual image blur area by calling a compute shader in the image processing engine.

[0136] Optionally, generating the blur projection image covering the virtual image blur area by calling the compute shader in the image processing engine includes:

[0137] generating a cover image with the same size as the virtual image blur area by the compute shader in the image processing engine;

[0138] replacing the material of the cover image with a Gaussian blur material ball to generate the blur projection image covering the virtual image blur area by the compute shader in the image processing engine.

[0139] Optionally, the compute shader is a component in the image processing engine that is independent of the rendering pipeline but can read and write operations on GPU (Graphics Processing Unit) resources. Accessing the GPU using the compute shader can implement a data parallel algorithm to improve image processing efficiency.

[0140] In the example shown in FIG. 4, if the left and right virtual image blur areas are both rectangular, the image covering the virtual image blur area can be directly replaced with a Gaussian blur material ball by the compute shader.

[0141] In the example shown in FIG. 3, in addition to the material replacement by the compute shader, shape conversion is also required, from a rectangle to a non-rectangle, to adapt to the arc-shaped boundary in FIG. 3.

[0142] Optionally, determining the virtual image blur area in the imaging plane according to the viewpoint position and the human eye observation range further includes:

[0143] determining the left and right boundaries of the visible area on the imaging plane according to the viewpoint position and the visible radius;

[0144] According to the boundary of the imaging plane, the left boundary and the right boundary of the visible area, the left virtual image blur area and the right virtual image blur area are determined in the imaging plane.

[0145] It can be understood that the determination method of the virtual image blur area is provided herein when the observation range of the human eye is circular. When the observation range of the human eye is circular, the visible radius of the user can be obtained. The visible radius can be a preset fixed value, or can be dynamically adjusted according to the observation situation of the user. For example, when the 3D element is far away in the real world, the radius is reduced, and vice versa. The radius is increased. This is only an exemplary description.

[0146] Optionally, the image area in the original projection image matched with the virtual image blur area is blurred to obtain a target projection image, including:

[0147] The calculation shader in the image processing engine is used to generate a cover image with the same size as the virtual image blur area, replace the material of the cover image with a Gaussian blur material ball, and convert the shape of the cover image into a target shape matched with the left boundary and the right boundary of the visible area.

[0148] The cover center point position of the blurred projection image in the original projection image is calculated, and the blurred projection image is overlaid on the original projection image according to the cover center point position to obtain a target projection image.

[0149] Optionally, for the virtual image blur area with an arc-shaped boundary, in addition to material replacement, shape conversion is also needed through the calculation shader during the blurring process.

[0150] S270, the cover center point position of the blurred projection image in the original projection image is calculated, and the blurred projection image is overlaid on the original projection image according to the cover center point position to obtain a target projection image.

[0151] As in the previous example, the left side coverage center point position can be calculated by the following formula: imageL.anchoredPosition = (vagueXL - imageLwidth / 2, imageL.anchoredPosition.y), and the right side coverage center point position can be calculated by the following formula: imageR.anchoredPosition = (vagueXR - imageRwidth / 2, imageR.anchoredPosition.y); wherein imageLwidth is the left side blurred projection image width, imageRwidth is the right side blurred projection image width, and imageL.anchoredPosition.y and imageR.anchoredPosition.y are the heights of the left side blurred projection image and the right side blurred projection image, respectively.

[0152] S280, projecting the target projection image into a display screen for display.

[0153] Optionally, the target projection image is projected into a display screen for display by a three-dimensional head-up display device.

[0154] Optionally, the target projection image comprises a three-dimensional head-up display image.

[0155] The advantage of such an arrangement is that by means of image partition blurring of the three-dimensional head-up display image, a clear 3D projection element can be shown to the user when the driver uses the three-dimensional head-up display, thereby improving the user's driving experience.

[0156] Fig. 8 is a flowchart of another optional image partition blurring method. In the specific application scenario shown in Fig. 8, the user's eye position is detected in real time, and the screen position coordinates observed by the human eye are extracted by means of certain experimental data extraction of relevant parameters. The virtual image screen range that can be observed by the human eye is calculated through experiments and the human eye field of view range. The human eye observation position coordinates and the human eye observation range are used as parameters to calculate the blurring area required outside the human eye observation screen area, and the area outside the observation screen area is processed by a shader to achieve Gaussian blurring. The virtual image blurring area is dynamically adjusted by means of real-time human eye observation by the DMS, thereby improving the display effect of the naked eye 3D content part.

[0157] The technical scheme of the embodiment of the application determines the viewpoint position of the user in the imaging plane according to the left and right eye position data acquired in real time, determines the virtual image blur area in the imaging plane according to the viewpoint position and the human eye observation range, performs blur processing on the image area matched with the virtual image blur area in the original projection image to be displayed matched with the imaging plane, obtains the target projection image, and projects the target projection image into the display screen for display. In this way, the image area to be blurred can be determined according to the real-time position of the human eye, and then the projection image is projected and displayed after blur processing, so that the image area outside the field of view in the user's visual angle is blurred, the user can better focus on the image in the field of view, the 3D content display effect in the field of view is effectively improved, and the blur area can be dynamically adjusted, so that the user can achieve better viewing effect at any viewing position.

[0158] Embodiment three

[0159] FIG. 9 is a structural schematic diagram of an image partition blur device provided by the embodiment three of the application. As shown in FIG. 9, the device includes a viewpoint position determination module 310, a virtual image blur area determination module 320, a projection image generation module 330, and an image projection module 340.

[0160] The viewpoint position determination module 310 is configured to acquire the left and right eye position data of the user in real time, and determine the viewpoint position of the user in the imaging plane according to the left and right eye position data.

[0161] The virtual image blur area determination module 320 is configured to determine the virtual image blur area in the imaging plane according to the viewpoint position and the human eye observation range.

[0162] The projection image generation module 330 is configured to acquire the original projection image to be displayed matched with the imaging plane, and perform blur processing on the image area matched with the virtual image blur area in the original projection image, to obtain the target projection image.

[0163] The image projection module 340 is configured to project the target projection image into the display screen for display.

[0164] The technical scheme of the embodiments of the present application determines the user's view point position in the imaging plane according to the real-time acquired left and right eye position data, determines the virtual image blur area in the imaging plane according to the view point position and the human eye observation range, performs blur processing on the image area in the original projection image to be displayed which matches the virtual image blur area, obtains a target projection image, and projects and displays the target projection image in the display screen, so that the image area outside the field of view in the user's visual angle is blurred, the user can better focus on the image in the field of view, the 3D content display effect in the field of view is effectively improved, and the blur area can be dynamically adjusted, so that the user can achieve a better viewing effect at any viewing position.

[0165] On the basis of the above embodiments, the view point position determination module 310 can comprise:

[0166] An original view point position determination unit configured to determine an original view point position according to the left and right eye position data of the user collected by the DMS in real time.

[0167] An imaging plane view point position determination unit configured to determine the view point position of the user in the imaging plane according to the calibration result of the view point position determined by the DMS collection in the imaging plane and the original view point position.

[0168] On the basis of the above embodiments, the imaging plane view point position determination unit can be specifically configured to:

[0169] Obtain the horizontal and vertical coordinate offset of the view point position determined by the DMS collection in the imaging plane.

[0170] When the user gazes at the calibration point in the imaging plane, the horizontal and vertical coordinate offset is calibrated by the calibration view point position determined by the DMS collection.

[0171] Determine the view point position of the user in the imaging plane according to the horizontal and vertical coordinate offset, the original view point position, the size information of the imaging plane and the view point position range determined by the DMS collection.

[0172] On the basis of the above embodiments, the imaging plane view point position determination unit can be specifically configured to:

[0173] Calculate a first proportion value and a second proportion value according to the size information of the imaging plane and the view point position range determined by the DMS collection, wherein the first proportion value is the ratio between the width of the imaging plane and the horizontal coordinate range of the view point, and the second proportion value is the ratio between the height of the imaging plane and the vertical coordinate range of the view point.

[0174] The horizontal coordinate of the viewpoint position of the user in the imaging plane is calculated according to the formula (viewpointX+OffsetX)*DeltaX-ScreenWidth / 2, and the vertical coordinate of the viewpoint position of the user in the imaging plane is calculated according to the formula (viewpointY-OffsetY)*DeltaY-ScreenHeight / 2;

[0175] viewpointX and viewpointY are respectively the horizontal and vertical coordinate values in the original viewpoint position, OffsetX and OffsetY are respectively the horizontal and vertical coordinate offsets, DeltaX and DeltaY are respectively the first and second scale values, ScreenWidth is the width of the imaging plane, and ScreenHeight is the height of the imaging plane.

[0176] On the basis of the above embodiments, the virtual image blur area determination module 320 can include:

[0177] A visible width determination unit configured to determine a visible width on the imaging plane according to the human eye observation range.

[0178] A coordinate value determination unit configured to determine, according to the horizontal coordinate value of the user's viewpoint in the viewpoint position and the visible width, a horizontal coordinate value of a left-side blur boundary and a horizontal coordinate value of a right-side blur boundary in the imaging plane.

[0179] A region division unit configured to determine, according to the boundary of the imaging plane, the horizontal coordinate value of the left-side blur boundary and the horizontal coordinate value of the right-side blur boundary, a left-side virtual image blur area and a right-side virtual image blur area in the imaging plane.

[0180] On the basis of the above embodiments, the coordinate value determination unit can be specifically configured to:

[0181] The horizontal coordinate value vagueXL of the left-side blur boundary and the horizontal coordinate value vagueXR of the right-side blur boundary in the imaging plane are calculated according to the formula vagueXL=EyeRect.anchoredPosition.x-△Width / 2 and the formula vagueXR=EyeRect.anchoredPosition.x+△Width / 2;

[0182] wherein EyeRect.anchoredPosition.x is the horizontal coordinate value of the user's viewpoint in the viewpoint position, and △Width is the visible width.

[0183] On the basis of the above embodiments, the projection image generation module 330 can include:

[0184] a blur projection image generation unit configured to generate a blur projection image covering the virtual image blur area by invoking a compute shader in the image processing engine;

[0185] a blur projection image covering unit configured to calculate a covering center point position of the blur projection image in the original projection image, and cover the blur projection image on the original projection image according to the covering center point position to obtain a target projection image.

[0186] On the basis of the above embodiments, the blur projection image generation unit can be specifically configured to:

[0187] generate a covering image with the same size as the virtual image blur area by a compute shader in the image processing engine;

[0188] generate a blur projection image covering the virtual image blur area by replacing the material of the covering image with a material ball of Gaussian blur by a compute shader in the image processing engine.

[0189] On the basis of the above embodiments, the virtual image blur area determination module 320 can be further specifically configured to:

[0190] determine a left side boundary and a right side boundary of the visible area on the imaging plane according to the viewpoint position and the visible radius;

[0191] determine a left side virtual image blur area and a right side virtual image blur area in the imaging plane according to the boundaries of the imaging plane, the left side boundary and the right side boundary of the visible area.

[0192] On the basis of the above embodiments, the projection image generation module 330 can be further specifically configured to:

[0193] generate a covering image with the same size as the virtual image blur area by a compute shader in the image processing engine, replace the material of the covering image with a material ball of Gaussian blur, and convert the shape of the covering image into a target shape matching the left side boundary and the right side boundary of the visible area;

[0194] calculate a covering center point position of the blur projection image in the original projection image, and cover the blur projection image on the original projection image according to the covering center point position to obtain a target projection image.

[0195] On the basis of the above embodiments, the image projection module 340 is configured to project the target projection image into a display screen for display by a three-dimensional head-up display device.

[0196] On the basis of the above embodiments, the target projection image comprises a three-dimensional head-up display image.

[0197] The image partition blurring apparatus provided by the embodiments of the present application can perform the image partition blurring method provided by any of the embodiments of the present application, and has the corresponding function modules and effects of performing the method.

[0198] Embodiment Four

[0199] FIG. 10 shows a structural schematic diagram of a projection device 10 that can be used to implement embodiments of the present application. The projection device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The projection device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown in the figures, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present application as described and / or claimed in this document.

[0200] As shown in FIG. 10, the projection device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the projection device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0201] Various components in the projection device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the projection device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0202] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the image partition blur method as described in embodiments of the present application. That is:

[0203] obtaining left and right eye position data of a user in real time, and determining a viewpoint position of the user in an imaging plane according to the left and right eye position data;

[0204] determining a virtual image blur region in the imaging plane according to the viewpoint position and a human eye observation range;

[0205] obtaining an original projection image to be displayed that matches the imaging plane, and performing blur processing on an image region in the original projection image that matches the virtual image blur region to obtain a target projection image;

[0206] projecting the target projection image into a display screen for display.

[0207] In some embodiments, the image partition blur method can be implemented as a computer program that is tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the projection device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the image partition blur method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the image partition blur method by any other appropriate means, such as by means of firmware.

[0208] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0209] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program

[0210] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0211] To provide for interaction with a user, the systems and techniques described here can be implemented on a projection device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the projection device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0212] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0213] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0214] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in different orders, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

Claims

1. An image partition blurring method, comprising: acquiring left and right eye position data of a user in real time, and determining a viewpoint position of the user in an imaging plane according to the left and right eye position data; determining a virtual image blurring area in the imaging plane according to the viewpoint position and a human eye observation range; acquiring an original projection image to be displayed matched with the imaging plane, and performing blurring processing on an image area in the original projection image matched with the virtual image blurring area to obtain a target projection image; projecting the target projection image to a display screen for display.

2. The method of claim 1, wherein, acquiring left and right eye position data of a user in real time, and determining a viewpoint position of the user in an imaging plane according to the left and right eye position data, comprising: determining an original viewpoint position according to left and right eye position data of the user acquired in real time by a driver monitoring system (DMS); determining the viewpoint position of the user in the imaging plane according to a calibration result of the viewpoint position acquired by the DMS in the imaging plane and the original viewpoint position.

3. The method of claim 2, wherein, determining the viewpoint position of the user in the imaging plane according to a calibration result of the viewpoint position acquired by the DMS in the imaging plane and the original viewpoint position, comprising: acquiring a horizontal and vertical coordinate offset of the viewpoint position acquired by the DMS in the imaging plane; wherein the horizontal and vertical coordinate offset is calibrated by acquiring the calibration viewpoint position determined by the DMS when the user is observed to gaze at a calibration point in the imaging plane; determining the viewpoint position of the user in the imaging plane according to the horizontal and vertical coordinate offset, the original viewpoint position, size information of the imaging plane, and a viewpoint position range determined by the DMS.

4. The method of claim 3, wherein, determining the viewpoint position of the user in the imaging plane according to the horizontal and vertical coordinate offset, the original viewpoint position, size information of the imaging plane, and a viewpoint position range determined by the DMS, comprising: calculating a first scale value and a second scale value according to the size information of the imaging plane and the viewpoint position range determined by the DMS; wherein the first scale value is a ratio between a width of the imaging plane and a horizontal coordinate range of the viewpoint position, and the second scale value is a ratio between a height of the imaging plane and a vertical coordinate range of the viewpoint position; calculating a horizontal coordinate of the viewpoint position of the user in the imaging plane according to the formula (viewpointX+OffsetX)*DeltaX-ScreenWidth / 2, and calculating a vertical coordinate of the viewpoint position of the user in the imaging plane according to the formula (viewpointY-OffsetY)*DeltaY-ScreenHeight / 2; wherein viewpointX and viewpointY are horizontal and vertical coordinate values in the original viewpoint position, OffsetX and OffsetY are the horizontal and vertical coordinate offsets, DeltaX and DeltaY are the first scale value and the second scale value, ScreenWidth is the width of the imaging plane, and ScreenHeight is the height of the imaging plane.

5. The method of claim 1, wherein, determining a virtual image blurring area in the imaging plane according to the viewpoint position and a human eye observation range, comprising: determining a visible width on the imaging plane according to the human eye observation range; determining a horizontal coordinate value of a left side to-be-blurred boundary and a horizontal coordinate value of a right side to-be-blurred boundary in the imaging plane respectively according to a horizontal coordinate value of the user's viewpoint in the viewpoint position and the visible width; determining a left side virtual image blurred area and a right side virtual image blurred area in the imaging plane according to the boundary of the imaging plane, the horizontal coordinate value of the left side to-be-blurred boundary and the horizontal coordinate value of the right side to-be-blurred boundary.

6. The method of claim 5, wherein, determining a horizontal coordinate value of a left side to-be-blurred boundary and a horizontal coordinate value of a right side to-be-blurred boundary in the imaging plane respectively according to a horizontal coordinate value of the user's viewpoint in the viewpoint position and the visible width, comprising: calculating the horizontal coordinate value of the left side to-be-blurred boundary vagueXL and the horizontal coordinate value of the right side to-be-blurred boundary vagueXR in the imaging plane according to the formula vagueXL = EyeRect.anchoredPosition.x - △Width / 2 and the formula vagueXR = EyeRect.anchoredPosition.x + △Width / 2; wherein, EyeRect.anchoredPosition.x is the horizontal coordinate value of the user's viewpoint in the viewpoint position, and △Width is the visible width.

7. The method of claim 6, wherein, performing blur processing on the image area in the original projection image matched with the virtual image blurred area to obtain a target projection image, comprising: generating a blurred projection image covering the virtual image blurred area by calling a calculation shader in the image processing engine; calculating a covering center point position of the blurred projection image in the original projection image, and covering the blurred projection image on the original projection image according to the covering center point position to obtain the target projection image.

8. The method of claim 7, wherein, generating a blurred projection image covering the virtual image blurred area by calling a calculation shader in the image processing engine, comprising: generating a covering image with the same size as the virtual image blurred area by the calculation shader in the image processing engine; generating a blurred projection image covering the virtual image blurred area by replacing the material of the covering image with a material ball of Gaussian blur by the calculation shader in the image processing engine.

9. The method of claim 1, wherein, determining a virtual image blurred area in the imaging plane according to the viewpoint position and the human eye observation range, comprising: determining a left side boundary and a right side boundary of a visible area in the imaging plane respectively according to the viewpoint position and a visible radius; determining a left side virtual image blurred area and a right side virtual image blurred area in the imaging plane according to the boundary of the imaging plane, the left side boundary and the right side boundary of the visible area.

10. The method of claim 9, wherein, performing blur processing on the image area in the original projection image matched with the virtual image blurred area to obtain a target projection image, comprising: generating a covering image with the same size as the virtual image blurred area by the calculation shader in the image processing engine, replacing the material of the covering image with a material ball of Gaussian blur, and converting the shape of the covering image into a target shape matched with the left side boundary and the right side boundary of the visible area. The center point position of the fuzzy projection image covering in the original projection image is calculated, and the fuzzy projection image is covered on the original projection image according to the center point position of the covering, to obtain a target projection image.

11. The method of any one of claims 1-10, wherein, The target projection image is projected into a display screen for display, including: The target projection image is projected into a display screen for display by a three-dimensional head-up display device.

12. The method of any one of claims 1-10, wherein, The target projection image includes a three-dimensional head-up display image.

13. An image partition fuzzy device, comprising: a view point position determination module configured to acquire left and right eye position data of a user in real time, and determine a view point position of the user in an imaging plane according to the left and right eye position data; a virtual image fuzzy area determination module configured to determine a virtual image fuzzy area in the imaging plane according to the view point position and a human eye observation range; a projection image generation module configured to acquire an original projection image to be displayed matched with the imaging plane, and perform fuzzy processing on an image area in the original projection image matched with the virtual image fuzzy area, to obtain a target projection image; an image projection module configured to project the target projection image into a display screen for display.

14. A projection device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image partition fuzzy method in any one of claims 1-12.

15. A computer readable storage medium, the computer readable storage medium storing computer instructions for causing a processor to execute the image partition fuzzy method in any one of claims 1-12 when executed by the processor.

Images