Naked-eye 3D uncertain region image fusion method, system, apparatus, and device

Abstract

A naked-eye 3D uncertain region image fusion method, a system, an apparatus, and a device. The method comprises: after display of a current projection image is completed, acquiring a target static region in a next projection image; and if it is determined that the current projection image comprises a dynamic 3D fused object to be moved, acquiring a target dynamic region of the dynamic 3D fused object in the next projection image (S110); on the basis of the target static region and the target dynamic region, determining target pixel point positions respectively corresponding to target threads, and using each target thread to perform rendering processing on a corresponding target pixel point to obtain the next projection image (S120); and projecting the next projection image onto a display screen for projection display (S130).

Description

Naked-eye 3D indefinite area image fusion method, system, device and equipment

[0001] The present application claims priority to the Chinese patent application No. 202411040091.1 filed on July 31, 2024 with the China Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of 3D head-up display technology, for example, to a naked-eye 3D indefinite area image fusion method, system, device and equipment. BACKGROUND

[0003] Naked-eye 3D display technology refers to a kind of 3D display technology that users can directly enjoy three-dimensional images with naked eyes without wearing special 3D glasses and other auxiliary equipment, presenting 3D effect. Naked-eye 3D display technology has a wide range of applications, such as movie playing, interactive projection, stage performance and head-up display, etc.

[0004] In the head-up display application scenario of a vehicle, naked-eye 3D display technology can provide more rich and intuitive information display for the driver, increasing driving safety and convenience, especially the naked-eye 3D display combined with augmented reality effect, which can provide 3D holographic images in head-up display, helping the driver to obtain road information in a lifelike way.

[0005] In the process of implementing the present application, it is found that the related art has the following problems: The head-up display device of a vehicle generally carries a processing system that can be used by a mobile terminal such as an Android system or a Harmony system, but the processing system of the related art generally provides sufficient computing power for 2D image processing. For 3D images, the GPU (Graphics Processing Unit) calculates the pixel points of the entire screen display area to achieve image fusion. In the process of ensuring real-time performance, the device may not have enough computing power, which cannot provide sufficient computing power support for full-frame 3D images, thereby possibly causing a picture lag situation. Meanwhile, the head-up display image may contain elements displayed in a static area and elements displayed in different dynamic areas. Therefore, the present application proposes a method of simultaneously implementing image fusion in static and dynamic areas while occupying fewer resources. SUMMARY

[0006] The present application provides a naked-eye 3D indefinite area image fusion method, system, device and equipment, which solves the problem of consuming too much computing power for processing full-frame 3D images, adapts to projection devices with lower computing power, and improves picture smoothness.

[0007] According to an aspect of the present application, a naked-eye 3D indefinite area image fusion method is provided, comprising:

[0008] After the display of the current projection image is completed, a target static area in a next projection image is acquired; and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, a target dynamic area of the dynamic 3D fusion object in the next projection image is acquired;

[0009] According to the target static area and the target dynamic area, target pixel point positions corresponding to respective target threads are determined, and the target threads are used to perform rendering processing on the corresponding target pixel points to obtain the next projection image.

[0010] The next projection image is projected into a display screen for projection display.

[0011] Optionally, the target static area in the next projection image is acquired, comprising:

[0012] According to the static partition description information, the target static area in the next projection image is determined; wherein the static partition description information comprises position information and size information of the target static area.

[0013] The advantage of such a setting is that by determining the static area in the next projection image, the area using the fusion algorithm can be reduced, thereby solving the problem of insufficient computing power of the processing system when processing a full-frame 3D image in the related art, adapting to projection equipment with lower computing power, and ensuring the smoothness of the 3D picture.

[0014] Optionally, according to the target static area and the target dynamic area, the target pixel point positions to be rendered are determined, comprising:

[0015] According to the static partition description information of each target static area, a first thread and a first pixel point position corresponding to each first thread are determined;

[0016] According to the target dynamic area, dynamic partition description information and screen anchoring information are extracted, and according to the dynamic partition description information and the screen anchoring information, a plurality of dynamic parameters are acquired;

[0017] According to each dynamic parameter and the screen anchoring information, a second pixel point position corresponding to each second thread is determined.

[0018] The advantage of such a setting is that the parallelism of image processing can be improved by using a single thread to process an image of a unique pixel point, the characteristics of a computing shader are fully utilized, and the image processing efficiency is improved.

[0019] Optionally, the dynamic partition description information comprises a first coordinate of a center point of the target dynamic region in the projection image, and size information of the target dynamic region in the projection image.

[0020] The screen anchor information comprises size information of the projection image and a second coordinate of the anchor point in the projection image.

[0021] In this way, the number of calculation parameters of the target dynamic region can be reduced by setting the dynamic partition description information and the screen anchor information, so as to ensure the calculation efficiency and the accuracy of the target dynamic region calculation.

[0022] Optionally, according to the dynamic partition description information and the screen anchor information, a plurality of dynamic parameters are obtained, comprising:

[0023] According to the size information of the target dynamic region in the projection image, a region width and a region height of the target dynamic region in the projection image are determined.

[0024] According to the first coordinate and the second coordinate, a horizontal coordinate difference and a vertical coordinate difference between the center point of the target dynamic region and the anchor point are determined.

[0025] In this way, the calculation amount of the dynamic parameters is reduced by calculating the dynamic parameters according to the dynamic partition description information and the screen anchor information, so as to ensure the real-time performance of the dynamic parameter updating and reduce the projection delay of the 3D fusion object.

[0026] Optionally, according to the dynamic parameters and the screen anchor information, a second pixel point position corresponding to each second thread is determined, comprising:

[0027] A calculation shader in the image processing engine is called to determine the second pixel point position corresponding to each second thread according to the following formula: id.x1=id.x2+LocalX+ScreenWidth / 2-ΔWidth / 2; id.y1=id.y2+LocalY+ScreenHeight / 2-ΔHeight / 2.

[0028] wherein id.x1 and id.y1 are horizontal and vertical coordinates of the second pixel point, id.x2 and id.y2 are horizontal and vertical coordinates of a pixel point processed by the second thread in the current projection image, LocalX and LocalY are horizontal and vertical relative coordinates between a center point of the target dynamic region and the anchor point, ScreenWidth is a screen width of the next projection image, ScreenHeight is a screen height of the next projection image, ΔWidth is a region width of the target dynamic region in the next projection image, and ΔHeight is a reference height of the target dynamic region in the next projection image.

[0029] The above formula is used to calculate the position of the second pixel point corresponding to each second thread, which can quickly update the pixel point to be rendered by the second thread in combination with the dynamic parameters, and the calculation is simple and real-time.

[0030] Optionally, after the plurality of dynamic parameters are obtained according to the target partition description information and the screen anchor information, the method further comprises:

[0031] According to the target dynamic region and the dynamic parameters, each second thread used for image fusion of the target dynamic region is determined in a compute shader.

[0032] The above arrangement can minimize the occupation of thread groups, maximize the use of threads in the thread groups, avoid occupying thread groups of other tasks, and maximize the computing performance of the compute shader.

[0033] Optionally, according to the target dynamic region and the dynamic parameters, each second thread used for image fusion of the target dynamic region is determined in a compute shader, comprising:

[0034] A thread arrangement mode in each thread group predefined in the compute shader is determined.

[0035] According to the thread arrangement mode, the region width and the region height, a second thread group used for image fusion of the target dynamic region is defined in the compute shader, and a plurality of second threads are determined according to the second thread group.

[0036] The above arrangement can minimize the occupation of thread groups, maximize the use of threads in the thread groups, avoid occupying thread groups of other tasks, maximize the computing performance of the compute shader, improve the parallelism of image processing, and improve the image processing efficiency.

[0037] Optionally, the target threads in the computing shader are used to render the unique target pixel points corresponding to the target threads according to the fusion rule to obtain a rendering result value of the target pixel points, and after the rendering of each target thread is completed, the next projection image is determined to complete the image fusion processing.

[0038] The advantage of such an arrangement is that by one-to-one correspondence between threads and pixel points, the single thread renders only a unique pixel point at a time, which can fully utilize the characteristics of the computing shader, improve the parallelism of image processing, and improve the efficiency of image processing.

[0039] Optionally, the target static area includes a static 3D fusion object, and the static 3D fusion object includes a vehicle state prompt element.

[0040] The dynamic 3D fusion object includes an augmented reality (AR) prompt element, and the AR prompt element specifically includes an AR navigation element and an AR obstacle prompt element.

[0041] The advantage of such an arrangement is that by displaying the vehicle state prompt element in the form of a static 3D fusion object and displaying the AR navigation element and the AR obstacle prompt element in the form of a dynamic 3D fusion object, the virtual reality can be used to provide a good driving experience for the user, and the 3D displayed AR prompt element can improve the indication accuracy, readability, and interactivity.

[0042] According to another aspect of the present application, a head-up display system is provided, which includes a display screen and a head-up display; the head-up display includes a control component;

[0043] The control component is configured to perform the naked-eye 3D indefinite area image fusion method according to any embodiment of the present application.

[0044] Optionally, the head-up display further includes an optical grating device and a mirror device.

[0045] The optical grating device is configured to split the projection image to generate a split image, so as to realize three-dimensional imaging.

[0046] The mirror device is configured to project the split projection image into the display screen according to a preset angle for head-up display.

[0047] The advantage of such an arrangement is that by providing the optical grating device and the mirror device in the head-up display, the head-up display device can project clear 3D content, and adding 3D elements to the head-up display can improve the driving experience of the user.

[0048] According to another aspect of the present application, a naked-eye 3D indefinite area image fusion device is provided, which includes:

[0049] a region determining module, configured to, after display of a current projection image is completed, acquire a target static region in a next projection image, and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, acquire a target dynamic region of the dynamic 3D fusion object in the next projection image;

[0050] an image fusion module, configured to determine target pixel point positions corresponding to respective target threads according to the target static region and the target dynamic region, and perform rendering processing on corresponding target pixel points by using the target threads to obtain the next projection image;

[0051] a projection display module, configured to project the next projection image into a display screen for projection display.

[0052] According to another aspect of the present application, a projection device is provided, which comprises:

[0053] at least one processor; and

[0054] a memory connected in communication with the at least one processor; wherein

[0055] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the naked-eye 3D indefinite region image fusion method according to any one of the embodiments of the present application.

[0056] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the naked-eye 3D indefinite region image fusion method according to any one of the embodiments of the present application when executed by the processor.

[0057] The technical scheme of the embodiment of the application is that, after the display of the current projection image is completed, the target static region in the next projection image is acquired, and when it is determined that the current projection image contains a dynamic 3D fusion object to be moved, the target dynamic region in the next projection image is acquired, the target pixel point positions corresponding to each target thread are determined according to the target static region and the target dynamic region, and the target pixel points corresponding to the target threads are rendered and processed, so that the next projection image is obtained and projected into the display screen for projection display. In this way, the target pixel points processed by the target threads can be dynamically allocated, the static region and the dynamic region in the naked-eye 3D image where the target to be displayed exists are processed by the relatively fixed threads, other parts of the image are not processed, and then the region using the fusion algorithm is reduced, thereby solving the problem of insufficient computing power when the processing system of the related art processes the full-width 3D image, and the application is suitable for the projection equipment with low computing power, and the fluency of the 3D picture is ensured.

[0058] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0060] FIG. 1 is a flowchart of a naked-eye 3D indefinite region image fusion method according to an embodiment of the application;

[0061] FIG. 2 is a schematic diagram of a head-up display imaging process according to an embodiment of the application;

[0062] FIG. 3 is a schematic diagram of image fusion region division according to an embodiment of the application;

[0063] FIG. 4 is a schematic diagram of a static region according to an embodiment of the application;

[0064] FIG. 5 is a schematic diagram of a dynamic region according to an embodiment of the application;

[0065] FIG. 6 is a schematic diagram of a naked-eye 3D imaging principle according to an embodiment of the application;

[0066] FIG. 7 is a schematic diagram of the projection effect of a naked-eye 3D image in a head-up display scene according to an embodiment of the application;

[0067] FIG. 8 is a flow chart of a static area image fusion method of naked-eye 3D according to Embodiment Two of the present application;

[0068] FIG. 9 is a schematic diagram of a thread group arrangement mode according to the present application;

[0069] FIG. 10 is a schematic diagram of a thread arrangement mode according to the present application;

[0070] FIG. 11 is a diagram of the relationship between a thread group and threads according to the present application;

[0071] FIG. 12 is a flow chart of a static area image fusion method of naked-eye 3D according to Embodiment Two of the present application;

[0072] FIG. 13 is a flow chart of a dynamic area image fusion method of naked-eye 3D according to Embodiment Two of the present application;

[0073] FIG. 14 is a schematic diagram of the structure of a head-up display system according to Embodiment Three of the present application;

[0074] FIG. 15 is a schematic diagram of the structure of an indefinite area image fusion device of naked-eye 3D according to Embodiment Four of the present application;

[0075] FIG. 16 is a schematic diagram of the structure of a projection device implementing an indefinite area image fusion method of naked-eye 3D according to Embodiment Four of the present application. DETAILED DESCRIPTION

[0076] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of the present application.

[0077] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0078] The present application can be applied to various projection scenarios, such as movie playing, interactive projection, stage performance, and head-up display, etc. The present application takes the head-up display of a vehicle as an example to specifically describe the dynamic area image fusion method of naked-eye 3D, wherein the vehicle is intended to represent various forms of vehicles. For example, the vehicle can include, but is not limited to, land vehicles such as cars, aerial vehicles (or referred to as aircraft), or water or underwater vehicles, etc.

[0079] Optionally, in the application of the head-up display of the vehicle, the head-up display device of the vehicle can realize the processing and projection display of the projection image. The front display part of the naked-eye 3D display technology needs to calculate and generate the projection image through the pixel imaging unit, so that the left eye and the right eye of a person see different pictures and fuse into a 3D effect picture in the brain.

[0080] Embodiment one

[0081] FIG. 1 is a flowchart of a naked-eye 3D indefinite area image fusion method provided by the present application. The present embodiment can be applied to the case of only performing image fusion processing on the area with a 3D fusion object in the 3D projection image. The method can be executed by a naked-eye 3D indefinite area image fusion device, which can be realized in the form of hardware and / or software and generally can be configured in a projection device with image processing function. As shown in FIG. 1, the method includes:

[0082] S110, after completing the display of the current projection image, obtaining the target static area in the next projection image; and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, obtaining the target dynamic area of the dynamic 3D fusion object in the next projection image.

[0083] FIG. 2 is a schematic diagram of an optional head-up display imaging process. As shown in FIG. 2, the light source in the HUD (Head-Up display, head-up display device) emits light imaging and is projected onto the windshield of the vehicle via the plane mirror and the curved mirror. The windshield is an optional imaging element. If the vehicle is configured with other imaging elements, it can also be projected into other imaging elements. The eyebox refers to the range of imaging light that can be accepted by the height of the driver's eyes. The actual head-up display content observed in the driver's view angle is actually a virtual image on a virtual image plane in space. Through the modification of the projection image, the virtual image on the virtual image plane can be adjusted, and finally the image change effect of the driver's view angle is realized.

[0084] Optionally, in most naked-eye 3D projection display scenarios, the projection content is video or dynamic image, and whether it is video or dynamic image, it is actually to project multiple frames of images in a short time. For example, if the scene is a head-up display in a vehicle, the projected image can be a 3D head-up display image, and each frame of projected image includes the 3D head-up display content at the current time point.

[0085] Optionally, in order to solve the problem of insufficient computing power of the processing system, the image fusion algorithm is split into full-screen image fusion and indefinite area image fusion. For pictures that do not need full-screen display, indefinite area image fusion can be performed in the selected indefinite area in the projection image. FIG. 3 is a schematic diagram of an optional image fusion area division. As shown in FIG. 3, the indefinite area can be divided into a static irregular area and a dynamic irregular area. It can be understood that the static irregular area can refer to an area in the projection image whose shape and position are unchanged, but the specific display content in the area can be changed. The dynamic irregular area can refer to the display area of the 3D fusion object whose display position is constantly moving in the projection image. For the moving 3D fusion object, its position in the display picture is different at different time points. In each frame of image, the area displaying the 3D fusion object to be moved can be regarded as a dynamic irregular area. Moreover, the shape of the static irregular area and the dynamic irregular area is arbitrary. In order to facilitate explanation, they are collectively referred to as static area and dynamic area in this application.

[0086] FIG. 4 is a schematic diagram of an optional static area. As shown in FIG. 4, fusion A, fusion B and fusion C are three static areas in the projection image. The shape of the static area is not limited. FIG. 4 is only illustrative. In the case where the total number of threads of the processing system is greater than the total number of threads required for image fusion, the position, number and shape of the static area are not limited.

[0087] Optionally, the static area is generally a pre-designed area. For example, in a head-up display application, the vehicle state information such as vehicle speed is fixedly displayed at a position. Therefore, the area displaying the speed prompt element can be regarded as a static area. According to the preset static partition description information, the position information and size information of the target static area in the next projection image can be determined. The display time information of the static area can also be included in the static partition description information. For example, a picture can be continuously displayed in a certain static area for 10 minutes, and the picture disappears after 10 minutes. This is only illustrative.

[0088] Fig. 5 is a schematic diagram of an optional dynamic area. As shown in Fig. 5, Position A can be a current dynamic area in a current projection image, in which a 3D fusion object to be moved is displayed. Position B and Position C show two optional dynamic areas. If the 3D fusion object to be moved needs to be moved to Position B in a next projection image, Position B is a target dynamic area in the next projection image. The 3D fusion object to be moved can also be moved to Position B and Position C at the same time, and then Position B and Position C are two different target dynamic areas. The shape and number of the dynamic areas are not limited herein, and Fig. 5 is only used for illustrative purposes. In a case where the total number of threads of the processing system is greater than the total number of threads required for image fusion, the dynamic areas can be moved to different area positions as required, and the shape and number of the dynamic areas are not limited.

[0089] Optionally, the 3D fusion object can refer to a 3D target to be displayed in an image. Taking a heads-up display in a vehicle as an example, an AR navigation element in a heads-up display screen can be a 3D target, and an AR obstacle prompt element can also be a 3D target. When there are multiple different obstacles, each obstacle corresponds to an AR obstacle prompt element, and each AR obstacle prompt element can be a 3D target. The dynamic 3D fusion object can refer to a 3D fusion object whose position can be moved, and the static 3D fusion object can refer to a display object in a static area in a projection image.

[0090] Optionally, the target static area can be one or multiple. The display content in the target static area can be determined according to real-time data of the vehicle. For example, in a heads-up display application scenario of a vehicle, the display content in the target static area can be a vehicle speed display icon, and the display content in the target static area can be determined according to real-time vehicle speed sent by a sensor.

[0091] Optionally, the target dynamic area of the dynamic 3D fusion object in a next projection image can be determined according to pre-set 3D projection content. The target dynamic area of the dynamic 3D fusion object in the next projection image can be determined according to real-time collected information. For example, in a heads-up display application scenario of a vehicle, the area of a heads-up display element in a next projection image, i.e., the target dynamic area, can be determined according to real-time collected vehicle information.

[0092] S120, determining target pixel point positions corresponding to each target thread according to the target static area and the target dynamic area, and performing rendering processing on the corresponding target pixel points by using the target thread to obtain a next projection image.

[0093] Optionally, the target pixel position to be rendered is determined according to the target static region and the target dynamic region, and the target pixel position to be rendered comprises:

[0094] According to the static partition description information of each target static region, a first thread is determined, and a first pixel position corresponding to each first thread is determined.

[0095] According to the target dynamic region, dynamic partition description information and screen anchor information are extracted, and a plurality of dynamic parameters are obtained according to the dynamic partition description information and the screen anchor information.

[0096] According to each dynamic parameter and the screen anchor information, a second pixel position corresponding to each second thread is determined.

[0097] Optionally, the first thread and the second thread are threads in an image processing engine for image rendering processing of a unique pixel, that is, a single thread can only render a unique pixel in a single image fusion processing. The first pixel can refer to each pixel in the static region, and the second pixel can refer to each pixel in the dynamic region. When the static region and the dynamic region coincide, the pixel in the coincident region can only be processed as one of the first pixel or the second pixel, that is, each pixel can only be rendered once in a single image fusion processing.

[0098] Optionally, the first thread and the second thread together serve as a target thread, and the first pixel and the second pixel together serve as a target pixel.

[0099] Optionally, after the target dynamic region of the dynamic 3D fusion object in the next projection image is determined, the dynamic partition description information and the screen anchor information can be determined.

[0100] Optionally, the dynamic partition description information comprises a first coordinate of a center point of the target dynamic region in the projection image and size information of the target dynamic region in the projection image; and the screen anchor information comprises size information of the projection image and a second coordinate of an anchor point in the projection image.

[0101] Optionally, the shape of the target dynamic region can be a rectangle, a triangle, a circle, an ellipse, or an irregular shape, which is not limited here.

[0102] Optionally, the size information selected is different for different shapes of the target dynamic region. For example, for a rectangular, triangular or elliptical target dynamic region, the size information selects the region width and the region height of the target dynamic region. For a circular target dynamic region, the size information can select the radius of the circle. For an irregularly shaped target dynamic region, a minimum circumscribed frame of the target dynamic region can be determined first using a regular shape, and then the width and height of the minimum circumscribed frame of the target dynamic region are determined. Here, only several optional size information acquisition methods are shown, and the size information is not specifically limited.

[0103] Optionally, the size information of the target dynamic region in the projection image can be the same as or different from the shape and size information of the target dynamic region in the next projection image, depending on the specific display requirements of the 3D projection image.

[0104] Optionally, based on conventional projection technology, the projection image is generally a regular image, and the size information of the projection image can include the picture height and the picture width of the projection image. Of course, the projection image can also be an irregular image. When the projection image is an irregular image, the size information of the projection image can be the size information of the minimum circumscribed frame of the projection image. Here, the size information of the projection image is not specifically limited.

[0105] Optionally, the anchor point can refer to a point in the projection image that is pre-marked and can serve as a reference point in the projection image. The anchor point can be selected as the center point of the projection image, the corner points around the projection image, etc., but is not limited thereto.

[0106] Optionally, after the pixel points corresponding to the first thread and the second thread are determined, the corresponding pixel points are rendered by the first thread and the second thread, respectively.

[0107] It can be understood that, in order to reduce the resource occupation of image fusion, based on the threads for image fusion of the static region and the dynamic region in the current projection image, the first thread and the second thread for image fusion of the next projection image are determined. The specific determination method is as follows. If the number of pixel points occupied by the static region and the dynamic region in the current projection image is the same as that in the next projection image, the threads for image fusion of the current projection image are divided into the first thread and the second thread. If the number of pixel points occupied by the static region or the dynamic region in the current projection image is less than that in the next projection image, new threads are confirmed based on the threads for image fusion of the current projection image, and the new threads are collectively used as the first thread or the second thread. If the number of pixel points occupied by the static region or the dynamic region in the current projection image is greater than that in the next projection image, part of the threads for image fusion of the current projection image are selected as the first thread and the second thread. Here, the selection is not limited.

[0108] Optionally, the target pixel points are rendered simultaneously by the target threads, and after the rendering of all the target pixel points is completed, the finally generated image is taken as the next projection image.

[0109] S130, projecting the next projection image into the display screen for projection display.

[0110] FIG. 6 is a schematic diagram of an optional naked-eye 3D imaging principle. In FIG. 6, the HUD is optically designed so that the left eye of the user sees image P1 and the right eye sees image P2 through the imaging structure, such as the windshield. Due to the binocular disparity formed by the image P1 and the image P2, the user feels the object has a sense of depth and space, and the right eye sees the image for the right eye, which is synthesized into a stereoscopic picture with a sense of depth in the user's brain. The naked-eye 3D imaging can change the position between the two images, adjust the binocular disparity, and make the user feel that the virtual image distance has changed (the actual distance of the virtual image does not change). The closer the two images are, the closer the user feels the virtual image distance. Conversely, the farther the two images are, the farther the user feels the virtual image distance.

[0111] Optionally, the target static region of the next projection image includes the left-eye visible image and the right-eye visible image of the static 3D fusion object; and the target dynamic region of the next projection image includes the left-eye visible image and the right-eye visible image of the dynamic 3D fusion object.

[0112] Optionally, when the application scenario is the head-up display function of the vehicle, the display screen can be an imaging element configured on the vehicle. The imaging element can include a front windshield of the vehicle, a reflective layer film arranged in the vehicle, or an imaging window arranged in the vehicle, but is not limited thereto. The imaging through the windshield is W-HUD (Windshield-HUD, windshield head-up display), and the imaging through the imaging window is C-HUD (Combiner HUD, combiner head-up display). Optionally, the imaging window is generally made of transparent material (transparent to visible light) or an imaging plate with a certain curvature.

[0113] Fig. 7 is a schematic diagram of a projection effect of an optional dynamic 3D fusion object in a heads-up display scene, as shown in Fig. 7, the middle black box can be regarded as a dynamic area, and the dynamic 3D fusion object in the middle black box is a navigation arrow in an AR navigation element. The ghosting effect indicated by the arrow in the figure is actually a superposition effect of left-eye and right-eye images. The arrow is 3D dynamic in the perspective of the driver's eyes, and the rest can be regarded as a simulated segmentation of the vehicle front windshield and other areas in the vehicle in the perspective of the driver, which can be ignored. The AR navigation element can be displayed in different areas in the picture, and the shape of the AR navigation element displayed in each area can also be different, for example, a straight arrow is displayed first, and then a left-turn arrow is displayed. The position of the AR navigation area displayed in each projection image can be regarded as a target dynamic area of the projection image.

[0114] Optionally, the projection image is a 3D heads-up display image; the display screen is an imaging element configured on the vehicle; and the imaging element includes a front windshield of the vehicle, an emission film layer arranged in the vehicle, or an imaging window arranged in the vehicle.

[0115] The technical scheme of the embodiment of the present application can dynamically allocate the target pixel points processed by the target thread by obtaining the target static area in the next projection image after completing the display of the current projection image, obtaining the target dynamic area in the next projection image if the current projection image contains the dynamic 3D fusion object to be moved, determining the target pixel point positions corresponding to each target thread according to the target static area and the target dynamic area, and rendering the corresponding target pixel points by using the target thread. The 3D image fusion processing is performed on the static area and the dynamic area in the naked-eye 3D image where the target to be displayed exists, and the other parts of the image are not processed, thereby reducing the area using the fusion algorithm, and solving the problem of insufficient computing power of the processing system of the related art when processing the full-width 3D image, adapting to the projection equipment with lower computing power, and ensuring the smoothness of the 3D picture.

[0116] Embodiment Two

[0117] Fig. 8 is a flowchart of a naked-eye 3D static area image fusion method according to an embodiment of the present application. The embodiment is based on the above-mentioned embodiment and specifically describes the naked-eye 3D static area image fusion method. As shown in Fig. 8, the method comprises the following steps:

[0118] S210, after completing the display of the current projection image, determining a target static area in the next projection image according to the static partition description information; and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, obtaining a target dynamic area of the dynamic 3D fusion object in the next projection image.

[0119] Optionally, the static partition description information comprises position information and size information of the target static area.

[0120] Optionally, the target static area comprises a static 3D fusion object, and the static 3D fusion object comprises a vehicle state prompt element.

[0121] The dynamic 3D fusion object comprises an augmented reality (AR) prompt element, and the AR prompt element specifically comprises an AR navigation element and an AR obstacle prompt element.

[0122] Optionally, when the application scenario is a head-up display function of the vehicle, the static 3D fusion object is the vehicle state prompt element, and the vehicle state prompt element is generally generated according to the state of the vehicle itself and is an element that does not need to be moved and displayed, such as a speed, a light state, an engine state, and the like.

[0123] Optionally, when the application scenario is the head-up display function of the vehicle, the dynamic 3D fusion object can be the AR prompt element, and the AR prompt element specifically comprises the AR navigation element and the AR obstacle prompt element, but is not limited to the AR prompt element exemplified above.

[0124] Optionally, the AR prompt element can refer to an element that provides a prompt to a user in a head-up display manner by using an augmented reality technology and combining real-time vehicle auxiliary information, the AR navigation element can refer to a navigation element that guides a direction of a user by using a virtual model, an arrow, a dashed line, and the like, and the AR obstacle prompt element can refer to an element that prompts an obstacle around the vehicle to the user by using an indicator such as a triangle and a horizontal line.

[0125] It can be understood that, as a virtual image imaging plane (which can be referred to as a mirror imaging principle) of the projection image, when a light source in the HUD projects the projection image on an imaging element of the vehicle, the projection image actually observed in a driver's visual angle is actually located on the virtual image plane in space, and by controlling imaging sizes and imaging positions of the static 3D fusion object and the dynamic 3D fusion object in the virtual image plane, a virtual image of the static 3D fusion object and the dynamic 3D fusion object is finally presented in the driver's visual angle, and a fusion effect of the virtual image and a real-world road environment is finally presented.

[0126] Optionally, the position information of the target static region can indicate the position coordinates of at least one positioning point in the target static region on the projection image, and can be used to determine the size information of the target static region. For different shapes of the target static region, the size information selected is also different. For example, for a rectangular, triangular or elliptical target static region, the size information selected is the width and height of the target static region. For a circular target static region, the size information selected can be the radius of the circle. For an irregularly shaped target static region, a minimum circumscribed frame of the target static region can be determined first by using a regular shape, and then the width and height of the minimum circumscribed frame of the target static region can be determined. Here, only several optional size information acquisition methods are shown, and the size information is not specifically limited.

[0127] S220, determining a first thread according to the static partition description information of each target static region, and a first pixel point position corresponding to each first thread.

[0128] Optionally, each first thread in the computing shader corresponds to each pixel point in the target static region, and the first thread is used for image fusion processing on the corresponding pixel point in the target static region.

[0129] Optionally, the naked eye 3D irregular region image fusion method proposed in the present application can be called by an image processing engine in a projection device to process a computing shader. The image processing engine can create and render a 3D model. In the computing shader, a plurality of regularly arranged thread groups are included, and each thread group includes a plurality of regularly arranged threads. Based on the properties of the image processing engine itself, each thread in the computing shader can process one pixel point in the projection image, and each thread can be processed in parallel. Therefore, for complex image processing of a plurality of pixel points, the processing of each thread in the computing shader can be performed simultaneously to greatly save the calculation time.

[0130] Optionally, the number of thread groups in the computing shader can be specified by pre-configured thread group attributes. A thread group can be understood as a combination of a plurality of threads. A thread group can include a plurality of threads. The usage of the thread group attribute is as follows: the first parameter is a kernel index value, and the last three parameters are the number of thread groups in the x, y and z axes.

[0131] Continuing the previous example, FIG. 9 is a schematic diagram of an optional thread group arrangement mode. In the example shown in FIG. 9, it is set that cs.Dispatch(kernal, 4, 4, 2). Each cube in FIG. 9 can be regarded as a thread group. The thread group arrangement mode is that there are 4 thread groups in the X axis direction, 4 thread groups in the Y axis direction, and 2 thread groups in the Z axis direction. FIG. 9 is only used for illustrative purposes, and the actual thread group arrangement mode is not limited.

[0132] Optionally, the number of threads in the thread group can also be specified in the compute shader by a pre-configured thread property, and the arrangement of threads in the thread group is similar to the cuboid arrangement, and the thread property specifies the number of threads in the x, y and z directions.

[0133] Continuing the previous example, FIG. 10 is a schematic diagram of an alternative thread arrangement mode, in which [numthreads(8, 4, 2)] is specified, which specifies 8, 4 and 2 threads in the x, y and z directions, respectively, so that there are 64 threads in a thread group. One small cube in FIG. 10 represents one thread.

[0134] Optionally, FIG. 11 is a diagram of the relationship between a thread group and threads, as shown in FIG. 11, the right side is a schematic diagram of a thread group arrangement mode, and the left side is a schematic diagram of a thread arrangement mode. The right image is an expansion of one thread in the schematic diagram of the thread arrangement mode on the left side. When there are 32 thread groups in the image on the left side, and there are 64 threads in each thread group in the image on the right side, there are a total of 32x64=2048 threads in the image processing engine.

[0135] Optionally, after the size information of the target static region is determined, the number of pixel points in the target static region can be determined according to the size information. According to the characteristics of the image processing engine, each first thread in the compute shader corresponds to one pixel point in the target static region, and the first thread is used to perform image fusion processing on the corresponding pixel point in the target static region. Therefore, after the number of pixel points is obtained, the number of first threads required can be determined.

[0136] S230, according to the target dynamic region, extracting dynamic partition description information and screen anchor information, and according to the dynamic partition description information and the screen anchor information, obtaining a plurality of dynamic parameters.

[0137] Optionally, the dynamic partition description information includes a first coordinate of a center point of the target dynamic region in the projection image, and size information of the target dynamic region in the projection image.

[0138] The screen anchor information includes size information of the projection image and a second coordinate of an anchor point in the projection image.

[0139] Optionally, according to the dynamic partition description information and the screen anchor information, a plurality of dynamic parameters are obtained, including:

[0140] According to the size information of the target dynamic region in the projection image, the region width and the region height of the target dynamic region in the projection image are determined.

[0141] According to the first coordinate and the second coordinate, a horizontal coordinate difference and a vertical coordinate difference between the center point of the target dynamic region and the anchor point are determined.

[0142] Optionally, the horizontal coordinate difference between the center point and the anchor point is obtained by subtracting the horizontal coordinate in the second coordinate from the horizontal coordinate in the first coordinate, and the vertical coordinate difference between the center point and the anchor point is obtained by subtracting the vertical coordinate in the second coordinate from the vertical coordinate in the first coordinate.

[0143] In a specific application scenario, integer type (int) dynamic parameter region width ΔWidth, region height ΔHeight, horizontal coordinate difference LocalX and vertical coordinate difference LocalY can be defined, and the parameters are assigned to the imaging region Image. The specific definition can be represented as ImageRectTransform.sizeDelta = new Vector2(ΔWidth, ΔHeight) and ImageRectTransform.anchoredPosition = new Vector2(PositionX, PositionY), wherein ImageRectTransform.anchoredPosition is the position of the 3D fusion object relative to the anchor point, and ImageRectTransform.sizeDelta is the object size of the 3D fusion object, both of which are properties in the image processing engine.

[0144] S240, according to the target dynamic region and the dynamic parameters, determining each second thread for image fusion of the target dynamic region in the computing shader.

[0145] Optionally, according to the target dynamic region and the dynamic parameters, each second thread for image fusion of the target dynamic region in the computing shader is determined, comprising:

[0146] determining a thread arrangement mode in each thread group predefined in the computing shader;

[0147] defining a second thread group for image fusion of the target dynamic region in the computing shader according to the thread arrangement mode, the region width and the region height, and determining a plurality of second threads according to the second thread group.

[0148] Optionally, each second thread in the computing shader corresponds to each pixel point in the target dynamic region one by one, and the second thread is used for image fusion processing of the corresponding pixel point in the target dynamic region.

[0149] In one specific application scenario, the second thread group can be defined by CS.Dispatch(kernal, DeltaWidth / numberThreasdX, DeltaHeight / numberThreasdY, 1), where numberThreasdX is the number of threads in the X-axis of the thread group, numberThreasdY is the number of threads in the Y-axis of the thread group, the kernel keyword is used to define a computing kernel, which is a function that can be executed in parallel, and it accepts a set of input data and returns a set of output data. In Compute Shader, one or more computing kernels can be defined, and then the computing task can be executed by calling these kernels.

[0150] S250, determine the second pixel point positions corresponding to the second threads respectively according to the dynamic parameters and the screen anchor information.

[0151] Optionally, the second pixel point positions corresponding to the second threads respectively are determined according to the dynamic parameters and the screen anchor information, including:

[0152] The computing shader in the image processing engine is called to determine the second pixel point positions corresponding to the second threads respectively according to the following formula: id.x1=id.x2+LocalX+ScreenWidth / 2-DeltaWidth / 2; id.y1=id.y2+LocalY+ScreenHeight / 2-DeltaHeight / 2.

[0153] Where id.x1 and id.y1 are the horizontal and vertical coordinates of the second pixel point, id.x2 and id.y2 are the horizontal and vertical coordinates of the pixel point processed by the second thread in the current projection image, LocalX and LocalY are the horizontal and vertical relative coordinates between the center point of the target dynamic region and the anchor point, ScreenWidth is the picture width of the next projection image, ScreenHeight is the picture height of the next projection image, DeltaWidth is the region width of the target dynamic region in the next projection image, and DeltaHeight is the reference height of the target dynamic region in the next projection image.

[0154] S260, render the corresponding target pixel points using the target threads to obtain the next projection image.

[0155] Optionally, the target threads in the computing shader are used to render the unique target pixel points corresponding thereto according to the fusion rule to obtain the rendering result values of the target pixel points. When all the target threads complete the rendering, the next projection image is determined to complete the image fusion processing.

[0156] S270, project the next projection image into the display screen for projection display.

[0157] Optionally, Fig. 12 is another optional static area image fusion flowchart of naked-eye 3D, Fig. 12 shows an optional image fusion application scenario of naked-eye 3D static area, that is, the size and position of the target static area are obtained through the static partition image fusion algorithm, and then the total number of Shader threads required is determined, the size and position of the partition and the total number of Shader threads are jointly input into the 3D image fusion model established in advance, the subscript, the fusion pixel value and the slope are obtained, and then the fusion imaging result of the static partition image is generated according to the total number of Shader threads, the subscript, the fusion pixel value and the slope.

[0158] Optionally, Fig. 13 is a dynamic area image fusion flowchart of an optional naked-eye 3D, Fig. 13 shows another optional dynamic area image fusion application scenario of naked-eye 3D, that is, the dynamic partition size parameter and the dynamic partition position parameter are obtained first, which can be similar to the dynamic area size information and the coordinate difference between the dynamic area center point and the anchor point in the present application, and the total number of Shader threads is determined according to the dynamic partition size parameter, the dynamic partition position parameter and the dynamic partition size parameter are jointly input into the 3D image fusion model established in advance, the subscript, the fusion pixel value and the slope are obtained, and then the fusion imaging result of the dynamic partition image is generated according to the total number of Shader threads, the subscript, the fusion pixel value and the slope.

[0159] The technical scheme of the embodiment of the present application can dynamically allocate the target pixel points processed by the target threads by obtaining the target static area in the next projection image after completing the display of the current projection image, obtaining the target dynamic area in the next projection image when it is determined that the current projection image contains a dynamic 3D fusion object to be moved, determining the target pixel point positions corresponding to each target thread according to the target static area and the target dynamic area, and using the target threads to render the corresponding target pixel points, obtaining the next projection image and projecting it into the display screen for projection display. The static area and the dynamic area in the naked-eye 3D image where the target to be displayed exists are processed by using the relatively fixed threads for image fusion, and other parts of the image are not processed, thereby reducing the area using the fusion algorithm, thereby solving the problem of insufficient computing power of the processing system of the related art when processing the full-width 3D image, adapting to the projection equipment with lower computing power, and ensuring the smoothness of the 3D picture.

[0160] Embodiment three

[0161] Fig. 14 is a structural schematic diagram of a head-up display system according to Embodiment Three of the present application. As shown in Fig. 14, the head-up display system includes a display screen 310 and a head-up display 320; the head-up display 320 includes a control component 321, an optical grating device 322 and a mirror device 323.

[0162] The control component 321 is configured to perform the naked-eye 3D indefinite area image fusion method according to any one of the embodiments of the present application. That is:

[0163] After the display of the current projection image is completed, a target static area in a next projection image is obtained; and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, a target dynamic area of the dynamic 3D fusion object in the next projection image is obtained;

[0164] According to the target static area and the target dynamic area, target pixel point positions corresponding to respective target threads are determined, and the target threads are used to perform rendering processing on the corresponding target pixel points to obtain the next projection image;

[0165] The next projection image is projected into the display screen for projection display.

[0166] The optical grating device 322 can be configured to split the projection image to generate a split image, so as to realize three-dimensional imaging.

[0167] The mirror device 323 can be configured to project the split projection image into the display screen according to a preset angle for head-up display.

[0168] It can be understood that after the image rendering is completed, the rendered image can be split by the optical grating device 322, and the split projection image includes an image visible to the left eye of the driver and an image visible to the right eye of the driver, so as to achieve the effect of three-dimensional imaging.

[0169] Optionally, the mirror device 323 can include a plane mirror, a curved mirror or the like, and the split image emitted by the image source is reflected or refracted by the mirror device 323, so as to be projected into the display screen.

[0170] The technical scheme of the embodiments of the present application can provide a head-up display system which can realize image fusion with less computing power and project three-dimensional projection images to users, thereby improving the user experience.

[0171] Embodiment Four

[0172] Fig. 15 is a structural schematic diagram of an irregular region image fusion device for naked-eye 3D according to an embodiment of the present application. As shown in Fig. 15, the device comprises a region determining module 410, an image fusion module 420, and a projection display module 430.

[0173] The region determining module 410 is configured to, after the display of the current projection image is completed, acquire a target static region in a next projection image, and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, acquire a target dynamic region of the dynamic 3D fusion object in the next projection image.

[0174] The image fusion module 420 is configured to determine target pixel point positions corresponding to respective target threads according to the target static region and the target dynamic region, and perform rendering processing on the corresponding target pixel points by using the target threads to obtain the next projection image.

[0175] The projection display module 430 is configured to project and display the next projection image on a display screen.

[0176] The technical scheme of the embodiment of the present application can dynamically allocate target pixel points processed by target threads by the following manner: after the display of the current projection image is completed, a target static region in a next projection image is acquired, and if it is determined that the current projection image contains a dynamic 3D fusion object to be moved, a target dynamic region in the next projection image is acquired, target pixel point positions corresponding to respective target threads are determined according to the target static region and the target dynamic region, and the corresponding target pixel points are rendered by using the target threads to obtain the next projection image and project and display the next projection image on a display screen. This can perform image fusion processing on static regions and dynamic regions in a naked-eye 3D image where a target to be displayed exists by using relatively fixed threads, and does not process other parts of the image, thereby reducing the region where the fusion algorithm is used, and thus solving the problem of insufficient computing power when a processing system of the related art processes a full-frame 3D image, and adapting to a projection device with lower computing power to ensure the fluency of a 3D picture.

[0177] On the basis of the above-mentioned embodiments, the region determining module 410 can comprise a static region determining unit configured to:

[0178] acquire the target static region in the next projection image according to static partition description information, wherein the static partition description information comprises position information and size information of the target static region.

[0179] On the basis of the above-mentioned embodiments, the image fusion module 420 can comprise a pixel point determining unit and a pixel point rendering unit.

[0180] The pixel point determination unit can include:

[0181] The first pixel point determination subunit is configured to determine a first thread according to the static partition description information of each target static region, and a first pixel point position corresponding to each first thread, respectively.

[0182] The dynamic parameter acquisition subunit is configured to extract dynamic partition description information and screen anchor information according to the target dynamic region, and acquire a plurality of dynamic parameters according to the dynamic partition description information and the screen anchor information.

[0183] The second pixel point determination subunit is configured to determine a second pixel point position corresponding to each second thread according to each dynamic parameter and the screen anchor information.

[0184] On the basis of each of the above embodiments, the dynamic partition description information includes a first coordinate of a center point of the target dynamic region in the projection image, and size information of the target dynamic region in the projection image.

[0185] The screen anchor information includes size information of the projection image and a second coordinate of an anchor point in the projection image.

[0186] On the basis of each of the above embodiments, the dynamic parameter acquisition subunit is configured to:

[0187] determine a region width and a region height of the target dynamic region in the projection image according to the size information of the target dynamic region in the projection image;

[0188] determine a horizontal coordinate difference and a vertical coordinate difference between the center point of the target dynamic region and the anchor point according to the first coordinate and the second coordinate.

[0189] On the basis of each of the above embodiments, the second pixel point determination subunit is configured to:

[0190] invoke a calculation shader in an image processing engine to determine the second pixel point position corresponding to each second thread according to the following formula: id.x1=id.x2+LocalX+ScreenWidth / 2-ΔWidth / 2; id.y1=id.y2+LocalY+ScreenHeight / 2-ΔHeight / 2.

[0191] Wherein, id.x1 and id.y1 are horizontal and vertical coordinates of the second pixel point respectively, id.x2 and id.y2 are horizontal and vertical coordinates of the pixel point processed by the second thread in the current projection image respectively, LocalX and LocalY are horizontal and vertical relative coordinates between the center point of the target dynamic region and the anchor point respectively, ScreenWidth is the picture width of the next projection image, ScreenHeight is the picture height of the next projection image, ΔWidth is the region width of the target dynamic region in the next projection image, and ΔHeight is the reference height of the target dynamic region in the next projection image.

[0192] On the basis of the above embodiments, a second thread determination unit can be further included and configured to:

[0193] According to the target dynamic region and the dynamic parameter, each second thread for image fusion of the target dynamic region is determined in a compute shader.

[0194] On the basis of the above embodiments, the second thread determination unit is configured to:

[0195] Determine a thread arrangement mode in each thread group predefined in the compute shader;

[0196] According to the thread arrangement mode, the region width and the region height, a second thread group for image fusion of the target dynamic region is defined in the compute shader, and a plurality of second threads are determined according to the second thread group.

[0197] On the basis of the above embodiments, the target thread in the compute shader is used to render the unique target pixel point corresponding thereto according to a fusion rule to obtain a rendering result value of the target pixel point, and after each target thread completes the rendering, the next projection image is determined to complete the image fusion processing.

[0198] On the basis of the above embodiments, the target static region includes a static 3D fusion object, and the static 3D fusion object includes a vehicle state prompt element.

[0199] The dynamic 3D fusion object includes an augmented reality (AR) prompt element, and the AR prompt element specifically includes an AR navigation element and an AR obstacle prompt element.

[0200] The naked-eye 3D indefinite region image fusion device provided in the embodiments of the present application can execute the naked-eye 3D indefinite region image fusion method provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0201] Embodiment five

[0202] FIG. 16 shows a structural diagram of a projection device 10 that can be used to implement embodiments of the present application. The projection device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The projection device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0203] As shown in FIG. 16, the projection device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected to the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the projection device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0204] Various components in the projection device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the projection device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0205] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the foveated image fusion method for naked-eye 3D as described in embodiments of the present application. That is:

[0206] acquiring a target static region in a next projection image after finishing displaying the current projection image; and acquiring a target dynamic region of a dynamic 3D fusion object to be moved in the next projection image if it is determined that the dynamic 3D fusion object to be moved is contained in the current projection image;

[0207] determining target pixel point positions corresponding to the target threads respectively according to the target static region and the target dynamic region, and rendering the corresponding target pixel points by using the target threads to obtain the next projection image;

[0208] projecting the next projection image onto a display screen for projection display.

[0209] In some embodiments, the naked-eye 3D indefinite region image fusion method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the projection device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the naked-eye 3D indefinite region image fusion method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the naked-eye 3D indefinite region image fusion method by any other appropriate means (for example, by means of firmware).

[0210] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0211] Computer programs used to practice the methods of the application can be written in any combination of one or more programming languages. These computer programs can be implemented on general-purpose computers, special purpose computers, or other programmable data processing apparatus to produce the functions / acts specified in the flow diagrams and / or block diagrams. Computer programs can be applied to a data changed on the processing unit to produce a result. The computer programs can be implemented entirely on a machine, partially on a machine, partially on a machine as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0212] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store the computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0213] To provide for interaction with a user, the systems and techniques described here can be implemented on a projection device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the projection device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0214] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0215] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0216] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved.

Claims

1. A naked-eye 3D irregular region image fusion method, comprising: after completing display of a current projection image, obtaining a target static region in a next projection image; and if it is determined that a dynamic 3D fusion object to be moved is contained in the current projection image, obtaining a target dynamic region of the dynamic 3D fusion object in the next projection image; determining target pixel point positions corresponding to respective target threads according to the target static region and the target dynamic region, and rendering corresponding target pixel points using the target threads to obtain the next projection image; projecting the next projection image into a display screen for projection display.

2. The method of claim 1, wherein, Obtaining a target static region in a next projection image comprises: determining the target static region in the next projection image according to static partition description information; wherein the static partition description information comprises position information and size information of the target static region.

3. The method of claim 2, wherein, Determining target pixel point positions to be rendered according to the target static region and the target dynamic region comprises: determining first threads and first pixel point positions corresponding to respective first threads according to static partition description information of respective target static regions; extracting dynamic partition description information and screen anchoring information according to the target dynamic region, and obtaining a plurality of dynamic parameters according to the dynamic partition description information and the screen anchoring information; determining second pixel point positions corresponding to respective second threads according to the dynamic parameters and the screen anchoring information.

4. The method of claim 3, wherein, The dynamic partition description information comprises a first coordinate of a center point of the target dynamic region in the projection image, and size information of the target dynamic region in the projection image; the screen anchoring information comprises size information of the projection image and a second coordinate of an anchor point in the projection image.

5. The method of claim 4, wherein, Obtaining a plurality of dynamic parameters according to the dynamic partition description information and the screen anchoring information comprises: determining a region width and a region height of the target dynamic region in the projection image according to size information of the target dynamic region in the projection image; determining a horizontal coordinate difference and a vertical coordinate difference between the center point of the target dynamic region and the anchor point according to the first coordinate and the second coordinate.

6. The method of claim 5, wherein, Determining second pixel point positions corresponding to respective second threads according to the dynamic parameters and the screen anchoring information comprises: calling a calculation shader in an image processing engine to determine second pixel point positions corresponding to respective second threads according to the following formulae: id.x1 = id.x2 + LocalX + ScreenWidth / 2 - ΔWidth / 2; id.y1 = id.y2 + LocalY + ScreenHeight / 2 - ΔHeight / 2; Wherein, id.x1 and id.y1 are horizontal and vertical coordinates of the second pixel point, id.x2 and id.y2 are horizontal and vertical coordinates of the pixel point processed by the second thread in the current projection image, LocalX and LocalY are horizontal and vertical relative coordinates between the center point of the target dynamic region and the anchor point, ScreenWidth is the screen width of the next projection image, ScreenHeight is the screen height of the next projection image, ΔWidth is the region width of the target dynamic region in the next projection image, and ΔHeight is the reference height of the target dynamic region in the next projection image.

7. The method of claim 6, after obtaining the plurality of dynamic parameters according to the target partition description information and the screen anchor information, the method further comprises: determining, in a compute shader, a plurality of second threads for image blending of the target dynamic region according to the target dynamic region and the dynamic parameters.

8. The method of claim 7, wherein, determining, in a compute shader, a plurality of second threads for image blending of the target dynamic region according to the target dynamic region and the dynamic parameters, comprising: determining a thread arrangement mode in each thread group predefined in the compute shader; defining, in the compute shader, a second thread group for image blending of the target dynamic region according to the thread arrangement mode, the region width and the region height, and determining a plurality of second threads according to the second thread group.

9. The method of claim 1, wherein, a target thread in the compute shader is used to render a unique target pixel point corresponding to the target thread according to a blending rule to obtain a rendering result value of the target pixel point, and after each target thread completes the rendering, the next projection image is determined to complete the image blending processing.

10. The method of claim 1, wherein, The target static region comprises a static 3D blending object, and the static 3D blending object comprises a vehicle state prompt element. The dynamic 3D blending object comprises an augmented reality (AR) prompt element, and the AR prompt element specifically comprises an AR navigation element and an AR obstacle prompt element.

11. A head-up display system comprising a display screen and a head-up display, wherein the head-up display comprises a control component. wherein, The control component is configured to perform the naked-eye 3D indefinite region image blending method of any one of claims 1-10.

12. The system of claim 11, wherein, The head-up display further comprises an optical grating device and a mirror device. The optical grating device is configured to split the projection image to generate a split image to realize three-dimensional imaging. The mirror device is configured to project the split projection image into the display screen according to a preset angle to realize head-up display.

13. A naked-eye 3D indefinite region image blending device, comprising: a region determination module configured to, after completing display of a current projection image, obtain a target static region in a next projection image; and if it is determined that the current projection image comprises a dynamic 3D blending object to be moved, obtain a target dynamic region of the dynamic 3D blending object in the next projection image. ​ The image fusion module is configured to determine target pixel point positions corresponding to each target thread according to the target static area and the target dynamic area, and perform rendering processing on the corresponding target pixel points by using the target thread to obtain a next projection image. The projection display module is configured to project the next projection image to a display screen for projection display.

14. A projection device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the naked-eye 3D indefinite area image fusion method of any one of claims 1-10.

15. A computer readable storage medium, the computer readable storage medium stores computer instructions, the computer instructions are used to enable the processor to implement the naked-eye 3D indefinite area image fusion method of any one of claims 1-10 when executed.

Images