A floating 3D imaging method, imaging system and vehicle

Abstract

This application provides a suspended 3D imaging method, imaging system, and vehicle, relating to the field of suspended 3D imaging technology for intelligent cockpits. The imaging method includes acquiring multiple parallax images of a digital 3D model and forming a suspended 3D image of the digital 3D model based on the multiple parallax images. Acquiring multiple parallax images of the digital 3D model includes: acquiring N sub-images of the i-th region of the digital 3D model, where i≥1; forming a first image sequence of the i-th region based on the N sub-images; and redistributing the sub-image arrangement order of the first image sequence to form a second image sequence of the i-th region. The second image sequence has a different arrangement order than the first image sequence. The initial arrangement order of the sub-images has been rearranged to restore the correct mapping relationship between the left and right eye images, ensuring that the left and right eyes can receive the correct parallax information after the light signal is reflected by the micromirror array plate, thus solving the depth inversion problem introduced by the micromirror array plate.

Description

Technical Field

[0001] This application relates to the field of intelligent cockpit floating 3D imaging technology, and in particular to a floating 3D imaging method, a floating 3D imaging system, and a vehicle. Background Technology

[0002] 3D display technology has significant application value and broad development prospects in numerous fields. However, traditional 3D displays always rely on physical screens, which weakens the immersive nature of 3D visualization. In contrast, suspended 3D display technology, achieved by combining suspended display elements with 3D technology, eliminates the need for a physical screen and can provide a more immersive and realistic 3D viewing experience. These characteristics are highly valuable in areas such as human-computer interaction in smart cockpits. However, current suspended 3D display technology is still limited by issues such as depth inversion. Summary of the Invention

[0003] In view of this, this application provides a suspended 3D imaging method, imaging system, and vehicle, as follows:

[0004] In a first aspect, this application provides a suspended 3D imaging method, including:

[0005] A digital 3D model is provided, and multiple disparity images of the digital 3D model are obtained, wherein the multiple disparity images correspond to different regions of the digital 3D model.

[0006] An imaging module is used to form a floating 3D image of the digital 3D model based on the multiple parallax images; the imaging module includes a 3D display component and a micromirror array plate, the 3D display component forms N viewpoints of the digital 3D model based on the parallax images, and the micromirror array plate forms a floating 3D image of the digital 3D model based on the N viewpoints;

[0007] Obtaining multiple disparity images of the digital 3D model includes:

[0008] Obtain N sub-images of the i-th region of the digital 3D model, where i ≥ 1, and the i-th region is any one of the different regions of the digital 3D model. The N sub-images represent the images of the N viewpoints of the i-th region, respectively.

[0009] A first image sequence for the i-th region is formed based on the N sub-images, wherein the N sub-images are arranged in a first preset order.

[0010] The first image sequence is reordered to form a second image sequence for the i-th region. In the second image sequence, the N sub-images are arranged in a second preset order, which is different from the first preset order.

[0011] Secondly, this application provides a suspended 3D imaging system, which executes the suspended 3D imaging method described in any of the above claims to form a suspended 3D image of a digital three-dimensional model. The imaging system includes an imaging module, which includes a three-dimensional display component and a micromirror array plate.

[0012] The imaging module forms a floating 3D image of the digital 3D model based on multiple parallax images of the digital 3D model, and the multiple parallax images correspond to different regions of the digital 3D model respectively; wherein, the 3D display component forms N viewpoints of the digital 3D model based on the parallax images, and the micromirror array plate forms a floating 3D image of the digital 3D model based on the N viewpoints.

[0013] The parallax image includes N sub-images, each of which corresponds to one of the N viewpoints, and the N sub-images form a second image sequence arranged in a second preset order; wherein, the second image sequence is obtained by redistributing the sub-image arrangement order of the first image sequence, the N sub-images in the first image sequence are arranged in a first preset order, and the second preset order is different from the first preset order.

[0014] Thirdly, this application provides a vehicle, including an intelligent cockpit domain controller and a floating 3D imaging system, wherein the floating 3D imaging system is any of the floating 3D imaging methods described above.

[0015] The intelligent cockpit domain controller is used to provide at least one of the digital 3D models, acquire multiple parallax images of the digital 3D models, and redistribute the sub-image arrangement order of the first image sequence to form the second image sequence; wherein, the at least one digital 3D model includes a preset digital 3D model and a digital 3D model formed based on the user image;

[0016] The intelligent cockpit domain controller is also used to perform 3D floating imaging of the digital 3D model and multimodal human-computer interaction with the user. The multimodal human-computer interaction includes realizing 3D floating imaging of the digital 3D model and human-computer interaction with the user based on the user's facial expressions, gestures, voice and emotions.

[0017] Compared with related technologies, the beneficial effects of the technical solution in this application are as follows:

[0018] This imaging method involves acquiring multiple disparity images of a digital 3D model and forming a suspended 3D image of the digital 3D model based on these multiple disparity images. Acquiring multiple disparity images of the digital 3D model includes: acquiring N sub-images of the i-th region of the digital 3D model, where i ≥ 1, and the i-th region is any one of the different regions of the digital 3D model; the N sub-images representing images of N viewpoints in the i-th region; forming a first image sequence of the i-th region based on the N sub-images, in which the N sub-images are arranged in a first preset order; and redistributing the sub-image arrangement order of the first image sequence to form a second image sequence of the i-th region, in which the N sub-images are arranged in a second preset order, which is different from the first preset order. This imaging method can rearrange the initial arrangement order of the sub-images in the disparity images, restoring the correct mapping relationship between the left and right eye images, ensuring that the left and right eyes can receive the correct disparity information after the light signal is reflected by the micromirror array plate, and solving the depth inversion problem introduced by the micromirror array plate. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] The structures, proportions, sizes, etc., shown in the accompanying drawings are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the implementation conditions of this application. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size should still fall within the scope of the technical content disclosed in this application, provided that they do not affect the effects and purposes that this application can produce.

[0021] Figure 1 A flowchart of a suspended 3D imaging method provided in this application;

[0022] Figure 2 A flowchart of another suspended 3D imaging method provided in this application;

[0023] Figure 3 and Figure 4 These are schematic diagrams of the optical paths before and after the sub-images are rearranged in order.

[0024] Figure 5 This represents the relative relationship between the first image sequence and the second image sequence;

[0025] Figure 6 This is a schematic diagram of the structure of a 3D display component;

[0026] Figure 7 This is a schematic diagram of the structure of a micromirror array plate;

[0027] Figure 8 This is a schematic diagram illustrating the relationship between the disparity image, the first image sequence, and the second image sequence of a digital 3D model.

[0028] Figure 9 This is a schematic diagram illustrating the effect of a suspended 3D image formed using the suspended 3D imaging method provided in this application. Detailed Implementation

[0029] The embodiments of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0030] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] As described in the background section, floating 3D display technology can provide a more immersive and realistic 3D viewing experience, but it is limited by the depth inversion phenomenon and also suffers from problems such as poor image quality.

[0032] Suspended display elements typically include dihedral reflector arrays, retroreflective devices, freeform mirrors, parabolic mirrors, microlens arrays, and micromirror array plates. Among these, micromirror array plates, with their simpler and lower-cost planar structure, have become highly attractive suspended display elements. However, current suspended 3D displays based on micromirror array plates suffer from image drift and visual discomfort due to the depth inversion introduced by the micromirror array plates.

[0033] Based on the above, this application provides a suspended 3D imaging method, such as... Figure 1 As shown, Figure 1 A flowchart of a levitation 3D imaging method provided in this application is included, the imaging method comprising:

[0034] S1: Provide a digital 3D model, obtain multiple disparity images of the digital 3D model, and obtain the disparity images of each region of the digital 3D model.

[0035] S2: Utilizing an imaging module based on multiple parallax images, a suspended 3D image of the digital 3D model is formed. The imaging module includes a 3D display component and a Micro-Mirror Array Plate (MMAP). The 3D display component forms N viewpoints of the digital 3D model based on the parallax images, and the MMAP plate forms a suspended 3D image of the digital 3D model based on the N viewpoints, thus forming a suspended 3D image of the digital 3D model. In other words, this suspended 3D imaging method achieves suspended 3D imaging based on a 3D display component combined with a MMAP plate.

[0036] like Figure 2 As shown, acquiring multiple disparity images of a digital 3D model includes:

[0037] S11: Obtain N sub-images of the i-th region of the digital 3D model, where i ≥ 1. The i-th region can be any one of the different regions of the digital 3D model, and the N sub-images represent the images of the i-th region from N viewpoints. It should be noted that the N sub-images correspond one-to-one with the N viewpoints, and the N viewpoints represent N different perspectives.

[0038] S12: Form a first image sequence for the i-th region based on N sub-images, wherein the N sub-images are arranged in a first preset order.

[0039] S13: The sub-images of the first image sequence are reordered to form a second image sequence for the i-th region. In the second image sequence, N sub-images are arranged in a second preset order, which is different from the first preset order. It should be noted that the i-th region can be any region of the digital 3D model, that is, the disparity image of any region of the digital 3D model can be obtained based on the steps described in S11~S13, thus obtaining disparity images corresponding to multiple regions of the digital 3D model, i.e., obtaining multiple disparity images of the digital 3D model. In other words, this imaging method can use the Parallax Image Reassignment Method (PIRM) to reorder the sub-images in the disparity image.

[0040] Human perception of stereoscopic depth relies on the correspondence between the images of the left and right eyes. When a point in an image scene is projected onto each eye, the brain interprets the difference in their relative positions as spatial depth. However, if the optical paths swap the images that should be transmitted to the left and right eyes, the correspondence is reversed. For example, a point that should be inside the screen is perceived as outside, and vice versa. This reversal leads to a "pseudo-stereoscopic" effect, where raised areas appear sunken and sunken areas appear raised, resulting in an inverted sense of spatial depth. In suspended 3D imaging displays based on micromirror arrays (MMAPs), the conjugate mirror configuration alters the light propagation path, causing the left and right eye views to be interchanged.

[0041] like Figure 3 As shown, light emitted from virtual point 1 propagates through the 3D display component 10 to the micromirror array (MMAP), converges at the real point 2 outside the mirror image, and then diverges towards the human eye. This virtual point can be understood as a parallax image, or a sub-image within a parallax image. Specifically, light emitted from the parallax image propagates through the 3D display component 10 to the MMAP, converges at the real point 2 outside the mirror image, and then diverges towards the human eye. Normally, the left eye 3 receives violet light, and the right eye 4 receives orange light. However, due to the conjugate mirror configuration altering the light propagation path, the violet light carrying the parallax information of the left eye... The orange light, carrying the right eye's parallax, entered the right eye (4). It entered the left eye 3. Then, it was formed by both eyes based on the purple light carrying the disparity information of the left eye. and orange light carrying right-eye parallax The binocular parallax point formed by connecting these two rays corresponds precisely to the real point 2 in front of the mirror image, thus creating depth inversion. Therefore, restoring the correct mapping relationship between the left and right eye images is crucial for maintaining accurate depth perception in levitation 3D imaging.

[0042] In this application, the imaging method rearranges the order of sub-images in the first image sequence to obtain a second image sequence with a different order. In other words, this imaging method rearranges the initial order of sub-images in the parallax image to restore the correct mapping relationship between the left and right eye images, ensuring that both eyes receive the correct parallax information after the light signal is reflected by the micromirror array (MMAP), thus solving the depth inversion problem introduced by the MMAP. Specifically, as follows... Figure 4 As shown, the second image sequence is obtained by redistributing the sub-image order of the first image sequence. Based on the second image sequence, the purple rays carrying left-eye disparity information can be swapped. and orange light carrying right-eye parallax The parallax mapping enables the purple light carrying the parallax information of the left eye to... Orange light entering the left eye (3), carrying the parallax of the right eye. Entering the right eye (4), the intersection of these two light rays now corresponds to virtual point 1 on the screen, preserving the original depth relationship. It should be noted that... Figure 3 In the diagram, "10" represents the virtual 3D display component, "5" and "6" represent the virtual left eye and right eye respectively, and "D" represents the distance between the human eye and the 3D display component.

[0043] Furthermore, as described above, this imaging method addresses the depth inversion problem in suspended 3D imaging by rearranging the order of sub-images within the parallax image during the sub-image encoding stage, rather than relying on optical filtering or physical structural constraints. Therefore, this imaging method can generate stable, drift-free suspended 3D images with good imaging quality while maintaining a compact architecture (without mechanical movement or a bulk medium), achieving natural depth perception.

[0044] In one embodiment of this application, the nth sub-image in the second preset order is the (N+1-n)th sub-image in the first preset order, where n≥1. For example, This represents the nth sub-image in the second image sequence arranged according to a second preset order. This represents the nth sub-image in the first image sequence arranged in a first preset order. It should be noted that the above... This represents the position of each sub-image in its respective image sequence, specifically the "coordinate position of the sub-image mapped to the pixel region" as described later.

[0045] Specifically, such as Figure 5 As shown, Figure 5 Image a in the middle represents the first image sequence, which includes the 1st, 2nd, 3rd, ..., nth, ..., Nth sub-images. Figure 5 Image b represents the second image sequence, which includes sub-images 1, 2, 3, ..., n, ..., N. However, according to... Figure 5 As shown in Figures a and b, this imaging method utilizes a disparity image redistribution method to reverse the order of sub-images in the disparity image, thereby restoring the correct mapping relationship between the left and right eye images. This ensures that both eyes can receive the correct disparity information after the light signal is reflected by the micromirror array (MMAP), solving the depth inversion problem introduced by the MMAP. It should be noted that... Figure 5 The "200" in the text represents a cylindrical lens grating.

[0046] In one embodiment of this application, such as Figure 6 As shown, Figure 6 This is a schematic diagram of a 3D display assembly, which includes a display panel 100 and a lenticular lens grating 200. The plane of the lenticular lens grating 200 is parallel to the plane of the display panel 100. This 3D display assembly is a transmissive 3D display assembly, also known as a lenticular 3D display. It is known that each 3D point in a digital 3D model is associated with a pair of rays passing through the lenticular lens grating 200 and the micromirror array (MMAP), with their physical intersection located in front of the MMAP. Before applying PIRM, the viewpoint assignments of these rays are interchanged, so a scene rendered behind the screen is perceived as being in front of the air plane (pseudo-stereoscopic depth). After PIRM processing, the same physical intersection is paired to the correct left and right parallax, which can be interpreted as a virtual point located at the target depth behind the screen.

[0047] Using a 3D display component to form N viewpoints based on parallax images includes:

[0048] The display panel 100 is divided into multiple pixel regions, each corresponding to a parallax image. The parallax image is mapped to its corresponding pixel region, forming an interlaced image of the digital 3D model on the display panel 100. Specifically, the display panel 100 encodes sub-images using an interlaced image format. This interlaced image format forms a 2D image on the display panel 100 used to achieve a floating 3D image display of the digital 3D model. This 2D image can be understood as the image material source for achieving the floating 3D image display of the digital 3D model. Each pixel region includes N sub-pixel regions, each corresponding to a sub-image. The N sub-images are arranged in a second preset order within the pixel region to restore the correct mapping relationship between the left and right eye images.

[0049] The lenticular lens grating 200 forms N viewpoints based on the interlaced images displayed on the display panel 100, providing a foundation for the micromirror array (MMAP) to form a suspended 3D image of the digital 3D model. It should be noted that the specific number of the aforementioned N viewpoints and their spatial distribution are determined by the lenticular lens grating 200 to obtain parallax images of different regions of the digital 3D model. Light rays from different sub-images in the parallax images are refracted through the lenticular lens grating 200 to the corresponding viewpoints; specifically, light rays from different sub-pixel regions in the parallax images are refracted through the lenticular lens grating 200 to the corresponding viewpoints.

[0050] In one embodiment of this application, the following continues... Figure 6 As shown, the display panel 100 includes multiple columns of pixel units 120 arranged along a first direction, and the pixel unit columns 120 extend along a second direction. The pixel unit columns 120 include at least one of red light-emitting columns, green light-emitting columns, and blue light-emitting columns, typically including red light-emitting columns, green light-emitting columns, and blue light-emitting columns. The first direction and the second direction intersect, for example, the first direction and the second direction are perpendicular.

[0051] The lenticular lens grating 200 includes a plurality of periodically arranged lenticular lenses 220. The lenticular lenses 220 extend along a third direction, which intersects with a second direction, and there is a first preset angle between the third direction and the second direction. That is, the lenticular lenses 220 in the lenticular lens grating 200 extend obliquely relative to the pixel unit column 120.

[0052] The cylindrical lens 220 corresponds to at least two columns of pixel units 120, meaning that the tilted cylindrical lens 220 can correspond to multiple columns of pixel units 120. This allows the cylindrical lens 220 to correspond to more sub-pixels, thereby generating more viewpoints along the first direction (e.g., N=80), increasing visual density and supporting simultaneous observation by multiple people. It should be noted that if the first direction is defined as horizontal and the second direction as vertical, then the cylindrical lens 220 tilted relative to the vertical direction can utilize its corresponding vertical sub-pixels to generate more horizontally distributed viewpoints, increasing visual density and supporting simultaneous observation by multiple people.

[0053] In one embodiment of this application, forming a first image sequence for the i-th region based on N sub-images includes:

[0054] Based on the position of the sub-pixel region in the pixel region corresponding to the i-th region, the number of viewpoints, the number of sub-pixel regions corresponding to the cylindrical lens, and the first preset angle, the mapping relationship between the sub-pixel region in the pixel region corresponding to the i-th region and the sub-image is obtained.

[0055] Based on the mapping relationship between sub-pixel regions and sub-images, sub-images are mapped to corresponding sub-pixel regions to form the first image sequence of the i-th region based on N sub-images.

[0056] The position of the sub-pixel region in the pixel region corresponding to the i-th region mentioned above is denoted as (k, l). The sub-pixels in the specific pixel region are arranged in an array, so the position of the sub-pixel is (k, l) (which is also the position of the sub-image in the aforementioned first image sequence). This means the sub-pixel in the k-th row and l-th column of the pixel region; the number of viewpoints is denoted as N; the number of sub-pixel regions corresponding to the cylindrical lens 220 is denoted as X; the first preset included angle is denoted as the tilt angle of the cylindrical lens 220, and the mapping relationship between sub-pixel regions and sub-images is denoted as α. Where n represents the disparity index, specifically the nth sub-image in the first image sequence; mod represents the modulo operation, applicable to integer indices. Through the mapping relationship between the sub-pixel regions and sub-images described above, it is possible to calculate which sub-image corresponds to each sub-pixel region in the pixel region, and thus map the sub-image to the corresponding sub-pixel region.

[0057] This application also provides a suspended 3D imaging system that performs the suspended 3D imaging method described in any of the above embodiments to form a suspended 3D image of a digital three-dimensional model. The imaging system includes an imaging module, which includes a 3D display component and a micromirror array (MMAP).

[0058] The imaging module forms a floating 3D image of the digital 3D model based on multiple parallax images. Each parallax image corresponds to a different region of the digital 3D model, meaning there is a one-to-one correspondence between the parallax images and different regions of the digital 3D model. The 3D display component forms N viewpoints of the digital 3D model based on the parallax images, and the micromirror array (MMAP) plate forms a floating 3D image of the digital 3D model based on these N viewpoints.

[0059] The parallax image comprises N sub-images, each corresponding to one of the N viewpoints, and the N sub-images form a second image sequence arranged in a second preset order. The second image sequence is obtained by redistributing the sub-image order of a first image sequence, where the N sub-images in the first image sequence are arranged in a first preset order, and the second preset order differs from the first preset order.

[0060] As described above, this imaging system performs levitation 3D display based on the second image sequence obtained by redistributing the sub-image arrangement order, enabling the purple light carrying left-eye parallax information to... Orange light entering the left eye, carrying the parallax of the right eye. Upon entering the right eye, the intersection of these two light rays re-corresponds to a virtual point on the screen, preserving the original depth relationship. In other words, this imaging system performs a floating 3D display based on the second image sequence obtained by redistributing the sub-image arrangement order, restoring the correct mapping relationship between the left and right eye images. This ensures that both eyes can receive the correct parallax information after the light signal is reflected by the micromirror array (MMAP), thus solving the depth inversion problem introduced by the MMAP.

[0061] Furthermore, this imaging system performs levitation 3D display based on a second image sequence obtained by sequentially redistributing sub-images. This means it addresses the depth inversion problem in levitation 3D imaging during the sub-image encoding stage, rather than relying on optical filtering or physical structural constraints. Therefore, the imaging system can generate stable, drift-free levitation 3D images with good imaging quality while maintaining a compact architecture (without mechanical movement or a bulk medium), achieving natural depth perception.

[0062] In one embodiment of this application, the nth sub-image in the second preset order is the (N+1-n)th sub-image in the first preset order, where n≥1. It should be noted that the correspondence between the first image sequence and the second image sequence has been described in detail in the foregoing imaging method embodiments and will not be repeated here.

[0063] In one embodiment of this application, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a 3D display component, which includes a display panel 100 and a lenticular lens grating 200. The plane of the lenticular lens grating 200 is parallel to the plane of the display panel 100.

[0064] The display panel 100 is divided into multiple pixel regions, each corresponding to a parallax image. The parallax image is mapped to the corresponding pixel region, forming an interlaced image of a digital 3D model on the display panel 100. Each pixel region includes N sub-pixel regions, each corresponding to a sub-image. The sub-image is mapped to the corresponding sub-pixel region, and the N sub-images are arranged in the pixel region according to a second preset order to restore the correct mapping relationship between the left and right eye images.

[0065] The lenticular lens grating 200 forms a 3D image of a digital three-dimensional model based on the interlaced images displayed on the display panel 100, providing a basis for the micromirror array plate MMAP to form a floating 3D image of a digital three-dimensional model based on N viewpoints.

[0066] In one embodiment of this application, the following continues... Figure 6 As shown, the display panel 100 includes multiple columns of pixel units 120 arranged along a first direction, and the pixel unit columns 120 extend along a second direction. The pixel unit columns 120 include at least one of red light-emitting columns, green light-emitting columns, and blue light-emitting columns, typically including red light-emitting columns, green light-emitting columns, and blue light-emitting columns. The first direction and the second direction intersect, for example, the first direction and the second direction are perpendicular.

[0067] The lenticular lens grating 200 includes a plurality of periodically arranged lenticular lenses 220. The lenticular lenses 220 extend along a third direction, which intersects with a second direction, and there is a first preset angle between the third direction and the second direction. That is, the lenticular lenses 220 in the lenticular lens grating 200 extend obliquely relative to the pixel unit column 120.

[0068] The cylindrical lens 220 corresponds to at least two columns of pixel units 120, meaning that the tilted cylindrical lens 220 can correspond to multiple columns of pixel units 120. This allows the cylindrical lens 220 to correspond to more sub-pixels, thereby generating more viewpoints along the first direction (e.g., N=80), increasing visual density and supporting simultaneous observation by multiple people. It should be noted that if the first direction is defined as horizontal and the second direction as vertical, then the cylindrical lens 220 tilted relative to the vertical direction can utilize its corresponding vertical sub-pixels to generate more horizontally distributed viewpoints, increasing visual density and supporting simultaneous observation by multiple people.

[0069] It should be noted that, Figure 6 The red light emitting unit, green light emitting unit and blue light emitting unit shown in the figure constitute a pixel unit, and the aforementioned sub-pixel region is composed of at least one pixel unit.

[0070] In one embodiment of this application, such as Figure 7 As shown in Figure a, the micromirror array plate (MMAP) includes a first micromirror array layer 320 and a second micromirror array layer 340. Both layers include multiple reflective micromirrors; specifically, the first micromirror array layer 320 includes multiple reflective micromirrors 320-1, and the second micromirror array layer 340 also includes multiple reflective micromirrors 340-1. There are gaps between adjacent reflective micromirrors 320-1 in the first micromirror array layer 320, and there are also gaps between adjacent reflective micromirrors 340-1 in the second micromirror array layer 340.

[0071] In this configuration, the reflecting micromirrors in the first micromirror array layer 320 and the reflecting micromirrors in the second micromirror array layer 340 are arranged orthogonally. Specifically, the reflecting micromirrors 320-1 in the first micromirror array layer 320 are parallel to each other, and the reflecting micromirrors 340-1 in the second micromirror array layer 340 are parallel to each other, and the reflecting micromirrors 320-1 in the first micromirror array layer 320 and the reflecting micromirrors 340-1 in the second micromirror array layer 340 are arranged orthogonally. Figure 7As shown in Figure b, when light enters the micromirror array plate MMAP at an appropriate angle, it can first be reflected by the reflective surface of the reflective micromirror 320-1 in the first micromirror array layer 320 to the reflective surface of the reflective micromirror 340-1 in the second micromirror array layer 340, and then reflected out by the reflective micromirror 340-1 in the second micromirror array layer 340. That is, when light enters the micromirror array plate MMAP at an appropriate angle, it can first be reflected by the first layer of reflective micromirrors to the second layer of reflective micromirrors, and then reflected out by the second layer of reflective micromirrors.

[0072] In one embodiment of this application, the plane containing the micromirror array plate (MMAP) is inclined relative to the plane containing the 3D display component, and a second preset angle exists between the plane containing the micromirror array plate (MMAP) and the plane containing the 3D display component. It should be noted that the plane containing the micromirror array plate (MMAP) refers to the plane containing both the first micromirror array layer 320 and the second micromirror array layer 340 being parallel to this plane, but not the plane containing the reflecting micromirrors 320-1 in the first micromirror array layer 320 and the second micromirror array layer 340-1. For example, in this application, the plane containing the reflecting micromirrors 320-1 in the first micromirror array layer 320 and the second micromirror array layer 340-1 is perpendicular to the plane containing the micromirror array plate (MMAP). The plane containing the 3D display component refers to the plane containing both the display panel 100 and the lenticular lens grating 200 being parallel to this plane.

[0073] To gain a clearer understanding of the suspended 3D imaging method and imaging system provided in this application, a specific embodiment will be described below.

[0074] This imaging system integrates a lenticular 3D display assembly and a micromirror array (MMAP). The lenticular 3D display assembly uses a 7.9-inch display panel 100 with a resolution of 1536×2048 and a pixel pitch of 0.078×0.078 mm. It employs an RGB vertical stripe subpixel arrangement, meaning the aforementioned pixel unit column 120 extends vertically (the second direction) and is arranged horizontally (the first direction). The lenticular lens grating 200 has a line count of 17.49 lines / mm, and the tilt angle of the lenticular lens relative to the pixel unit column 120 is 5.95°, i.e., a first preset angle of 5.95°. The micromirror array (MMAP) consists of a first micromirror array layer 320 and a second micromirror array layer 340 arranged orthogonally. The micromirror intercept is 50 μm, meaning the distance between adjacent reflecting micromirrors in the micromirror array layers (first micromirror array layer 320 and second micromirror array layer 340) is 50 μm, forming a square reflective array. A lenticular 3D display component is mounted at a 45° angle to the MMAP, i.e., a second preset angle of 45°, constituting the imaging module of the suspended 3D display system. The 3D image, i.e., the digital 3D model, is suspended approximately 3 cm above the MMAP, tilted at 45° relative to the horizontal plane, with a 60° field of view and a depth of field range of approximately ±5 cm (relative to the aerial display plane). The entire imaging system is approximately 15 cm high.

[0075] The aforementioned digital 3D model can be rendered using Blender software. The digital 3D model is placed in front of a horizontally arranged array of cameras, each camera capturing the 3D scene from its own viewpoint to generate corresponding sub-images. It should be noted that the aforementioned horizontally arranged camera array refers to virtual cameras; that is, the sub-images are not obtained through real camera captures, but rather through a simulated camera capture process. For example, in a computer graphics (CG) environment, the imaging process of a real camera is completely simulated to obtain sub-images from N viewpoints.

[0076] like Figure 8 As shown, Figure 8 Figure a in the middle represents a digital 3D model. Figure 8 Figure b shows the disparity image of the i-th region of the digital 3D model. The i-th region can be, for example... Figure 8 As shown in the dashed box in Figure a, this parallax image comprises 80 sub-images acquired from 80 different viewpoints. The resolution of the parallax image can be 3840×4096, and the resolution of each sub-image can be 384×512. The 80 sub-images are arranged in a left-to-right, top-to-bottom order, i.e., in an 8-row × 10-column array, corresponding to viewpoints 1 to 80. Figure 8Figure c shows the parallax image after applying PIRM processing, compared to... Figure 8 Compared to Figure b, Figure 8 In Figure c, the starting and ending sequences of the sub-images in the disparity image are reversed, presenting a reverse order. This ensures that the right contour of the 3D suspended image formed by the digital 3D model is visible from the left perspective, and the left contour of the 3D suspended image formed by the digital 3D model is visible from the right perspective, guaranteeing the accuracy of binocular disparity. This matching achieves good binocular fusion and significantly reduces visual fatigue caused by depth conflict. Specifically, as shown... Figure 9 As shown in Figure a. Figure 9 Figure a shows a 3D image of a 3D digital model formed using the levitation 3D imaging method provided in this application. Figure 9 In Figure a, (1), (2), and (3) represent the right outline of the 3D floating image formed by the digital 3D model as seen from the left-hand perspective, the front outline of the 3D floating image formed by the digital 3D model as seen from the front-hand perspective, and the left outline of the 3D floating image formed by the digital 3D model as seen from the right-hand perspective, respectively. For example... Figure 9 As shown in Figure b. Figure 9 Figure b shows a suspended 3D image of a three-dimensional digital model formed by an existing imaging method that does not utilize the imaging method provided in this application for left and right eye parallax correction. Figure 9 In Figure b, (4), (5), and (6) represent the right outline of the three-dimensional levitation image formed by the digital three-dimensional model as observed from the left-hand perspective, the front outline of the three-dimensional levitation image formed by the digital three-dimensional model as observed from the front-hand perspective, and the left outline of the three-dimensional levitation image formed by the digital three-dimensional model as observed from the right-hand perspective, respectively. Figure 9 As can be seen, the imaging method described in this application rearranges the initial arrangement order of sub-images in the parallax image, which can restore the correct mapping relationship between the left and right eye images, thereby ensuring that the left and right eyes can receive the correct parallax information after the light signal is reflected by the micromirror array plate (MMAP), thus solving the depth inversion problem introduced by the micromirror array plate (MMAP).

[0077] In addition, according to Figure 9 It can also be seen that, Figure 9 In Figure a, the 3D levitation images observed from different viewing angles did not exhibit significant drift. This means that as the human eye moves, the 3D levitation images generated by the digital 3D model maintain spatial stability in the air without any drift, eliminating artifacts caused by uncorrected mapping and providing a natural and consistent depth perception. Compared to the uncorrected 3D levitation images, the PIRM-corrected 3D levitation images show advantages in PSNR and SSIM metrics, demonstrating higher reconstruction fidelity and more accurate depth perception.

[0078] It should be noted that, Figure 8 Figure a in the middle and Figure 9 The image shown is merely an illustration of a digital 3D model; the actual digital 3D model can be numbers, people, animals, cartoon characters, etc. This application does not impose any limitations on this, and the specific model will be determined on a case-by-case basis.

[0079] This application also provides a vehicle including an intelligent cockpit domain controller and a floating 3D imaging system, wherein the floating 3D imaging system is the floating 3D imaging system described in any of the above embodiments.

[0080] The intelligent cockpit domain controller provides at least one digital 3D model, acquires multiple parallax images of the digital 3D model, and redistributes the sub-image order of a first image sequence to form a second image sequence. The at least one digital 3D model includes a preset digital 3D model and a digital 3D model based on the user's image.

[0081] The intelligent cockpit domain controller is also used for floating 3D imaging of digital 3D models and multimodal human-computer interaction with users. Multimodal human-computer interaction includes floating 3D imaging of digital 3D models and human-computer interaction with users based on their facial expressions, gestures, voice and emotions.

[0082] As described above, the intelligent cockpit domain controller provides a digital 3D model and, in conjunction with a suspended 3D imaging system, renders a personalized 3D suspended image of the digital 3D model in the air, achieving immersive, spatially anchored AI interaction. Specifically, the intelligent cockpit domain controller can be configured with Large Language Models (LLMs), enabling the suspended 3D imaging system to combine with the large language model to achieve immersive interaction. Furthermore, unlike traditional AI assistants limited to flat screens, this system can shape the AI ​​assistant into a suspended 3D virtual avatar in the air, achieving natural, spatially anchored human-computer interaction.

[0083] Furthermore, the intelligent cockpit domain controller can capture multimodal input from users via microphones, cameras, and infrared sensors, such as facial expressions, gestures, voice, and emotional information. This data is processed by a large language model and processing modules. The large language model parses the user's intent and generates a text response, which is then converted into synchronized speech and gestures using advanced text-to-speech synthesis technology (including lip-sync). This allows the suspended 3D virtual avatar to dynamically respond, displaying facial expressions, movements, and even tactile feedback, providing a human-like and vivid interactive experience. The aforementioned processing modules support personalization, allowing users to create their own personalized virtual avatars by referencing photos or text descriptions. They can also capture images of users (e.g., drivers or passengers) through cameras, identify their clothing styles, and customize digital 3D models with unique avatars, clothing, and voice tones. Users can also upload personal photos or videos, enabling the generated 3D suspended image of the digital 3D model to move in sync with music. Thus, this vehicle achieves aerial 3D visualization of auxiliary tools such as voice assistants and integrates them with AI-driven interaction, bringing a more immersive, intelligent, and emotionally resonant new paradigm of human-computer interaction to intelligent cockpit and other application scenarios.

[0084] It should be noted that the aforementioned floating 3D imaging system can be placed on the vehicle's control panel to enable real-time interaction with the driver or front passenger, or it can be placed in the armrest or rear center armrest to facilitate comprehensive entertainment and interaction for the driver, front passenger, and rear passengers.

[0085] In one embodiment of this application, the vehicle further includes a streaming rearview mirror system (Camera-Monitor System, CMS), a head-up display (HUD), and pixel headlights. The intelligent cockpit domain controller is used to receive and process images captured by the streaming rearview mirror system and output them to the head-up display system; receive and process head-up display data and output it to the head-up display system; and control the display of the pixel headlights. It should be noted that the aforementioned head-up display system includes at least one of a panoramic head-up display (PHUD) module, an augmented reality head-up display (AR-HUD) module, and a holographic head-up display (3D-HUD) module. The panoramic head-up display (PHUD) content may include vehicle performance parameters, navigation images, entertainment, song lyrics, and time information; the augmented reality head-up display (AR-HUD) content may include 3D guide signs, AR images, AR guide signs for augmented reality, and information on interaction with the real world; the holographic head-up display (3D-HUD) eliminates the need for a screen and provides a more immersive and realistic 3D viewing experience.

[0086] The intelligent cockpit domain controller is also used for: 1) creating a floating 3D digital model based on images captured by the streaming rearview mirror system for human-computer interaction; 2) creating a floating 3D digital model based on head-up display data for human-computer interaction; and 3) creating a floating 3D digital model based on pixel headlight displays for human-computer interaction.

[0087] Specifically, the PHUD (Head-Up Display) module is responsible for presenting static vehicle information and navigation guidance, while integrating information from the CMS (Content Management System) to achieve "multi-functionality on one screen." The AR-HUD works in conjunction with the PHUD to fuse virtual information with actual road conditions using AR, enabling the identification of targets such as vehicles and pedestrians ahead. The pixel headlights work in conjunction with the AR-HUD to provide functions such as light carpet navigation and pedestrian priority prompts. The floating 3D imaging system further integrates the capabilities of the aforementioned CMS, HUD, and pixel headlights, providing natural human-computer interaction for drivers and passengers. It extracts and identifies relevant information from the AR-HUD, PHUD, CMS, and pixel headlights (including colored pixel headlights), providing real-time road condition reminders, pedestrian warnings, and speed reduction prompts to the driver. Meanwhile, the ambient lighting inside the vehicle enhances the ambiance, and combined with the floating 3D images of the digital 3D model and human-computer interaction, it provides a comfortable atmosphere and emotional value for the driver and passengers.

[0088] In summary, this application provides a suspended 3D imaging method, imaging system, and vehicle. The imaging method includes acquiring multiple parallax images of a digital 3D model and forming a suspended 3D image of the digital 3D model based on the multiple parallax images. Acquiring multiple parallax images of the digital 3D model includes: acquiring N sub-images of the i-th region of the digital 3D model, where i ≥ 1, and the i-th region is any one of the different regions of the digital 3D model; the N sub-images representing images of N viewpoints in the i-th region; forming a first image sequence of the i-th region based on the N sub-images, in which the N sub-images are arranged in a first preset order; and redistributing the sub-image arrangement order of the first image sequence to form a second image sequence of the i-th region, in which the N sub-images are arranged in a second preset order, which is different from the first preset order. This imaging method can rearrange the initial arrangement order of the sub-images in the parallax image, restoring the correct mapping relationship between the left and right eye images, ensuring that the left and right eyes can receive the correct parallax information after the light signal is reflected by the micromirror array plate, and solving the depth inversion problem introduced by the micromirror array plate.

[0089] The various embodiments in this specification are described in a progressive, parallel, or combined manner. Each embodiment focuses on its differences from other embodiments, and similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0090] It should be noted that, in the description of this application, the terms "upper," "lower," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. When a component is considered to be "connected" to another component, it can be directly connected to the other component or there may be a component centrally located at the same time.

[0091] It should also be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes the aforementioned element.

[0092] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of suspended 3D imaging, characterized in that, include: A digital 3D model is provided, and multiple disparity images of the digital 3D model are obtained, wherein the multiple disparity images correspond to different regions of the digital 3D model. An imaging module is used to form a floating 3D image of the digital 3D model based on the multiple parallax images; the imaging module includes a 3D display component and a micromirror array plate, the 3D display component forms N viewpoints of the digital 3D model based on the parallax images, and the micromirror array plate forms a floating 3D image of the digital 3D model based on the N viewpoints; Obtaining multiple disparity images of the digital 3D model includes: Obtain N sub-images of the i-th region of the digital 3D model, where i ≥ 1, and the i-th region is any one of the different regions of the digital 3D model. The N sub-images represent the images of the N viewpoints of the i-th region, respectively. A first image sequence for the i-th region is formed based on the N sub-images, wherein the N sub-images are arranged in a first preset order. The first image sequence is reordered to form a second image sequence for the i-th region. In the second image sequence, the N sub-images are arranged in a second preset order, which is different from the first preset order. The three-dimensional display component includes a display panel and a lenticular lens grating. The plane containing the lenticular lens grating is parallel to the plane containing the display panel. The N viewpoints used by the three-dimensional display component to form the digital three-dimensional model include: The display panel is divided into multiple pixel regions, each pixel region corresponding to a parallax image. The parallax image is mapped to the corresponding pixel region to form an interlaced image of the digital 3D model on the display panel. Each pixel region includes N sub-pixel regions, each sub-pixel region corresponding to a sub-image, and the N sub-images are arranged in the pixel region according to the second preset order. The lenticular lens grating forms the N viewpoints based on the interlaced images displayed on the display panel.

2. The method of claim 1, wherein, The nth sub-image in the second preset order is the (N+1-n)th sub-image in the first preset order, where n≥1.

3. The method of claim 1, wherein, The display panel includes multiple columns of pixel units arranged along a first direction, the pixel unit columns extending along a second direction, and the pixel unit columns including at least one of red light emitting columns, green light emitting columns, and blue light emitting columns, wherein the first direction and the second direction intersect. The lenticular lens grating includes a plurality of lenticular lenses arranged periodically, the lenticular lenses extending along a third direction, the third direction intersecting the second direction, and the third direction having a first preset angle with the second direction; The cylindrical lens corresponds to at least two columns of the pixel unit; The lenticular lens grating forms the N viewpoints based on the interlaced images displayed on the display panel, including: The cylindrical lens transmits light beams from different parallax images in different directions, forming the N viewpoints.

4. The method of claim 3, wherein, The first image sequence for forming the i-th region based on the N sub-images includes: Based on the position of the sub-pixel region in the pixel region corresponding to the i-th region, the number of viewpoints, the number of sub-pixel regions corresponding to the cylindrical lens, and the first preset angle, the mapping relationship between the sub-pixel region in the pixel region corresponding to the i-th region and the sub-image is obtained; Based on the mapping relationship between the sub-pixel regions and the sub-images, the sub-images are mapped to the corresponding sub-pixel regions to form a first image sequence of the i-th region based on the N sub-images.

5. A suspended 3D imaging system characterized in that, The imaging system performs the levitation 3D imaging method according to any one of claims 1-4 to form a levitation 3D image of a digital three-dimensional model. The imaging system includes an imaging module, which includes a three-dimensional display component and a micromirror array plate. The imaging module forms a floating 3D image of the digital 3D model based on multiple parallax images of the digital 3D model, and the multiple parallax images correspond to different regions of the digital 3D model respectively; wherein, the 3D display component forms N viewpoints of the digital 3D model based on the parallax images, and the micromirror array plate forms a floating 3D image of the digital 3D model based on the N viewpoints. The parallax image includes N sub-images, each of which corresponds to one of the N viewpoints, and the N sub-images form a second image sequence arranged in a second preset order; wherein, the second image sequence is obtained by redistributing the sub-image arrangement order of a first image sequence, the N sub-images in the first image sequence are arranged in a first preset order, and the second preset order is different from the first preset order; The three-dimensional display component includes a display panel and a lenticular lens grating. The plane containing the lenticular lens grating is parallel to the plane containing the display panel. The N viewpoints used by the three-dimensional display component to form the digital three-dimensional model include: The display panel is divided into multiple pixel regions, each pixel region corresponding to a parallax image. The parallax image is mapped to the corresponding pixel region to form an interlaced image of the digital 3D model on the display panel. Each pixel region includes N sub-pixel regions, each sub-pixel region corresponding to a sub-image, and the N sub-images are arranged in the pixel region according to the second preset order. The lenticular lens grating forms the N viewpoints based on the interlaced images displayed on the display panel.

6. The floating 3D imaging system of claim 5, wherein, The micromirror array plate includes a first micromirror array layer and a second micromirror array layer, and the first micromirror array layer and the second micromirror array layer include a plurality of reflective micromirrors; The reflective micromirrors in the first micromirror array layer are arranged orthogonally to the reflective micromirrors in the second micromirror array layer.

7. The floating 3D imaging system of claim 5, wherein, The plane where the micromirror array plate is located is inclined relative to the plane where the three-dimensional display component is located, and there is a second preset angle between the plane where the micromirror array plate is located and the plane where the three-dimensional display component is located.

8. A vehicle characterized by comprising: It includes an intelligent cockpit domain controller and a suspended 3D imaging system, wherein the suspended 3D imaging system is the suspended 3D imaging system according to any one of claims 5-7; The intelligent cockpit domain controller is used to provide at least one digital 3D model, acquire multiple parallax images of the digital 3D model, and redistribute the sub-image arrangement order of the first image sequence to form a second image sequence; wherein, the at least one digital 3D model includes a preset digital 3D model and a digital 3D model formed based on the user's image. The intelligent cockpit domain controller is also used to perform 3D floating imaging of the digital 3D model and multimodal human-computer interaction with the user. The multimodal human-computer interaction includes realizing 3D floating imaging of the digital 3D model and human-computer interaction with the user based on the user's facial expressions, gestures, voice and emotions.

9. The vehicle of claim 8, wherein, It also includes a streaming rearview mirror system, an in-vehicle head-up display system, and pixel headlights; The intelligent cockpit domain controller is used to receive and process images captured by the streaming media rearview mirror system and output them to the vehicle head-up display system; receive and process head-up display data and output it to the vehicle head-up display system; and control the display of the pixel headlights. The intelligent cockpit domain controller is also used to perform floating 3D imaging of the digital 3D model and human-computer interaction with the user based on the images captured by the streaming media rearview mirror system; and to perform floating 3D imaging of the digital 3D model and human-computer interaction with the user based on the head-up display data. Based on the display of the pixel headlights, the digital three-dimensional model is used for floating 3D imaging and human-computer interaction with the user.

Images