Method and system for optimizing projection display

Abstract

The present invention provides a method and system for optimizing projection display, wherein the method for optimizing projection display is characterized in that the method for optimizing projection display comprises the following steps: (a) collecting an image of a projection surface without illumination, and illuminating the image of the projection surface with projection light from a projector to obtain parameters of ambient light and the projection surface; (b) calculating a reflection spectrum of the projection surface based on the parameters collected by various projection lights; and (c) linearly mapping the reflection spectrum of the projection surface to a coordinate system of a projection screen after measuring the projection light path to obtain a projected adjustment image.

Description

Technical Field

[0001] This invention relates to the field of projection display technology, and more particularly to a method and system for optimizing projection displays. Background Technology

[0002] A projection device, also known as a projector, is an electronic device that projects image content onto a target area (such as a projection screen, a blank wall area, etc.). In existing technology, before projecting image content onto the target area, the projector needs to be dimmed to ensure accurate display. Currently, there are two main external factors affecting the projection effect: ambient light and the condition of the surface onto which the image is projected.

[0003] Different types of projectors have different projection methods and dimming methods. For example, if the projection surface is gradient color, it is difficult to adjust the projector according to the current dimming technology, which also makes it difficult for the projector's projection display effect to meet the ideal requirements. Summary of the Invention

[0004] A key advantage of this invention is that it provides a method and system for optimizing projection display, wherein the method optimizes projection display by linearly mapping the reflection spectrum of the projection surface to the coordinate system of the projection image after calculating the projection optical path, thereby obtaining a calibrated image of the projection (pixel level), thus improving the projection effect.

[0005] Another advantage of the present invention is that it provides a method and system for optimizing projection display, wherein the method for optimizing projection display can obtain pixel-level reflectivity information of the projection surface, thereby achieving the effect of solving background problems in different situations and displaying the original image during projection.

[0006] Another advantage of the present invention is that it provides a method and system for optimizing projection display, wherein the method for optimizing projection display can project pixel-level reflectivity results onto the coordinate system of the projector screen, thereby obtaining a pixel-level projection scheme that can precisely cancel out the background reflectivity texture and achieve a perfect projection effect.

[0007] Another advantage of the present invention is that it provides a method and system for optimizing projection display, wherein the method and system for optimizing projection display can perform spectral calculations on an NPU, ensuring high spectral accuracy while also having low power consumption, making it very suitable for mobile terminals such as mobile phones.

[0008] According to one aspect of the present invention, an optimized projection display method of the present invention, which can achieve the foregoing and other objects and advantages, includes the following steps:

[0009] (a) Acquire images of the projection surface under no illumination and images of the projection surface illuminated with uniform strong light from a projector to obtain parameters of ambient light and projection surface;

[0010] (b) Based on the parameters collected from various projected lights, the reflection spectrum of the projection surface is calculated; and

[0011] (c) The reflection spectrum of the projection surface is linearly mapped onto the coordinate system of the projection image after the projection light path is calculated, so as to obtain the calibrated image of the projection.

[0012] According to an embodiment of the present invention, step (b) of the method for optimizing projection display further includes:

[0013] (b.1) Based on the acquired projection surface image, determine whether the projection surface is a uniform color projection surface, a gradient color projection surface, or a strong texture projection surface; and

[0014] (b.2) If the projection surface is a uniform projection surface, the ambient light spectrum and the reflection spectrum of the projection surface are obtained based on the reflection spectrum corresponding to the color block of the image set in the projection surface; if the projection surface is a gradient color projection surface, the entire surface of the projection surface is linearly sampled and smoothed based on the expectation that the projection surface belongs to a gradient reflection surface, so as to obtain the reflection spectrum of the projection surface.

[0015] According to one embodiment of the present invention, prior to step (b.2) of the method for optimizing the projection display, the method further includes: performing a sampling multi-point reflectivity measurement and an ambient light measurement on the entire projection surface.

[0016] According to one embodiment of the present invention, in step (b.2) of the method for optimizing projection display, if the target projection surface is a gradient color, a mosaic repeating image of the basic calibration image is projected onto the full screen.

[0017] According to an embodiment of the present invention, in step (a) of the method for optimizing projection display, at least one set image is projected onto the projection surface, and the reflectance spectrum of its multi-channel spectral image is obtained and calculated to obtain the ambient spectrum and the reflectance of the projection surface.

[0018] According to one embodiment of the present invention, the set image is a single image comprising multiple color blocks of arbitrary color with different brightness, or multiple images comprising multiple color blocks of arbitrary color with different brightness, each of the multiple images comprising one or more color blocks, and the one or more color blocks having the same or different brightness and the same or different color.

[0019] According to an embodiment of the present invention, step (a) of the method for optimizing projection display further includes the following steps:

[0020] (a.1) Determine the base light spectrum of a set image, said set image comprising at least two color patches of arbitrary color with different brightness; and

[0021] (a.2) Project the set image onto the target area of ​​the projection surface and obtain the reflection spectrum of the set image reflected from the target area of ​​the projection surface.

[0022] According to one embodiment of the present invention, before step (a.1) of the method for optimizing projection display, the method further includes: acquiring the substrate light spectrum of the set image, wherein the set image includes a plurality of color blocks of arbitrary colors with different brightness.

[0023] According to an embodiment of the present invention, in step (a.1) of the method for optimizing projection display, determining the base light color of the set image includes:

[0024] Projecting substrate light of a known spectrum onto the target area of ​​the projection surface; and

[0025] The reflection spectrum of the substrate light in the target area of ​​the projection surface is obtained as the substrate light spectrum.

[0026] According to one embodiment of the present invention, the RGB color of the known spectrum of the substrate light is (0, 0, 0).

[0027] According to an embodiment of the present invention, in step (a.2) of the method for optimizing projection display, the reflection spectrum of each region corresponding to each color block among the plurality of color blocks is defined based on the reflection spectrum of the set image.

[0028] According to an embodiment of the present invention, in step (a.2) of the method for optimizing projection display, obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image includes:

[0029] The reflectance spectrum relationship of the region corresponding to each color patch is determined as follows:

[0030] Reflectance spectrum n = (substrate light spectrum + ambient light spectrum) * reflectance;

[0031] Reflectance spectrum 1 = (Region 1 spectrum + Ambient light spectrum) * Reflectance;

[0032] Reflectance spectrum 2 = (Region 2 spectrum + Ambient light spectrum) * Reflectance;

[0033] ...

[0034] Reflectance spectrum n-1 = (Regional n-1 spectrum + Ambient light spectrum) * Reflectance.

[0035] According to an embodiment of the present invention, in step (a.2) of the method for optimizing projection display, obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image further includes:

[0036] Define the loss function as:

[0037] Loss = ||Reflectance Spectrum 1 - (Region 1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum 2 - (Region 2 Spectrum + Ambient Light Spectrum) * Reflectance|| + ... + ||Reflectance Spectrum n-1 - (Region n-1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance n - (Substrate Light Spectrum + Ambient Light Spectrum) * Reflectance||, where ||·|| represents the norm of a vector or matrix.

[0038] According to one embodiment of the present invention, the ambient light spectrum and the reflectance of the target area of ​​the projection surface are solved from the relationship using a neural network modeling method.

[0039] According to an embodiment of the present invention, in step (a) of the method for optimizing projection display, the corresponding image information is obtained according to the characteristics of the spatial domain, including the following steps: simultaneously acquiring an image, which includes at least a number of color blocks of arbitrary color with different brightness, wherein the spectral information of the corresponding different colors is also known.

[0040] According to an embodiment of the present invention, in step (a) of the method for optimizing projection display, corresponding image information is obtained according to the characteristics of the time domain. Only one color block of the corresponding color is obtained in each image, so that each color can fill the entire image. The corresponding spectral information can be obtained in the corresponding pixel behind each image, and more pixel elements can be obtained at the same time. The pixel element is the spectral information of the smallest unit in each color block.

[0041] According to another aspect of this application, this application further provides a projection display system, including:

[0042] Projector; and

[0043] The projector and the spectral imaging sensor are electrically connected. The spectral imaging sensor acquires the substrate light spectrum and the reflection spectrum information of the set image. The projector calculates the reflection spectrum of the projection surface based on the parameters collected by various projection lights, and linearly maps the reflection spectrum of the projection surface to the coordinate system of the projection image after measuring the projection light path, so as to obtain the calibrated image of the projection.

[0044] The further objects and advantages of the invention will become fully apparent from the following description and accompanying drawings.

[0045] These and other objects, features and advantages of the present invention will become fully apparent from the following detailed description and accompanying drawings. Attached Figure Description

[0046] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. In the drawings, unless otherwise specified, the same reference numerals are used to denote the same parts. Wherein:

[0047] Figure 1 This is a schematic diagram with a black background illustrating a method for optimizing projection display based on a first preferred embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of a spectral image projected by a method for optimizing projection display based on a first preferred embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of a method for obtaining reflectance spectrum based on a first preferred embodiment of the present invention for optimizing projection display.

[0050] Figure 4 This is a schematic diagram of the reflection spectrum measured in each region of the figure above, which is a method for optimizing projection display according to a first preferred embodiment of the present invention.

[0051] Figure 5 This is a schematic diagram of the projection surface of a method for optimizing projection display according to a first preferred embodiment of the present invention.

[0052] Figure 6 This is a schematic diagram of the basic calibration screen for a method of optimizing projection display according to a first preferred embodiment of the present invention.

[0053] Figure 7 This is a schematic diagram of the projection result on the gradient color projection surface of the target projection surface area according to a method for optimizing projection display according to a first preferred embodiment of the present invention.

[0054] Figure 8 This is a schematic diagram of the spectral response of a spectral imaging sensor in different bands, representing a method for optimizing projection display according to a first preferred embodiment of the present invention.

[0055] Figure 9 This is a schematic diagram of the steps of a method for optimizing projection display according to a first preferred embodiment of the present invention.

[0056] Figure 10 This is a schematic diagram of a spectral imaging device for a projection display system according to a first preferred embodiment of the present invention. Detailed Implementation

[0057] It should be noted that the embodiments shown in the accompanying drawings are merely examples used to specifically and vividly explain and illustrate the concept of the present invention. They are not necessarily drawn to scale in terms of size and structure, nor do they constitute a limitation on the concept of the present invention.

[0058] The directional terms such as up, down, left, right, front, back, front, back, top, and bottom mentioned or possibly used in this specification are defined relative to the structures shown in the various accompanying drawings. They are relative concepts and may therefore vary depending on their location and usage. Therefore, these or other directional terms should not be interpreted as restrictive.

[0059] Refer to the accompanying drawings in this application specification. Figures 1 to 9 As shown, a method for optimizing projection display according to a first preferred embodiment of this application will be described below. The method optimizes projection display by using ambient light information and reflectivity information of the projection surface to resolve background issues in different situations during projection, thereby displaying the original image. The method optimizes projection display by projecting the reflectivity result onto the projector's screen coordinate system, thereby obtaining a projection scheme that precisely cancels out background reflectivity textures, achieving a perfect projection effect.

[0060] Specifically, in this preferred embodiment of the application, an image of the projection surface under no illumination is first acquired, and then the image of the projection surface is illuminated with uniform strong light from a projector to obtain the parameters of the ambient light and the projection surface, i.e., the ambient light spectrum and the reflection spectrum of the projection surface are obtained. In this application, the target area projected by the projector includes, but is not limited to, wall surfaces, fabric surfaces, etc. Since the reflectivity of the target area projected on different materials is different, the light projected by the human eye will have various color distortions after reflection.

[0061] In the optimized projection display method of this preferred embodiment of the application, at least one set image is projected onto the target projection surface, and the reflectance spectrum of the set image is obtained and calculated to obtain the ambient spectrum and the reflectance of the target projection surface.

[0062] As an example, in this preferred embodiment of the application, at least one set image is projected onto the projection surface, and the reflectance spectrum of its multi-channel spectral image is obtained using a multispectral image acquisition device. Calculations are then performed to obtain the environmental spectrum and the reflectance of the projection surface.

[0063] In this preferred embodiment of the application, the setting image includes at least the following features: the setting image may be a single image comprising multiple color blocks of arbitrary color with different brightness, or multiple images each containing multiple color blocks of arbitrary color with different brightness. Each of the multiple images may contain one or more color blocks of arbitrary brightness and arbitrary color. For example, the multiple color blocks may have the same brightness or different brightness, or the multiple color blocks may have the same color or different colors, and the multiple color blocks may also have the same brightness and different colors, or the same color and different brightness, etc. In this embodiment of the application, it is only necessary that the setting image includes multiple color blocks of arbitrary color with different brightness, without limiting the number of images of the setting image, nor limiting the brightness and color of the color blocks contained in each image.

[0064] Therefore, in this preferred embodiment of the application, it is necessary to obtain the spectrum of the substrate light in advance. The spectrum of the substrate light can be obtained by labeling specific spectral colors or by calculating based on other known spectra.

[0065] As an example, in a specific example of this application, light of a known spectrum (0, 0, 0) color is projected onto the target area, and the reflection spectrum at that time is obtained, thereby obtaining the reflectivity of the substrate light.

[0066] like Figure 1 As shown, the base color of the set image is acquired when the projector is not turned on, which can be presented as, for example... Figure 1 The background shown is black. It is worth noting that when the projector projects the set image onto the target area of ​​the projection surface (e.g., a wall or cloth), due to the base brightness of the projector and the presence of ambient light (e.g., in a non-completely dark environment), the black area will also have a certain brightness. At the same time, the color areas corresponding to the various color blocks with different brightness in the set image will also have color shifts.

[0067] Therefore, in this embodiment, it is necessary to first obtain the base light spectrum of the set image in advance. This can be obtained through pre-calibration. For example, the specific calibration is as follows: project light with a known RGB color of (0, 0, 0) onto the target area of ​​the projection surface and obtain the reflection spectrum at this time, thereby obtaining the reflectivity of the base light. Of course, other known spectra can also be preset for calculation.

[0068] Those skilled in the art will understand that, in the embodiments of this application, it is only necessary that the spectrum of the substrate light is known, without limiting the color coordinate values ​​of the substrate light's color space. For example, it can be any color coordinate value of the RGB color space or other color spaces. In the embodiments of this application, the color coordinates for obtaining the known spectrum of the substrate light, such as RGB coordinates, are adjustable. The known spectrum of the substrate light can also be spectral information parameters mapped from the color space in other ways.

[0069] In the method for optimizing projection display according to the embodiments of this application, determining the base light color of at least one set image includes: projecting base light of a known spectrum onto a target area of ​​the projection surface; and obtaining the reflection spectrum of the base light in the target area of ​​the projection surface as the base light spectrum.

[0070] The set image also includes several color blocks of arbitrary colors with different brightness, wherein the spectral information of the corresponding different colors is also known. Furthermore, when the projector is turned on, the set image includes the reflection of the base light and the reflection of each preset color block, as well as the reflection spectrum of ambient light on the projected target area.

[0071] like Figure 2 The image shown is a measured spectral image. The test area can be square, circular, or any shape. It can be a spaced area or a continuous area.

[0072] In the method for optimizing projection display according to the embodiments of this application, the set image is projected onto a target area, and the reflection spectrum of the set image reflected from the target projection area is obtained.

[0073] Specifically, when the set image is projected onto the target area of ​​the projection surface, since the set image includes multiple color blocks of arbitrary colors with different brightness, the reflection spectrum of the set image includes the reflection spectrum of the base light and the reflection spectrum of each color block. In addition, the ambient light also has a reflection spectrum for the target area of ​​the projection surface.

[0074] In the method for optimizing projection display according to the embodiments of this application, the reflection spectrum of the set image includes the reflection spectrum of the substrate light, the reflection spectrum of multiple color blocks in the set image, and the reflection spectrum of ambient light.

[0075] like Figure 2 The image shown is a measured reflectance spectrum of the set image. The test area can be square, circular, or any shape, and can be spaced out or continuous. Furthermore, the projected colors can be white or a combination of various colors, such as four rows of color blocks with decreasing intensity from left to right. Figure 3The various colors shown can be white, RGB, or other layouts. Figure 2 The illustration shows a schematic diagram of a first example of the reflection spectrum of a set image in a method for obtaining parameters of ambient light and projection surface according to an embodiment of this application.

[0076] like Figure 4 As shown, the reflectance spectra measured in each region of the defined image are used. Based on the reflectance spectra corresponding to the multiple color patches in the defined image, the ambient light spectrum and the reflectance spectrum of the projection surface are obtained.

[0077] First, define the reflection spectrum of each region corresponding to each of the multiple color patches, such as... Figure 4 As shown, 1, 2, ..., n-1 are the main test color patches, and n is the base color patch. The reflectance spectrum of each color patch can be obtained by a spectral imaging sensor, or the spectrum of the test light for each color patch can be a preset spectrum.

[0078] Then, the reflectance spectrum of each region corresponding to each color block is obtained, and the reflectance spectrum relationship of each region corresponding to each color block is as follows:

[0079] Reflectance spectrum n = (substrate light spectrum + ambient light spectrum) * reflectance

[0080] Reflectance spectrum 1 = (Region 1 spectrum + Ambient light spectrum) * Reflectance

[0081] Reflectance spectrum 2 = (Region 2 spectrum + Ambient light spectrum) * Reflectance

[0082] ...

[0083] Reflectance spectrum n-1 = (Region n-1 spectrum + Ambient light spectrum) * Reflectance

[0084] In the embodiments of this application, since the ambient light spectrum and reflectivity are parameters that change each time, they are also the main objects of solution according to the embodiments of this application.

[0085] As can be seen, the above problem is a problem of solving a noisy linear process, which can be solved using various algorithms. The method here is not limited to a single approach of solving matrix equations to obtain the measured spectrum; generally, the least squares method or neural network modeling is used. Neural network modeling can perform spectral calculations on an NPU, ensuring high spectral accuracy while maintaining low power consumption. It is understood that the optimized projection display method described in this application can also be used in spectral imaging in fields such as home appliances and automobiles to calculate the spectrum of ambient light and the reflectance of the target background.

[0086] Since the environment in which the projector is used is not static, calculations of ambient light and reflectivity of the projected target area are required each time the projector is turned on. If the measured spectral information of the ambient light is consistent with the previous record and the acquired spectral information of the base light is also consistent, it can be further confirmed that the current environment and the projected target area have not changed, thus reducing the calculation process.

[0087] To achieve greater accuracy, the same calculations can be performed again to obtain the current ambient light spectrum and the reflectance spectrum of the projected target area. For example, the loss function Loss can be defined as:

[0088] Loss = ||Reflectance Spectrum 1 - (Region 1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum 2 - (Region 2 Spectrum + Ambient Light Spectrum) * Reflectance|| + ... + ||Reflectance Spectrum n-1 - (Region n-1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance n - (Substrate Light Spectrum + Ambient Light Spectrum) * Reflectance||, where ||·|| represents the norm of a vector or matrix.

[0089] By minimizing the loss using the least squares method, the best estimate of the reflectivity of the target area on the projection surface and the ambient light spectrum can be calculated.

[0090] The method for optimizing projection display according to the embodiments of this application takes into account that the projected image of the projector is affected by ambient light and the target area of ​​the projection, including but not limited to the material of the wall surface, etc. Different materials have different reflectivities of the target area to be projected, so the projected light seen by the human eye also has various color distortion problems after reflection. The method calculates the parameters of ambient light and projection surface based on the reflection spectrum of a set image including multiple color blocks in different areas of the projection surface, so as to conveniently and accurately obtain the parameters of ambient light and projection surface.

[0091] It is understandable that while RGB, RGBIR, and RGBW cameras have a relatively small number of channels, they can still obtain relatively coarse spectral information of ambient light and reflectivity information of the projected target area using the methods described above. However, because RGB, RGBIR, and RGBW cameras have limitations in the spectral range they can sense, they cannot achieve full-band response. Therefore, in this preferred embodiment of the application, a spectral imaging sensor can acquire spectral information of different bands, resulting in greater accuracy.

[0092] In the optimized projection display method described in this application, the acquired image can be obtained according to its temporal and spatial characteristics. If image information is acquired spatially, a single image is obtained, comprising at least several color patches of arbitrary colors with varying brightness, and the spectral information of the corresponding colors is known. This reduces computation time, allowing for faster calculation of the reflectivity of the projected area and the spectral information of the ambient light. If acquired temporally, only one color patch is obtained from each image, ensuring each color fills the entire image. The corresponding spectral information can be obtained from the pixels behind each image, and more pixel elements can be acquired, where each pixel element represents the spectral information of the smallest unit within each color patch. This allows for more accurate spectral information to be obtained from the optical disc pixels in each image using the spectral recovery algorithm described below.

[0093] If the projector's projection surface is a uniform projection surface, then based on the ambient light spectrum and the reflection spectrum of the projection surface obtained above, and according to the target reflection spectrum to be projected, the projection light path is calculated and linearly mapped onto the coordinate system of the projected image to obtain the calibrated projection image. This target reflection spectrum is set according to a specific strategy, such as based on user needs, changes in the ambient light spectrum, etc., and then calculated based on the resulting reflection spectrum. For example, the calibrated image is obtained by dividing the target reflection spectrum by the ambient light spectrum and the reflection spectrum of the projection surface. Figure 5 As shown, if the projection surface of the projector is a gradient color, such as an uneven color from top to bottom (e.g., darker at the top and lighter at the bottom in the image), or other unevenness, this is just one example. That is to say, unlike the preferred embodiment described above, in another optional embodiment of this application, the projection surface image illuminated by the projector is a gradient color projection surface.

[0094] If the target projection surface is determined to be gradient-colored, it can be assumed that the projection surface, within a small size range, is equivalent to a uniform-colored projection surface. That is, if the target projection surface is gradient-colored, the projection surface is divided into multiple small-sized projection surface units, where each individual image acquisition unit is considered a uniform-colored projection surface.

[0095] like Figure 6 As shown, a mosaic repeating image of the basic calibration screen is projected across the entire screen; the projection result on the gradient color projection surface in the target projection area is as follows. Figure 7 As shown, the projection surface here can be entirely non-uniform, gradually changing, or partially non-uniform. Based on the above judgment method, the relevant regions are obtained, and the calculation amount for non-uniform regions is further reduced through judgment.

[0096] It is understandable that in the above steps, it is also possible to omit the judgment of certain areas. If there are uneven areas within the projection surface, the above method can be used.

[0097] In the optimized projection display method described in this preferred embodiment of the application, a multi-point reflectance measurement and ambient light measurement are performed on the entire projection surface using sampling.

[0098] Based on the expectation that the projection surface is a gradient reflective surface, the entire projection surface is linearly sampled and smoothed to obtain the reflection spectrum of the projection surface.

[0099] The reflection spectrum of the projection surface is linearly mapped onto the coordinate system of the projected image by measuring the projection light path, thus obtaining the calibrated image of the projection (pixel level), thereby achieving the effect of perfect projection on a gradient projection surface.

[0100] It is worth mentioning that the optimized projection display method described in the preferred embodiment of this application can obtain pixel-level reflectivity information of the projection surface, thereby solving background problems in different situations during projection and displaying the effect of the original image.

[0101] Based on the optimized projection display method described in the preferred embodiments of this application, the pixel-level reflectance result of the imaging plane of the multispectral camera can be solved. Since the intrinsic and extrinsic parameters of the multispectral camera and the projection lens are known, we can accurately project the pixel-level reflectance result onto the coordinate system of the projector screen, thereby obtaining a pixel-level projection scheme that can precisely cancel out the background reflectance texture, achieving a perfect projection effect.

[0102] Below is the relationship of the reflection spectrum obtained for each channel in each region with a set small size.

[0103] Reflectance spectrum n = (Projector substrate light spectrum + Ambient light spectrum) * Reflectance

[0104] Reflectance spectrum 1 = (Projector test light spectrum 1 + Ambient light spectrum) * Reflectance

[0105] Reflectance spectrum² = (Projector test light spectrum² + Ambient light spectrum) * Reflectance

[0106] ...

[0107] Reflectance spectrum n-1 = (Projector test light spectrum n-1 + Ambient light spectrum) * Reflectance

[0108] As mentioned above, since the ambient light spectrum is a variable parameter that changes with each iteration, it is also the main object of the solution.

[0109] As can be seen, the above problem is a problem of solving a linear process with noise, which can be solved using various algorithms. The method here is not limited to a certain way of solving matrix equations to obtain the spectrum to be measured. Generally, the least squares method or neural network modeling is used. Among them, the neural network modeling method can realize spectral calculation on NPU, which can ensure high spectral accuracy while also having the characteristics of low power consumption.

[0110] Similar to the preferred embodiment described above, for greater accuracy, the same calculations can be performed to obtain the current ambient light spectrum and the reflectance spectrum of the projected target area. For example, the loss function Loss can be defined as:

[0111] Loss = ||Reflectance Spectrum 1 - (Region 1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum 2 - (Region 2 Spectrum + Ambient Light Spectrum) * Reflectance|| + ... + ||Reflectance Spectrum n-1 - (Region n-1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance n - (Substrate Light Spectrum + Ambient Light Spectrum) * Reflectance||, where ||·|| represents the norm of a vector or matrix.

[0112] By minimizing the loss using the least squares method, the best estimate of the reflectivity of the target area on the projection surface and the ambient light spectrum can be calculated.

[0113] like Figure 8 As shown, the projector described in the preferred embodiment of this application needs to calculate the ambient spectrum and reflectance to achieve better projection display effects. Existing RGB cameras, RGBIR cameras, and RGBW cameras can also calculate the ambient spectrum and reflectance using the above methods. However, if an RGB camera is used, the calculation of the ambient spectrum and reflectance will be limited to the QE range (sensing range) of the RGB camera, and ambient light outside the RGB sensing range will not be perceived. Ambient light also affects human perception; therefore, the lack of a complete visible light spectrum makes these methods less effective than the spectral camera approach.

[0114] like Figure 5 As shown, when light around 450nm is received, S cells respond most strongly, while M and L cells respond weakly. After post-processing this perceptual information, the human brain "labels" this type of light as blue. It can be seen that cone cells can only respond to light within approximately 400-700nm; therefore, the human eye can only see the color of light in this wavelength range, which is what we commonly refer to as visible light. However, the perception range provided by an RGB camera is much smaller than that of the human eye. If limited by the QE curves of the three perception ranges of an RGB camera, the accuracy of this process is at least 7-8 times higher than that of a traditional RGB camera.

[0115] Refer to the accompanying drawings in this application specification. Figure 9 The present application further elaborates on the method steps for optimizing projection display according to a specific embodiment, wherein the method includes the following steps:

[0116] (a) Acquire images of the projection surface under no illumination and images of the projection surface illuminated by the projector's projection light to obtain parameters of the ambient light and the projection surface;

[0117] (b) Based on the parameters collected from various projected lights, the reflection spectrum of the projection surface is calculated; and

[0118] (c) The reflection spectrum of the projection surface is linearly mapped onto the coordinate system of the projection image after the projection light path is calculated, so as to obtain the calibrated image of the projection.

[0119] The method for optimizing projection display described in this application further includes the following steps in step (a):

[0120] The base light spectrum of a set image is determined, the set image comprising multiple color patches of arbitrary color with different brightness.

[0121] Here, the set image can be a single image comprising multiple color blocks of arbitrary color with different brightness, or multiple images each containing multiple color blocks of arbitrary color with different brightness. Furthermore, each of the multiple images can contain one or more color blocks of arbitrary brightness and arbitrary color. For example, the multiple color blocks may have the same brightness or different brightness, or the multiple color blocks may have the same color or different colors. Additionally, the multiple color blocks may have the same brightness and different colors, or the same color and different brightness, and so on. That is, in this embodiment, it is only necessary that the set image includes multiple color blocks of arbitrary color with different brightness; the number of images in the set image is not limited, nor is the brightness and color of the color blocks contained in each image limited.

[0122] The base spectrum of the set image can be acquired, for example, when the projector is not turned on, and it can be presented as, for example... Figure 1 The background shown is black, here, Figure 1 This is a schematic diagram of different brightness color blocks of a set image according to an embodiment of this application. It is worth noting that, in reality, when the projector projects the set image onto the target area of ​​the projection surface (e.g., a wall or cloth), due to the base brightness of the projector and the presence of ambient light (e.g., in a non-completely dark environment), the black areas will also have a certain brightness, and at the same time, the color of each square area corresponding to the multiple color blocks of different brightness in the set image will also be shifted.

[0123] Therefore, in this embodiment, it is necessary to first obtain the base light spectrum of the set image in advance. This can be obtained through pre-calibration. For example, the specific calibration is as follows: project light with a known RGB color of (0, 0, 0) onto the target area of ​​the projection surface and obtain the reflection spectrum at this time, thereby obtaining the reflectivity of the base light. Of course, other known spectra can also be preset for calculation.

[0124] Here, those skilled in the art will understand that in this embodiment, it is only necessary that the spectrum of the substrate light is known, without limiting the color coordinate values ​​of the substrate light's color space. For example, it can be any color coordinate value in the RGB color space or other color spaces. Furthermore, in this embodiment, the color coordinates for obtaining the known spectrum of the substrate light, such as RGB coordinates, are adjustable, and the known spectrum of the substrate light can also be spectral information parameters mapped from the color space in other ways.

[0125] Therefore, determining the base light color of at least one set image includes: projecting base light of a known spectrum onto a target area of ​​a projection surface; and obtaining the reflection spectrum of the base light in the target area of ​​the projection surface as the base light spectrum.

[0126] The set image is projected onto the target area of ​​the projection surface, and the reflection spectrum of the set image reflected from the target area of ​​the projection surface is obtained.

[0127] Specifically, when the set image is projected onto the target area of ​​the projection surface, since the set image includes multiple color blocks of arbitrary colors with different brightness, the reflection spectrum of the set image includes the reflection spectrum of the base light and the reflection spectrum of each color block. In addition, the ambient light also has a reflection spectrum for the target area of ​​the projection surface.

[0128] Therefore, the reflection spectrum of the set image includes the reflection spectrum of the base light, the reflection spectrum of the multiple color blocks in the set image, and the reflection spectrum of the ambient light.

[0129] Specifically, in the embodiments of this application, the measured reflectance spectrum image of the set image can be as follows: Figure 2 As shown, for example, the test area can be square, circular, or any shape, and can have intervals or be continuous areas; furthermore, the projected colors can be white or a combination of various colors, such as four rows of color blocks with decreasing intensity from left to right, and the colors can be white, RGB, or other layouts. Here, Figure 2 The illustration shows a schematic diagram of a first example of the reflectance spectrum of a set image in a method for obtaining parameters of ambient light and a projection surface according to an embodiment of this application. Figure 3The illustration shows a schematic diagram of a second example of the reflection spectrum of a set image in a method for obtaining parameters of ambient light and projection surface according to an embodiment of this application.

[0130] The ambient light spectrum and the reflection spectrum of the projection surface are obtained based on the reflection spectra corresponding to the multiple color blocks of the set image.

[0131] First, regarding such Figure 2 and Figure 3 The reflection spectrum of the set image shown defines the reflection spectrum of each region corresponding to each color patch in the plurality of color patches, such as... Figure 4 As shown, 1, 2, ..., n-1 are the main test color patches, and n is the base color patch. The reflectance spectrum of each color patch can be obtained using a spectral imaging sensor, or the spectrum of the test light for each color patch can be a preset spectrum. Here, Figure 4 The illustration shows schematic diagrams of various regions defined in the reflection spectrum of a set image according to an embodiment of this application.

[0132] Then, the reflectance spectrum of each region corresponding to each color block is obtained, and the reflectance spectrum relationship of each region corresponding to each color block is as follows:

[0133] Reflectance spectrum n = (substrate light spectrum + ambient light spectrum) * reflectance

[0134] Reflectance spectrum 1 = (Region 1 spectrum + Ambient light spectrum) * Reflectance

[0135] Reflectance spectrum 2 = (Region 2 spectrum + Ambient light spectrum) * Reflectance

[0136] ...

[0137] Reflectance spectrum n-1 = (Region n-1 spectrum + Ambient light spectrum) * Reflectance

[0138] In the embodiments of this application, since the ambient light spectrum and reflectivity are parameters that change each time, they are also the main objects of solution according to the embodiments of this application.

[0139] As can be seen, the above problem is a problem of solving a linear process with noise. Various solution algorithms can be used to solve it. The method here is not limited to a certain way of solving matrix equations to obtain the spectrum to be measured. Generally, the least squares method or neural network modeling is used.

[0140] For example, the loss function Loss can be defined as:

[0141] Loss = ||Reflectance Spectrum 1 - (Region 2 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum 2 - (Region 2 Spectrum + Ambient Light Spectrum) * Reflectance|| + ... + ||Reflectance Spectrum n-1 - (Region n-1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum n - (Substrate Light Spectrum + Ambient Light Spectrum) * Reflectance||, where ||·|| represents the norm of a vector or matrix.

[0142] Then, by minimizing the loss using the least squares method, the best estimate of the reflectance of the target area of ​​the projection surface and the ambient light spectrum can be calculated.

[0143] Alternatively, the solution can be obtained through neural network modeling. Since neural network modeling can perform spectral calculations on an NPU, it offers high spectral accuracy while maintaining low power consumption, making it ideal for mobile devices such as smartphones. Furthermore, it can be used in spectral imaging in fields such as home appliances and automobiles to calculate the reflectance of ambient light spectra and target areas on projection surfaces.

[0144] Here, considering that the environment used by the user is not constant each time, the method of obtaining the parameters of ambient light and projection surface according to the embodiment of this application can be performed every time the projector is turned on, so as to calculate the ambient light spectrum and the reflectivity of the target area of ​​the projection surface. Of course, if the measured ambient light spectrum information is consistent with the previous record and the obtained substrate light spectrum information is also consistent, it can be further confirmed that the current environment and target area have not changed, which can reduce the calculation process.

[0145] Of course, to ensure accuracy, calculations can be performed according to the method in the embodiments of this application to obtain the current ambient light spectrum and the reflection spectrum of the target area of ​​the projection surface.

[0146] Therefore, the method according to the embodiments of this application takes into account that the projected image of the projector is affected by ambient light and the target area of ​​the projection, including but not limited to the material of the wall surface, etc. That is, the reflectivity of the target area of ​​different materials is different, so the projected light seen by the human eye also has various color distortion problems after reflection. By calculating the parameters of ambient light and projection surface based on the reflection spectrum of the set image including multiple color blocks in different areas of the projection surface, the parameters of ambient light and projection surface can be obtained conveniently and accurately.

[0147] In step (b) of the method for optimizing the projection display, the method further includes:

[0148] (b.1) Based on the acquired projection surface image, determine whether the projection surface is a uniform color projection surface, a gradient color projection surface, or a strong texture projection surface; and

[0149] (b.2) If the projection surface is a uniform projection surface, the ambient light spectrum and the reflection spectrum of the projection surface are obtained based on the reflection spectrum corresponding to the color block of the image set in the projection surface; if the projection surface is a gradient color projection surface, the entire surface of the projection surface is linearly sampled and smoothed based on the expectation that the projection surface belongs to a gradient reflection surface, so as to obtain the reflection spectrum of the projection surface.

[0150] Prior to step (b.2) of the method for optimizing the projection display, the method further includes: performing a multi-point reflectance measurement and an ambient light measurement on the entire projection surface.

[0151] In step (b.2) of the method for optimizing the projection display, if the target projection surface is a gradient color, a mosaic repeating image of the basic calibration image is projected onto the full screen.

[0152] In step (a) of the method for optimizing projection display, at least one set image is projected onto the projection surface, and the reflectance spectrum of its multi-channel spectral image is obtained. Calculations are then performed to obtain the ambient spectrum and the reflectance of the projection surface.

[0153] The set image is a single image comprising multiple color blocks of arbitrary color with different brightness, or multiple images comprising multiple color blocks of arbitrary color with different brightness. Each of the multiple images contains one or more color blocks, and the one or more color blocks have the same or different brightness and the same or different color.

[0154] The method for optimizing projection display, step (a), further includes the following steps:

[0155] (a.1) Determine the base light spectrum of a set image, said set image comprising at least two color patches of arbitrary color with different brightness; and

[0156] (a.2) Project the set image onto the target area of ​​the projection surface and obtain the reflection spectrum of the set image reflected from the target area of ​​the projection surface.

[0157] It should be noted that the set image is a single image comprising multiple color patches of arbitrary color with different brightness, or multiple images each containing multiple color patches of arbitrary color with different brightness. Each of the multiple images contains one or more color patches, and the one or more color patches have the same or different brightness and the same or different color. In step (a.1) of the method for optimizing projection display, determining the base light color of the set image includes:

[0158] Projecting a substrate light with a known spectrum onto the target area of ​​the projection surface; and

[0159] The reflection spectrum of the substrate light in the target area of ​​the projection surface is obtained as the substrate light spectrum.

[0160] Preferably, in this preferred embodiment of the present application, the RGB color of the known spectrum of the substrate light is (0, 0, 0).

[0161] In step (a.2) of the method for optimizing projection display, the reflection spectrum of each region corresponding to each color block among the plurality of color blocks is defined based on the reflection spectrum of the set image.

[0162] In step (a.2) of the method for optimizing projection display, obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image includes:

[0163] The reflectance spectrum relationship of the region corresponding to each color patch is determined as follows:

[0164] Reflectance spectrum n = (substrate light spectrum + ambient light spectrum) * reflectance;

[0165] Reflectance spectrum 1 = (Region 1 spectrum + Ambient light spectrum) * Reflectance;

[0166] Reflectance spectrum 2 = (Region 2 spectrum + Ambient light spectrum) * Reflectance;

[0167] ...

[0168] Reflectance spectrum n-1 = (Regional n-1 spectrum + Ambient light spectrum) * Reflectance.

[0169] In step (a.2) of the method for optimizing projection display, obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image further includes:

[0170] Define the loss function as:

[0171] Loss = ||Reflectance Spectrum 1 - (Region 1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum 2 - (Region 2 Spectrum + Ambient Light Spectrum) * Reflectance|| + ... + ||Reflectance Spectrum n-1 - (Region n-1 Spectrum + Ambient Light Spectrum) * Reflectance|| + ||Reflectance Spectrum n - (Substrate Light Spectrum + Ambient Light Spectrum) * Reflectance||, where ||·|| represents the norm of a vector or matrix;

[0172] The optimal estimate of the ambient light spectrum and the reflectance of the target area of ​​the projection surface is calculated by minimizing the loss function using the least squares method.

[0173] Preferably, in this preferred embodiment of the present application, the ambient light spectrum and the reflectance of the target area of ​​the projection surface are solved from the relationship using a neural network modeling method.

[0174] In step (a) of the method for optimizing projection display, the corresponding image information is obtained according to the characteristics of the spatial domain, including the following steps: simultaneously acquiring an image, which includes at least several color blocks of arbitrary colors with different brightness, wherein the spectral information of the corresponding different colors is also known.

[0175] In step (a) of the method for optimizing projection display, the corresponding image information is obtained according to the characteristics of the time domain. Only one color block of the corresponding color is obtained in each image, so that each color can fill the entire image. The corresponding spectral information can be obtained in the corresponding pixel behind each image, and more pixel elements can be obtained. The pixel element is the spectral information of the smallest unit in each color block.

[0176] Refer to the accompanying drawings in this application specification. Figure 10 As shown, a projection display system according to another aspect of this application is illustrated in the following description. The projection display system includes a projector and a spectral imaging sensor, wherein the projector and the spectral imaging sensor are electrically connected. The spectral imaging sensor acquires the substrate light spectrum and the reflection spectrum information of a set image. The projector calculates the reflection spectrum of the projection surface based on parameters acquired from various projected lights, and linearly maps the reflection spectrum of the projection surface onto the coordinate system of the projected image after measuring the projection light path, to obtain a calibrated projected image.

[0177] Figure 10 A schematic configuration diagram of a spectral imaging sensor according to an embodiment of this application is shown. Figure 10 As shown, in the spectral imaging device according to the embodiments of this application, the optical system is optional and may be an optical system such as a lens assembly or a homogenizing assembly. The filter structure is a broadband filter structure in the frequency domain or wavelength domain. The pass spectra of different wavelengths of the filter structure are not completely the same at different locations. The filter structure can be a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties. For example, in the embodiments of this application, the filter structure can be the light modulation layer in Chinese Patent CN201921223201.2, and the image sensor (i.e., photodetector array) can be a CMOS image sensor (CIS), CCD, array photodetector, etc. In addition, the optional data processing unit can be a processing unit such as an MCU, CPU, GPU, FPGA, NPU, ASIC, etc., which can export the data generated by the image sensor to the outside for processing.

[0178] Specifically, after the image sensor measures light intensity information, it sends the data to the data processing unit for reconstruction calculation. The specific process is explained below:

[0179] The intensity signals of incident light at different wavelengths λ are denoted as x(λ), and the transmission spectrum curve of the filter structure is denoted as T(λ). The filter (filter structure) has m sets of structural units, and the transmission spectrum of each set of structural units is different. Overall, the filter structure can be denoted as Ti(λ) (i=1,2,3,…,m). Each set of structural units has a corresponding physical pixel below it, which detects the light intensity bi modulated by the filtered light structure. In the embodiments of this application, for example, one physical pixel can correspond to one set of structural units, but it is not limited to this. Multiple physical pixels can also be grouped together to correspond to one set of structural units. Therefore, in the spectral imaging device according to the embodiments of this application, multiple sets of structural units constitute a "spectral pixel". Furthermore, at least one spectral pixel can be used to reconstruct an image. It should be noted that the number of effective transmission spectra (the transmission spectrum used for spectral recovery is called the effective transmission spectrum) Ti(λ) of the filter structure may not be the same as the number of structural units. The transmission spectrum of the filter structure is set, tested, or calculated manually according to certain rules based on the needs of identification or recovery (for example, the transmission spectrum obtained by testing each structural unit is the effective transmission spectrum). Therefore, the number of effective transmission spectra of the filter structure may be less than the number of structural units, or even more than the number of structural units; for example, a certain transmission spectrum curve is not necessarily determined by a set of structural units.

[0180] The relationship between the spectral distribution of incident light and the measurements from the image sensor can be expressed by the following formula:

[0181] Yi=∫x(λ)*Ti(λ)*T(λ)dλ

[0182] After discretization, we get:

[0183] Yi=Σ(x(λ)*Ti(λ)*T(λ))

[0184] Where T(λ) is the response of the image sensor, denoted as:

[0185] Yi(λ)=Xi(λ)*T(λ),

[0186] The above equation can then be extended into matrix form.

[0187] Where Ti (i = 1, 2, 3, ..., m) is the response of the image sensor after the light under test passes through the filter structure, corresponding to the light intensity measurement values ​​of the image sensor corresponding to each of the m structural units. When one physical pixel corresponds to one structural unit, it can be understood as the light intensity measurement values ​​corresponding to m "physical pixels", which is a vector of length m. X is the system's response to light of different wavelengths, determined by the transmittance of the filter structure and the quantum efficiency of the image sensor. X is a matrix, where each row vector corresponds to a set of structural units' responses to incident light of different wavelengths. Here, the incident light is sampled discretely and uniformly, with a total of n sampling points. The number of columns in X is the same as the number of sampling points of the incident light. Here, X(λ) is the light intensity of the incident light at different wavelengths λ, which is the incident light spectrum to be measured.

[0188] In some examples, unlike the above, the filter structure can be directly formed on the upper surface of the image sensor, such as quantum dots, nanowires, etc. It directly forms the filter structure or material (nanowires, quantum dots, etc.) in the photosensitive area of ​​the sensor. Taking the filter structure as an example, it can be understood that when the raw materials of the image sensor are processed to form the image sensor, the filter structure is formed on the upper surface of the raw materials. The transmission spectrum and the response of the image sensor are integrated; that is, it can be understood that the response of the detector and the transmission spectrum are the same curve. In this case, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0189] Yi=Σ(x(λ)*Ti(λ))

[0190] Furthermore, at least one filter structure for modulating incident light is provided on the image sensor with the filter structure. This can be understood as replacing the image sensor (i.e., photodetector array) in the first embodiment (which could be a CMOS image sensor (CIS), CCD, array photodetector, etc.) with the image sensor integrated with the filter structure in the second embodiment.

[0191] At this point, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0192] Yi=∫x(λ)*Ti(λ)*Ri(λ)dλ

[0193] After discretization, we get:

[0194] Yi=Σ(x(λ)*Ti(λ)*Ri(λ))

[0195] That is, in this embodiment, Yi(λ)=Ti(λ)*Xi(λ)

[0196] Furthermore, the spectral imaging device 200 further includes an optical system, such as a lens. The principle of acquiring spectral imaging is as follows: after the target is acquired by the spectral imaging sensor, it passes through the camera lens, filter, and spectral chip, and is converted into an electrical signal for processing, ultimately obtaining the spectral information of the corresponding object being photographed.

[0197] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

[0198] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations fall within the protection scope of this invention.

Claims

1. A method for optimizing projection display, characterized in that: The method for optimizing projection display comprises the following steps: (a) collecting an image of the projection surface without illumination, and illuminating the image of the projection surface with projection light from a projector to obtain parameters of the ambient light and the projection surface; (b) calculating the reflection spectrum of the projection surface based on the parameters collected from various projection lights; as well as (c) The reflection spectrum of the projection surface is linearly mapped to the coordinate system of the projection screen after measuring the projection light path to obtain the projected adjustment image.

2. The method for optimizing projection display according to claim 1, wherein in step (b) of the method for optimizing projection display, further comprising: (b.1) judging, based on the collected projection surface image, whether the type of the projection surface is a uniform color projection surface, a gradient color projection surface, or a strong texture projection surface; and (b.2) If the projection surface is a uniform light projection surface, the ambient light spectrum and the reflection spectrum of the projection surface are obtained based on the reflection spectrum corresponding to the color block of the set image in the projection surface; If the projection surface is a gradient color projection surface, based on the expectation that the projection surface is a gradient reflection surface, linear sampling and smoothing are performed on the entire surface of the projection surface to obtain the reflection spectrum of the projection surface.

3. The method for optimizing projection display according to claim 2, wherein before step (b.2) of the method for optimizing projection display, further comprising: Perform sampled multi-point reflectivity measurement and ambient light measurement on the entire projection surface.

4. The method for optimizing projection display according to claim 3, wherein in step (b.2) of the method for optimizing projection display, if the target projection surface is a gradient color, a mosaic repeated image of the basic calibration picture is projected on the full screen.

5. The method for optimizing projection display according to claim 4, wherein in step (a) of the method for optimizing projection display, at least one set image is projected onto a projection surface, and a reflectance spectrum of its multi-channel spectral image is obtained, and calculation is performed to obtain an ambient spectrum and a reflectance of the projection surface.

6. The method for optimizing projection display according to claim 5, wherein the set image is a single image including a plurality of color blocks of arbitrary colors with different brightnesses, or a plurality of images each including a plurality of color blocks of arbitrary colors with different brightnesses, each of the plurality of images including one or more color blocks, and the one or more color blocks are of the same or different brightnesses and the same or different colors.

7. The method for optimizing projection display according to claim 5, wherein step (a) of the method for optimizing projection display further comprises the following steps: (a.1) determining a base light spectrum of a set image, wherein the set image includes at least two color blocks of arbitrary colors with different brightness; and (a.2) Projecting the set image onto a target area of ​​a projection surface, and acquiring a reflection spectrum of the set image reflected from the target area of ​​the projection surface.

8. The method for optimizing projection display according to claim 7, wherein before step (a.1) of the method for optimizing projection display, the method further comprises: A base light spectrum of the setting image is obtained, wherein the setting image includes a plurality of color blocks of arbitrary colors with different brightness.

9. The method for optimizing projection display according to claim 8, wherein in step (a.1) of the method for optimizing projection display, determining the base light color of the set image comprises: Projecting base light of known spectrum onto a target area of ​​the projection surface; as well as A reflection spectrum of the base light in the target area of ​​the projection surface is obtained as the base light spectrum.

10. The method for optimizing projection display according to claim 9, wherein the RGB color of the base light of the known spectrum is (0, 0, 0) color.

11. The method for optimizing projection display according to claim 9, wherein in step (a.2) of the method for optimizing projection display, the reflectance spectrum of each area corresponding to each color block in the multiple color blocks is defined based on the reflectance spectrum of the set image.

12. The method for optimizing projection display according to claim 7, wherein in step (a.2) of the method for optimizing projection display, obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the multiple color blocks of the set image comprises: Determine the reflectance spectrum relationship of the area corresponding to each color block as follows: Reflection spectrum n = (base light spectrum + ambient light spectrum) * reflectivity; Reflectance spectrum 1 = (region 1 spectrum + ambient light spectrum) * reflectivity; Reflectance spectrum 2 = (region 2 spectrum + ambient light spectrum) * reflectivity; …… Reflection spectrum n-1 = (region n-1 spectrum + ambient light spectrum) * reflectivity.

13. The method for optimizing projection display according to claim 12, wherein in step (a.2) of the method for optimizing projection display, acquiring the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectrum corresponding to the plurality of color blocks of the set image further comprises: Define the loss function Loss as: Loss = || reflection spectrum 1 - (region 1 spectrum + ambient light spectrum) * reflectivity || + || reflection spectrum 2 - (region 2 spectrum + ambient light spectrum) * reflectivity || + ... + || reflection spectrum n-1 - (region n-1 spectrum + ambient light spectrum) * reflectivity || + || reflection spectrum n - (base light spectrum + ambient light spectrum) * reflectivity ||, where ||·|| represents the norm of a vector or matrix.

14. The method for optimizing projection display according to claim 12, wherein the ambient light spectrum and the reflectivity of the target area of ​​the projection surface are solved from the relationship by a neural network modeling method.

15. The method for optimizing projection display according to claim 5, wherein in step (a) of the method for optimizing projection display, corresponding image information is obtained according to the characteristics of the spatial domain, comprising the following steps: simultaneously obtaining an image, wherein at least a plurality of color blocks of arbitrary colors with different brightnesses are included, wherein the corresponding spectral information of the different colors is also known.

16. According to the method for optimizing projection display of claim 5, in step (a) of the method for optimizing projection display, corresponding image information is obtained according to the characteristics of the time domain, and only the corresponding color block of one color is obtained in each image, so that each color can cover the entire image, and the corresponding spectral information can be obtained in the corresponding pixels behind each image, and more pixel elements can be obtained at the same time, and the pixel element is the spectral information of the smallest unit corresponding to each color block.

17. A projection display system, characterized in that: include: Projector; and A spectral imaging sensor, wherein the projector and the spectral imaging sensor are electrically connected, and the spectral imaging sensor is used to obtain the base light spectrum and the reflection spectrum information of the set image. The projector obtains the reflection spectrum of the projection surface based on the type of the projection surface, and linearly maps the reflection spectrum of the projection surface to the coordinate system of the projection screen after measuring the projection light path, so as to obtain a projected adjustment image.

Images