Projection control method, spectral imaging equipment and projection equipment

Abstract

The invention relates to a projection control method, spectral imaging equipment and projection equipment. The projection control method comprises the following steps: acquiring a spectral image of a predetermined area including a projection light area of projection equipment through spectral imaging; determining whether the projection light area contains a predetermined object based on the spectral image; and in response to determining that the projection light area contains the predetermined object, performing control corresponding to the predetermined object on the projection equipment. Thus, whether the spectral image contains the spectral imaging information of the predetermined object or not is determined based on the spectral image of the projection light area of the projection equipment, projection control is performed based on the predetermined object, and the control effect of the projection equipment can be improved.

Description

Technical Field

[0001] The present application relates to the field of optical processing technology, and more specifically, to a projection control method, a spectral imaging device and a projection device. Background Art

[0002] A projection device, also known as a projector or projector, is an electronic device that can project image content onto a target area (e.g., a projection screen area, a blank area on a wall, etc.). Current projection devices need to be at a certain distance from the projection area to project a normal projection image, and when a user enters the projection area projected by the projection device, the projection light will cause damage to the user's eyes. Therefore, how to accurately and quickly identify users or other objects during the projection process of the projection device has become a research focus of projection devices.

[0003] The current micro / miniaturized spectral sensing / spectral imaging technology usually adopts the following working mode: using sensors to acquire optical signals, and then performing different degrees of data processing to finally obtain spectral information. The sensors used in this process can obtain information in the frequency domain of the light to be measured. The implementation methods include: a light detector array with a light modulation structure, or a combination of a filter (or filter structure) array and a light detector array to obtain spectral information of different bands, and then calculate the information of the incident light based on the acquired spectral information and known modulation response parameters; using this computational reconstruction technology, the light path of the splitting space in traditional spectral technology can be avoided, and a smaller spectrometer or spectral camera can be realized.

[0004] Since micro / miniaturized spectrometers or micro / miniaturized spectral imaging devices are emerging technologies, they can be widely used in practical applications. Currently, environmental monitoring can also be carried out by setting micro spectral sensors in the projection field and other audio-visual equipment.

[0005] Therefore, it is desirable to provide a projection control solution based on spectral imaging technology. Summary of the invention

[0006] The embodiments of the present application provide a projection control method, a spectral imaging device and a projection device, which can determine whether the spectral image contains spectral imaging information of a predetermined object based on the spectral image of the projection light area of ​​the projection device, so as to perform projection control based on the predetermined object, thereby improving the control effect of the projection device.

[0007] According to one aspect of the present application, a projection control method is provided, comprising: acquiring a spectral image of a predetermined area of ​​a projection device including a projection light area through spectral imaging; determining whether the projection light area includes a predetermined object based on the spectral image; and, in response to determining that the projection light area includes the predetermined object, controlling the projection device corresponding to the predetermined object.

[0008] In the above-mentioned projection control method, determining whether the projection light area contains a predetermined object based on the spectral image includes: judging whether there is a predetermined object in the projection light area of ​​the projection device by whether there is a set response of a set spectrum band in the spectral information of the spectral image obtained by spectral imaging.

[0009] In the above projection control method, the set spectrum segment is set in the non-visible spectrum segment.

[0010] In the above projection control method, the spectral band range of the spectral imaging device used for spectral imaging is 780-2200nm.

[0011] In the above projection control method, determining whether the projection light area includes a predetermined object based on the spectral image includes: determining whether spectral imaging information of the predetermined object is included based on whether an imaging boundary corresponding to the predetermined object exists in the spectral image.

[0012] In the above-mentioned projection control method, determining whether the spectral imaging information of the predetermined object is included based on whether there is an imaging boundary corresponding to the predetermined object in the spectral image includes: in response to the existence of an imaging boundary matching the predetermined object in the spectral image, determining that the projected light area includes the predetermined object; and, in response to the absence of an imaging boundary matching the predetermined object in the spectral image, determining whether the spectral imaging information of the spectral image includes the spectral imaging information of the predetermined object.

[0013] In the above-mentioned projection control method, in response to determining that the projection light area contains a predetermined object, controlling the projection device corresponding to the predetermined object includes: in response to determining that the projection light area contains a human body, controlling the projection device to enter an eye protection mode.

[0014] In the above-mentioned projection control method, in response to determining that the projection light area contains a predetermined object, controlling the projection device corresponding to the predetermined object includes: in response to determining based on spectral imaging information that the predetermined object includes a target object, controlling the projection device to enter an eye protection mode, and the target object includes a human face or human skin; and, in response to the predetermined object not including the target object, maintaining the current state of the projection device.

[0015] In the above-mentioned projection control method, the predetermined area includes a larger range than the projection light area.

[0016] In the above-mentioned projection control method, in response to determining that the projection light area contains a predetermined object, controlling the projection device corresponding to the predetermined object includes: presetting the boundary position of the projection light area of ​​the projection device in the spectral image; and, in response to the predetermined object being contained in the projection light area, turning on the eye protection mode of the projection device.

[0017] In the above-mentioned projection control method, in response to a predetermined object being contained in the projection light area, after starting the eye protection mode of the projection device, it further includes: determining the moving direction of the predetermined object based on a second spectral image after a predetermined time; in response to the predetermined object leaving the projection light area, turning off the eye protection mode of the projection device; and, in response to the predetermined object entering the boundary position of the projection light area, turning on the eye protection mode of the projection device.

[0018] In the above projection control method, determining whether the projection light area includes a predetermined object based on the spectrum image includes: determining whether the projection light area includes a predetermined object based on a predetermined area image of the spectrum image.

[0019] In the above projection control method, the predetermined area image includes one or more area images corresponding to a set portion of the predetermined object.

[0020] According to another aspect of the present application, a spectral imaging device is provided, which is applied to the projection control method as described above to acquire the spectral image.

[0021] According to another aspect of the present application, a projection device is provided, comprising the spectral imaging device as described above.

[0022] The projection control method, spectral imaging device and projection device provided in the embodiments of the present application can determine whether the spectral image of the projection light area of ​​the projection device contains spectral imaging information of a predetermined object based on the spectral image of the area, so as to perform projection control based on the predetermined object, thereby improving the control effect of the projection device. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the detailed description of the preferred specific embodiments below, various other advantages and benefits of the present application will become clear to those of ordinary skill in the art. The drawings in the specification are only used for the purpose of illustrating the preferred embodiments and are not considered to be limitations of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work. Moreover, the same reference numerals are used to represent the same components throughout the drawings.

[0024] Figure 1 A schematic flow chart of a projection control method according to an embodiment of the present application is illustrated;

[0025] Figure 2 A schematic diagram illustrating an example of human body detection in a projection control method according to an embodiment of the present application is illustrated;

[0026] Figure 3 The diagram shows Figure 2 A schematic diagram of the detection effects of different bands in the human body detection example shown;

[0027] Figure 4 A schematic diagram of an infrared spectrum image of a palm in a projection control method according to an embodiment of the present application is illustrated;

[0028] Figure 5 The projection control method according to the embodiment of the present application is illustrated. Figure 4 A schematic diagram of the spectral information of the spectral image shown;

[0029] Figure 6 A schematic diagram illustrating another example of a predetermined area of ​​a projection control method according to an embodiment of the present application is illustrated;

[0030] Figure 7 A schematic diagram illustrating an example of a projection area and a boundary setting area according to a projection control method according to an embodiment of the present application;

[0031] Figure 8 A schematic diagram illustrating an example of a filter structure and a spectral image of a spectral imaging sensor used in a projection control method according to an embodiment of the present application is shown;

[0032] Fig. 9 A schematic configuration diagram of a spectral imaging device according to an embodiment of the present application is illustrated;

[0033] Fig.10 A schematic diagram of an image sensor chip of a spectral imaging device for calculating a reconstructed spectrum according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0035] Schematic method

[0036] Figure 1 The figure illustrates a schematic flow chart of a projection control method according to an embodiment of the present application.

[0037] like Figure 1 As shown, the projection control method according to the embodiment of the present application includes the following steps.

[0038] Step S110, obtaining a spectral image of a predetermined area of ​​the projection device including the projection light area through spectral imaging. In the embodiment of the present application, since in the spectral image obtained through spectral imaging, if it is a predetermined object in the display content on the screen projected by the projection device, such as information of a person, is not visible or imaged within the set spectral range of the spectral image, only the predetermined object in the actual environment, such as a human body, a part of a human body or an animal, can be detected by imaging information within the set spectral range, such as Figure 2 shown. Figure 2 A schematic diagram illustrating an example of human body detection in a projection control method according to an embodiment of the present application is illustrated.

[0039] Step S120, determining whether the projected light area contains a predetermined object based on the spectral image. That is, as described above, since the display content on the screen projected by the projection device does not affect the spectral imaging information within the set spectral range of the spectral image, it is possible to determine whether there is a predetermined object, such as a human body, a part of a human body, or an animal, in the projected light area of ​​the projection device by determining whether the spectral image contains the spectral imaging information of the predetermined object.

[0040] That is, the spectral image obtained by spectral imaging includes both the spectral imaging information of the projection image projected by the projection device onto the screen and the spectral imaging information of the environment. Therefore, the spectral imaging information of the environment can be used to determine whether the spectral imaging information of the predetermined object is included to determine whether the projected light area includes the predetermined object.

[0041] Specifically, in the embodiment of the present application, it is possible to determine whether there is a predetermined object in the projection light area of ​​the projection device by using the spectrum information of the spectrum image obtained by the spectrum imaging to determine whether there is a set response of the set spectrum band. In addition, in order to avoid being affected by the projected image, the set spectrum band can be set in the non-visible spectrum band. Here, in the embodiment of the present application, the spectrum imaging device for spectrum imaging, such as the spectrum imaging sensor, can generally use a spectrum band range of 780-2200nm.

[0042] Figure 3 The diagram shows Figure 2 The schematic diagram of the detection effect of different bands in the human detection example shown in FIG. Figure 3 As shown, when the visible light band is used, the human body in the projected image and the human body in the projection light area are detected at the same time, while when the near-infrared band is used, only the human body in the projection light area is detected.

[0043] Experiments have confirmed that in an infrared image acquired by a spectral imaging device, such as a spectral imaging sensor, images presented in visible light in other projection areas will not be displayed, such as Figure 4 and Figure 5 Here, Figure 4 FIG. 1 is a schematic diagram of an infrared spectrum image of a palm in a projection control method according to an embodiment of the present application. Figure 5 The projection control method according to the embodiment of the present application is illustrated. Figure 4 The schematic diagram of the spectral information of the spectral image shown.

[0044] And, if Figure 4 and Figure 5 As shown, through the spectral image, the imaging boundaries generated corresponding to different predetermined objects can be used to judge different objects. For example, when it matches the preset imaging boundary of the human body, it can be directly judged through the spectral image of infrared imaging. If it cannot be judged through the preset imaging boundary image, it can be further judged based on the acquired spectral information.

[0045] Therefore, in the projection control method according to an embodiment of the present application, determining whether the projection light area contains a predetermined object based on the spectral image includes: judging whether there is a predetermined object in the projection light area of ​​the projection device by determining whether there is a set response of a set spectrum band in the spectral information of the spectral image obtained by spectral imaging.

[0046] In the above projection control method, the set spectrum segment is set in the non-visible spectrum segment.

[0047] In the above projection control method, the spectral band range of the spectral imaging device used for spectral imaging is 780-2200nm.

[0048] Furthermore, in the projection control method according to an embodiment of the present application, determining whether the projection light area includes a predetermined object based on the spectral image includes: determining whether spectral imaging information of the predetermined object is included based on whether an imaging boundary corresponding to the predetermined object exists in the spectral image.

[0049] In the above-mentioned projection control method, determining whether the spectral imaging information of the predetermined object is included based on whether there is an imaging boundary corresponding to the predetermined object in the spectral image includes: in response to the existence of an imaging boundary matching the predetermined object in the spectral image, determining that the projected light area includes the predetermined object; and, in response to the absence of an imaging boundary matching the predetermined object in the spectral image, determining whether the spectral imaging information of the spectral image includes the spectral imaging information of the predetermined object.

[0050] Step S130, in response to determining that the projection light area contains a predetermined object, the projection device is controlled corresponding to the predetermined object. For example, when the projection control method according to an embodiment of the present application is applied to the eye protection mode of the projection device, the spectral imaging information of the comparison spectral image can be judged to determine whether the spectral imaging information of the human body is included to determine the environment, such as whether there is a human body in the projection light area. In this way, when a user enters the projection light area of ​​the projection device, the projection device can be controlled to accurately turn on the eye protection mode to protect the user's eyes from damage by the projection light of the projection device.

[0051] Therefore, in the projection control method according to an embodiment of the present application, in response to determining that the projection light area contains a predetermined object, controlling the projection device corresponding to the predetermined object includes: in response to determining that the projection light area contains a human body, controlling the projection device to enter an eye protection mode.

[0052] In addition, the projection control method according to the embodiment of the present application can also control the projection device according to the type of the predetermined object. Specifically, when it is determined that the environment includes the predetermined object, the spectral imaging information of the predetermined object in the environment covered by the projection light area of ​​the projection device can be further obtained, and the spectral imaging information obtained can be compared; when it is determined through the comparison that the spectral imaging includes the first spectral information of the target object, the eye protection mode is started, wherein the first spectral information at least includes the spectral information of the face.

[0053] That is, the projection control method according to the embodiment of the present application can be used to further determine whether the spectral imaging information of the acquired spectral image includes spectral information including a face. If the face information happens to be in the captured image, the projection needs to be turned off immediately, that is, the eye protection mode is turned on. Correspondingly, if the spectral imaging information of the acquired spectral image does not contain spectral information of a target object such as skin or a face, the eye protection mode does not need to be turned on.

[0054] Therefore, in the projection control method according to an embodiment of the present application, in response to determining that the projected light area contains a predetermined object, controlling the projection device corresponding to the predetermined object includes: in response to determining based on spectral imaging information that the predetermined object includes a target object, controlling the projection device to enter an eye protection mode, and the target object includes a human face or human skin; and, in response to the predetermined object not including the target object, maintaining the current state of the projection device.

[0055] In addition, in the projection control method according to the embodiment of the present application, the predetermined area may include a larger range in addition to the projection light area, so that the spectral imaging device, such as a spectral imaging sensor, obtains a spectral image of a larger range, such as a spectral image of an area covered by the projection light of the projection device, as follows: Figure 6 shown. Figure 6 A schematic diagram illustrating another example of a predetermined area of ​​the projection control method according to an embodiment of the present application is illustrated.

[0056] Here, by the predetermined area including a larger range than the projection light area, a target object in a set area outside the projection light coverage area can be acquired in time, for example, Figure 6 As shown, human body 4 is a person in the content displayed on the projection screen of the projection device, and human body 3 is a person moving in the actual environment. Here, since the spectral imaging sensor can set the field of view (FOV) at different angles, it is possible to make a judgment on the eye protection mode before detecting that the target object enters the area projected by the projection device.

[0057] That is, if Figure 7 As shown, the imaging position of the projection light area of ​​the projection device is preset in the spectral image, and when the existence of the predetermined object is photographed within the boundary setting position, the eye protection mode is activated. Furthermore, when the existence of the predetermined object is photographed within the boundary setting position, the eye protection mode is activated, and further, the next spectral image after the set time is further determined, and the moving direction of the target object is determined by the second spectral image obtained. If the target object leaves the set area, the eye protection mode is not activated, and if the target object enters the boundary of the projection, the eye protection mode is activated. Figure 7 A schematic diagram illustrating an example of a projection area and a boundary setting area according to a projection control method according to an embodiment of the present application is illustrated.

[0058] Therefore, in the projection control method according to the embodiment of the present application, the predetermined area includes a larger range than the projection light area.

[0059] In the above-mentioned projection control method, in response to determining that the projection light area contains a predetermined object, controlling the projection device corresponding to the predetermined object includes: presetting the boundary position of the projection light area of ​​the projection device in the spectral image; and in response to the predetermined object being contained in the projection light area, turning on the eye protection mode of the projection device.

[0060] In the above-mentioned projection control method, in response to the predetermined object being contained in the projection light area, after starting the eye protection mode of the projection device, the method further includes: determining the moving direction of the predetermined object based on the second spectral image after a predetermined time; in response to the predetermined object leaving the projection light area, turning off the eye protection mode of the projection device; and, in response to the predetermined object entering the boundary position of the projection light area, turning on the eye protection mode of the projection device.

[0061] Furthermore, in the projection control method according to the embodiment of the present application, since the spectral imaging device, such as the spectral imaging sensor, can identify spectral information in different regions, the spectral information of the set position of the target object in the corresponding spectral image can be obtained according to the position information of the head in the imaging area, and it is determined whether the spectral information corresponding to the position is the first set spectral information.

[0062] In addition, since the spectral information of the skin is similar, in order to confuse the spectral information of other arms or legs with the spectral information of the face and avoid misjudgment, judgment can be made based on the spectral information of different areas in the spectral image.

[0063] Therefore, in the projection control method according to the embodiment of the present application, determining whether the projection light area includes a predetermined object based on the spectrum image includes: determining whether the projection light area includes a predetermined object based on a predetermined area image of the spectrum image.

[0064] In the above projection control method, the predetermined area image includes one or more area images corresponding to a set portion of the predetermined object.

[0065] In addition, in the projection control method according to the embodiment of the present application, the parameters of the current ambient light can be further determined, including at least the parameters of the set wavelength band, and the others can be brightness, spectrum information, etc. For example, if there is no infrared light to assist spectral imaging in a completely dark environment, such as by blocking curtains, then fill light needs to be performed when a spectrum without the set wavelength band is detected, and the wavelength band of the fill light source at least contains the set spectrum band, for example, it can be not less than the range of the set spectrum band.

[0066] For example, the fill light source can be set to an IR infrared fill light source, and the light emitted by the infrared fill light source is irradiated onto the target object. After absorption and reflection, the corresponding spectral image is obtained, so as to further determine whether the spectral image includes spectral imaging information of the predetermined object being detected or further determine whether the corresponding predetermined object is included.

[0067] Here, as mentioned above, spectral imaging can be implemented by a predetermined spectral imaging device or an infrared imaging device, wherein the spectral imaging device, for example, a spectral imaging sensor may include Figure 8 If it is a simple infrared sensor, it is enough to read directly from the image whether the image contains the predetermined object. If a spectral imaging sensor is used, it can not only obtain spectral images of different bands, but also obtain spectral information of any set position in the spectral image, such as Figure 8 shown. Figure 8 A schematic diagram of a filter structure and a spectral image example of a spectral imaging sensor used in a projection control method according to an embodiment of the present application is illustrated.

[0068] Schematic spectral imaging equipment

[0069] As described above, in the projection control method according to the embodiment of the present application, a spectral imaging device is required to generate a spectral image. Fig. 9 FIG. 1 is a schematic configuration diagram of a spectral imaging device according to an embodiment of the present application. Fig. 9 As shown, in the spectral imaging device according to the embodiment of the present application, the optical system is optional, which may be an optical system such as a lens assembly, a homogenizing assembly, etc. The filter structure is a broadband filter structure in the frequency domain or the wavelength domain. The pass spectra of different wavelengths of the filter structures in different places are not exactly the same. The filter structure can be a structure or material with filtering characteristics such as a metasurface, a photonic crystal, a nanocolumn, a multilayer film, a dye, a quantum dot, a MEMS (microelectromechanical system), an FP etalon, a cavity layer, a waveguide layer, a diffraction element, etc. For example, in an embodiment of the present application, the filter structure can be the light modulation layer in Chinese patent CN201921223201.2

[0070] The image sensor (i.e., the photodetector array) may be a CMOS image sensor (CIS), a CCD, an array photodetector, etc. In addition, the optional data processing unit may be a processing unit such as an MCU, a CPU, a GPU, an FPGA, an NPU, an ASIC, etc., which may export the data generated by the image sensor to an external device for processing.

[0071] For example, after the image sensor measures the light intensity information, it is transmitted to the data processing unit for recovery calculation. The specific description of the process is as follows:

[0072] The intensity signal of the incident light at different wavelengths λ is recorded as x(λ), and the transmission spectrum curve of the filter structure is recorded as T(λ). There are m groups of structural units on the filter (filter structure), and the transmission spectrum of each group of structural units is different. Generally speaking, the filter structure can be recorded as Ti(λ) (i=1,2,3,…,m). There are corresponding physical pixels under each group of structural units to detect the light intensity bi modulated by the filter structure. In a specific embodiment of the present application, one physical pixel, that is, one physical pixel corresponds to a group of structural units, but it is not limited to this. In other embodiments, multiple physical pixels can also correspond to a group of structural units. Therefore, in the spectral imaging device according to the embodiment of the present application, multiple groups of structural units constitute a "spectral pixel". Furthermore, the present invention can use at least one spectral pixel to restore the image. It should be noted that the number of effective transmission spectra (the transmission spectrum used for spectral restoration is called the effective transmission spectrum) Ti(λ) of the filter structure may be inconsistent with the number of structural units. The transmission spectrum of the filter structure is artificially set, tested, or calculated according to certain rules based on the needs of identification or restoration (for example, the transmission spectrum of each structural unit tested above is the effective transmission spectrum). Therefore, the number of effective transmission spectra of the filter structure may be less than the number of structural units, or even more than the number of structural units. In this variant embodiment, a certain transmission spectrum curve is not necessarily determined by a group of structural units.

[0073] The relationship between the spectral distribution of the incident light and the measured value of the image sensor can be expressed as follows:

[0074] bi=∫x(λ)*T i(λ)*R(λ)dλ

[0075] Discretize it again and get:

[0076] bi=Σ(x(λ)*Ti(λ)*R(λ))

[0077] Where R(λ) is the response of the image sensor, expressed as:

[0078] Ai(λ)=T i(λ)*R(λ),

[0079] The above formula can be expanded into matrix form:

[0080]

[0081] Among them, bi (i = 1, 2, 3, ..., m) is the response of the image sensor after the light to be measured passes through the filter structure, which corresponds to the light intensity measurement value of the image sensor corresponding to m structural units. When a physical pixel corresponds to a structural unit, it can be understood as the light intensity measurement value corresponding to m "physical pixels", which is a vector of length m. A is the system's response to light of different wavelengths, which is determined by two factors: the transmittance of the filter structure and the quantum efficiency of the image sensor. A is a matrix, and each row vector corresponds to the response of a group of structural units to incident light of different wavelengths. Here, the incident light is discretely and uniformly sampled, with a total of n sampling points. The number of columns of A is the same as the number of sampling points of the incident light. Here, x (λ) is the light intensity of the incident light at different wavelengths λ, that is, the incident light spectrum to be measured.

[0082] In some embodiments, the difference from the above embodiments is that the filter structure can be directly formed on the upper surface of the image sensor, such as quantum dots, nanowires, etc., which directly form a filter structure or material (nanowires, quantum dots, etc.) on the photosensitive area of ​​the sensor. Taking the filter structure as an example, at this time, it can be understood that when the raw material of the image sensor is processed to form the image sensor, the filter structure is formed on the upper surface of the raw material. The transmission spectrum and the response of the image sensor are integrated, that is, it can be understood that the response of the detector and the transmission spectrum are the same curve. At this time, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0083] bi=Σ(x(λ)*Ri(λ))

[0084] That is, in this embodiment, the transmission spectrum Ai(λ)=Ri(λ)

[0085] Furthermore, it can also be a combination of the above two embodiments, that is, at least one filter structure for modulating incident light is provided on the image sensor with the filter structure. It can be understood that the image sensor (i.e., the light detector array) in the first embodiment can be a CMOS image sensor (CIS), CCD, array light detector, etc., replaced with the image sensor with the filter structure integrated in the second embodiment.

[0086] At this time, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0087] bi=∫x(λ)*T i(λ)*Ri(λ)dλ

[0088] Discretize it again and get:

[0089] bi=Σ(x(λ)*Ti(λ)*Ri(λ))

[0090] That is, in this embodiment, Ai(λ)=Ti(λ)*Ri(λ)

[0091] Exemplary Methods

[0092] If multi-channel spectral imaging is considered, the imaging principle of each spectral pixel of a spectral imaging device, such as a snapshot spectral camera, can be expressed by the following equation:

[0093] AX=B

[0094] By reconstructing the spectral information of the incident light through the above calculation, the spectral information of the image acquired by the spectral imaging sensor can be obtained.

[0095] Therefore, Figure 8 As shown, each filter unit is provided with a plurality of filter structures, and the filter units corresponding to different areas can obtain corresponding spectral responses, so corresponding spectral information can be obtained at a set position of the image.

[0096] Fig.10 FIG. 1 is a schematic diagram of an image sensor chip of a spectral imaging device for calculating a reconstructed spectrum according to an embodiment of the present application. Fig.10 As shown, the image sensor chip of the spectral imaging device is an image acquisition chip and object imaging recognition device disclosed by the inventor of the present application in Chinese patent CN201921222710.3. Based on the content of Chinese patent CN201921222710.3, it can be known that it includes a light modulation layer, an image sensing layer and at least two groups of pixel confirmation modules, the light modulation layer is located above the image sensing layer, each group of the pixel confirmation modules includes a modulation unit and a sensing unit, and each of the modulation units and each of the sensing units are respectively arranged on the light modulation layer and the image sensing layer correspondingly.

[0097] It is worth mentioning that in the preferred embodiment provided in the present application, the spectral chip of the spectral imaging detection device can be but is not limited to a computationally reconstructed spectral imaging chip. As an example, the spectral chip 31 includes a photodetection layer 110 and a light modulation layer 120 located on a sensing path of the photodetection layer 110, and the photodetection layer 110 is configured to obtain an optical signal modulated by the light modulation layer 120.

[0098] In this preferred embodiment of the present invention, at least one modulator unit is respectively provided in each of the modulation units, and each of the modulation units is respectively provided with a plurality of modulation holes penetrating the optical modulation layer, and each of the modulation holes in the same modulation unit is arranged into a two-dimensional graphic structure with a specific arrangement rule. In some specific examples, the sensing unit includes at least one sensing unit, each of the sensing units is arranged in a matrix, each of the sensing units is respectively provided with at least one image sensor, and all of the sensing units are respectively electrically connected through the signal processing circuit layer. The optical modulation layer 120 includes at least one modulation unit 121 and at least one non-modulation unit, and each of the modulation units 121 and each of the non-modulation units respectively correspond to at least one sensing unit of the photoelectric detection layer, that is, the modulation unit 121 and the sensing unit 111 can be set in a one-to-one correspondence, or a one-to-many arrangement, or even a many-to-one arrangement, and the non-modulation unit and the sensing unit 111 can be set in a one-to-one correspondence, or a one-to-many arrangement, or even a many-to-one arrangement, including at least two sensing units corresponding to different modulation units.

[0099] In a specific embodiment, each modulation unit 121 is arranged corresponding to a sensing unit 111, so that the spectrum received by the modulation unit is formed into a pixel point by using the image sensor 311 in the sensing unit, and the intensity distribution on a pixel point can be obtained through the algorithm. The integration of multiple pixel points corresponding to different groups of pixel confirmation modules can form an image containing multiple spectrum information.

[0100] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0101] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0102] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0103] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0104] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A projection control method, comprising: Acquiring a spectral image of a predetermined area of ​​the projection device including a projection light area by spectral imaging; determining whether the projected light area includes a predetermined object based on the spectral image; as well as In response to determining that the projection light area includes a predetermined object, the projection device is controlled corresponding to the predetermined object.

2. The projection control method according to claim 1, wherein: Determining whether the projection light area includes a predetermined object based on the spectral image includes: Whether there is a predetermined object in the projection light area of ​​the projection device is determined by whether there is a set response of a set spectrum band in the spectrum information of the spectrum image acquired by the spectrum imaging.

3. The projection control method according to claim 2, wherein: The set spectrum segment is set in the non-visible spectrum segment.

4. The projection control method according to claim 2, wherein: The spectral band range of the spectral imaging device used for spectral imaging is 780-2200nm.

5. The projection control method according to claim 1, wherein: Determining whether the projection light area includes a predetermined object based on the spectral image includes: Whether the spectral imaging information of the predetermined object is included is determined based on whether an imaging boundary corresponding to the predetermined object exists in the spectral image.

6. The projection control method according to claim 5, wherein: Determining whether the spectral imaging information of the predetermined object is contained based on whether there is an imaging boundary corresponding to the predetermined object in the spectral image includes: In response to the presence of an imaging boundary matching the predetermined object in the spectral image, determining that the projected light region includes the predetermined object; and, In response to the absence of an imaging boundary matching the predetermined object in the spectral image, it is determined whether the spectral imaging information of the spectral image includes the spectral imaging information of the predetermined object.

7. The projection control method according to claim 1, wherein: In response to determining that the projection light area includes a predetermined object, controlling the projection device corresponding to the predetermined object includes: In response to determining that the projection light area includes a human body, the projection device is controlled to enter an eye protection mode.

8. The projection control method according to claim 1, wherein: In response to determining that the projection light area includes a predetermined object, controlling the projection device corresponding to the predetermined object includes: In response to determining that the predetermined object includes a target object based on the spectral imaging information, controlling the projection device to enter an eye protection mode, wherein the target object includes a human face or human skin; and In response to the predetermined object not including the target object, maintaining the current state of the projection device.

9. The projection control method according to claim 1, wherein: The predetermined area includes a larger range than the projected light area.

10. The projection control method according to claim 9, wherein: In response to determining that the projection light area includes a predetermined object, controlling the projection device corresponding to the predetermined object includes: Presetting a boundary position of a projection light area of ​​a projection device in the spectral image; and, In response to a predetermined object being included in the projection light area, an eye protection mode of the projection device is turned on.

11. The projection control method according to claim 10, wherein: In response to a predetermined object being included in the projection light area, the method of starting the eye protection mode of the projection device further comprises: determining a moving direction of the predetermined object based on a second spectral image after a predetermined time; In response to the predetermined object leaving the projection light area, turning off the eye protection mode of the projection device; and, In response to the predetermined object entering a boundary position of the projection light area, an eye protection mode of the projection device is turned on.

12. The projection control method according to claim 1, wherein: Determining whether the projection light area includes a predetermined object based on the spectrum image includes: determining whether the projection light area includes a predetermined object based on a predetermined area image of the spectrum image.

13. The projection control method according to claim 12, wherein: The predetermined area image includes one or more area images corresponding to a set portion of the predetermined object.

14. A spectral imaging device, applied to the projection control method according to any one of claims 1 to 13, for acquiring the spectral image.

15. A projection device, comprising the spectral imaging device according to claim 14.

Images