Multispectral detection method, apparatus, and electronic device

Abstract

This application discloses a multispectral detection method, apparatus, and electronic device, belonging to the field of data processing technology. The multispectral detection method is applied to a multispectral detection apparatus, which includes an imaging module, the imaging module including a multispectral component, and the method includes: acquiring the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component; and outputting object information of the object within the field of view of the imaging module based on the energy value of light in each wavelength band.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a multispectral detection method, apparatus, and electronic equipment. Background Technology

[0002] With the development of mobile terminals, more and more functions are integrated into them, including object recognition.

[0003] Currently, the process of object recognition on mobile terminals involves: capturing an image of the object using a camera module; and then performing object recognition methods such as shape detection, texture recognition, and color recognition on the image to identify the captured object. However, current methods for object recognition using camera modules are primarily designed for tangible objects. For non-tangible objects such as gases, which lack shape, color, and texture features, camera modules typically cannot detect them, resulting in lower accuracy for object detection on mobile terminals. Summary of the Invention

[0004] The purpose of this application is to provide a multispectral detection method, apparatus, and electronic device that can detect more objects and improve the accuracy of object detection.

[0005] In a first aspect, embodiments of this application provide a multispectral detection method applied to a multispectral detection device, the multispectral detection device including an imaging module, the imaging module including multispectral components, and the method comprising:

[0006] Obtain the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component;

[0007] Based on the energy value of light in each wavelength band, the camera module outputs object information of the subject within its field of view.

[0008] Secondly, embodiments of this application provide a multispectral detection device, the device comprising:

[0009] The camera module includes a multispectral component;

[0010] An acquisition module is used to acquire the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component;

[0011] The output module is used to output object information of the subject within the field of view of the shooting module based on the energy value of light in each wavelength band.

[0012] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0013] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0015] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0016] In this embodiment, object detection is achieved by acquiring the energy values ​​of multiple wavelengths of light transmitted through the multispectral component in the imaging module, and then outputting object information of the subject within the imaging module's field of view based on the energy value of each wavelength. Since the light incident on the multispectral component is the reflected light from the subject within the imaging module's field of view, the energy values ​​of these multiple wavelengths reflect the subject's reflection of light across these wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module allows for the detection of more objects and improves the accuracy of object detection compared to related technologies. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the AF shooting module provided in the embodiments of this application;

[0018] Figure 2 This is one of the flowcharts of the multispectral detection method provided in the embodiments of this application;

[0019] Figure 3 This is one of the structural schematic diagrams of the shooting module provided in the embodiments of this application;

[0020] Figure 4 This is a second schematic diagram of the structure of the shooting module provided in the embodiments of this application;

[0021] Figure 5 This is the third schematic diagram of the structure of the shooting module provided in the embodiments of this application;

[0022] Figure 6 This is a schematic diagram of the pixel arrangement in the image sensor provided in the embodiments of this application;

[0023] Figure 7 This is one of the schematic diagrams showing the arrangement of the multispectral components and infrared filters provided in the embodiments of this application;

[0024] Figure 8 This is a second schematic diagram showing the arrangement of the multispectral components and infrared filters provided in the embodiments of this application;

[0025] Figure 9 This is the third schematic diagram of the structure of the shooting module provided in the embodiments of this application;

[0026] Figure 10 This is a schematic diagram of the arrangement of pixels and spectral filters in the image sensor provided in the embodiments of this application;

[0027] Figure 11 This is a schematic diagram of the arrangement of infrared filters provided in the embodiments of this application;

[0028] Figure 12 This is the second flowchart of the multispectral detection method provided in the embodiments of this application;

[0029] Figure 13 This is one of the image illustrations including object information markers provided in the embodiments of this application;

[0030] Figure 14 This is the second schematic diagram of an image including object information markers provided in the embodiments of this application;

[0031] Figure 15 This is the third schematic diagram of an image including object information markers provided in the embodiments of this application;

[0032] Figure 16 This is the fourth schematic diagram of an image including object information markers provided in the embodiments of this application;

[0033] Figure 17 This is the fifth of the image illustrations including object information markers provided in the embodiments of this application;

[0034] Figure 18 This is the sixth of the image illustrations including object information markers provided in the embodiments of this application;

[0035] Figure 19 This is a block diagram of the multispectral detection device provided in the embodiments of this application;

[0036] Figure 20 This is a block diagram of the electronic device provided in the embodiments of this application;

[0037] Figure 21 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0039] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0040] The multispectral detection method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0041] The powerful data processing capabilities and compact portability of mobile devices have made them indispensable in people's lives. With the development of mobile devices, especially after the integration of camera modules (often referred to as "electronic eyes"), the functions integrated into mobile devices have become increasingly diverse. Examples include object imaging and object recognition.

[0042] Current shooting modules typically include CMOS camera modules (CCM) and autofocus (AF) shooting modules. This application provides illustrative descriptions of the structures of these two types of shooting modules.

[0043] A typical CCM (Chip Module) includes: a lens, a voice coil motor (VCM), an infrared filter (IR filter), an image sensor, a digital signal processor (DSP), and a flexible printed circuit (FPC). The image sensor can be a CMOS image sensor. The general workflow of a CCM is as follows: after the voice coil motor drives the lens to a precise focusing position, the light passing through the lens is filtered by the infrared filter and then incident on the pixel array of the image sensor. Each pixel in the pixel array can be a photosensitive device such as a photodiode. Each pixel converts the sensed light signal into an electrical signal and transmits it to an amplification circuit. The electrical signal is then amplified and converted into a digital signal matrix (the image) by an analog-to-digital (AD) converter. The image is then processed by the digital signal processor and compressed for storage.

[0044] For AF shooting modules, please refer to... Figure 1 This illustrates a structural schematic diagram of an AF shooting module provided in an embodiment of this application. Figure 1 As shown, the AF shooting module 100 includes: a protective film 101, a lens 102, a voice coil motor 103, a bracket 104, an infrared filter 105, an image sensor 106, a flexible circuit board 107, and a connecting cable 108. The protective film 101 is used to prevent lens damage. The bracket 104 is used to fix the lens. The general working process of the AF shooting module is as follows: after the voice coil motor 103 drives the lens 102 to a accurately focused position, the light passing through the lens 102 is filtered by the infrared filter 105 and then incident on the pixel array of the image sensor 106. Each pixel in the pixel array can be a photosensitive device such as a photodiode. Each pixel converts the sensed light signal into an electrical signal and transmits it to the flexible circuit board 107. The flexible circuit board 107 digitally processes the received electrical signals to generate an image and compresses and stores the image.

[0045] For object recognition in mobile terminals, current camera modules can only capture images of objects and output them. The mobile terminal's processor then uses object recognition methods such as shape detection, texture recognition, and color recognition to identify objects within the camera module's field of view. However, current object recognition methods primarily target concrete objects. For non-concrete objects such as gases, which lack shape, color, and texture features, the camera module typically cannot detect them, resulting in low accuracy in object detection on mobile terminals.

[0046] refer to Figure 2 This document illustrates a flowchart of a multispectral detection method provided in an embodiment of this application. This multispectral detection method can solve the aforementioned technical problems. The multispectral detection method can be applied to a multispectral detection device. Optionally, the multispectral detection device can be an electronic device. For example, the electronic device may include a mobile phone, tablet computer, etc. The multispectral detection device includes a shooting module, which includes a multispectral component. This multispectral component is used to filter light rays whose wavelengths are outside the target wavelength band, so that light rays of the target wavelength band can pass through. Figure 2 As shown, multispectral detection methods include:

[0047] Step 201: Obtain the energy values ​​of light in multiple wavelengths transmitted through the multispectral component.

[0048] In some embodiments of this application, multiple bands may include each band in the full spectrum. Alternatively, multiple bands may include some bands in the full spectrum. The electronic device can control the multispectral component to filter incident light, so that the multispectral component transmits light in multiple bands respectively. That is, the electronic device uses multispectral technology to split the light incident on the imaging module into multiple bands using the multispectral component. The electronic device detects the energy value of the light transmitted through each band of the multispectral component to obtain the energy values ​​of the light transmitted through the multiple bands of the multispectral component. The light incident on the imaging module is typically full-spectrum light or wide-band light. The light transmitted through the multispectral component is typically narrow-band light. Optionally, the multispectral component can be a Fabry-Perot interferometer (FPI) multispectral component or multiple spectral filters.

[0049] Step 202: Based on the energy value of light in each band, output the object information of the subject within the field of view of the shooting module.

[0050] When light from sources such as sunlight and electric lights shines on an object, the object selectively absorbs different wavelengths of light due to its structural properties and reflects the unabsorbed light. Because different objects have different structural properties, their reflection of different wavelengths of light will vary. These structural properties can refer to the composition or structure of the object's molecules, atoms, and other substances. Therefore, when the same light source shines on different objects, the reflection of multiple wavelengths of light by any given object is unique, and the greater the degree of reflection of a particular wavelength, the greater the energy value of the reflected light for that wavelength.

[0051] In this embodiment, the light incident on the imaging module is the reflected light from the object within the imaging field of view of the imaging module. The electronic device can analyze the object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module. Optionally, the object information may include at least one of the following: the name of the object, the density information of the object, etc.

[0052] In summary, the multispectral detection method provided in this application acquires the energy values ​​of multiple wavelengths of light transmitted through the multispectral component in the imaging module, and then outputs object information of the subject within the imaging module's field of view based on the energy value of each wavelength, thereby achieving object detection. Since the light incident on the multispectral component is the reflected light from the subject within the imaging module's field of view, the energy values ​​of the acquired wavelengths reflect the subject's reflection of those wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module, compared to related technologies, allows for the detection of more objects and improves the accuracy of object detection.

[0053] The imaging module involved in the embodiments of this application may also include an image sensor. The structure of the imaging module can be varied, and the following examples illustrate several possibilities.

[0054] The imaging module with multispectral components located outside the image sensor includes the following structures:

[0055] The structure of the shooting module in some embodiments of this application is as follows: Figure 3 As shown, it may include: a lens 301, a multispectral component 302, an infrared filter 303, and an image sensor 304 arranged sequentially according to the transmission direction of the light incident on the shooting module.

[0056] The structure of the shooting module in some embodiments of this application is as follows: Figure 4 As shown, it may include: a lens 401, an infrared filter 402, a multispectral component 403, and an image sensor 404 arranged sequentially according to the transmission direction of the light incident on the shooting module.

[0057] The structure of the shooting module in some embodiments of this application is as follows: Figure 5 As shown, it may include: a multispectral component 501, a lens 502, an infrared filter 503, and an image sensor 504 arranged sequentially according to the transmission direction of the light incident on the imaging module.

[0058] In the structures of the aforementioned imaging modules, the projections of the infrared filter and the multispectral component onto the target plane do not overlap. The image sensor includes full-spectrum pixels, which can transmit light across the entire wavelength range. Optionally, the full-spectrum pixels can be pixels without a color filter array (CFA). The multispectral component is positioned on the light-incident side of the full-spectrum pixels, meaning that the projections of the multispectral component and the full-spectrum pixels onto the target plane overlap. The target plane is parallel to the light-emitting surface of the imaging module. Optionally, the multispectral component can be an FPI multispectral component or a spectral filter.

[0059] In this embodiment, the image sensor may contain at least one full-spectrum pixel. When there are multiple full-spectrum pixels, the number of multispectral components may be at least one. A multispectral component may be disposed on the incident light side of at least one full-spectrum pixel.

[0060] In one alternative implementation, the number of multispectral components can be equal to the number of full-spectrum pixels. Each multispectral component is positioned on the incident light side of a corresponding full-spectrum pixel.

[0061] As an example, multiple full-spectrum pixels are distributed within the pixel array of an image sensor. That is, some scattered pixels in the multiple pixels constituting the pixel array are replaced by full-spectrum pixels. The multiple pixels constituting the pixel array may include color pixels, which can be red, green, and blue (RGB) pixels.

[0062] In another example, multiple full-spectrum pixels can form at least two groups of full-spectrum pixels, with each group containing at least two full-spectrum pixels. Each group of full-spectrum pixels can be spaced apart within the pixel array of the image sensor, and the full-spectrum pixels included in each group can be arranged in an array.

[0063] For example, please refer to Figure 6 This illustrates a schematic diagram of the pixel arrangement in an image sensor provided in an embodiment of this application. For example... Figure 6 As shown, the image sensor comprises a 16×16 pixel matrix consisting of 64 full-spectrum pixels (c) and 192 color pixels (r1, r2, r3, gr1, gr2, gr4, gb1, gb3, gb4, b2, b3, b4). r represents red pixels, gr represents red-blue pixels, gb represents blue-green pixels, and b represents blue pixels. The 64 full-spectrum pixels are grouped into sets of four (c). Each group of full-spectrum pixels is spaced apart within the pixel array of the image sensor, and the four full-spectrum pixels in each group are arranged in a 2×2 array.

[0064] In an image sensor, pixels are like Figure 6 In the arrangement shown, the imaging module includes multispectral components and infrared filters that can be used as follows: Figure 7 The arrangement is shown. (As shown in the image.) Figure 7 As shown, the imaging module includes 64 FPI multispectral components and 192 infrared filters (IRr1, IRr2, IRr3, IRgr1, IRgr2, IRgr4, IRgb1, IRgb3, IRgb4, IRb2, IRb3, IRb4). IRr represents the infrared filter for transmitting red light. IRgr represents the infrared filter for transmitting red-green light. IRgb represents the infrared filter for transmitting blue-green light. IRb represents the infrared filter for transmitting blue light. Each of the 192 infrared filters is positioned on the incident light side of a corresponding color pixel. Each of the 64 FPI multispectral components is positioned on the incident light side of a corresponding full-spectrum pixel. It should be noted that... Figure 7 In this context, FPI stands for FPI multispectral component.

[0065] In another alternative implementation, the number of full-spectrum pixels can be an integer multiple of the number of multispectral components. Each multispectral component is respectively disposed on the incident light side of a group of full-spectrum pixels, and the number of full-spectrum pixels in each group of full-spectrum pixels is at least two. Each group of full-spectrum pixels can be spaced apart in the pixel array of the image sensor, and the full-spectrum pixels included in each group of full-spectrum pixels can be arranged in an array.

[0066] For example, in an image sensor, pixels are like Figure 6 In the arrangement shown, the imaging module includes multispectral components and infrared filters that can be used as follows: Figure 8 The arrangement is shown. (As shown in the image.) Figure 8 As shown, the imaging module includes 16 FPI multispectral components and 192 infrared filters (IRr1, IRr2, IRr3, IRgr1, IRgr2, IRgr4, IRgb1, IRgb3, IRgb4, IRb2, IRb3, IRb4). Each of the 192 infrared filters is positioned on the incident light side of a corresponding color pixel. Each of the 16 FPI multispectral components is positioned on the incident light side of a group of full-spectrum pixels. It should be noted that... Figure 8 In this context, FPI stands for FPI multispectral component.

[0067] The imaging module, with its multispectral components located inside the image sensor, includes the following structure:

[0068] The structure of the shooting module in some embodiments of this application is as follows: Figure 9As shown, it may include: a lens 901, an infrared filter 902, and an image sensor 903 arranged sequentially according to the transmission direction of the light incident on the shooting module.

[0069] The image sensor may include at least two multispectral components, which are spectral filters. The at least two spectral filters form at least two filter groups, each filter group including at least four spectral filters, with each spectral filter in each filter group transmitting light in a different wavelength band. The at least two filter groups are spaced apart on the pixels of the image sensor. That is, in the pixel array of the image sensor, some of the spaced pixels are replaced with spectral filters. For example, the wavelength bands of light transmitted by the four spectral filters in each filter group may be 100-300 nm, 301-500 nm, 501-700 nm, and 701-900 nm, respectively.

[0070] For example, please refer to Figure 10 This illustrates a schematic diagram of the pixel and spectral filter arrangement in another image sensor provided in an embodiment of this application. Figure 10 As shown, the image sensor comprises a 16×16 pixel matrix consisting of 64 spectral filters (s1, s2, s3, s4) and 192 color pixels (r1, r2, r3, gr1, gr2, gr4, gb1, gb3, gb4, b2, b3, b4). The 64 spectral filters form 16 filter groups, each including spectral filter s1, s2, s3, and s4. Spectral filters s1, s2, s3, and s4 are used to filter light of different wavelengths, allowing light of different bands to pass through. Each filter group is spaced apart within the pixel array of the image sensor, and the four spectral filters in each filter group are arranged in a 2×2 array.

[0071] In image sensors, pixels and spectral filters, such as Figure 10 In the arrangement shown, the infrared filter included in the imaging module can be as follows: Figure 11 The arrangement is shown. (As shown in the image.) Figure 11 As shown, the imaging module includes 192 infrared filters (IRr1, IRr2, IRr3, IRgr1, IRgr2, IRgr4, IRgb1, IRgb3, IRgb4, IRb2, IRb3, IRb4). Each of the 192 infrared filters is positioned on the light-incident side of a corresponding color pixel. The infrared filter layer formed by these 192 filters is a patterned layer, with the light-incident surface of the spectral filters mapped onto the hollowed-out area of ​​the infrared filter layer.

[0072] It should be noted that the shooting module may also include at least one of the following: lens protective film, voice coil motor, lens bracket, flexible circuit board, and shooting module connecting cable, etc.

[0073] This application also provides a multispectral detection method. This multispectral detection method can be applied to a multispectral detection device. Optionally, the multispectral detection device can be an electronic device. For example, the electronic device may include a mobile phone, tablet computer, etc. The multispectral detection device includes any of the imaging modules with the structure provided in this application embodiment (e.g., Figure 3 , Figure 4 , Figure 5 or Figure 9 The imaging module shown includes a multispectral component, which filters incident light with wavelengths outside the target wavelength band, allowing light of the target wavelength band to pass through. The multispectral detection method includes:

[0074] Step 201: Obtain the energy values ​​of light in multiple wavelengths transmitted through the multispectral component.

[0075] In some embodiments of this application, the multiple bands may include each band in the entire wavelength range. Alternatively, the multiple bands may include some bands in the entire wavelength range. The electronic device can control the multispectral component to filter incident light, so that the multispectral component can transmit light in multiple bands respectively. That is, the electronic device uses multispectral technology to split the light incident on the imaging module into multiple bands using the multispectral component. The electronic device detects the energy value of the light transmitted through each band of the multispectral component to obtain the energy values ​​of the light transmitted through the multiple bands of the multispectral component.

[0076] Step 202: Based on the energy value of light in each band, output the object information of the subject within the field of view of the shooting module.

[0077] It should be noted that, as mentioned earlier, any object reflects light of different wavelengths differently due to its structural properties. Therefore, the reflection of an object across multiple wavelengths of light can be used to identify the object. In some embodiments of this application, the light incident on the shooting module is the reflected light from the object within the shooting field of view of the shooting module. The electronic device can employ multispectral technology, utilizing the light filtering capability of multispectral components to collect the energy values ​​of multiple wavelengths of light incident on the shooting module, in order to analyze the reflection of the object against multiple different wavelengths of light, and thus identify the object. Multispectral technology can be divided into temporal multispectral technology and spatial multispectral technology. Temporal multispectral technology can be understood as acquiring the reflection of an object against multiple wavelengths of light multiple times by acquiring the reflection of an object against one wavelength of light at a time. Spatial multispectral technology can be understood as simultaneously acquiring the reflection of an object against multiple wavelengths of light. This application embodiment illustrately illustrates the process by which an electronic device uses temporal multispectral technology and spatial multispectral technology to determine the object information of the object being photographed.

[0078] In one optional implementation, the electronic device employs temporal multispectral technology to determine object information of the subject being photographed. In this case, the imaging module also includes an image sensor comprising at least one full-spectrum pixel. Furthermore, the multispectral component included in the imaging module is an FPI (Fiber Optic Induction Phase) multispectral component. The FPI multispectral component is disposed on the light-incident side of the full-spectrum pixel. The FPI multispectral component includes a Fabry-Perot cavity mover and a Fabry-Perot cavity stator. The FPI multispectral component is an optical resonant cavity, and the wavelength of transmitted light can be controlled by changing the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator. Optionally, the imaging module can be as described above. Figures 3 to 5 Any of the shooting modules shown.

[0079] Optionally, the process of obtaining the energy values ​​of light in multiple bands transmitted through the multispectral component in step 201 above may include steps 2011A to 2012A.

[0080] In step 2011A, the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator is adjusted sequentially according to the transmission distance corresponding to each band, so that the light of each band passes through the FPI multispectral component sequentially and is incident on the full-spectrum pixel. The transmission distance is the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator.

[0081] In this embodiment of the application, the electronic device can collect the reflection of light from the object being photographed across multiple wavelengths in order to analyze the object information of the object being photographed.

[0082] Optionally, the electronic device can sequentially adjust the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator according to the transmission distance corresponding to each of the multiple bands, so that the target light passes through the FPI multispectral component and is incident on the full-spectrum pixel. The target light is the light in the band corresponding to the adjusted distance. Here, the transmission distance corresponding to the band refers to the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator when the FPI multispectral component passes light in that band. The target sequence can be an increasing order of wavelength, a decreasing order of wavelength, an increasing order of transmission distance, or a decreasing order of transmission distance, etc.

[0083] For example, multiple wavelength bands correspond to different transmission distances, and these transmission distances differ from the initial distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator by positive integer multiples of the target distance increment. The electronic device can sequentially increase the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator from the initial distance to the target distance increment, so that light from multiple wavelength bands sequentially passes through the FPI multispectral component and is incident on the full-spectrum pixel.

[0084] Specifically, multiple bands include a first band, a second band, and a third band. The first transmission distance corresponding to the first band differs from the initial distance by a target distance increment. The second transmission distance corresponding to the second band differs from the initial distance by twice the target distance increment. The third transmission distance corresponding to the third band differs from the initial distance by three times the target distance increment. Assuming the initial distance is 0, the electronic device adjusts the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator three times, starting from 0, increasing the distance between them by the target distance increment each time. This allows the light from the first, second, and third bands to sequentially pass through the FPI multispectral component and be incident on the full-spectrum pixel.

[0085] In another example, the electronic device pre-stores transmission distances corresponding to multiple wavelength bands. The electronic device sequentially adjusts the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator in ascending order of the stored transmission distances. Specifically, the electronic device stores a first transmission distance corresponding to a first wavelength band and a second transmission distance corresponding to a second wavelength band. The electronic device sequentially adjusts the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator to the first and second transmission distances, so that light from the first and second wavelength bands sequentially passes through the FPI multispectral component and is incident on the full-spectrum pixel.

[0086] In step 2012A, the energy value of the light incident on each band of the full-spectrum pixel is obtained.

[0087] In some embodiments of this application, each time the electronic device adjusts the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator, so that light of a certain wavelength corresponding to the adjusted distance passes through the FPI multispectral component and is incident on the full-spectrum pixel, the energy value of the light of that wavelength incident on the full-spectrum pixel can be obtained, thereby obtaining the energy value of the light of each wavelength incident on the full-spectrum pixel.

[0088] For example, suppose multiple bands include a first band and a second band. The electronic device first adjusts the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator to the transmission distance corresponding to the first band, and then obtains the energy value of the light incident on the full-spectrum pixel in the first band. Then, the electronic device adjusts the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator to the transmission distance corresponding to the second band, and obtains the energy value of the light incident on the full-spectrum pixel in the second band.

[0089] The image sensor may include at least one full-spectrum pixel. When there is only one full-spectrum pixel, the electronic device acquires the energy value of each wavelength of light incident on the full-spectrum pixel. When there are multiple full-spectrum pixels, the electronic device may acquire the energy value of each wavelength of light incident on each full-spectrum pixel.

[0090] In this embodiment, the electronic device can, after acquiring the energy values ​​of light in multiple bands transmitted through the multispectral component, output object information of the object within the field of view of the imaging module based on the energy value of light in each band. The object information includes at least one of the following: object name, object density information, etc.

[0091] In this embodiment, by sequentially adjusting the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator in the FPI multispectral component according to the transmission distance corresponding to each band, light of each band is sequentially transmitted through the FPI multispectral component and incident on the full-spectrum pixel, thereby obtaining the energy value of the light of each band incident on the full-spectrum pixel. In this implementation, since the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator in one FPI multispectral component can be adjusted to allow light of multiple bands to be sequentially incident on the full-spectrum pixel, thus obtaining the energy value of the light of each band incident on the full-spectrum pixel, a smaller number of FPI multispectral components, or even just one FPI multispectral component, can be used in the imaging module to ensure the acquisition of the energy value of the light of each band in multiple bands incident on the imaging module, reducing the manufacturing cost of the imaging module.

[0092] For cases where the object information of the subject being photographed includes the object name and the number of full-spectrum pixels is one, the process of outputting the object information of the subject within the field of view of the shooting module in step 202, based on the energy value of light in each band, may include the following steps 2021A to 2022A.

[0093] In step 2021A, the object information corresponding to the full-spectrum pixel is obtained based on the energy value of the light incident on each band of the full-spectrum pixel.

[0094] In a first alternative implementation, the electronic device may pre-store a set of reflectance spectra. This set of reflectance spectra includes the reflectance spectrum of at least one object and object attribute information corresponding to each reflectance spectrum. For example, the spectral reflectance set includes the correspondence between the reflectance spectrum of at least one object and its object attribute information. The process by which the electronic device obtains the object information corresponding to a full-spectrum pixel based on the energy value of light incident on each band of the full-spectrum pixel may include steps 2021A1 to 2021A3.

[0095] In step 2021A1, based on the energy value of light in each band of the full-spectrum pixel, the energy curve corresponding to the full-spectrum pixel is output. The energy curve is a curve showing how the energy value of light changes with wavelength.

[0096] Optionally, the electronic device can plot an energy curve showing the energy value of the light incident on the full-spectrum pixel as the wavelength increases or decreases, based on the energy value of the light in each wavelength band incident on the full-spectrum pixel. For example, the energy curve could be a curve showing the energy value of the light incident on the full-spectrum pixel as the wavelength increases.

[0097] In step 2021A2, the target reflection spectrum that matches the curve shape of the energy curve is found from the pre-stored set of reflection spectra, and the target reflection spectrum corresponding to the full-spectrum pixel is obtained.

[0098] In one alternative implementation, the electronic device can calculate the similarity between each reflectance spectrum in the reflectance spectrum set and the energy curve. The reflectance spectrum with the highest similarity to the energy curve is taken as the target reflectance spectrum, thus obtaining the target reflectance spectrum corresponding to the full-spectrum pixel.

[0099] For example, an electronic device can use a target algorithm to calculate the similarity between each reflectance spectrum in the reflectance spectrum set and the energy curve, and select the reflectance spectrum with the highest similarity value as the target reflectance spectrum. The target algorithm can be the DTW algorithm, LCSS algorithm, or EDR algorithm, etc.

[0100] In another alternative implementation, the electronic device can acquire the first band corresponding to the peak value and the second band corresponding to the trough value in the energy curve. Furthermore, the electronic device acquires the third band corresponding to the peak value and the fourth band corresponding to the trough value in each reflection spectrum set. The reflection spectrum in which the first and third bands are the same, and the third and fourth bands are the same, is taken as the target reflection spectrum.

[0101] For example, suppose the light incident on a full-spectrum pixel includes multiple wavelengths: a first ray with a wavelength of 100-300 nm, a second ray with a wavelength of 301-500 nm, a third ray with a wavelength of 501-700 nm, and a fourth ray with a wavelength of 701-900 nm, and the energy values ​​of the first, third, second, and fourth rays decrease sequentially. The electronic device outputs an energy curve showing the energy value of the light as a function of wavelength, based on the energy values ​​of the first to fourth rays incident on the full-spectrum pixel. The first wavelengths corresponding to the peaks in this energy curve are 100-300 nm and 501-700 nm, respectively. The second wavelengths corresponding to the valleys are 301-500 nm and 701-900 nm, respectively. The electronic device then searches for the target reflection spectrum from the reflection spectrum set, identifying the wavelength corresponding to the peak as the first wavelength and the wavelength corresponding to the valley as the second wavelength, to obtain the target reflection spectrum of the full-spectrum pixel.

[0102] In step 2021A3, the object attribute information corresponding to the target reflectance spectrum is determined as the object information of the full-spectrum pixels corresponding to the target reflectance spectrum.

[0103] In some embodiments of this application, the electronic device determines the object attribute information corresponding to the target reflectance spectrum in the reflectance spectrum set as the object information of the full-spectrum pixel corresponding to the target reflectance spectrum. For example, suppose the object attribute information corresponding to the target reflectance spectrum is formaldehyde. The electronic device determines the corresponding information of the full-spectrum pixel corresponding to the target reflectance spectrum as formaldehyde.

[0104] In this embodiment, the light incident on the shooting module is the reflected light from the object within the shooting range of the shooting module. The energy curve is the curve showing the energy value of the light incident on the full-spectrum pixel as a function of wavelength. The reflection spectrum is the curve showing the reflectivity of the object as a function of wavelength. The reflectivity of the object can be expressed as the proportion of reflected energy to total energy. Total energy can refer to the total energy of the light incident on the object. Reflected energy can refer to the energy of the reflected light from the object. Since the illumination light of the object within the shooting field of view of the shooting module is usually from the same light source, the total energy of the light incident on the object can be considered equal. Therefore, the shape of the energy curve of the object can be considered the same as the shape of the reflection spectrum of the object. Therefore, the electronic device can effectively detect the information of the object to which the energy curve belongs by matching the shape of the energy curve with the shape of the reflection spectrum of the known object information in the reflection spectrum set, and thus effectively determine the object information of the full-spectrum pixel corresponding to the energy curve. Furthermore, any object exhibits selective reflection of light of different wavelengths, meaning that any object has a unique reflection spectrum. Therefore, the method of determining object information by matching the energy curve of the photographed object with the reflectance spectrum of known object information can detect more objects compared to related technologies, thus further improving the accuracy of object detection.

[0105] In a second alternative implementation, the electronic device can pre-store the energy values ​​of reflected light from at least one object across multiple wavelengths, as well as the object's attribute information. The process by which the electronic device obtains the object information corresponding to a full-spectrum pixel based on the energy values ​​of light incident on each wavelength of the full-spectrum pixel can include: the electronic device directly calculates the difference between the pre-stored energy values ​​of reflected light from the object for each wavelength and the energy values ​​of light incident on each wavelength of the full-spectrum pixel; identifying objects whose errors between any two differences across multiple wavelengths fall within the error range as target objects; and determining the object information corresponding to the full-spectrum pixel based on the object attribute information of the target objects.

[0106] For example, an electronic device stores the energy value A of the reflected light from a first object in a first band, the energy value B of the reflected light from the first object in a second band, and object attribute information of the first object. Assume the energy value C of the light in the first band incident on a full-spectrum pixel, and the energy value D of the light in the second band incident on the full-spectrum pixel. Calculate the first difference between energy value A and energy value C, and the second difference between energy value B and energy value D. Since the reflection of light by any object is unique, regardless of the energy value of the light incident on the object, the object selectively reflects the light incident on it in the same way; that is, the energy values ​​of multiple bands in the reflected light from the object change in the same way. Therefore, if the error between the first difference and the second difference is within the error range, it indicates that the energy value changes of each band in the light incident on the full-spectrum pixel and the reflected light from the first object are similar. The electronic device can then determine the object attribute information of the first object as the object information of the full-spectrum pixel.

[0107] In step 2022A, object information markers are displayed in the image regions corresponding to full-spectrum pixels in the original image acquired by the imaging module. These object information markers are used to mark the object information corresponding to the full-spectrum pixels. The original image is an image of the object captured by the imaging module within its field of view.

[0108] In this embodiment, each pixel in the original image corresponds to a pixel of the image sensor or a full-spectrum pixel. The image region in the original image corresponding to the full-spectrum pixel can refer to the region where the pixel corresponding to the full-spectrum pixel is located in the original image.

[0109] Optionally, the electronic device may store a correspondence between objects and fill materials. The fill material may include at least one of the following: solid color fill material, texture fill material, or gradient fill material. After determining the object information corresponding to the full-spectrum pixels, the electronic device may determine the fill material corresponding to the object indicated by the object information as the fill material corresponding to the full-spectrum pixels. The image area in the original image corresponding to the full-spectrum pixels is filled with this fill material to display object information markers in the image area corresponding to the full-spectrum pixels in the original image.

[0110] For example, suppose the object information corresponding to the full-spectrum pixels determined by the electronic device indicates that the object's fill material is red. The electronic device fills the image region in the original image corresponding to the full-spectrum pixels with red.

[0111] In this embodiment, the method of displaying object information markers in the image area corresponding to full-spectrum pixels in the original image acquired by the shooting module not only more intuitively reflects the shooting objects within the shooting field of view, but also, because the position of the object information marker is the corresponding position of the full-spectrum pixels in the original image, the marked original image can more intuitively reflect the specific position of the object indicated by the determined object information within the shooting field of view, thus improving the display effect of the identified object.

[0112] Optionally, in some embodiments of this application, the object information of the full-spectrum pixels determined by the electronic device may further include object concentration information. When the object information includes object name information and object concentration information, the process of displaying object information markers in the image region corresponding to the full-spectrum pixels in the original image acquired by the imaging module in step 2022A may include:

[0113] Based on the energy value of the light incident on the full-spectrum pixel, object information markers are displayed in the image region corresponding to the full-spectrum pixel in the original image. These object information markers are used not only to mark the object information corresponding to the full-spectrum pixel, but also to mark the object concentration information corresponding to the full-spectrum pixel.

[0114] In this context, a higher energy value corresponds to a higher object density, and different object densities correspond to different object information marker parameters. The marker parameter can refer to the degree of marking of the object information marker. Optionally, when the fill material corresponding to the object information marker is a solid color fill material, the marker parameter refers to the color saturation parameter. When the fill material corresponding to the object information marker is a texture fill material, the marker parameter refers to the texture density parameter. When the fill material corresponding to the object information marker is a gradient fill material, the marker parameter refers to the gradient density parameter.

[0115] Optionally, the process by which an electronic device displays an object information marker in the image region corresponding to the full-spectrum pixel in the original image based on the energy value of the light incident on the full-spectrum pixel may include steps 2022A1 to 2022A3.

[0116] In step 2022A1, the fill material corresponding to the full spectrum pixels is determined based on the fill material corresponding to the object indicated by the object name.

[0117] Optionally, the electronic device may store a correspondence between objects and fill materials. The fill materials may include at least one of the following: solid color fill materials, texture fill materials, and gradient fill materials. After determining the object information corresponding to a full-spectrum pixel, the electronic device can determine the fill material corresponding to the object indicated by that object information as the fill material corresponding to the full-spectrum pixel.

[0118] In step 2022A2, the marking parameters of the fill material corresponding to the full-spectrum pixel are determined based on the energy value of the light incident on the full-spectrum pixel.

[0119] In some embodiments of this application, the marking parameter may refer to the marking degree parameter of the object information mark. Optionally, the higher the energy value of the light incident on the full-spectrum pixel, the higher the value of the marking parameter of the fill material corresponding to the object information mark. When the fill material corresponding to the object information mark is a solid color fill material, the higher the energy value, the higher the color saturation of the fill material. When the fill material corresponding to the object information mark is a texture fill material, the higher the energy value, the higher the texture density of the fill material. When the fill material corresponding to the object information mark is a gradient fill material, the higher the energy value, the higher the gradient density of the fill material.

[0120] In one optional implementation, the higher the energy value of each ray incident on a full-spectrum pixel, the larger the area of ​​the shape enclosed by the energy curve. The electronic device can store multiple area ranges, fill materials, and corresponding marker parameters. The values ​​within the area range represent the area of ​​the shape enclosed by the energy curve. Optionally, the higher the energy value of the ray incident on the full-spectrum pixel, the larger the area of ​​the shape enclosed by the energy curve, and the larger the value of the marker parameter of the fill material corresponding to the object information marker. When the fill material corresponding to the object information marker is a solid color fill material, the larger the shape enclosed by the energy curve, the higher the color saturation of the fill material. When the fill material corresponding to the object information marker is a textured fill material, the larger the shape enclosed by the energy curve, the higher the texture density of the fill material. When the fill material corresponding to the object information marker is a gradient fill material, the larger the shape enclosed by the energy curve, the higher the gradient density of the fill material.

[0121] The process by which an electronic device determines the marking parameters of the fill material corresponding to a full-spectrum pixel based on the energy value of the light incident on the full-spectrum pixel may include: the electronic device calculating the area of ​​the shape enclosed by the energy curves corresponding to the full-spectrum pixel based on the energy value of the light incident on each band of the full-spectrum pixel; determining the area range to which this area belongs; and defining the marking parameters corresponding to both this area range and the fill material as the marking parameters corresponding to the full-spectrum pixel.

[0122] For example, suppose the electronic device stores the correspondence between a first area range, a solid color fill material, and a first color saturation; the correspondence between a second area range, a solid color fill material, and a second color saturation; the correspondence between a first area range, a texture fill material, and a first texture density; and the correspondence between a second area range, a texture fill material, and a second texture density. Assume the object information corresponding to the full-spectrum pixel indicates that the object's fill material is red in the solid color fill material. The electronic device outputs an energy curve based on the energy value of light incident on each wavelength band of the full-spectrum pixel. In the coordinate system of this energy curve, the horizontal axis represents the wavelength, and the vertical axis represents the energy value of the light. The electronic device uses definite integrals to calculate the area of ​​the shape enclosed by the energy curve and the horizontal axis, and determines the area range to which this area belongs. If the area range to which the area belongs is the first area range, then the electronic device determines the first color saturation, which corresponds to both the first area range and the solid color fill material, as the marker parameter corresponding to the full-spectrum pixel.

[0123] In another alternative implementation, the electronic device can store multiple numerical ranges, fill materials, and correspondences between marker parameters. The values ​​within a range represent the difference between a first value and a second value. The first value is the energy of light incident on the target wavelength of the full-spectrum pixel. The second value is the reflectance value in the target reflection spectrum corresponding to the target wavelength for that full-spectrum pixel. The target wavelength can be any wavelength.

[0124] When the fill material corresponding to the object information marker is a solid color fill material, the higher the energy value of the light of the target wavelength incident on the full-spectrum pixel, and the greater the difference between the first and second values, the higher the color saturation of the fill material. When the fill material corresponding to the object information marker is a texture fill material, the higher the energy value of the light of the target wavelength incident on the full-spectrum pixel, and the greater the difference between the first and second values, the higher the texture density of the fill material. When the fill material corresponding to the object information marker is a gradient fill material, the higher the energy value of the light of the target wavelength incident on the full-spectrum pixel, and the greater the difference between the first and second values, the higher the gradient density of the fill material.

[0125] For example, assume the target wavelength is 300 nanometers. The electronic device stores the correspondence between a first numerical range, a solid color fill material, and a first color saturation; a second numerical range, a solid color fill material, and a second color saturation; a first numerical range, a texture fill material, and a first texture density; and a second numerical range, a texture fill material, and a second texture density. Assume the object information corresponding to the full-spectrum pixel indicates that the object's fill material is red in the solid color fill material. The electronic device calculates the difference between the energy value of light incident on the full-spectrum pixel at a wavelength of 300 nanometers and the reflectance in the target reflection spectrum corresponding to the 300-nanometer wavelength. It determines the numerical range to which this difference belongs. If the numerical range to which the difference belongs is the first numerical range, the electronic device determines the first color saturation, which corresponds to both the first numerical range and the solid color fill material, as the marker parameter corresponding to the full-spectrum pixel.

[0126] In step 2022A3, according to the marking parameters of the fill material corresponding to the full-spectrum pixels, the image area corresponding to the full-spectrum pixels in the original image acquired by the shooting module is filled with the fill material corresponding to the object indicated by the object name.

[0127] In some embodiments of this application, each pixel in the original image corresponds to a pixel of the image sensor or a full-spectrum pixel. The image region in the original image corresponding to the full-spectrum pixel may refer to the region where the pixel corresponding to the full-spectrum pixel is located. The electronic device fills the region in the original image with the fill material corresponding to the full-spectrum pixel, according to the marking parameters of the fill material corresponding to the full-spectrum pixel, using the fill material corresponding to the object indicated by the object name.

[0128] In this embodiment, the light incident on the shooting module is the light from the object within the shooting field of view of the shooting module. The positive correlation between the density of the object within the shooting field of view and the energy value of the reflected light from the object is utilized. Based on the energy value of the light incident on the full-spectrum pixels, fill material with marker parameters reflecting different concentrations is used to fill the original image, so that the filled original image not only intuitively reflects the objects present within the shooting field of view, but also more intuitively reflects the density of the objects.

[0129] For cases where the object information of the subject being photographed includes the object name and the number of full-spectrum pixels is multiple, the process of outputting the object information of the subject within the field of view of the shooting module in step 202, based on the energy value of the light in each band, may include the following steps 2021B to 2022B.

[0130] In step 2021B, the object information corresponding to each full-spectrum pixel is obtained based on the energy value of the light incident on each band of each full-spectrum pixel.

[0131] The explanation and implementation of step 2021B can be found in the explanation and implementation of step 2021A mentioned above, and will not be repeated in this embodiment.

[0132] It should be noted that the process by which an electronic device obtains object information corresponding to each full-spectrum pixel based on the energy value of light incident on each band of each full-spectrum pixel may include steps 2021B1 to 2021B3.

[0133] In step 2021B1, based on the energy value of light in each band of each full-spectrum pixel, the energy curve corresponding to each full-spectrum pixel is output.

[0134] In some embodiments of this application, the electronic device can output the energy curve corresponding to a single full-spectrum pixel based on the energy value of light in each band of a single full-spectrum pixel, so as to obtain the energy curve corresponding to each full-spectrum pixel among multiple full-spectrum pixels. The implementation method of the electronic device outputting the energy curve corresponding to a single full-spectrum pixel based on the energy value of light in each band of a single full-spectrum pixel can refer to the implementation method of the aforementioned step 2021A1, and will not be elaborated upon in this embodiment.

[0135] In step 2021B2, the target reflection spectrum that matches the curve shape of each energy curve is found from the pre-stored set of reflection spectra to obtain the target reflection spectrum corresponding to each full-spectrum pixel.

[0136] In some embodiments of this application, the electronic device can search for a target reflection spectrum that matches the curve shape of a single energy curve from a pre-stored set of reflection spectra, obtain the target reflection spectrum of the full-spectrum pixel corresponding to the single energy curve, and then obtain the target reflection spectrum corresponding to each of the multiple full-spectrum pixels. The implementation method of the electronic device searching for a target reflection spectrum that matches the curve shape of a single energy curve from a pre-stored set of reflection spectra to obtain the target reflection spectrum of the full-spectrum pixel corresponding to the single energy curve can refer to the implementation method of step 2021A2 described above, and will not be elaborated further in this embodiment.

[0137] In step 2021B3, the object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the full-spectrum pixel corresponding to the reflectance spectrum of a single target, so as to obtain the object information corresponding to each full-spectrum pixel.

[0138] In some embodiments of this application, the way in which the electronic device determines the object attribute information corresponding to a single target reflection spectrum as the object information of the full-spectrum pixel corresponding to the single target reflection spectrum can refer to the implementation of the aforementioned step 2021A3, and this application embodiment will not elaborate on this.

[0139] In step 2022B, object information markers are displayed in the image region corresponding to each full-spectrum pixel in the original image acquired by the imaging module. These object information markers are used to mark the object information corresponding to the full-spectrum pixels. The original image is an image of the object captured by the imaging module within its field of view.

[0140] In some embodiments of this application, the electronic device displays object information markers in the image region corresponding to each full-spectrum pixel in the original image, which can be implemented by referring to the aforementioned implementation of step 2022A. This application embodiment will not elaborate on this.

[0141] In some embodiments of this application, when there are multiple full-spectrum pixels, the electronic device can determine the object name corresponding to each full-spectrum pixel by matching the energy curve corresponding to the full-spectrum pixel with the reflectance spectrum of known object information. Compared to determining the object name corresponding to only one full-spectrum pixel, this increases the object detection range within the shooting field of view. Furthermore, displaying object information markers in the image area corresponding to each full-spectrum pixel in the original image acquired by the shooting module not only more intuitively reflects the objects present within the shooting field of view, but also, because the object information markers are located at the corresponding positions of the full-spectrum pixels in the original image, the marked original image can more intuitively reflect the specific location of the object indicated by the determined object information within the shooting field of view, improving the display effect of object recognition.

[0142] As previously stated, in this embodiment, the object information of the full-spectrum pixel determined by the electronic device may further include object concentration information. When the object information includes object name information and object concentration information, step 2022B, which involves displaying object information markers in the image region corresponding to each full-spectrum pixel in the original image acquired by the imaging module, may include:

[0143] Based on the energy value of the light incident on each full-spectrum pixel, object information markers are displayed in the image region corresponding to each full-spectrum pixel in the original image. These object information markers are used not only to mark the object information corresponding to the full-spectrum pixel but also to mark the object density information. The higher the energy value, the greater the object density; different object densities correspond to object information markers with different marking parameters. The marking parameters can refer to the degree of marking of the object information markers. Optionally, when the fill material corresponding to the object information marker is a solid color fill material, the marking parameter refers to the color saturation parameter. When the fill material corresponding to the object information marker is a texture fill material, the marking parameter refers to the texture density parameter. When the fill material corresponding to the object information marker is a gradient fill material, the marking parameter refers to the gradient density parameter.

[0144] Optionally, the process by which an electronic device displays an object information marker in the image region corresponding to each full-spectrum pixel in the original image based on the energy value of the light incident on each full-spectrum pixel may include steps 2022B1 to 2022B3.

[0145] In step 2022B1, the fill material corresponding to each full-spectrum pixel is determined based on the fill material corresponding to the object indicated by the object name.

[0146] In some embodiments of this application, the electronic device determines the fill material corresponding to each full-spectrum pixel based on the fill material corresponding to the object indicated by the object name. The implementation method can refer to the implementation method of the aforementioned step 2022A1, and this application embodiment will not elaborate on this.

[0147] In step 2022B2, the marking parameters of the fill material corresponding to each full-spectrum pixel are determined based on the energy value of the light incident on each full-spectrum pixel.

[0148] In some embodiments of this application, the electronic device determines the marking parameters of the fill material corresponding to a single full-spectrum pixel based on the energy value of the light incident on that single full-spectrum pixel, thereby obtaining the marking parameters of the fill material corresponding to each full-spectrum pixel. The implementation method of the electronic device determining the marking parameters of the fill material corresponding to a single full-spectrum pixel based on the energy value of the light incident on that single full-spectrum pixel can refer to the implementation method of the aforementioned step 2022A2, and will not be elaborated upon in this embodiment.

[0149] In step 2022B3, according to the marking parameters of the fill material corresponding to each full-spectrum pixel, the image area corresponding to each full-spectrum pixel in the original image acquired by the shooting module is filled with the fill material corresponding to the object indicated by the object name.

[0150] In some embodiments of this application, the electronic device fills the image region corresponding to each full-spectrum pixel in the original image acquired by the shooting module with the filling material corresponding to the object indicated by the object name according to the marking parameters of the filling material corresponding to each full-spectrum pixel. The implementation method can refer to the implementation method of the aforementioned step 2022A3, and this application embodiment will not elaborate on this.

[0151] In some embodiments of this application, the positive correlation between the concentration of objects within the shooting field of view and the energy value of the reflected light from the objects is utilized. Based on the energy value of the light incident on each full-spectrum pixel (the reflected light from the objects), fill material reflecting different concentration marker parameters is used to fill the original image. This allows the filled original image to not only more intuitively reflect the objects present in more locations within the shooting field of view, but also to more intuitively reflect the concentration of the objects. Furthermore, because the object information markers are located at the corresponding positions of the full-spectrum pixels in the original image, the marked original image can more intuitively reflect the specific location of the object indicated by the determined object information within the shooting field of view, improving the display effect of object recognition.

[0152] In another alternative implementation, the electronic device uses spatial multispectral technology to determine the object information of the subject being photographed.

[0153] With the multispectral components located inside the image sensor, the number of multispectral components in the imaging module is at least two, and the multispectral components are spectral filters. The at least two spectral filters form at least two filter groups. Each filter group includes at least four spectral filters. The imaging module may also include an image sensor, with the at least two filter groups spaced apart on the pixels of the image sensor. Each spectral filter in each filter group transmits light in a different wavelength band. The wavelength band that each spectral filter can transmit is a fixed wavelength band. Optionally, the imaging module can be as described above. Figure 9 The shooting module shown.

[0154] Optionally, the process of obtaining the energy values ​​of light in multiple bands transmitted through the multispectral component in step 201 above may include the following step 2011C.

[0155] In step 2011C, the energy value of light passing through each band of each filter group is obtained.

[0156] In some embodiments of this application, the electronic device acquires the energy value of light transmitted through each spectral filter in each of at least two filter groups, so as to obtain the energy value of light in each of the multiple bands corresponding to each filter group.

[0157] In some embodiments of this application, the imaging module comprises at least two filter groups consisting of at least two spectral filters, and each spectral filter in each filter group transmits light in a different wavelength band. Through this arrangement of spectral filters in the imaging module, the electronic device can simultaneously acquire the energy values ​​of light incident on multiple wavelength bands corresponding to multiple spectral filters within the filter group. Compared to the aforementioned method of acquiring the energy values ​​of light incident on multiple wavelength bands using an FPI multispectral component, this eliminates the need to wait for multiple adjustments to the distance between the Fabry-Perot cavity mover and stator in the FPI multispectral component, reducing the time required to acquire the energy values ​​of light in multiple wavelength bands and improving energy value acquisition efficiency.

[0158] When the object information of the subject being photographed includes the object name, the process of outputting the object information of the subject within the field of view of the shooting module in step 202 above, based on the energy value of the light in each band, may include the following steps 2021C to 2022C.

[0159] In step 2021C, the object information corresponding to each filter group is obtained based on the energy value of the light transmitted through each band of each filter group.

[0160] In some embodiments of this application, the electronic device obtains the object information corresponding to a single filter group based on the energy value of light passing through each wavelength band of the single filter group, and then obtains the object information corresponding to each filter group. The implementation method of the electronic device obtaining the object information corresponding to a single filter group based on the energy value of light passing through each wavelength band of the single filter group can refer to the implementation method of the aforementioned step 2021A, and will not be elaborated upon in this embodiment.

[0161] It is worth noting that the process by which an electronic device obtains object information corresponding to each filter group based on the energy value of light passing through each band of each filter group may include the following steps 2021C1 to 2021C3.

[0162] In step 2021C1, based on the energy value of light in each wavelength band of each filter group, the energy curve corresponding to each filter group is output. The energy curve is a curve showing the change in the energy value of light incident on the filter group as a function of wavelength.

[0163] In some embodiments of this application, the electronic device can output the energy curve corresponding to the single filter group based on the energy value of light in each band of the single filter group, so as to obtain the energy curve corresponding to each filter group. The implementation method of the electronic device outputting the energy curve corresponding to the single filter group based on the energy value of light in each band of the single filter group can refer to the implementation method of the aforementioned step 2021A1, and will not be elaborated further in this embodiment.

[0164] In step 2021C2, the target reflectance spectrum that matches the curve shape of each energy curve is found from the pre-stored reflectance spectrum set, thus obtaining the target reflectance spectrum corresponding to each filter group. The reflectance spectrum set includes the reflectance spectrum of at least one object and the object attribute information corresponding to each reflectance spectrum.

[0165] In some embodiments of this application, the electronic device can search for a target reflection spectrum that matches the curve shape of a single energy curve from a pre-stored set of reflection spectra, thereby obtaining the target reflection spectrum of the filter group corresponding to the single energy curve, and further obtaining the target reflection spectrum corresponding to each filter group. The implementation method of the electronic device searching for a target reflection spectrum that matches the curve shape of a single energy curve to obtain the target reflection spectrum of the filter group corresponding to the single energy curve can refer to the implementation method of the aforementioned step 2021A2, and will not be elaborated upon in this embodiment.

[0166] In step 2021C3, the object attribute information corresponding to the reflection spectrum of a single target is determined as the object information of the filter group corresponding to the reflection spectrum of a single target, so as to obtain the object information corresponding to each filter group.

[0167] In some embodiments of this application, the electronic device can determine the object attribute information corresponding to a single target reflection spectrum as the object information of the filter group corresponding to the single target reflection spectrum. The implementation method of step 2021A3 described above can be referred to, and this application embodiment will not elaborate on this.

[0168] In step 2022C, object information markers are displayed in the image area corresponding to each filter group in the original image acquired by the imaging module. These object information markers are used to mark the object information corresponding to each filter group. The original image is an image of the object captured by the imaging module within its field of view.

[0169] In some embodiments of this application, the electronic device can display an object information marker corresponding to a single filter in the image area corresponding to a single filter group in the original image, so as to display the object information marker in the image area corresponding to each filter group in the original image. The implementation method of the electronic device displaying the object information marker corresponding to a single filter in the image area corresponding to a single filter group in the original image can refer to the implementation method of step 2022A described above, and will not be elaborated further in this embodiment.

[0170] It should be noted that each pixel in the original image corresponds to a pixel (including full-spectrum pixels) of the image sensor, and the filters are disposed on the pixels of the image sensor. Therefore, the image region in the original image corresponding to a single filter group can refer to the region of the pixels in the original image corresponding to each spectral filter included in that single filter group. The electronic device displays object information markers in the region of the pixels in the original image corresponding to each spectral filter included in each filter group.

[0171] In some embodiments of this application, the method of displaying object information markers in the image area corresponding to each filter group in the original image acquired by the shooting module not only more intuitively reflects the shooting objects within the shooting field of view, but also, because the position of the object information marker is the corresponding position of the filter group corresponding to the object information in the original image, the marked original image can more intuitively reflect the specific position of the object indicated by the determined object information within the shooting field of view, thus improving the display effect of the identified object.

[0172] As previously stated, the object information of the full-spectrum pixels determined by the electronic device in this embodiment may further include object concentration information. When the object information includes object name information and object concentration information, the process of displaying object information markers in the image area corresponding to each filter group in the original image acquired by the imaging module in step 2022C may include:

[0173] Based on the energy value of the light incident on each filter group, object information markers are displayed in the image region corresponding to each filter group in the original image. These object information markers are used not only to mark the object information corresponding to the filter group but also to mark the object concentration information. Specifically, the higher the energy value, the higher the object concentration, and different object concentrations correspond to object information markers with different marking parameters.

[0174] Optionally, the process by which the electronic device displays object information markers in the image area corresponding to each filter group in the original image based on the energy value of the light incident on each filter group may include steps 2022C1 to 2022C3.

[0175] In step 2022C1, the fill material corresponding to each filter group is determined according to the fill material corresponding to the object indicated by the object name.

[0176] In some embodiments of this application, the electronic device determines the filling material corresponding to each filter group based on the filling material corresponding to the object indicated by the object name. The implementation method can refer to the implementation method of the aforementioned step 2022A1, and this embodiment of the application will not elaborate on this.

[0177] In step 2022C2, the marking parameters of the filling material corresponding to each filter group are determined based on the energy value of the light incident on each filter group.

[0178] In some embodiments of this application, the electronic device determines the marking parameters of the filling material corresponding to a single filter group based on the energy value of the light incident on the single filter group, thereby obtaining the marking parameters of the filling material corresponding to each filter group. The implementation method of the electronic device determining the marking parameters of the filling material corresponding to a single filter group based on the energy value of the light incident on the single filter group can refer to the implementation method of the aforementioned step 2022A2, and will not be elaborated upon in this embodiment.

[0179] In step 2022C3, according to the marking parameters of the fill material corresponding to each filter group, the image area corresponding to each filter group in the original image acquired by the shooting module is filled with the fill material corresponding to the object indicated by the object name.

[0180] In some embodiments of this application, the electronic device fills the image area corresponding to each filter group in the original image captured by the shooting module with the filling material corresponding to the object indicated by the object name according to the marking parameters of the filling material corresponding to each filter group. The implementation method can refer to the implementation method of the aforementioned step 2022A3, and this application embodiment will not elaborate on this.

[0181] In some embodiments of this application, the positive correlation between the density of objects within the shooting field of view and the energy value of the reflected light from the objects is utilized. Based on the energy value of the light incident on each filter group (the reflected light from the objects), fill material reflecting different concentration marker parameters is used to fill the original image. This allows the filled original image to not only more intuitively reflect the objects present in more locations within the shooting field of view, but also to more intuitively reflect the density of the objects. Furthermore, because the location of the object information marker is the corresponding position of the full-spectrum pixel in the original image corresponding to the object information, the marked original image can more intuitively reflect the specific location of the object indicated by the determined object information within the shooting field of view, improving the display effect of object recognition.

[0182] When the multispectral components are located outside the image sensor, there are at least two multispectral components. The imaging module also includes an image sensor, which includes at least two full-spectrum pixels. Each multispectral component is correspondingly disposed on the light-incident side of one full-spectrum pixel. The at least two multispectral components form at least one multispectral component group. Each multispectral component group includes at least two multispectral components, and each multispectral component in each multispectral component group transmits light in a different wavelength band. The wavelength band of light that can be transmitted by each multispectral component is a fixed wavelength band. It should be noted that when the multispectral component is an FPI multispectral component, the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator in each multispectral component is fixed. Optionally, the imaging module can be as described above. Figures 3 to 5 Any of the shooting modules shown, wherein the number of multispectral components in the shooting module is equal to the number of full-spectrum pixels.

[0183] Optionally, the process of obtaining the energy values ​​of light in multiple bands transmitted through the multispectral component in step 201 above may include the following step 2011D.

[0184] In step 2011D, the energy value of the light incident on each target full-spectrum pixel is obtained, where the target full-spectrum pixel is the full-spectrum pixel corresponding to a multispectral component group.

[0185] In some embodiments of this application, the electronic device acquires the energy value of the light transmitted by each multispectral component in each of at least two multispectral component groups, so as to obtain the energy value of the light in each of the multiple bands corresponding to each multispectral component group.

[0186] When the object information of the subject being photographed includes the object name, the process of outputting the object information of the subject within the field of view of the shooting module in step 202 above, based on the energy value of the light in each band, may include the following steps 2021D to 2022D.

[0187] In step 2021D, the object information corresponding to each multispectral component group is obtained based on the energy value of the light of each target full-spectrum pixel.

[0188] The explanation and implementation of this step can be found in the explanation and implementation of step 2021C above, and will not be repeated in this embodiment. It should be noted that the light incident on the full-spectrum pixel through the multispectral component can be equivalent to the light of the aforementioned incident spectral filter.

[0189] It is worth noting that the process by which an electronic device obtains object information corresponding to each multispectral component group based on the energy value of the light of each target full-spectrum pixel may include the following steps 2021D1 to 2021D3.

[0190] In step 2021D1, based on the energy value of the light from each target full-spectrum pixel, the energy curve corresponding to each multispectral component group is output. The energy curve is the curve showing the change of the energy value of the light incident on the full-spectrum pixel of the multispectral component group with wavelength.

[0191] In step 2021D2, the target reflection spectrum that matches the curve shape of each energy curve is found from the pre-stored reflection spectrum set to obtain the target reflection spectrum corresponding to each multispectral component group. The reflection spectrum set includes the reflection spectrum of at least one object and the object attribute information corresponding to each reflection spectrum.

[0192] In step 2021D3, the object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the multispectral component group corresponding to the reflectance spectrum of a single target, so as to obtain the object information corresponding to each multispectral component group.

[0193] The explanations and implementation methods of steps 2021D1, 2021D2 and 2021D3 can be referred to the explanations and implementation methods of the aforementioned steps 2021C1, 2021C2 and 2021C3 in turn. The embodiments of this application will not be repeated here.

[0194] In step 2022D, object information markers are displayed in the image region corresponding to each target full-spectrum pixel in the original image acquired by the imaging module. These object information markers are used to mark the object information corresponding to the target full-spectrum pixels. The original image is an image of the object captured by the imaging module within its field of view.

[0195] In some embodiments of this application, each pixel in the original image corresponds to a pixel of the image sensor or a pixel comprising the full spectrum. The image region in the original image corresponding to each target full spectrum pixel may refer to the region where the pixel corresponding to each target full spectrum pixel is located in the original image. The electronic device may display object information markers in the region where the pixel corresponding to each target full spectrum pixel is located in the original image.

[0196] As previously stated, in this embodiment, the object information of the full-spectrum pixel determined by the electronic device may further include object concentration information. When the object information includes object name information and object concentration information, the process described in step 2022D of displaying object information markers in the image region corresponding to each target full-spectrum pixel in the original image acquired by the imaging module may include:

[0197] Based on the energy value of the light incident on each full-spectrum pixel, object information markers are displayed in the image region corresponding to each full-spectrum pixel in the original image. These object information markers are also used to indicate the object concentration information corresponding to the full-spectrum pixel. Higher energy values ​​indicate higher object concentration, and different object concentrations correspond to object information markers with different marking parameters.

[0198] Optionally, the process by which an electronic device displays an object information marker in the image region corresponding to each full-spectrum pixel in the original image based on the energy value of the light incident on each full-spectrum pixel may include steps 2022D1 to 2022D3.

[0199] In step 2022D1, the fill material corresponding to each full-spectrum pixel is determined based on the fill material corresponding to the object indicated by the object name.

[0200] In step 2022D2, the marking parameters of the fill material corresponding to each full-spectrum pixel are determined based on the energy value of the light incident on each full-spectrum pixel.

[0201] In step 2022D3, according to the marking parameters of the fill material corresponding to each full-spectrum pixel, the image area corresponding to each full-spectrum pixel in the original image acquired by the shooting module is filled with the fill material corresponding to the object indicated by the object name.

[0202] The fill material varies for different objects and includes at least one of the following: solid color fill material, texture fill material, or gradient fill material. When the fill material is a solid color fill material, the higher the energy value, the higher the color saturation of the fill material. When the fill material is a texture fill material, the higher the energy value, the higher the texture density of the fill material. When the fill material is a gradient fill material, the higher the energy value, the higher the gradient density of the fill material.

[0203] The explanations and implementation methods of steps 2022D1, 2022D2 and 2022D3 can be referred to the explanations and implementation methods of steps 2022C1, 2022C2 and 2022C3 mentioned above, and will not be repeated in this embodiment.

[0204] In this embodiment, the positive correlation between the concentration of objects within the shooting field of view and the energy value of the reflected light from the objects is utilized. Based on the energy value of the light incident on each multispectral component group (the reflected light from the objects), fill material reflecting different concentration marker parameters is used to fill the original image. This allows the filled original image to not only more intuitively reflect the objects present in more locations within the shooting field of view, but also to more intuitively reflect the concentration of the objects. Furthermore, because the location of the object information marker is the corresponding position of the target full-spectrum pixel in the original image, the marked original image can more intuitively reflect the specific location of the object indicated by the determined object information within the shooting field of view, improving the display effect of object recognition.

[0205] In summary, the multispectral detection method provided in this application acquires the energy values ​​of multiple wavelengths of light transmitted through the multispectral component in the imaging module, and then outputs object information of the subject within the imaging module's field of view based on the energy value of each wavelength, thereby achieving object detection. Since the light incident on the multispectral component is the reflected light from the subject within the imaging module's field of view, the energy values ​​of the acquired wavelengths reflect the subject's reflection of those wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module, compared to related technologies, allows for the detection of more objects and improves the accuracy of object detection.

[0206] Please refer to Figure 12 This document illustrates a flowchart of another multispectral detection method provided in an embodiment of this application. This multispectral detection method can be applied to environmental monitoring scenarios. The multispectral detection method is applied to a multispectral detection device. Optionally, the multispectral detection device can be an electronic device. For example, the electronic device may include a mobile phone, tablet computer, etc. The multispectral detection device includes any of the imaging modules with the structures provided in the embodiments of this application (e.g., [example module name]). Figure 3 , Figure 4 , Figure 5 or Figure 9 (The shooting module shown). Figure 12 As shown, multispectral detection methods include:

[0207] Step 1201: Obtain the geographical location of the electronic device.

[0208] Alternatively, electronic devices can obtain their geographical location information in real time or periodically through the Global Positioning System (GPS).

[0209] Step 1202: If the geographical location is at a preset location, output an alarm message. The alarm message is used to notify the user that there is a harmful object at the geographical location of the electronic device, and the user can detect the object information of the harmful object through the electronic device.

[0210] In some embodiments of this application, the preset location may refer to an unsafe location in the environment where hazardous substances may exist. For example, the preset location may be a newly constructed building, a chemical plant, a nuclear radiation area, etc. The preset location may be a non-safe location manually set by the user. Alternatively, the preset location may be a non-safe location automatically identified by the electronic device. Optionally, the electronic device may utilize big data technology to search for non-safe locations.

[0211] After acquiring its geographical location, the electronic device can determine whether that location is within a preset location. If the location is within the preset location, it indicates that the user's current environment may contain hazardous substances. The electronic device can then output an alarm message. For example, if the electronic device is within the preset location, it can vibrate or ring and display an alarm message. The alarm message could state: "Hazardous substances may exist in the current environment; the electronic device can detect hazardous objects."

[0212] Optionally, the electronic device may also output an alarm message when the geographical location is at a preset location and the air pressure at the location of the electronic device is within the dangerous air pressure range corresponding to the preset location.

[0213] In some embodiments of this application, hazardous substances may be distributed in the environment at a certain height. For example, in a new building, floors 20 to 30 are newly renovated floors. The hazardous substance formaldehyde is mainly distributed in the environment of floors 20 to 30 of the new building.

[0214] The electronic device, knowing its geographical location is within a preset area, can obtain the air pressure value at its location and determine whether this pressure value is within the hazardous pressure range corresponding to that location. If the air pressure value at the electronic device's location is within the hazardous pressure range, it indicates that the user's current environment is highly likely to contain hazardous substances, and the electronic device can output an alarm message. If the air pressure value at the electronic device's location is not within the hazardous pressure range, it indicates that the user's current environment may not contain hazardous substances, and the electronic device may not output an alarm message. Optionally, the electronic device may include a barometer. The electronic device can read the air pressure value at its location using the barometer.

[0215] The hazardous pressure range corresponding to the preset location can be manually set by the user. Alternatively, the hazardous pressure range corresponding to the preset location can be automatically obtained by the electronic device. Optionally, the electronic device can use big data technology to search for the distribution of hazardous objects at the preset location and generate the pressure range corresponding to the environmental height where the hazardous objects may exist.

[0216] In this embodiment, since hazardous substances may be mainly distributed in an environment at a certain height within unsafe locations, for example, floors 20 to 30 of a new building are newly renovated. Formaldehyde, a hazardous substance, is mainly distributed in the environment of floors 20 to 30 of this new building. Therefore, when the electronic device is located at a preset location and the air pressure at that location is within the dangerous air pressure range corresponding to that preset location, the method of outputting alarm information takes into account the height of the hazardous substance distribution location, improving the accuracy of the electronic identification in determining whether hazardous substances exist at its current location. This, in turn, improves the accuracy of the timing of alarm information output and avoids invalid alarm information output.

[0217] In an optional implementation, before outputting an alarm message in step 1202 if the geographical location is at a preset location, the multispectral detection method may further include: the electronic device determining whether its geographical location is at a frequently used location. A frequently used location can refer to a location where the electronic device is frequently located. For example, a frequently used location could be the user's home location, workplace location, etc.

[0218] If the geographical location is not in a frequently used location, it indicates a probability that the electronic device is currently in an unsafe location. The electronic device can determine whether the geographical location is in a preset location. If the geographical location is in a frequently used location, it indicates that there is no probability that the electronic device is currently in an unsafe location. The electronic device may choose not to perform any action.

[0219] Step 1203: Obtain the energy values ​​of light in multiple bands transmitted through the multispectral component.

[0220] The explanation and implementation of this step can be found in the explanation and implementation of step 201 above, and will not be repeated in this embodiment.

[0221] Step 1204: Based on the energy value of light in each band, output the object information of the subject within the field of view of the shooting module.

[0222] The explanation and implementation of this step can refer to the explanation and implementation of step 202 above, and will not be repeated in this embodiment. It should be noted that since the multispectral detection method in this embodiment is mainly applied to environmental monitoring scenarios, the reflectance spectrum set stored in the electronic device may include: the reflectance spectrum of at least one hazardous object and the object attribute information of the hazardous object. Optionally, the hazardous object may be a substance containing functional groups such as formaldehyde, methylene, carbonyl (aldehyde, ketone, carboxylic acid), cyano, hydroxyl, and amino. For example, the hazardous object may be acetylene, ethane, xylene, isopropanol, ammonia, acetaldehyde, toluene, carbon dioxide, diborane, dimethyl sulfide, methanol, benzene, chloroform, ethyl acetate, cyclohexane, dimethyl sulfide, propionaldehyde, styrene, hydrogen sulfide, ethanol, and acetone.

[0223] The embodiments of this application illustrate, with the following examples, Figure 12 The multispectral detection method shown is further illustrated. For example, in the imaging module of a multispectral detection device, the image sensor includes multiple full-spectrum pixels, and these multiple full-spectrum pixels are distributed within the pixel array of the image sensor. Optionally, the imaging module can be... Figure 3 , Figure 4 or Figure 5 The shooting module shown.

[0224] The electronic device can obtain its real-time geographical location. If it determines that the location is not in a commonly used location, it will check if the location is within a preset location. If the location is within the preset location and the air pressure at the electronic device's location is within the corresponding hazardous air pressure range, the electronic device can vibrate or ring and display an alarm message to alert the user that there may be hazardous substances in the current environment, and that the user can remove the electronic device to detect hazardous objects.

[0225] The electronic device acquires the energy values ​​of light passing through multiple wavelengths of a multispectral component to gather spectral information about the current environment. The electronic device stores a set of reflectance spectra, which includes the reflectance spectrum of at least one hazardous object and its attribute information. After acquiring the energy values ​​of light passing through the multispectral component and incident on each full-spectrum pixel, the electronic device outputs an energy curve corresponding to each full-spectrum pixel based on the energy values ​​of each wavelength of light incident on each full-spectrum pixel. The electronic device can then search for a target reflectance spectrum from the pre-stored reflectance spectrum set that matches the curve shape of each energy curve to obtain the target reflectance spectrum corresponding to each full-spectrum pixel. The object attribute information corresponding to a single target reflectance spectrum is then identified as the object information for that single target reflectance spectrum's corresponding full-spectrum pixel, thus obtaining the object information for each full-spectrum pixel. The electronic device displays object information markers in the image region corresponding to each full-spectrum pixel in the raw image captured by the imaging module. These object information markers are used not only to mark the object information corresponding to the full-spectrum pixel but also to mark the object concentration information corresponding to the full-spectrum pixel.

[0226] For a specific example, suppose an electronic device matches three target reflectance spectra. The object attribute information corresponding to these target reflectance spectra is three toxic gases: formaldehyde, acetaldehyde, and carbon dioxide. Within the field of view, there are three concentrations of formaldehyde, two concentrations of acetaldehyde, and two concentrations of carbon dioxide. Assume the fill material corresponding to formaldehyde is blue, acetaldehyde is red, and carbon dioxide is green. The labeling parameters for the fill material corresponding to formaldehyde are three different saturation parameters. The labeling parameters for the fill material corresponding to acetaldehyde are two different saturation parameters. The labeling parameters for the fill material corresponding to carbon dioxide are two different saturation parameters. For example... Figure 13 As shown, in the original image displaying object information markers, the first image region I is filled with blue of a first saturation. The second image region II is filled with blue of a second saturation. The third image region III is filled with blue of a third saturation. The first, second, and third saturations decrease sequentially. The fourth image region IV is filled with red of a fourth saturation. The fifth image region V is filled with red of a fifth saturation. The fourth saturation is greater than the fifth saturation. The sixth image region VI is filled with green of a sixth saturation. The seventh image region VII is filled with green of a seventh saturation. The sixth saturation is greater than the seventh saturation. It should be noted that... Figure 13In the image, the first image region I, the second image region II, and the third image region III all use dot fill to represent blue fill material. The higher the dot density in these regions, the higher the saturation of the fill material. The fourth image region IV and the fifth image region V both use horizontal line fill to represent red fill material. The higher the horizontal line density in these regions, the higher the saturation of the fill material. The sixth image region VI and the seventh image region VII both use diagonal line fill to represent green fill material. The higher the diagonal line density in these regions, the higher the saturation of the fill material.

[0227] Depend on Figure 13 It can be seen that formaldehyde exists in three areas: the area in the upper right corner of the image is of moderate severity, the area in the lower right corner is of mild severity, and the area in the lower center is the most severe. It's easy to understand that the original image displaying object information can provide route guidance. This image can guide the user to leave the current location from the area with the lowest concentration or the area free of harmful substances.

[0228] In this embodiment, when the electronic device determines that its environment may contain hazardous substances, it can not only issue an alarm but also detect hazardous substances, achieving automatic environmental detection and enhancing the service functions of the imaging module. Furthermore, the original image marked with object information tags visualizes the detection of hazardous substances, more intuitively reflecting the hazardous substances present in the current environment of the electronic device, as well as their specific locations and concentrations, thus improving the display effect of the identified objects.

[0229] This application also provides a multispectral detection method. This multispectral detection method can be applied to scenarios such as facial skin condition detection and food health status detection. The multispectral detection method is applied to a multispectral detection device. Optionally, the multispectral detection device can be an electronic device. For example, the electronic device may include a mobile phone, tablet computer, etc. The multispectral detection device includes any of the imaging modules with the structure provided in this application embodiment (e.g., Figure 3 , Figure 4 , Figure 5 or Figure 9 Multispectral detection methods include:

[0230] The system receives the user's first input and, in response, displays the object to be detected on the shooting preview interface. The first input can be used to display the shooting preview interface. For example, the first input can be a click, swipe, or voice input targeting a shooting icon.

[0231] The electronic device acquires the energy values ​​of light passing through multiple wavelengths of a multispectral component to obtain the spectral information of the object to be detected. Based on the energy value of the light in each wavelength band, the electronic device outputs object information. When the object to be detected is a face, the object name information can be the face representing the target health level. When the object to be detected is food, the object name information can be the food representing the target health level.

[0232] Optionally, the example given is an image sensor in the shooting module that includes multiple full-spectrum pixels, and these multiple full-spectrum pixels are distributed within the pixel array of the image sensor. For example, the shooting module can be... Figure 3 , Figure 4 or Figure 5 The shooting module shown.

[0233] The electronic device stores a set of reflectance spectra. This set includes: reflectance spectra of multiple faces at different health levels, along with object attribute information, where the object attribute information refers to the face at the target health level to which the reflectance spectrum belongs. The set also includes: reflectance spectra of multiple foods at different health levels, along with object attribute information, where the object attribute information refers to the food at the target health level to which the reflectance spectrum belongs. Different health levels of faces can refer to faces with different skin types, or faces carrying cosmetics of different qualities. The classification of skin type or cosmetic quality can be determined based on the specific circumstances. Different health levels of food can refer to foods at different degrees of spoilage. The classification of food spoilage degrees can be determined based on the specific circumstances.

[0234] The process by which an electronic device acquires the energy values ​​of light passing through multiple bands of a multispectral component and outputs object information of the target object based on the energy value of each band of light can include: acquiring the energy values ​​of light passing through the multispectral component and incident on each full-spectrum pixel; and outputting an energy curve corresponding to each full-spectrum pixel based on the energy value of light passing through each band of light incident on each full-spectrum pixel. The electronic device can search for a target reflection spectrum that matches the curve shape of each energy curve from a pre-stored set of reflection spectra to obtain the target reflection spectrum corresponding to each full-spectrum pixel. The object attribute information corresponding to a single target reflection spectrum is then determined as the object information of the full-spectrum pixel corresponding to that single target reflection spectrum, thus obtaining the object information corresponding to each full-spectrum pixel.

[0235] In the raw image captured by the camera module, the electronic device displays object information markers for the image region corresponding to each full-spectrum pixel. These markers are used to label the object information corresponding to the full-spectrum pixels. Optionally, faces with different health levels correspond to different solid-color fill materials. Foods with different health levels can also correspond to different solid-color fill materials.

[0236] As an example, suppose there are four health levels: first, second, third, and fourth, with corresponding solid color fill materials of blue, red, yellow, and green, respectively. The electronic device fills the image region corresponding to each full-spectrum pixel in the original image with the fill material corresponding to the face of the target health level to display object information markers in the original image. For example... Figure 14 As shown, the original image is Figure 14 The image on the left shows the original image with object information markers. Figure 14 The image on the right. Assume that the object information of the full-spectrum pixels corresponding to the first image region 1401 in the original image is a face of the fourth health level, and the object information of the full-spectrum pixels corresponding to the second image region 1402 is a face of the third health level. In the original image displaying the object information markers, the first image region 1401 is filled with green, and the second image region 1402 is filled with yellow. Figure 14 In the image, the first image area 1401 is filled with white to indicate that the fill material is green. The second image area 1402 is filled with a right diagonal line to indicate that the fill material is yellow.

[0237] like Figure 15 As shown, the original image is Figure 15 The image on the left shows the original image with object information markers. Figure 15 The image on the right. Assume the object information for the full-spectrum pixels corresponding to the face region in the original image is a face of health level three. In the original image displaying object information markers, the face region corresponding to the full-spectrum pixels is filled with yellow. Figure 15 In the image, the face area is filled with a left diagonal line, and the fill material is yellow.

[0238] like Figure 16 As shown, the original image is Figure 16 The image on the left shows the original image with object information markers. Figure 16 The image on the right. Assume the object information for the full-spectrum pixels corresponding to the face region in the original image is a face of second health level. In the original image displaying object information markers, the face region corresponding to the full-spectrum pixels is filled with red. Figure 16 In the image, the face area is filled with horizontal lines, and the fill material is red.

[0239] like Figure 17 As shown, the original image is Figure 17 The image on the left shows the original image with object information markers. Figure 17The image on the right. Assume that the object information of the full-spectrum pixels corresponding to the first image region 1701 in the original image is a face of the first health level, and the object information of the full-spectrum pixels corresponding to the second image region 1702 is a face of the third health level. In the original image displaying the object information markers, the first image region 1701 is filled with blue, and the second image region 1702 is filled with yellow. Figure 17 In the image, the first image area 1701 is filled with white to indicate that the fill material is blue. The second image area 1702 is filled with black to indicate that the fill material is yellow.

[0240] As another example, suppose there are five health levels for bottled food: a first health level, a second health level, a third health level, a fourth health level, and a fifth health level. The corresponding solid color fill materials are red, yellow, green, blue, and purple, respectively. The electronic device fills the image area corresponding to each full-spectrum pixel in the original image with the fill material corresponding to the target health level of the bottled food, thus displaying object information markers in the original image.

[0241] refer to Figure 18 Assume that the object information of the full-spectrum pixels corresponding to the first image region 1801 in the original image is bottled food of the fifth health level; the object information of the full-spectrum pixels corresponding to the second image region 1802 is bottled food of the second health level; the object information of the full-spectrum pixels corresponding to the third image region 1803 is bottled food of the third health level; the object information of the full-spectrum pixels corresponding to the fourth image region 1804 is bottled food of the fourth health level; and the object information of the full-spectrum pixels corresponding to the fifth image region 1805 is bottled food of the fifth health level. Figure 18 As shown, in the original image displaying object information markers, the first image area 1801 is filled with red, the second image area 1802 is filled with yellow, the third image area 1803 is filled with green, the fourth image area 1804 is filled with blue, and the fifth image area 1805 is filled with purple. Figure 18 In the image, the first image area 1801 uses a right diagonal fill to indicate that the fill material is red. The second image area 1802 uses a horizontal fill to indicate that the fill material is yellow. The third image area 1803 uses a left diagonal fill to indicate that the fill material is green. The fourth image area 1804 uses a dot fill to indicate that the fill material is blue. The fifth image area 1805 uses a cross-line fill to indicate that the fill material is purple.

[0242] In some embodiments of this application, the electronic device acquires the energy values ​​of light (i.e., reflected light from the object to be detected) passing through multiple wavelengths of light from the multispectral component in the imaging module, and outputs the health level of the object to be detected based on the energy value of each wavelength, thereby achieving state detection of the object. Furthermore, the original image marked with object information tags visualizes the corresponding health level of the object to be detected, more intuitively reflecting its health status, improving the display effect of object recognition, and providing a powerful and user-friendly imaging module service.

[0243] It should be noted that the multispectral detection device is used as an electronic device in the above-described embodiments of the present application for illustrative purposes.

[0244] In summary, the multispectral detection method provided in this application acquires the energy values ​​of multiple wavelengths of light transmitted through the multispectral component in the imaging module, and then outputs object information of the subject within the imaging module's field of view based on the energy value of each wavelength, thereby achieving object detection. Since the light incident on the multispectral component is the reflected light from the subject within the imaging module's field of view, the energy values ​​of the acquired wavelengths reflect the subject's reflection of those wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module, compared to related technologies, allows for the detection of more objects and improves the accuracy of object detection.

[0245] The multispectral detection method provided in this application can be executed by a multispectral detection device. This application uses a multispectral detection device to execute the multispectral detection method as an example to illustrate the multispectral detection device provided in this application.

[0246] Please refer to Figure 19 This illustration shows a schematic diagram of the structure of a multispectral detection device provided in an embodiment of this application. Figure 19 As shown, the multispectral detection device 1900 includes: an imaging module 1901, an acquisition module 1902, and an output module 1903.

[0247] The 1901 camera module includes a multispectral component;

[0248] The acquisition module 1902 is used to acquire the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component;

[0249] The output module 1903 is used to output object information of the subject within the field of view of the shooting module based on the energy value of light in each band.

[0250] Optionally, the multispectral component is an FPI multispectral component, which includes a Fabry-Perot cavity mover and a Fabry-Perot cavity stator. The imaging module also includes an image sensor, which includes full-spectrum pixels. The FPI multispectral component is disposed on the incident light side of the full-spectrum pixels. The acquisition module 1902 is also used for:

[0251] According to the transmission distance corresponding to each band, the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator are adjusted sequentially so that the light of each band passes through the FPI multispectral component sequentially and is incident on the full-spectrum pixel. The transmission distance is the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator.

[0252] Obtain the energy value of light incident on each band of the full spectrum pixel.

[0253] Optionally, the object information includes the object name and the number of full-spectrum pixels is at least two;

[0254] The acquisition module 1902 is also used to acquire the energy value of light incident on each band of each full-spectrum pixel;

[0255] Output module 1903 is also used for:

[0256] Based on the energy value of the light in each band incident on each full-spectrum pixel, the object information corresponding to each full-spectrum pixel is obtained;

[0257] In the original image captured by the shooting module, an object information marker is displayed in the image area corresponding to each full-spectrum pixel. The object information marker is used to mark the object information corresponding to the full-spectrum pixel. The original image is the image of the shooting object captured by the shooting module within the shooting field of view.

[0258] Optionally, the output module 1903 is also used for:

[0259] Based on the energy value of light in each band of each full-spectrum pixel, the energy curve corresponding to each full-spectrum pixel is output. The energy curve is the curve of the energy value of light changing with wavelength.

[0260] From the pre-stored set of reflectance spectra, find the target reflectance spectrum that matches the curve shape of each energy curve to obtain the target reflectance spectrum corresponding to each full-spectrum pixel. The set of reflectance spectra includes the reflectance spectrum of at least one object and the object attribute information corresponding to each reflectance spectrum.

[0261] The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the full-spectrum pixels corresponding to the reflectance spectrum of a single target, so as to obtain the object information corresponding to each full-spectrum pixel.

[0262] Optionally, the object information includes an object name, the number of multispectral components is at least two, the multispectral components are spectral filters, and the at least two spectral filters form at least two filter groups; each filter group includes at least four spectral filters, the imaging module also includes an image sensor, the at least two filter groups are spaced apart on the pixels of the image sensor, and each spectral filter in each filter group transmits light in a different wavelength band;

[0263] The acquisition module 1902 is also used to acquire the energy value of light passing through each wavelength of each filter group;

[0264] Output module 1903 is also used for:

[0265] Based on the energy value of light passing through each wavelength band of each filter group, the object information corresponding to each filter group is obtained;

[0266] In the original image captured by the shooting module, an object information marker is displayed in the image area corresponding to each filter group. The object information marker is used to mark the object information corresponding to the filter group. The original image is the image of the shooting object captured by the shooting module within the shooting field of view.

[0267] Optionally, the object information includes the object name, the number of multispectral components is at least two, the shooting module also includes an image sensor, the image sensor includes at least two full-spectrum pixels, each multispectral component is respectively disposed on the light-incident side of a full-spectrum pixel, at least two multispectral components form at least one multispectral component group, each multispectral component group includes at least two multispectral components, and each multispectral component in each multispectral component group transmits light in a different wavelength band.

[0268] The acquisition module 1902 is also used to acquire the energy value of the light incident on each target full-spectrum pixel, where the target full-spectrum pixel is the full-spectrum pixel corresponding to a multispectral component group;

[0269] Output module 1903 is also used for:

[0270] Based on the energy value of the light rays of each target full-spectrum pixel, the object information corresponding to each multispectral component group is obtained;

[0271] In the original image captured by the shooting module, an object information marker is displayed in the image area corresponding to each target full-spectrum pixel. The object information marker is used to mark the object information corresponding to the target full-spectrum pixel. The original image is the image of the shooting object captured by the shooting module within the shooting field of view.

[0272] Optionally, the output module 1903 is also used for:

[0273] Based on the energy value of the light from each target full-spectrum pixel, the energy curve corresponding to each multispectral component group is output. The energy curve is the curve of the energy value of the light incident on the full-spectrum pixel of the multispectral component group as a function of wavelength.

[0274] From the pre-stored set of reflectance spectra, find the target reflectance spectrum that matches the curve shape of each energy curve to obtain the target reflectance spectrum corresponding to each multispectral component group. The set of reflectance spectra includes the reflectance spectrum of at least one object and the object attribute information corresponding to each reflectance spectrum.

[0275] The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the multispectral component group corresponding to the reflectance spectrum of a single target, so as to obtain the object information corresponding to each multispectral component group.

[0276] Optionally, the object information also includes object concentration information; the output module 1903 is also used for:

[0277] Based on the energy value of the light incident on each full-spectrum pixel, object information markers are displayed in the image region corresponding to each full-spectrum pixel in the original image. The object information markers are also used to mark the object concentration information corresponding to the full-spectrum pixel; where the larger the energy value, the greater the object concentration, and different object concentrations correspond to object information markers with different marking parameters.

[0278] Optionally, the output module 1903 is also used for:

[0279] Based on the fill material corresponding to the object indicated by the object name, determine the fill material corresponding to each full-spectrum pixel;

[0280] Based on the energy value of the light incident on each full-spectrum pixel, determine the marking parameters of the fill material corresponding to each full-spectrum pixel;

[0281] According to the marking parameters of the fill material corresponding to each full-spectrum pixel, the image area corresponding to each full-spectrum pixel in the original image captured by the shooting module is filled with the fill material corresponding to the object indicated by the object name;

[0282] The fill material varies for different objects and includes at least one of the following: solid color fill material, texture fill material, or gradient fill material. When the fill material is a solid color fill material, the higher the energy value, the higher the color saturation of the fill material. When the fill material is a texture fill material, the higher the energy value, the higher the texture density of the fill material. When the fill material is a gradient fill material, the higher the energy value, the higher the gradient density of the fill material.

[0283] Optionally, the acquisition module 1902 is also used to acquire the geographical location of the electronic device;

[0284] The output module 1903 is also used to output alarm information when the geographical location is at a preset location. The alarm information is used to remind the user that there are harmful objects in the geographical location of the electronic device, and the object information of the harmful objects can be detected by the electronic device.

[0285] Optionally, the output module 1903 is also used to output alarm information when the geographical location is at a preset location and the air pressure value at the location of the electronic device is within the dangerous air pressure range corresponding to the preset location.

[0286] In summary, the multispectral detection device provided in this application acquires the energy values ​​of multiple wavelengths of light transmitted through the multispectral component in the imaging module, and outputs object information of the object within the imaging module's field of view based on the energy value of each wavelength, thus achieving object detection. Since the light incident on the multispectral component is the reflected light from the object within the imaging module's field of view, the energy values ​​of the acquired wavelengths reflect the object's reflection of those wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module allows for the detection of more objects and improves the accuracy of object detection compared to related technologies.

[0287] The multispectral detection device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0288] The multispectral detection device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0289] The multispectral detection device provided in this application embodiment can achieve... Figure 2 or Figure 12 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0290] Optionally, such as Figure 20 As shown, this application embodiment also provides an electronic device 2000, including a processor 2001 and a memory 2002. The memory 2002 stores a program or instructions that can run on the processor 2001. When the program or instructions are executed by the processor 2001, they implement the various steps of the above-described multispectral detection method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0291] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0292] Figure 21This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. The electronic device 2100 includes a camera module, which includes a multispectral component. The electronic device 2100 also includes, but is not limited to, components such as: a radio frequency unit 2101, a network module 2102, an audio output unit 2103, an input unit 2104, a sensor 2105, a display unit 2106, a user input unit 2107, an interface unit 2108, a memory 2109, and a processor 2110.

[0293] Those skilled in the art will understand that the electronic device 2100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 2110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 21 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0294] The processor 2110 is used to acquire the energy values ​​of light in multiple bands transmitted through the multispectral component; and to output object information of the subject within the field of view of the shooting module based on the energy value of light in each band.

[0295] In this embodiment, object detection is achieved by acquiring the energy values ​​of multiple wavelengths of light transmitted through the multispectral component of the imaging module, and then outputting object information of the subject within the imaging module's field of view based on the energy value of each wavelength. Since the light incident on the multispectral component is the reflected light from the subject within the imaging module's field of view, the energy values ​​of these multiple wavelengths reflect the subject's reflection of light across those wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module allows for the detection of more objects and improves the accuracy of object detection compared to related technologies.

[0296] Optionally, the multispectral component is an FPI multispectral component, which includes a Fabry-Perot cavity mover and a Fabry-Perot cavity stator. The imaging module further includes an image sensor, which includes full-spectrum pixels. The FPI multispectral component is disposed on the incident light side of the full-spectrum pixels. The processor 2110 is further configured to:

[0297] According to the transmission distance corresponding to each band, the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator is adjusted sequentially so that the light of each band passes through the FPI multispectral component sequentially and is incident on the full-spectrum pixel. The transmission distance is the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator.

[0298] Obtain the energy value of light incident on each band of the full-spectrum pixel.

[0299] Optionally, the object information includes an object name, and the number of full-spectrum pixels is at least two; the processor 2110 is further configured to:

[0300] Obtain the energy value of light incident on each wavelength of each full-spectrum pixel;

[0301] Based on the energy value of the light in each band incident on each full-spectrum pixel, the object information corresponding to each full-spectrum pixel is obtained;

[0302] In the original image captured by the shooting module, an object information marker is displayed in the image area corresponding to each full-spectrum pixel. The object information marker is used to mark the object information corresponding to the full-spectrum pixel. The original image is an image of the object captured by the shooting module within the shooting field of view.

[0303] Optionally, the processor 2110 is also used for:

[0304] Based on the energy value of light in each band of each full-spectrum pixel, the energy curve corresponding to each full-spectrum pixel is output, and the energy curve is the curve of the energy value of light changing with wavelength;

[0305] From a pre-stored set of reflectance spectra, a target reflectance spectrum that matches the curve shape of each energy curve is found to obtain the target reflectance spectrum corresponding to each full-spectrum pixel. The set of reflectance spectra includes the reflectance spectrum of at least one object and object attribute information corresponding to each reflectance spectrum.

[0306] The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the full-spectrum pixel corresponding to the reflectance spectrum of the single target, so as to obtain the object information corresponding to each full-spectrum pixel.

[0307] Optionally, the object information includes an object name; the number of multispectral components is at least two; the multispectral components are spectral filters; at least two spectral filters form at least two filter groups; each filter group includes at least four spectral filters; the imaging module further includes an image sensor; the at least two filter groups are spaced apart on the pixels of the image sensor; each spectral filter in each filter group transmits light in a different wavelength band; the processor 2110 is further configured to:

[0308] Obtain the energy value of light passing through each wavelength band of each filter group;

[0309] Based on the energy value of light passing through each wavelength band of each filter group, the object information corresponding to each filter group is obtained;

[0310] In the original image captured by the shooting module, an object information marker is displayed in the image area corresponding to each filter group. The object information marker is used to mark the object information corresponding to the filter group. The original image is an image of the object captured by the shooting module within the shooting field of view.

[0311] Optionally, the object information includes an object name, the number of multispectral components is at least two, the shooting module further includes an image sensor, the image sensor includes at least two full-spectrum pixels, each multispectral component is respectively disposed on the incident light side of a full-spectrum pixel, at least two multispectral components form at least one multispectral component group, each multispectral component group includes at least two multispectral components, and each multispectral component in each multispectral component group transmits light in a different wavelength band; the processor 2110 is further configured to:

[0312] The energy value of the light incident on each target full-spectrum pixel is obtained, wherein the target full-spectrum pixel is a full-spectrum pixel corresponding to a multispectral component group;

[0313] Based on the energy value of the light rays of each target full-spectrum pixel, the object information corresponding to each multispectral component group is obtained;

[0314] In the original image acquired by the shooting module, an object information marker is displayed in the image area corresponding to each target full-spectrum pixel. The object information marker is used to mark the object information corresponding to the target full-spectrum pixel. The original image is an image of the object captured by the shooting module within the shooting field of view.

[0315] Optionally, the processor 2110 is also used for:

[0316] Based on the energy value of the light from each target full-spectrum pixel, the energy curve corresponding to each multispectral component group is output. The energy curve is the curve of the energy value of the light incident on the full-spectrum pixel of the multispectral component group as a function of wavelength.

[0317] From a pre-stored set of reflectance spectra, a target reflectance spectrum that matches the curve shape of each energy curve is found to obtain the target reflectance spectrum corresponding to each multispectral component group. The set of reflectance spectra includes the reflectance spectrum of at least one object and object attribute information corresponding to each reflectance spectrum.

[0318] The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the multispectral component group corresponding to the reflectance spectrum of the single target, so as to obtain the object information corresponding to each multispectral component group.

[0319] Optionally, the object information further includes object concentration information; the processor 2110 is further configured to:

[0320] Based on the energy value of the light incident on each full-spectrum pixel, an object information marker is displayed in the image region corresponding to each full-spectrum pixel in the original image. The object information marker is also used to mark the object concentration information corresponding to the full-spectrum pixel. The larger the energy value, the greater the object concentration, and different object concentrations correspond to object information markers with different marking parameters.

[0321] Optionally, the processor 2110 is also used for:

[0322] Based on the fill material corresponding to the object indicated by the object name, determine the fill material corresponding to each full-spectrum pixel;

[0323] Based on the energy value of the light incident on each full-spectrum pixel, determine the marking parameters of the fill material corresponding to each full-spectrum pixel;

[0324] According to the marking parameters of the fill material corresponding to each full-spectrum pixel, the image region corresponding to each full-spectrum pixel in the original image acquired by the shooting module is filled with the fill material corresponding to the object indicated by the object name;

[0325] The fill material varies for different objects, and the fill material includes at least one of the following: solid color fill material, texture fill material, and gradient fill material. When the fill material is a solid color fill material, the higher the energy value, the higher the color saturation of the fill material. When the fill material is a texture fill material, the higher the energy value, the higher the texture density of the fill material. When the fill material is a gradient fill material, the higher the energy value, the higher the gradient density of the fill material.

[0326] Optionally, the processor 2110 is also used for:

[0327] Obtain the geographical location of the electronic device;

[0328] If the geographical location is within a preset location, an alarm message is output. The alarm message is used to alert the user that there is a harmful object in the geographical location of the electronic device, and the user can detect the object information of the harmful object through the electronic device.

[0329] Optionally, the processor 2110 is also used for:

[0330] If the geographical location is within a preset location and the air pressure at the location of the electronic device is within the dangerous air pressure range corresponding to the preset location, an alarm message will be output.

[0331] In this embodiment, object detection is achieved by acquiring the energy values ​​of multiple wavelengths of light transmitted through the multispectral component of the imaging module, and then outputting object information of the subject within the imaging module's field of view based on the energy value of each wavelength. Since the light incident on the multispectral component is the reflected light from the subject within the imaging module's field of view, the energy values ​​of these multiple wavelengths reflect the subject's reflection of light across those wavelengths. Furthermore, all objects exhibit selective reflection of different wavelengths of light, and different objects reflect different wavelengths differently. Therefore, determining object information based on the energy values ​​of multiple wavelengths of light incident on the imaging module allows for the detection of more objects and improves the accuracy of object detection compared to related technologies.

[0332] It should be understood that, in this embodiment, the input unit 2104 may include a graphics processing unit (GPU) 21041 and a microphone 21042. The GPU 21041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 2106 may include a display panel 21061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 2107 includes at least one of a touch panel 21071 and other input devices 21072. The touch panel 21071 is also called a touch screen. The touch panel 21071 may include a touch detection device and a touch controller. Other input devices 21072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0333] The memory 2109 can be used to store software programs and various data. The memory 2109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 2109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 2109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0334] Processor 2110 may include one or more processing units; optionally, processor 2110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 2110.

[0335] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described multispectral detection method embodiments and achieve the same technical effects. To avoid repetition, these will not be described again here.

[0336] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0337] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described multispectral detection method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0338] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0339] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the multispectral detection method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0340] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0341] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0342] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A multispectral detection method, applied to a multispectral detection device, the multispectral detection device comprising an imaging module, characterized in that, The imaging module includes a multispectral component, and the method includes: Obtain the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component; Based on the energy value of light in each wavelength band, the object information of the object being photographed within the field of view of the shooting module is output; When the object information of the subject includes the object name and the number of full-spectrum pixels is multiple, the step of outputting the object information of the subject within the shooting field of view of the shooting module based on the energy value of light in each wavelength band includes: Based on the energy value of the light rays incident on each wavelength band of each full-spectrum pixel, the object information corresponding to each full-spectrum pixel is obtained; Based on the energy value of light in each band of each full-spectrum pixel, output the energy curve corresponding to each full-spectrum pixel; From the pre-stored set of reflectance spectra, find the target reflectance spectrum that matches the curve shape of each energy curve to obtain the target reflectance spectrum corresponding to each full-spectrum pixel; The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the full-spectrum pixel corresponding to the reflectance spectrum of a single target, so as to obtain the object information corresponding to each full-spectrum pixel; In the original image captured by the imaging module, an object information marker is displayed in the image region corresponding to each full-spectrum pixel. The object information marker is used to mark the object information corresponding to the full-spectrum pixel.

2. The method according to claim 1, characterized in that, The multispectral component is an FPI multispectral component, which includes a Fabry-Perot cavity mover and a Fabry-Perot cavity stator. The imaging module also includes an image sensor, which includes full-spectrum pixels. The FPI multispectral component is disposed on the light-incident side of the full-spectrum pixels. The step of obtaining the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component includes: According to the transmission distance corresponding to each band, the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator are adjusted sequentially so that the light of each band passes through the FPI multispectral component sequentially and is incident on the full-spectrum pixel. The transmission distance is the distance between the Fabry-Perot cavity mover and the Fabry-Perot cavity stator. Obtain the energy value of light incident on each band of the full-spectrum pixel.

3. The method according to claim 2, characterized in that, The object information includes the object name, and the number of full-spectrum pixels is at least two; The step of acquiring the energy value of light incident on each band of the full-spectrum pixel includes: Obtain the energy value of light incident on each wavelength of each full-spectrum pixel; The step of outputting object information of the subject within the field of view of the shooting module based on the energy value of light in each wavelength band includes: Based on the energy value of the light in each band incident on each full-spectrum pixel, the object information corresponding to each full-spectrum pixel is obtained; In the original image captured by the shooting module, an object information marker is displayed in the image region corresponding to each full-spectrum pixel. The object information marker is used to mark the object information corresponding to the full-spectrum pixel. The original image is an image of the object captured by the shooting module within the shooting field of view.

4. The method according to claim 3, characterized in that, The process of obtaining object information corresponding to each full-spectrum pixel based on the energy value of light incident on each band of the full-spectrum pixel includes: Based on the energy value of light in each band of each full-spectrum pixel, the energy curve corresponding to each full-spectrum pixel is output, and the energy curve is the curve of the energy value of light changing with wavelength; From a pre-stored set of reflectance spectra, a target reflectance spectrum that matches the curve shape of each energy curve is found to obtain the target reflectance spectrum corresponding to each full-spectrum pixel. The set of reflectance spectra includes the reflectance spectrum of at least one object and object attribute information corresponding to each reflectance spectrum. The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the full-spectrum pixel corresponding to the reflectance spectrum of the single target, so as to obtain the object information corresponding to each full-spectrum pixel.

5. The method according to claim 1, characterized in that, The object information includes an object name; the number of multispectral components is at least two; the multispectral components are spectral filters; at least two spectral filters form at least two filter groups; each filter group includes at least four spectral filters; the imaging module also includes an image sensor; the at least two filter groups are spaced apart on the pixels of the image sensor; each spectral filter in each filter group transmits light in a different wavelength band. The step of obtaining the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component includes: Obtain the energy value of light passing through each wavelength band of each filter group; The step of outputting object information of the subject within the field of view of the shooting module based on the energy value of light in each wavelength band includes: Based on the energy value of light passing through each wavelength band of each filter group, the object information corresponding to each filter group is obtained; In the original image captured by the shooting module, an object information marker is displayed in the image area corresponding to each filter group. The object information marker is used to mark the object information corresponding to the filter group. The original image is an image of the object captured by the shooting module within the shooting field of view.

6. The method according to claim 1, characterized in that, The object information includes the object name, the number of multispectral components is at least two, the shooting module also includes an image sensor, the image sensor includes at least two full-spectrum pixels, each multispectral component is respectively disposed on the light-incident side of a full-spectrum pixel, at least two multispectral components form at least one multispectral component group, each multispectral component group includes at least two multispectral components, and each multispectral component in each multispectral component group transmits light in a different wavelength band. The step of obtaining the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component includes: The energy value of the light incident on each target full-spectrum pixel is obtained, wherein the target full-spectrum pixel is a full-spectrum pixel corresponding to a multispectral component group; The step of outputting object information of the subject within the field of view of the shooting module based on the energy value of light in each wavelength band includes: Based on the energy value of the light rays from each target full-spectrum pixel, the object information corresponding to each multispectral component group is obtained; In the original image captured by the shooting module, an object information marker is displayed in the image region corresponding to each target full-spectrum pixel. The object information marker is used to mark the object information corresponding to the target full-spectrum pixel. The original image is an image of the object captured by the shooting module within the shooting field of view.

7. The method according to claim 6, characterized in that, The step of obtaining object information corresponding to each multispectral component group based on the energy value of the light from each target full-spectrum pixel includes: Based on the energy value of the light from each target full-spectrum pixel, the energy curve corresponding to each multispectral component group is output. The energy curve is the curve of the energy value of the light incident on the full-spectrum pixel of the multispectral component group as a function of wavelength. From a pre-stored set of reflectance spectra, a target reflectance spectrum that matches the curve shape of each energy curve is found to obtain the target reflectance spectrum corresponding to each multispectral component group. The set of reflectance spectra includes the reflectance spectrum of at least one object and object attribute information corresponding to each reflectance spectrum. The object attribute information corresponding to the reflectance spectrum of a single target is determined as the object information of the multispectral component group corresponding to the reflectance spectrum of the single target, so as to obtain the object information corresponding to each multispectral component group.

8. The method according to claim 3, characterized in that, The object information also includes object concentration information; The image region corresponding to each full-spectrum pixel in the original image captured by the shooting module displays object information markers, including: Based on the energy value of the light incident on each full-spectrum pixel, an object information marker is displayed in the image region corresponding to each full-spectrum pixel in the original image. The object information marker is also used to mark the object concentration information corresponding to the full-spectrum pixel. The higher the energy value, the higher the concentration of the object; different object concentrations correspond to object information tags with different tagging parameters.

9. The method according to claim 8, characterized in that, The step of displaying object information markers in the image region corresponding to each full-spectrum pixel in the original image based on the energy value of the light incident on each full-spectrum pixel includes: Based on the fill material corresponding to the object indicated by the object name, determine the fill material corresponding to each full-spectrum pixel; Based on the energy value of the light incident on each full-spectrum pixel, determine the marking parameters of the fill material corresponding to each full-spectrum pixel; According to the marking parameters of the fill material corresponding to each full-spectrum pixel, the image region corresponding to each full-spectrum pixel in the original image acquired by the shooting module is filled with the fill material corresponding to the object indicated by the object name; The fill material varies for different objects, and the fill material includes at least one of the following: solid color fill material, texture fill material, and gradient fill material. When the fill material is a solid color fill material, the higher the energy value, the higher the color saturation of the fill material. When the fill material is a texture fill material, the higher the energy value, the higher the texture density of the fill material. When the fill material is a gradient fill material, the higher the energy value, the higher the gradient density of the fill material.

10. The method according to claim 1, characterized in that, Before acquiring the energy values ​​of light transmitted through multiple wavelengths of the multispectral component, the method further includes: The location of an electronic device is obtained, wherein the electronic device includes a multispectral detection device; If the geographical location is within a preset location, an alarm message is output. The alarm message is used to alert the user that there is a harmful object in the geographical location of the electronic device, and the user can detect the object information of the harmful object through the electronic device.

11. The method according to claim 10, characterized in that, When the geographical location is at a preset location, an alarm message is output, including: If the geographical location is within a preset location and the air pressure at the location of the electronic device is within the dangerous air pressure range corresponding to the preset location, an alarm message will be output.

12. A multispectral detection device, characterized in that, The device includes: A camera module, the camera module including a multispectral component; An acquisition module is used to acquire the energy values ​​of light in multiple wavelength bands transmitted through the multispectral component; The output module is used to output object information of the subject within the field of view of the shooting module based on the energy value of light in each wavelength band. When the object information of the object being photographed includes an object name and the number of full-spectrum pixels is multiple, the output module is specifically used to obtain the object information corresponding to each full-spectrum pixel based on the energy value of the light incident on each band of each full-spectrum pixel; output the energy curve corresponding to each full-spectrum pixel based on the energy value of the light in each band of each full-spectrum pixel; search for the target reflection spectrum that matches the curve shape of each energy curve from the pre-stored set of reflection spectra to obtain the target reflection spectrum corresponding to each full-spectrum pixel; determine the object attribute information corresponding to a single target reflection spectrum as the object information of the full-spectrum pixel corresponding to the single target reflection spectrum to obtain the object information corresponding to each full-spectrum pixel; and display an object information marker in the image area corresponding to each full-spectrum pixel in the original image acquired by the shooting module, the object information marker being used to mark the object information corresponding to the full-spectrum pixel.

13. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the multispectral detection method as described in any one of claims 1 to 11.

Images