Imaging device, filter film, shape determination method, image stitching method and device

Abstract

This disclosure relates to an imaging device, a filter film, a shape determination method, an image stitching method, and an apparatus. The imaging device includes: a filter film comprising at least one filter strip and a chip target surface. The filter film is located above the chip target surface. Each filter strip has a corresponding target spectrum. The filter film is used to filter incident light to obtain an optical signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the optical signal, and to determine the spectral image under the current application scenario based on the image of each target spectral channel. The shape of each filter strip is determined according to information from the application scenario, so that the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image is a preset shape. The imaging device of this disclosure obtains the image of each target spectral channel with a preset shape by cylindrical projection transformation of the acquired spectral image, simplifying the image fusion process and improving image processing efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of imaging technology, and in particular to an imaging device, a filter film, a shape determination method, an image stitching method and device. Background Technology

[0002] With the development of spectral imaging technology, spectral imaging devices are being applied in various scenarios to obtain target images in different situations. For example, data can be acquired through spectral imaging devices, and target images, such as hyperspectral images, can be obtained through image fusion processing, such as data cube synthesis and image stitching.

[0003] However, based on the data collected by the imaging device in the relevant technology, the image fusion processing is complex and inefficient. Summary of the Invention

[0004] This disclosure presents an imaging device, a filter film, a method for determining the shape of filter strips, an image stitching method, and a device.

[0005] According to one aspect of this disclosure, an imaging apparatus is provided, comprising: a filter film including at least one filter stripe and a chip target surface, wherein the filter film is located above the chip target surface, and each filter stripe has a corresponding target spectrum.

[0006] The filter film is used to filter the incident light to obtain the optical signal of the target spectrum;

[0007] The chip target surface is used to generate images of target spectral channels based on the optical signal, and to determine the spectral image for the current application scenario based on the images of each target spectral channel.

[0008] The shape of each filter strip is determined based on the information of the application scenario, so that the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image is a preset shape.

[0009] In one possible implementation, the filter film comprises a quantum dot filter film.

[0010] In one possible implementation, the device includes: a filter film comprising a plurality of filter strips, wherein at least two filter strips have different shapes; or a plurality of filter films, wherein at least two filter films contain filter strips with different shapes.

[0011] In one possible implementation, the preset shape includes a rectangle, and the chip target surface includes a CCD image sensor or a CMOS image sensor.

[0012] According to another aspect of this disclosure, a method for determining the shape of a filter strip is provided, comprising:

[0013] Obtain information about the application scenario of the image to be acquired;

[0014] Based on the information of the application scenario, the shape of each filter strip in the imaging device is determined so that the cylindrical projection transformation is performed on the spectral image acquired by the imaging device under the application scenario, and the resulting image of each target spectral channel has a preset shape.

[0015] The imaging device includes: a filter film containing at least one filter strip and a chip target surface. The filter film is located above the chip target surface and is used to filter incident light to obtain a light signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the light signal and to determine the spectral image based on the image of each target spectral channel.

[0016] In one possible implementation, determining the shape of each filter strip in the imaging device based on information about the application scenario includes:

[0017] Based on the information of the application scenario, determine the correspondence between the coordinates of the spectral channel image in the cylindrical projection transformation coordinate system and the coordinates of the pixels on the chip target surface of the imaging device;

[0018] Based on the correspondence and the coordinates of the target spectral channel image, the coordinates of the corresponding target pixel on the chip target surface are determined, wherein the image of the target spectral channel has a preset shape;

[0019] The shape of the filter strip corresponding to the target spectrum is determined based on the coordinates of the target pixel.

[0020] In one possible implementation, the application scenario information includes at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotating axis of the broom, and the distance from the optical center of the imaging device to the rotating axis, wherein the application scenario information is determined by calibration.

[0021] In one possible implementation, the filter film comprises a quantum dot filter film.

[0022] According to another aspect of this disclosure, a filter film is provided, comprising at least one filter strip, wherein the shape of the filter strip is determined according to any of the above-described filter strip shape determination methods.

[0023] According to another aspect of this disclosure, an image stitching method is provided, comprising:

[0024] Acquire multiple frames of spectral images captured by the imaging device in the application scenario;

[0025] Perform cylindrical projection transformation on each frame of the spectral image to obtain the corresponding reference spectral image;

[0026] For two reference spectral images with relative displacement, image translation and stitching are performed based on the relative displacement to determine the stitched image.

[0027] The relative displacement is determined by feature matching of two reference spectral images, and the imaging device includes any of the imaging devices described above.

[0028] According to another aspect of this disclosure, a filter strip shape determining apparatus is provided, comprising:

[0029] The acquisition module is used to acquire information about the application scenario of the image to be acquired;

[0030] The determining module is used to determine the shape of each filter strip in the imaging device based on the information of the application scenario, so that the cylindrical projection transformation of the spectral image acquired by the imaging device under the application scenario yields an image of each target spectral channel with a preset shape.

[0031] The imaging device includes: a filter film containing at least one filter strip and a chip target surface. The filter film is located above the chip target surface and is used to filter incident light to obtain a light signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the light signal and to determine the spectral image based on the image of each target spectral channel.

[0032] In one possible implementation, the determining module is configured to:

[0033] Based on the information of the application scenario, determine the correspondence between the coordinates of the spectral channel image in the cylindrical projection transformation coordinate system and the coordinates of the pixels on the chip target surface of the imaging device;

[0034] Based on the correspondence and the coordinates of the target spectral channel image, the coordinates of the corresponding target pixel on the chip target surface are determined, wherein the image of the target spectral channel has a preset shape;

[0035] The shape of the filter strip corresponding to the target spectrum is determined based on the coordinates of the target pixel.

[0036] In one possible implementation, the application scenario information includes at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotating axis of the broom, and the distance from the optical center of the imaging device to the rotating axis, wherein the application scenario information is determined by calibration.

[0037] In one possible implementation, the filter film comprises a quantum dot filter film.

[0038] According to another aspect of this disclosure, an image stitching apparatus is provided, comprising:

[0039] The image acquisition module is used to acquire multiple frames of spectral images collected by the imaging device in the application scenario;

[0040] The image determination module is used to perform cylindrical projection transformation on each frame of spectral image to obtain the corresponding reference spectral image;

[0041] The stitching module is used to perform image translation and stitching based on the relative displacement of two reference spectral images with relative displacement, thereby determining the stitched image.

[0042] The relative displacement is determined by feature matching of two reference spectral images, and the imaging device includes any of the imaging devices described above.

[0043] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: perform the above-described filter strip shape determination method.

[0044] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the image stitching method described above.

[0045] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method for determining the shape of the filter strip.

[0046] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described image stitching method.

[0047] According to the imaging apparatus of the present disclosure, the image of each target spectral channel obtained by cylindrical projection transformation of the acquired spectral image has a preset shape, which can simplify the process of determining the target image and thus improve the processing efficiency.

[0048] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0049] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0051] Figure 1 A schematic diagram of an image acquired by a push-broom imaging device in the related art is shown.

[0052] Figure 2 This diagram illustrates an image obtained by a push-broom imaging device in the related art, after cylindrical projection transformation.

[0053] Figure 3 A schematic diagram of an imaging apparatus according to an embodiment of the present disclosure is shown.

[0054] Figure 4 A schematic diagram showing the shape of a plurality of filter strips of an imaging apparatus according to an embodiment of the present disclosure is provided.

[0055] Figure 5 A schematic diagram showing an image transformed by cylindrical projection from an image acquired by a scanning imaging apparatus according to an embodiment of the present disclosure.

[0056] Figure 6 A flowchart illustrating a method for determining the shape of a filter strip according to an embodiment of the present disclosure is shown.

[0057] Figure 7 A flowchart illustrating an image stitching method according to an embodiment of the present disclosure is shown.

[0058] Figure 8 A block diagram of a filter strip shape determination apparatus according to an embodiment of the present disclosure is shown.

[0059] Figure 9 A block diagram of an image stitching apparatus according to an embodiment of the present disclosure is shown.

[0060] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0061] Figure 11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0062] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0063] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0064] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0065] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0066] In related technologies, when acquiring images by rotating and pushing a broom through an imaging device (e.g., a spectral imaging device), the chip target surface of the imaging device may be parallel or non-parallel to the rotation axis of the rotating broom. When the chip target surface is not parallel to the rotation axis, it can be understood as performing a conical broom.

[0067] Figure 1 A schematic diagram of an image acquired by a push-broom imaging device in the related art is shown. Figure 2 This diagram illustrates an image obtained by a push-broom imaging device in the related art, after cylindrical projection transformation.

[0068] like Figure 1 , Figure 2 As shown, based on the images acquired by the conical pushbroom imaging device in related technologies, after cylindrical projection transformation, the shape of each spectral channel image becomes irregular, such as an irregular quadrilateral, resulting in a larger image area within the field of view. During image fusion processing based on the pushbroom-acquired images, such as data cube synthesis and image stitching, to determine the target image, it is necessary to determine the effective region in the image using a mask matrix and mark irrelevant invalid regions. This not only reduces the utilization rate of the chip target surface but also slows down image processing speed and reduces image processing efficiency.

[0069] Cylindrical projection transformation can be understood as establishing a cylindrical surface, for example, a cylindrical surface with the focal length of the imaging device as its radius. The image sequence data is projected onto this cylindrical surface and then transformed through an image fusion process to generate a cylindrical panoramic image.

[0070] To address the aforementioned issues, this disclosure provides an imaging device, a filter film, a method for determining the shape of filter strips, an image stitching method, and an apparatus. The imaging device uses cylindrical projection transformation to obtain images of each target spectral channel from spectral images acquired in the application scenario. These images have a preset shape, allowing for image translation and stitching. The images of the target spectral channels with the preset shapes represent the effective regions, improving the utilization rate of the chip target surface. Furthermore, it eliminates the need to determine the effective regions in the image and mark invalid regions, thereby simplifying the process of determining the target image and improving processing efficiency.

[0071] The filter strip shape determination method according to embodiments of this disclosure can determine the shape of each filter strip of an imaging device for different application scenarios, thereby improving the utilization rate of the chip target surface, simplifying the processing of the target image, and improving processing efficiency. For example, when the chip target surface of the imaging device may be parallel to the rotation axis of the rotating broom, the shape of each filter strip of the imaging device can be a regular shape, such as a rectangle. When the chip target surface of the imaging device may not be parallel to the rotation axis of the rotating broom, the shape of each filter strip of the corresponding imaging device can be determined according to the information of the application scenario.

[0072] According to the filter strip shape determination method of this disclosure, the shape of each filter strip of the imaging device in the application scenario can be determined, and a filter film can be determined based on the shape of each filter strip. The filter film includes at least one filter strip. Thus, the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image acquired by the imaging device including the filter film in the application scenario has a preset shape, allowing for image translation and stitching. The image of the target spectral channel with the preset shape is the effective area, which can improve the utilization rate of the chip target surface. Furthermore, it eliminates the need to determine the effective area in the image and mark the invalid area, thereby simplifying the process of determining the target image and improving processing efficiency.

[0073] According to the image stitching method of this disclosure, multiple frames of spectral images collected by the imaging device in the application scenario can be acquired, a cylindrical projection transformation can be performed on each frame of spectral image to obtain a corresponding reference spectral image, and two reference spectral images with relative displacement can be stitched together by image translation to improve the image stitching speed and thus improve the efficiency of determining the target image.

[0074] Figure 3 A schematic diagram of an imaging apparatus according to an embodiment of the present disclosure is shown. Figure 3 As shown, the imaging device includes: a filter film containing at least one filter stripe and a chip target surface, wherein the filter film is located above the chip target surface, and each filter stripe has a corresponding target spectrum.

[0075] The filter film is used to filter the incident light to obtain the optical signal of the target spectrum;

[0076] The chip target surface is used to generate images of target spectral channels based on the optical signal, and to determine the spectral image for the current application scenario based on the images of each target spectral channel.

[0077] The shape of each filter strip is determined based on the information of the application scenario, so that the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image is a preset shape.

[0078] According to the imaging apparatus of this disclosure, the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image acquired in the application scenario has a preset shape, which can simplify the processing of determining the target image. For example, it can simplify the processing of data cube synthesis and image stitching, thereby improving the processing efficiency of determining the target image.

[0079] It should be noted that, Figure 3 This is a schematic diagram of an imaging device. The imaging device may also include other components, such as... Figure 3 As shown, it also includes a substrate (e.g., encapsulation material), an adhesive layer, etc. The imaging device may also include... Figure 3 Other components not shown, such as base brackets, etc., are not limited in the structure and composition of the imaging device.

[0080] The incident light can be the light rays that strike the imaging device during image acquisition in the current application scenario, such as light rays reflected from the target to be detected or light rays emitted by the target to be detected.

[0081] The imaging device may include at least one filter film, and the filter film may contain at least one filter strip. For example, the imaging device may include one filter film containing multiple filter strips. The imaging device may also include multiple filter films, each filter film including at least one filter strip. For example, each filter film may include one filter strip (each filter strip is made into a separate filter film). This disclosure does not limit the number of filter films included in the imaging device or the correspondence between filter films and filter strips.

[0082] The filter film can be any filter capable of filtering light to obtain a target spectrum optical signal. For example, it can be a quantum dot filter film with good performance. The target spectrum is the target spectrum corresponding to the filter bands contained in the filter film, and the quantum dot filter film contains quantum dot filter bands. This disclosure does not impose any limitations on this.

[0083] The chip target can be any detector, image sensor, etc., capable of generating an image based on an optical signal. For example, it can be a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor. This disclosure does not limit it in this regard.

[0084] The filter film is located above the chip target surface. As long as the filter film can filter the light and the resulting optical signal can be received and processed by the chip target surface, this disclosure does not limit whether the filter film and the chip target surface are in direct contact. For example, the filter film can be in contact with the chip target surface, or it can be located above the chip target surface and at a target distance from the chip target surface, without directly contacting the chip target surface.

[0085] In one possible implementation, the filter film can be covered on the chip target surface by an adhesive layer to ensure the stability of the positional relationship between the filter film and the chip target surface in the imaging device.

[0086] The shape of each filter strip is predetermined based on the information of the current application scenario. The shape of each filter strip can be determined by making the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image acquired in the current application scenario the preset shape.

[0087] The preset shape can be any regular shape that is easy to splice, such as a rectangle.

[0088] In this way, when determining the target image based on the data collected by the imaging device, cylindrical projection transformation can be performed on each spectral image, and the resulting image of each target spectral channel is a regular shape that is easy to stitch together. Thus, the target image can be determined by image translation and stitching, which can improve processing efficiency.

[0089] As previously stated, the shape of each filter strip in the imaging apparatus of this disclosure is related to the information of the application scenario. Therefore, the shapes of the filter strips included in an imaging apparatus for acquiring images under different application scenarios can be different. For an imaging apparatus for acquiring images under the same application scenario, the shapes of the multiple filter strips included can be exactly the same, partially different, or completely different.

[0090] The shape of the filter strip can be regular. For example, to ensure that the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image acquired in the current application scenario has a preset shape, the shape of a certain filter strip is determined to be rectangular. The shape of the filter strip can also be irregular. For example, if the shape of a certain filter strip is irregular, the image of the target spectral channel corresponding to that filter strip obtained by cylindrical projection transformation of the spectral image acquired in the current application scenario can have a preset shape.

[0091] This disclosure does not limit the number of filter strips included in the imaging device, the shape of each filter strip, whether the shapes of multiple filter strips are the same, or whether the shape of each filter strip is regular.

[0092] In one possible implementation, the imaging device comprises a filter film with multiple filter strips, wherein at least two filter strips have different shapes; or comprises multiple filter films, wherein at least two filter films contain filter strips with different shapes.

[0093] In one possible implementation, the imaging device includes multiple filter strips, at least two of which correspond to different target spectra.

[0094] In one possible implementation, multiple filter strips can be arranged in an array above the chip target surface, with different filter strips not overlapping.

[0095] In one possible implementation, the chip target surface includes multiple pixels, which can be arranged in a row and column direction. The multiple pixels corresponding to a filter strip can be located in different rows and / or different columns, and different pixels in a row and / or a column of the chip target surface can also correspond to different filter strips.

[0096] Figure 4 A schematic diagram showing the shape of a plurality of filter strips of an imaging apparatus according to an embodiment of the present disclosure is provided. Figure 5 A schematic diagram showing an image transformed by cylindrical projection from an image acquired by a scanning imaging apparatus according to an embodiment of the present disclosure.

[0097] like Figure 4 As shown, the imaging device includes multiple filter strips, the shape of which is determined based on information about the application scenario. At least two filter strips have different shapes, and at least one filter strip has an irregular shape. Figure 5 As shown, including Figure 4 After cylindrical projection transformation, the images acquired by the imaging device with multiple filter strips shown are rectangular for each spectral channel.

[0098] According to the imaging apparatus of this disclosure, the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image acquired in the application scenario is of a preset shape, which can be used for image translation and stitching. The image of the target spectral channel of the preset shape is the effective area, which can improve the utilization rate of the chip target surface. Furthermore, there is no need to determine the effective area in the image or mark the invalid area, thereby simplifying the process of determining the target image and improving processing efficiency.

[0099] Figure 6 A flowchart illustrating a method for determining the shape of a filter strip according to an embodiment of the present disclosure is shown. Figure 6 As shown, the method includes:

[0100] In step S11, information about the application scenario of the image to be acquired is obtained;

[0101] In step S12, based on the information of the application scenario, the shape of each filter strip in the imaging device is determined, so that the cylindrical projection transformation is performed on the spectral image acquired by the imaging device under the application scenario, and the resulting image of each target spectral channel has a preset shape.

[0102] The imaging device includes: a filter film containing at least one filter strip and a chip target surface. The filter film is located above the chip target surface and is used to filter incident light to obtain a light signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the light signal and to determine the spectral image based on the image of each target spectral channel.

[0103] The filter strip shape determination method according to the embodiments of this disclosure can be used to determine the shape of each filter strip in an imaging device under different application scenarios, which can simplify the processing of data cube synthesis, image stitching and other processes, thereby improving processing efficiency.

[0104] The filter film includes a quantum dot filter film.

[0105] The information regarding the application scenario for acquiring the image to be captured can be determined through calibration. For example, calibration can be used to determine the device's parameter information, such as the intrinsic parameters of the imaging device and the extrinsic parameters related to the acquired image in the application scenario. The intrinsic parameters may include the focal length and principal point coordinates of the imaging device, while the extrinsic parameters may include the angle between the chip target surface and the rotation axis of the rotating broom, the distance from the optical center of the imaging device to the rotation axis, etc. This disclosure does not limit the method of determining the application scenario information, nor the type and content of the application scenario information.

[0106] In one possible implementation, the information of the application scenario may include at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotation axis of the rotating push-broom, and the distance from the optical center of the imaging device to the rotation axis.

[0107] Determining the shape of each filter strip in the imaging device based on the application scenario information can be achieved by determining the correspondence between the coordinates of the target spectral channel image of each preset shape and the coordinates of the corresponding target pixel on the chip target surface under a cylindrical projection transformation coordinate system. For example, the correspondence between the pixel coordinates on the chip target surface and the image of the spectral channel under the cylindrical projection transformation coordinate system can be determined, and this correspondence can be inversely transformed to determine the correspondence between the coordinates of the target spectral channel image of the preset shape and the coordinates of the corresponding target pixel on the chip target surface. In this way, the coordinates of the target pixel corresponding to each filter strip on the chip target surface can be determined, and the shape of the corresponding filter strip can be determined based on the coordinates of the target pixel.

[0108] Based on the information of the application scenario, the shape of each filter strip in the imaging device can be determined. Alternatively, the correspondence between the coordinates of the pixels on the chip target surface and the image of the spectral channel under the cylindrical projection transformation coordinate system can be determined. Based on the correspondence and the coordinates of the image of the target spectral channel with the preset shape, the coordinates of the corresponding target pixels on the chip target surface can be determined. Based on the coordinates of the target pixels, the shape of the filter strip corresponding to the target spectrum can be determined.

[0109] The shape of each filter strip can vary depending on the application scenario. This disclosure does not limit the method by which the shape of each filter strip in the imaging device is determined based on the information from the aforementioned application scenario.

[0110] In one possible implementation, determining the shape of each filter strip in the imaging device based on information about the application scenario may include:

[0111] Based on the information of the application scenario, determine the correspondence between the coordinates of the spectral channel image in the cylindrical projection transformation coordinate system and the coordinates of the pixels on the chip target surface of the imaging device;

[0112] Based on the correspondence and the coordinates of the target spectral channel image, the coordinates of the corresponding target pixel on the chip target surface are determined, wherein the image of the target spectral channel has a preset shape;

[0113] The shape of the filter strip corresponding to the target spectrum is determined based on the coordinates of the target pixel.

[0114] Based on the application scenario information, the correspondence between the coordinates of the spectral image acquired by the imaging device and the coordinates of the image of each spectral channel under the cylindrical projection transformation coordinate system can be determined. The coordinates of the acquired spectral image correspond to the coordinates of pixels on the chip target surface. Based on this correspondence and the coordinates of the image of each target spectral channel with a preset shape, the coordinates of the corresponding target pixel on the chip target surface can be determined. Based on the coordinates of the target pixel, the shape of the filter strip corresponding to the target spectrum can be determined; for example, the filter strip may cover the target pixel on the chip target surface.

[0115] In this way, the shape of each filter strip in the imaging device can be quickly determined.

[0116] This disclosure also provides a filter film comprising at least one filter strip, wherein the shape of the filter strip is determined according to any one of the filter strip shape determination methods described above.

[0117] The filter film may include filter strips corresponding to each target spectrum, and the shape of each filter strip is determined by the filter strip shape determination method described in any one of the above descriptions. The filter film may also include at least one of the filter strips, and multiple filter films including all the filter strips may be spliced ​​together to form an imaging device.

[0118] In this way, the image of each target spectral channel obtained by the cylindrical projection transformation of the spectral image acquired by the imaging device including the filter film in the application scenario has a preset shape, which can be used for image translation and stitching. The image of the target spectral channel with the preset shape is the effective area, which can improve the utilization rate of the chip target surface. Moreover, there is no need to determine the effective area in the image or mark the invalid area, thereby simplifying the process of determining the target image and improving processing efficiency.

[0119] Figure 7 A flowchart illustrating an image stitching method according to an embodiment of the present disclosure is shown, such as... Figure 7 As shown, the method includes:

[0120] In step S21, multiple frames of spectral images are acquired by the imaging device in the application scenario;

[0121] In step S22, cylindrical projection transformation is performed on each frame of spectral image to obtain the corresponding reference spectral image;

[0122] In step S23, for two reference spectral images with relative displacement, image translation and stitching are performed based on the relative displacement to determine the stitched image.

[0123] The relative displacement is determined by feature matching of two reference spectral images, and the imaging device includes the aforementioned imaging device.

[0124] According to the image stitching method of this disclosure, multiple frames of spectral images collected by the imaging device in the application scenario can be acquired, a cylindrical projection transformation can be performed on each frame of spectral image to obtain a corresponding reference spectral image, and two reference spectral images with relative displacement can be stitched together by image translation to improve the image stitching speed and thus improve the efficiency of determining the target image.

[0125] For example, multiple frames of spectral images acquired by the imaging device in the application scenario can be obtained. Each frame of spectral image can be transformed by cylindrical projection to obtain a corresponding reference spectral image. In the reference spectral image, the image of each target spectral channel has a preset shape, such as a rectangle.

[0126] Feature matching can be performed on any two reference spectral images to identify two reference spectral images with relative displacement. Based on this relative displacement, image translation and stitching can be performed to determine the stitched image.

[0127] This method can improve the efficiency of image stitching.

[0128] Figure 8 A block diagram of a filter strip shape determination apparatus according to an embodiment of the present disclosure is shown, such as Figure 8 As shown, the device includes:

[0129] The acquisition module 31 is used to acquire information about the application scenario of the image to be acquired;

[0130] The determining module 32 is used to determine the shape of each filter strip in the imaging device based on the information of the application scenario, so that the cylindrical projection transformation of the spectral image acquired by the imaging device under the application scenario yields an image of each target spectral channel with a preset shape.

[0131] The imaging device includes: a filter film containing at least one filter strip and a chip target surface. The filter film is located above the chip target surface and is used to filter incident light to obtain a light signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the light signal and to determine the spectral image based on the image of each target spectral channel.

[0132] In one possible implementation, the determining module is configured to:

[0133] Based on the information of the application scenario, determine the correspondence between the coordinates of the spectral channel image in the cylindrical projection transformation coordinate system and the coordinates of the pixels on the chip target surface of the imaging device;

[0134] Based on the correspondence and the coordinates of the target spectral channel image, the coordinates of the corresponding target pixel on the chip target surface are determined, wherein the image of the target spectral channel has a preset shape;

[0135] The shape of the filter strip corresponding to the target spectrum is determined based on the coordinates of the target pixel.

[0136] In one possible implementation, the application scenario information includes at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotating axis of the broom, and the distance from the optical center of the imaging device to the rotating axis, wherein the application scenario information is determined by calibration.

[0137] In one possible implementation, the filter film comprises a quantum dot filter film.

[0138] Figure 9 A block diagram of an image stitching apparatus according to an embodiment of the present disclosure is shown, such as Figure 9 As shown, the device includes:

[0139] Image acquisition module 41 is used to acquire multiple frames of spectral images collected by the imaging device in the application scenario;

[0140] The image determination module 42 is used to perform cylindrical projection transformation on each frame of spectral image to obtain the corresponding reference spectral image;

[0141] The stitching module 43 is used to perform image translation and stitching based on the relative displacement of two reference spectral images with relative displacement, and to determine the stitched image.

[0142] The relative displacement is determined by feature matching of two reference spectral images, and the imaging device includes the imaging device described above.

[0143] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0144] In addition, this disclosure also provides a filter strip shape determination device, electronic device, computer-readable storage medium, and program, all of which can be used to implement any of the filter strip shape determination methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section, and will not be repeated here.

[0145] In addition, this disclosure also provides an image stitching apparatus, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any of the image stitching methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0146] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0147] In some embodiments, the apparatus provided in this disclosure may have functions or include modules that can be used to perform the methods described in the above method embodiments. Specific implementations can be referred to the descriptions in the above method embodiments, and for brevity, will not be repeated here.

[0148] This disclosure also proposes a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the aforementioned filter strip shape determination method or image stitching method. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0149] This disclosure also proposes an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured for the above-described filter strip shape determination method or image stitching method.

[0150] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0151] Figure 10 This diagram illustrates a block diagram of an electronic device according to an embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0152] Reference Figure 10 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0153] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0154] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0155] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0156] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0157] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0158] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0159] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0160] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0161] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0162] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by a processor 820 of an electronic device 800 to perform the above-described method.

[0163] Figure 11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. For example, electronic device 1900 may be provided as a server. (Refer to...) Figure 11 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0164] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0165] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0166] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0167] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0168] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0169] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0170] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0171] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0172] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0174] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An imaging device, characterized in that, include: A filter film comprising at least one filter stripe and a chip target surface, wherein the filter film is located above the chip target surface, and each filter stripe has a corresponding target spectrum. The filter film is used to filter the incident light to obtain the optical signal of the target spectrum; The chip target surface is used to generate images of target spectral channels based on the optical signal, and to determine the spectral image for the current application scenario based on the images of each target spectral channel. The shape of each filter strip is determined based on the information of the application scenario, so that the image of each target spectral channel obtained by cylindrical projection transformation of the spectral image is a preset shape. The method for determining the shape of each filter strip includes: determining the correspondence between the coordinates of the pixels on the chip target surface and the image of the spectral channel under the cylindrical projection transformation coordinate system based on the information of the application scenario, and performing an inverse transformation on the correspondence to obtain the correspondence between the coordinates of the image of the target spectral channel of each preset shape and the coordinates of the corresponding target pixel on the chip target surface under the cylindrical projection transformation coordinate system; determining the coordinates of the target pixel corresponding to each filter strip on the chip target surface based on the correspondence between the coordinates of the image of the target spectral channel of each preset shape and the coordinates of the corresponding target pixel on the chip target surface; and determining the shape of the corresponding filter strip based on the coordinates of the target pixel. The application scenario information includes at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotating axis of the broom, and the distance from the optical center of the imaging device to the rotating axis. The application scenario information is determined through calibration.

2. The apparatus according to claim 1, characterized in that, The filter film includes a quantum dot filter film.

3. The apparatus according to claim 1, characterized in that, The device includes: a filter film comprising a plurality of filter strips, wherein at least two filter strips have different shapes; or a filter film comprising a plurality of filter films, wherein at least two filter films contain filter strips with different shapes.

4. The apparatus according to claim 1, characterized in that, The preset shape includes a rectangle, and the chip target surface includes a CCD image sensor or a CMOS image sensor.

5. A method for determining the shape of a filter strip, characterized in that, include: Obtain information about the application scenario of the image to be acquired; Based on the application scenario information, the shape of each filter strip in the imaging device is determined so that the spectral image acquired by the imaging device under the application scenario is transformed by cylindrical projection, and the image of each target spectral channel is obtained as a preset shape. Specifically, this includes: based on the application scenario information, determining the correspondence between the coordinates of pixels on the chip target surface and the image of the spectral channel under the cylindrical projection transformation coordinate system, and performing an inverse transformation on the correspondence to obtain the correspondence between the coordinates of the image of each preset spectral channel and the coordinates of the corresponding target pixel on the chip target surface under the cylindrical projection transformation coordinate system; based on the correspondence between the coordinates of the image of each preset spectral channel and the coordinates of the corresponding target pixel on the chip target surface, determining the coordinates of the target pixel corresponding to each filter strip on the chip target surface; and based on the coordinates of the target pixel, determining the shape of the corresponding filter strip. The imaging device includes: a filter film containing at least one filter strip and a chip target surface. The filter film is located above the chip target surface and is used to filter incident light to obtain a light signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the light signal and to determine the spectral image based on the image of each target spectral channel. The application scenario information includes at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotating axis of the broom, and the distance from the optical center of the imaging device to the rotating axis. The application scenario information is determined through calibration.

6. The method according to claim 5, characterized in that, Determining the shape of each filter strip in the imaging device based on the information of the application scenario includes: Based on the information of the application scenario, determine the correspondence between the coordinates of the spectral channel image in the cylindrical projection transformation coordinate system and the coordinates of the pixels on the chip target surface of the imaging device; Based on the correspondence and the coordinates of the target spectral channel image, the coordinates of the corresponding target pixel on the chip target surface are determined, wherein the image of the target spectral channel has a preset shape; The shape of the filter strip corresponding to the target spectrum is determined based on the coordinates of the target pixel.

7. The method according to claim 5, characterized in that, The filter film includes a quantum dot filter film.

8. A filter film, characterized in that, It includes at least one filter strip, wherein the shape of the filter strip is determined by the filter strip shape determination method according to any one of claims 5 to 7.

9. An image stitching method, characterized in that, include: Acquire multiple frames of spectral images captured by the imaging device in the application scenario; Perform cylindrical projection transformation on each frame of the spectral image to obtain the corresponding reference spectral image; For two reference spectral images with relative displacement, image translation and stitching are performed based on the relative displacement to determine the stitched image. The relative displacement is determined by feature matching of two reference spectral images, and the imaging device includes the imaging device according to any one of claims 1 to 4.

10. A filter strip shape determining device, characterized in that, include: The acquisition module is used to acquire information about the application scenario of the image to be acquired; The determining module is used to determine the shape of each filter strip in the imaging device based on the information of the application scenario, so that the image of each target spectral channel obtained by the imaging device under the application scenario through cylindrical projection transformation is a preset shape. Specifically, it includes: determining the correspondence between the coordinates of the pixels on the chip target surface and the image of the spectral channel under the cylindrical projection transformation coordinate system based on the information of the application scenario, and performing an inverse transformation on the correspondence to obtain the correspondence between the coordinates of the image of each preset spectral channel and the coordinates of the corresponding target pixel on the chip target surface under the cylindrical projection transformation coordinate system; determining the coordinates of the target pixel corresponding to each filter strip on the chip target surface based on the correspondence between the coordinates of the image of each preset spectral channel and the coordinates of the corresponding target pixel on the chip target surface; and determining the shape of the corresponding filter strip based on the coordinates of the target pixel. The imaging device includes: a filter film containing at least one filter strip and a chip target surface. The filter film is located above the chip target surface and is used to filter incident light to obtain a light signal of the target spectrum. The chip target surface is used to generate an image of the target spectral channel based on the light signal and to determine the spectral image based on the image of each target spectral channel. The application scenario information includes at least one of the following: the focal length of the imaging device, the principal point coordinates, the angle between the chip target surface and the rotating axis of the broom, and the distance from the optical center of the imaging device to the rotating axis. The application scenario information is determined through calibration.

11. An image stitching device, characterized in that, include: The image acquisition module is used to acquire multiple frames of spectral images collected by the imaging device in the application scenario; The image determination module is used to perform cylindrical projection transformation on each frame of spectral image to obtain the corresponding reference spectral image; The stitching module is used to perform image translation and stitching based on the relative displacement of two reference spectral images with relative displacement, thereby determining the stitched image. The relative displacement is determined by feature matching of two reference spectral images, and the imaging device includes the imaging device according to any one of claims 1 to 4.

12. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method described in any one of claims 5 to 7 or claim 9.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 5 to 7 or claim 9.

Images