Optical computing device

Abstract

The present invention achieves an optical computing device (1) capable of using optical computing to identify the class to which a target belongs and the position of the target in an image.　This optical computing device (1) comprises a first lens (11) which Fourier transforms a signal light, at least one optical diffraction element (13), and a second lens (12) which re-Fourier transforms the Fourier transformed signal light. The optical diffraction element (13) is designed such that: (a) when an input image IIN representing the signal light input into the first lens (11) includes N (N is an arbitrary natural number of 1 or more) targets P1, P2, ..., pN as objects, an output image IOUT representing the signal light output by the second lens (12) includes a marker qj corresponding to each target pj; and (b) for each marker qj included in the output image IOUT, (b1) a region Ai (j) which includes the marker qj among M predetermined regions A1, A2, ..., AM represents a class Ci(j) to which the target pj corresponding to the marker qj belongs among M predetermined classes C1, C2, ..., CM, and (b2) the position (xj, yj) of the marker qj in the region Ai(j) represents the position (Xj, Yj) of the target pj corresponding to the marker qj in the input image IIN.

Description

optical calculation device

[0001] This invention relates to an optical computing device.

[0002] The process of detecting or classifying objects included as subjects in an image is widely performed. Such processing is usually implemented as electrical calculations, but in order to perform the processing at a higher speed, implementation as optical calculations is being considered. For example, Non-Patent Document 1 discloses a technique for detecting objects included as subjects in an image using an optical diffraction element placed near the Fourier plane of a 4f optical system.

[0003] Tao Yan, et.al., "Fourier-space Diffractive Deep Neural Network", PHYSICAL REVIEW LETTERS 123, 023901 (2019)

[0004] However, images can contain subjects belonging to various classes. In such images, it is necessary to identify which class of subject is located where. However, the technology described in Non-Patent Document 1 can only identify the location of an object belonging to a specific class and cannot meet this requirement.

[0005] One aspect of the present invention has been made in view of the above-mentioned problems, and one of its objectives is to realize an optical computing device capable of determining, by optical calculation, which class of object is located where in an image.

[0006] An optical computing device according to one aspect of the present invention comprises a first lens that performs a Fourier transform on signal light, a second lens that performs a re-Fourier transform on the signal light that has been Fourier transformed by the first lens, and at least one optical diffraction element that acts on the signal after it has been Fourier transformed by the first lens and before it has been re-Fourier transformed by the second lens, wherein the optical diffraction element (a) an input image I represented by the signal light input to the first lens ＩＮ However, there are N objects p (where N is any natural number greater than or equal to 1). １ , p ２ , ..., p ＮWhen including the subject, the output image I representing the signal light output from the second lens ＯＵＴ has each target p ｊ corresponding mark q ｊ and (b) the output image I ＯＵＴ each mark q included in ｊ Regarding each mark q in, (b1) M predetermined regions A １ , A ２ , …, A Ｍ Among them, the region A including the mark q ｊ represents the class C ｉ（ｊ） to which the target p １ , C ２ , …, C Ｍ Among them, the target p corresponding to the mark q ｊ belongs, and (b2) the position (x ｊ ) of the mark q in the region A ｉ（ｊ） is the position (X ｉ（ｊ） ) of the mark q in the input image I ｊ corresponding to the target p ｊ , y ｊ ) is designed to represent (Y ＩＮ ). ｊ corresponding to the target p ｊ . ｊ , ｊ

[0007] According to one aspect of the present invention, it is possible to realize an optical arithmetic device that can specify the position of a target belonging to any class in an image by optical arithmetic.

[0008] ​This is a block diagram showing the configuration of an optical computing device according to one embodiment of the present invention. It shows the input and output images used in the optical computing device shown in Figure 1. (a) is a plan view showing the input image, (b) is a plan view showing the output image, and (c) to (f) are enlarged plan views of the regions corresponding to each class in the output image. This shows the configuration of the optical diffraction element in the optical computing device shown in Figure 1. (a) is a plan view of the optical diffraction element, and (b) is an enlarged perspective view of a part of the optical diffraction element. This is a block diagram showing a first modified example of the optical computing device shown in Figure 1. This is a block diagram showing a second modified example of the optical computing device shown in Figure 1. This is a block diagram showing a third modified example of the optical computing device shown in Figure 1. This is a flowchart showing the flow of the first post-processing performed by the processor in the optical computing device shown in Figure 1. (a) is a schematic plan view showing a specific example of the output image referenced in the post-processing shown in Figure 7, and (b) is a schematic plan view showing a specific example of the composite image generated in the post-processing shown in Figure 7. This is a flowchart showing the flow of the second post-processing performed by the processor in the optical computing device shown in Figure 1. (a) is a schematic diagram showing a specific example of an input image, and (b) is a schematic plan view showing a specific example of an output image corresponding to the input image shown in (a), which is referenced in the post-processing shown in Figure 9.

[0009] (Configuration of the Optical Computing Device) The configuration of the optical computing device 1 according to one embodiment of the present invention will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the optical computing device 1.

[0010] As shown in Figure 1, the optical computing device 1 includes a first lens 11, a second lens 12, and an optical diffraction element 13. In Figure 1, f is the focal length of the first lens 11, and f' is the focal length of the second lens.

[0011] The first lens 11 is configured to perform a Fourier transform on the signal light L. The second lens 12 is configured to perform a re-Fourier transform on the signal light L that has been Fourier transformed by the first lens 11. In this embodiment, the first lens 11 and the second lens 12 are optical lenses (convex lenses) that have positive power (light-gathering effect).

[0012] In this embodiment, the first lens 11 and the second lens 12 are arranged parallel to each other such that the optical axis of the first lens 11 and the optical axis of the second lens 12 coincide, and the front focus of the first lens 11 and the rear focus of the second lens 12 coincide. The optical system consisting of the first lens 11 and the second lens 12 arranged in this manner is called a 4f optical system. In a 4f optical system, an input plane P1, a Fourier plane P2, and an output plane P3 are defined. Here, the input plane P1 refers to a plane perpendicular to the optical axis of the first lens 11 at the rear focus of the first lens 11. The output plane P3 refers to a plane perpendicular to the optical axis of the second lens 12 at the front focus of the second lens 12. The Fourier plane P2 refers to a plane perpendicular to the optical axis of the first lens at the front focus of the first lens 11, that is, a plane perpendicular to the optical axis of the second lens 12 at the rear focus of the second lens 12.

[0013] For example, a spatial light modulator 14 is placed on the input surface P1. In this case, the first lens 11 is subjected to spatial intensity modulation or spatial phase modulation using the spatial light modulator 14 to input image I ＩＮ A signal light L superimposed on the spatial light modulator 14 is input. Alternatively, an object may be placed on the input surface P1 instead of the spatial light modulator 14, and the light transmitted through the object, or the light reflected or scattered by the object, may be input to the first lens 11 as the signal light L. On the other hand, an image sensor 15 may be placed on the output surface P3. In this case, the image sensor 15 receives the output image I represented by the signal light L output from the second lens 12. ＯＵＴ It detects the optical diffraction element 13 if it is not placed between the first lens 11 and the second lens 12. ＯＵＴ This is input image I ＩＮ This results in an image rotated 180°. In other words, the action of the second lens 12 is a combination of the inverse Fourier transform and a 180° rotation.

[0014] The optical diffraction element 13 processes the input image I ＩＮ The signal light L representing the output image I ＯＵＴ This configuration is for converting the input image I into a signal light L. ＩＮ Output image I ＯＵＴIt can also be said that this configuration is for performing optical calculations equivalent to the electrical calculations that derive the result. The optical diffraction element 13 is positioned between the first lens 11 and the second lens 12 so as to act on the signal L after it has been Fourier transformed by the first lens 11 and before it has been re-Fourier transformed by the second lens 12.

[0015] The optical diffraction element 13 is preferably positioned near the Fourier plane P2 so as to be parallel to the first lens 11 and the second lens 12, and more preferably positioned on the Fourier plane P2. Here, positioning the optical diffraction element 13 near the Fourier plane P2 means that the optical diffraction element 13 is positioned so as to intersect with a section on the optical axis of the first lens 11 where the distance from the principal point of the first lens 11 is between (3 / 4) × f and (5 / 4) × f. This allows the input image I ＩＮ The optical diffraction element 13 can be applied to the signal light L representing the spatial frequency image, which is obtained by performing a Fourier transform on the signal light L representing the spatial frequency image.

[0016] The function and configuration of the optical diffraction element 13 will be described later, with reference to a different diagram.

[0017] (Function of the optical diffraction element) The optical computing device 1 processes the target into M Class C １ , C ２ , ..., C Ｍ It is used to solve classification problems that require classifying into one of the following categories. Here, M is any natural number greater than or equal to 2. Class C １ , C ２ , ..., C Ｍ This can be any visually identifiable class, such as the type of object (an example of an object), the presence or absence of defects in the object (an example of an object type), the type of defects contained in the object (an example of an object type), or the type of defect (an example of an object).

[0018] The optical diffraction element 13 (a) the input image I represented by the signal light L input to the first lens 11 ＩＮ However, there are N objects p １ , p ２ , ..., p ＮWhen the subject includes the second lens 12, the output image I represents the signal light L output from the second lens 12. ＯＵＴ However, each target p ｊ The corresponding mark q ｊ (b) Output image I ＯＵＴ Each mark q included ｊ The following conditions (b1) and (b2) are satisfied for the design. Here, N is any natural number greater than or equal to 1, and j is each natural number between 1 and N (inclusive).

[0019] Condition (b1): M pre-set regions A １ , A ２ , ..., A Ｍ Among them, Mark Q ｊ Region A including ｉ（ｊ） However, there are M pre-set Class C １ , C ２ , ..., C Ｍ Among them, Mark Q ｊ The corresponding target p ｊ Class C to which it belongs ｉ（ｊ） This represents the expression where i(j) is any natural number between 1 and M (inclusive).

[0020] Condition (b2): Area A ｉ（ｊ） Mark q in ｊ Position (x ｊ , y ｊ ) is input image I ＩＮ Mark q in ｊ The corresponding target p ｊ Position (X ｊ , Y ｊ ) represents.

[0021] Note: Mark Q １ ,q ２ , ..., q Ｎ This is the output image I ＯＵＴ In mark q １ ,q ２ , ..., q Ｎ Any object that can be distinguished from the other regions is acceptable. For example, output image I ＯＵＴ If we consider this as a two-dimensional luminance distribution, the luminance peaks whose peak value is above a predetermined threshold are marked q. １ ,q ２ , ..., q Ｎcan be used. For each target p ｊ The peak corresponding to p １ , p ２ , …, p Ｎ becomes lower as the number N of them increases. Therefore, it is preferable to set the above threshold according to the number of the targets p １ , p ２ , …, p Ｎ .

[0022] An example of the input image I in (a) of FIG. 2 ＩＮ is shown. The input image I shown in (a) of FIG. 2 ＩＮ includes four targets p １ , p ２ , p ３ , p ４ as subjects.

[0023] An example of the output image I in (b) of FIG. 2 ＯＵＴ is shown. The output image I shown in (b) of FIG. 2 ＯＵＴ includes four marks q １ , q ２ , q ３ , q ４ corresponding to the four targets p １ , q ２ , q ３ , q ４ . Also, in the output image shown in (b) of FIG. 2, there are four regions A １ = battery, C ２ = resistor, C ３ = IC, C ４ = nut corresponding to the four targets p １ , A ２ , A​​​​​​​​​​​​​​​​​​​​​１ Class C １ This indicates that it belongs to the category, i.e., it is a battery. Figure 2(c) shows the output image I ＯＵＴ Region A １ Input image I ＩＮ This is an enlarged view scaled to the same size. Comparing Figure 2(a) and Figure 2(c), region A １ Mark q in １ Position (x １ , y １ ) is input image I ＩＮ The position of object p1 in X １ , Y １ It can be seen that this matches the input image I. ＩＮ At position (X １ , Y １ ) the subject p １ Class C １ It belongs to (is a battery) as shown in output image I ＯＵＴ It can be easily identified from this.

[0025] Also, the output image I shown in Figure 2(b) ＯＵＴ In this case, target p ２ The corresponding mark q ２ Class C ２ Corresponding region A ２ It is included in the output image I shown in Figure 2(b). ＯＵＴ is the target p ２ Class C ２ This indicates that it belongs to the category of resistors. Figure 2(d) shows the output image I ＯＵＴ Region A ２ Input image I ＩＮ This is an enlarged view scaled to the same size. Comparing Figure 2(a) and Figure 2(d), region A ２ Mark q in ２ Position (x ２ , y ２ ) is input image I ＩＮ In the object p ２ Position (X ２ , Y ２ It can be seen that this matches the input image I. ＩＮ At position (X ２, Y ２ ) the subject p ２ Class C ２ The output image I indicates that it belongs to (is a resistor). ＯＵＴ It can be easily identified from this.

[0026] Also, the output image I shown in Figure 2(b) ＯＵＴ In this case, target p ３ The corresponding mark q ３ Class C ４ Corresponding region A ４ It is included in the output image I shown in Figure 2(b). ＯＵＴ is the target p ３ Class C ４ This indicates that it belongs to the category, i.e., it is a nut. Figure 2(f) shows the output image I ＯＵＴ Region A ４ Input image I ＩＮ This is an enlarged view scaled to the same size. Comparing Figure 2(a) and Figure 2(f), region A ４ Mark q in ３ Position (x ３ , y ３ ) is input image I ＩＮ In the object p ３ Position (X ３ , Y ３ It can be seen that this matches the input image I. ＩＮ At position (X ３ , Y ３ ) the subject p ３ Class C ４ It belongs to (is a nut) as shown in output image I ＯＵＴ It can be easily identified from this.

[0027] Also, the output image I shown in Figure 2(b) ＯＵＴ In this case, target p ４ The corresponding mark q ４ Class C ３ Corresponding region A ３ It is included in the output image I shown in Figure 2(b). ＯＵＴ is the target p ４ Class C ３This indicates that it belongs to the category, i.e., it is an IC. Figure 2(e) shows the output image I ＯＵＴ Region A ３ Input image I ＩＮ This is an enlarged view scaled to the same size. Comparing Figure 2(a) and Figure 2(e), region A ３ Mark q in ４ Position (x ４ , y ４ ) is input image I ＩＮ In the object p ４ Position (X ４ , Y ４ It can be seen that this matches the input image I. ＩＮ At position (X ４ , Y ４ ) the subject p ４ Class C ３ It belongs to (is an IC), as shown in output image I ＯＵＴ It can be easily identified from this.

[0028] Note that there are M Class C １ , C ２ ..., C Ｍ Output image I corresponds to the region ＯＵＴ M regions A set above １ , A ２ , ..., A Ｍ These are, respectively, input image I ＩＮ It is preferable that the region is a similarly scaled-down region A. ｉ（ｊ） Mark q in ｊ Position (x ｊ , y ｊ ) From input image I ＩＮ Mark q in ｊ The corresponding target p ｊ Position (X ｊ , Y ｊ This makes it easier to identify the cause.

[0029] For example, Class C １ , C ２ ..., C Ｍ If the number of items is 4 or less, output image I ＯＵＴ The four regions A obtained by dividing the area into 2 x 2 = 4 equal parts. １ , A ２ , A３ , A ４ All or part of Class C ｉ It would be good to use this as the corresponding area. １ , C ２ ..., C Ｍ If the number of items is between 5 and 9, output image I ＯＵＴ The nine regions A obtained by dividing the area into 3 x 3 = 9 equal parts. １ , A ２ , ..., A ９ All or part of Class C ｉ It would be good to use this as the corresponding area. １ , C ２ ..., C Ｍ If the number of items is between 10 and 16, output image I ＯＵＴ Dividing the area into 4 x 4 = 16 equal parts gives us 16 regions A. １ , A ２ , ..., A １６ All or part of Class C ｉ It would be good to use this as the corresponding area.

[0030] (Configuration of the optical diffraction element) The configuration of the optical diffraction element 13 will be explained with reference to Figure 3. Figure 3(a) is a plan view of the optical diffraction element 13. Figure 3(b) is an enlarged perspective view of a part of the optical diffraction element 13 (the part enclosed by the dotted line in Figure 3(a)).

[0031] The optical diffraction element 13 is a transmissive microcell array composed of multiple microcells whose phase modulation amounts can be set independently of each other. The optical diffraction element 13 performs a predetermined optical calculation (transformation of a two-dimensional intensity distribution according to a predetermined transformation rule) by interfering with signal light L of different phases that have passed through each microcell. The signal light L that has passed through the optical diffraction element 13 represents the result of the optical calculation performed by the optical diffraction element 13.

[0032] In this specification, "microcell" refers to a cell with a size of less than 10 μm, for example. Furthermore, in this specification, "cell size" refers to the square root of the cell's area. For example, if the planar shape of a microcell is square, the cell size is the length of one side of the cell. The lower limit of the cell size is, for example, 1 nm.

[0033] The optical diffraction element 13 illustrated in Figure 3 is composed of 200 × 200 microcells arranged in a matrix. The planar shape of each microcell is a 500 nm × 500 nm square, and the planar shape of the optical diffraction element 13 is a 100 μm × 100 μm square.

[0034] (1) By independently setting the thickness of each microcell, or (2) by independently setting the refractive index of each microcell, the amount of phase change of light transmitted through each microcell can be independently set for each cell. In this embodiment, method (1), which can be realized by nanoimprint, is employed. In this case, each microcell is composed of a rectangular pillar having a square base with sides equal in length to the cell size, as shown in Figure 3(b). In this case, the amount of phase change of light transmitted through each microcell is determined according to the height of the pillar that constitutes that microcell. That is, the amount of phase change of light transmitted through a microcell composed of tall pillars is large, and the amount of phase change of light transmitted through a microcell composed of short pillars is small.

[0035] The thickness or refractive index of each microcell in the optical diffraction element 13 can be set, for example, using machine learning. The model used in this machine learning is, for example, a model that takes the two-dimensional intensity distribution at the input surface P1 of the signal light L as input and outputs the two-dimensional intensity distribution at the output surface P3 of the signal light L as output. This model includes the thickness or refractive index of each microcell in the optical diffraction element 13 as a parameter.

[0036] Alternatively, the optical diffraction layer formed in the dried gel may be used as the optical diffraction element 13. In this case, it is preferable to produce the optical diffraction element 13 by forming an optical diffraction layer in a swelling gel that shrinks while maintaining a similar shape by dehydration shrinkage, for example, a swelling gel used in the Implosion Fabrication method, and then drying the swelling gel. This makes it possible to realize an optical diffraction element 13 with high precision.

[0037] Alternatively, a microcell array having a cell structure similar to that of an MRAM (Magnetoresistive Random Access Memory) using the STT (Spin Transfer Torque) or SOT (Spin Orbit Torque) method may be used as the microcells constituting the optical diffraction element 13. Alternatively, a microcell array having a similar cell structure to that of an LCD (Liquid Crystal Display) may be used as the optical diffraction element 13. In these cases, the amount of phase modulation in each microcell constituting the optical diffraction element 13 can be electrically set.

[0038] (First Modified Example of Optical Computing Device) A first modified example of the optical computing device 1 (hereinafter referred to as "optical computing device 1A") will be described with reference to Figure 4. Figure 4 is a block diagram showing the configuration of the optical computing device 1A.

[0039] While the optical computing device 1 shown in Figure 1 is equipped with a single optical diffraction element 13, the optical computing device 1A shown in Figure 4 is equipped with multiple optical diffraction elements 13 (three in the illustrated example) arranged parallel to each other. All of these multiple optical diffraction elements 13 are arranged to intersect with a section on the optical axis of the first lens 11 where the distance from the principal point of the first lens 11 is between (3 / 4) × f and (5 / 4) × f.

[0040] In the optical computing device 1A shown in Figure 4, optical calculations are performed on the signal light L, which has been Fourier transformed by the first lens 11, by the first optical diffraction element 13, by the second optical diffraction element 13, and by the third optical diffraction element 13, in that order. Therefore, the optical computing device 1A shown in Figure 4 has the effect of being able to perform complex optical calculations.

[0041] (Second Modification of the Optical Computing Device) A second modification of the optical computing device 1 (hereinafter referred to as "optical computing device 1B") will be described with reference to Figure 5. Figure 5 is a block diagram showing the configuration of optical computing device 1B.

[0042] In the optical computing device 1 shown in Figure 1, optical lenses are used as the first lens 11 and the second lens 12, whereas in the optical computing device 1B shown in Figure 5, a transmission-type microcell array is used as the first lens 11 and the second lens 12.

[0043] The transmissive microcell arrays used as the first lens 11 and the second lens 12 are composed of multiple microcells whose phase modulation amounts can be set independently of each other. By interfering with each other with signal light L of different phases that have passed through each cell, the signal light L that has passed through itself is focused. The structure of the microcell arrays used as the first lens 11 and the second lens 12 is the same as the structure of the microcell array used as the optical diffraction element 13, except for the setting of the thickness or refractive index of each cell.

[0044] In the optical computing device 1B shown in Figure 5, the first lens 11, the second lens 12, and the optical diffraction element 13 are all composed of microcell arrays. Therefore, with the optical computing device 1B shown in Figure 5, the focal lengths of the first lens 11 and the second lens 12 can be freely set, resulting in the effect of miniaturization and weight reduction.

[0045] (Third Modification of the Optical Computing Device) A third modification of the optical computing device 1 (hereinafter referred to as "optical computing device 1C") will be described with reference to Figure 6. Figure 6 is a block diagram showing the configuration of optical computing device 1C.

[0046] In the optical computing device 1 shown in Figure 1, optical lenses are used as the first lens 11 and the second lens 12. In contrast, in the optical computing device 1C shown in Figure 6, a reflective microcell array that functions similarly to a concave mirror is used as the first lens 11 and the second lens 12. The optical computing device 1C further includes a mirror 16, and the first lens 11 and the second lens 12 are arranged such that their respective reflective surfaces face the reflective surface of the mirror 16, and that their respective reflective surfaces are contained within the same plane.

[0047] Furthermore, while the optical computing device 1 shown in Figure 1 uses a transmissive microcell array as the optical diffraction element 13, the optical computing device 1C shown in Figure 6 uses a reflective microcell array as the optical diffraction element 13. The optical diffraction element 13 is arranged such that its reflective surface faces the reflective surface of the mirror 16, and that its reflective surface is included in the same plane as the reflective surfaces of the first lens 11 and the second lens 12.

[0048] A reflective microcell array can be realized, for example, by providing a reflective layer on the back surface (the surface opposite to the surface where the pillar is formed) of the microcell array shown in Figure 3. The microcell array used as the first lens 11 and the second lens 12 is composed of a plurality of microcells whose phase modulation amount can be set independently of each other, and by interfering with each other the signal light L with different phases reflected from the back surface of each microcell, it focuses the signal light L reflected from itself. The microcell array used as the optical diffraction element 13 is composed of a plurality of microcells whose thickness or refractive index can be set independently of each other, and by interfering with each other the signal light L with different phases reflected from the back surface of each microcell, it performs a predetermined optical calculation. The signal light L reflected by the optical diffraction element 13 represents the result of the optical calculation performed by the optical diffraction element 13.

[0049] In the optical computing device 1C shown in Figure 6, the first lens 11, the second lens 12, and the optical diffraction element 13 are all composed of microcell arrays. Therefore, with the optical computing device 1C shown in Figure 6, the focal lengths of the first lens 11 and the second lens 12 can be freely set, resulting in the effect of miniaturization and weight reduction. Furthermore, in the optical computing device 1C shown in Figure 6, the optical path of the signal light L is folded back at the first lens 11, the second lens 12, the optical diffraction element 13, and the mirror 16. Therefore, with the optical computing device 1B shown in Figure 6, the effect of further miniaturization can be achieved.

[0050] (Post-processing of the output image) The optical processing unit 1 processes the output image I generated by the image sensor 15. ＯＵＴ The system may include a processor that performs post-processing by referring to image data representing the data. Specific examples of post-processing performed by this processor will be described below with reference to Figures 7 to 10.

[0051] Figure 7 is a flowchart showing the flow of the first post-processing step S1. As shown in Figure 7, post-processing step S1 includes a specific processing step S11, a synthesis processing step S12, and a superposition processing step S13.

[0052] The specific processing S11 refers to the image data generated by the image sensor 15 and outputs the image I ＯＵＴ Each mark q included ｊ Regarding Mark Q, ｊ Region A including ｉ（ｊ） This is the process of identifying [something].

[0053] The synthesis process S12 refers to the image data generated by the image sensor 15, and each is output as image I ＯＵＴ M subimages I, which are part of a larger image. １ , I ２ , ..., I Ｍ Each of these has M regions A １ , A ２ , ..., A Ｍ M subimages I corresponding to any of the following １ , I ２ , ..., I Ｍ This process generates a composite image I by superimposing the two images.

[0054] The superposition process S13 involves each mark q included in the composite image I. ｊ Regarding the region A identified in specific processing S11 ｉ（ｊ） Corresponding to Class C ｉ（ｊ） This process involves superimposing a label indicating the above onto the composite image I generated in the synthesis process S12.

[0055] Figure 8(a) shows the output image I ＯＵＴ This is a schematic diagram showing a specific example, and Figure 8(b) is a schematic diagram showing a specific example of the composite image I obtained by post-processing S1.

[0056] The user visually inspects the composite image I obtained by post-processing S1, and each mark q ｊ From position to target p ｊ The position of each mark q can be determined, and each mark q ｊ From the label attached to the target p ｊ Class C ｉ（ｊ） It is possible to obtain the following: In other words, according to post-processing S2, the input image I ＩＮ This allows users to know what is where.

[0057] Figure 9 is a flowchart showing the flow of the second post-processing step S2. Post-processing step S2 includes counting step S21, as shown in Figure 8.

[0058] The counting process S21 refers to the image data generated by the image sensor 15 and outputs the image I ＯＵＴ Each region A included ｉ Regarding N marks q １ ,q ２ , ..., q Ｎ Area A ｉ Number of marks included: N ｉ This is a process for counting.

[0059] Figure 10(a) shows input image I ＩＮ This is a schematic diagram illustrating a specific example, where Figure 10(b) shows the output image I ＯＵＴ This is a schematic diagram illustrating a specific example.

[0060] Output Image I ＯＵＴ N marks q included １ ,q２ , ..., q Ｎ Area A ｉ Number of marks included: N ｉ This is input image I ＩＮ N objects p included １ , p ２ , ..., p Ｎ Class C ｉ This matches the number of objects belonging to [the group]. Therefore, according to post-processing S2, the input image I ＩＮ This allows users to know what items are available and how many of each they are.

[0061] (Summary) [Aspect 1] A first lens that performs a Fourier transform on signal light, a second lens that performs a re-Fourier transform on the signal light that has been Fourier transformed by the first lens, and at least one optical diffraction element that acts on the signal after it has been Fourier transformed by the first lens and before it has been re-Fourier transformed by the second lens, wherein the optical diffraction element (a) the input image I represented by the signal light input to the first lens ＩＮ However, there are N objects p (where N is any natural number greater than or equal to 1). １ , p ２ , ..., p Ｎ When the subject includes the output image I represented by the signal light output from the second lens ＯＵＴ However, each target p ｊ The corresponding mark q ｊ (b) Output image I ＯＵＴ Each mark q included ｊ Regarding (b1), M predetermined regions A １ , A ２ , ..., A Ｍ Among them, Mark Q ｊ Region A including ｉ（ｊ） However, there are M predetermined Class C １ , C ２ ..., C Ｍ Among them, Mark Q ｊ The corresponding target p ｊ Class C to which it belongs ｉ（ｊ） (b2) Area A ｉ（ｊ） Mark q in ｊ Position (x ｊ , y ｊ ) is input image I ＩＮMark q in ｊ The corresponding target p ｊ Position (X ｊ , Y ｊ An optical computing device designed to represent ).

[0062] [Aspect 2] The optical computing device according to aspect 1, wherein the focal length of the first lens is f, and the optical diffraction element is arranged to intersect with a section on the optical axis of the first lens where the distance from the principal point of the first lens is (3 / 4) × f or more and (5 / 4) × f or less.

[0063] [Aspect 3] The optical diffracting element is composed of a plurality of cells in which the phase modulation amount can be set independently of each other, and the optical computing device according to aspect 1 or 2 performs a predetermined optical calculation on the signal light by (1) interfering with each other with the light transmitted through each cell, or (2) interfering with each other with the light reflected from the back surface of each cell.

[0064] [Aspect 4] An optical computing device according to any one of aspects 1 to 3, wherein one or both of the first lens and the second lens are composed of a plurality of cells in which the phase modulation amount can be set independently of each other, and the signal light is focused by (1) interfering with each other with the light transmitted through each cell, or (2) interfering with each other with the light reflected from the back surface of each cell.

[0065] [Aspect 5] The optical computing device according to any one of aspects 1 to 4, wherein the first lens and the second lens are configured to focus signal light that has passed through them, and the optical diffraction element is configured so that the signal light that has passed through it represents the result of the optical calculation.

[0066] [Aspect 6] The optical computing device according to any one of aspects 1 to 4, wherein the first lens and the second lens are configured to concentrate signal light reflected from themselves, and the optical diffraction element is configured such that the signal light reflected from itself represents the result of the optical calculation.

[0067] [Aspect 7] The signal light that has been re-Fourier transformed by the second lens is detected, and the output image I ＯＵＴAn image sensor that generates image data representing (1) an output image I ＯＵＴ Each mark q included ｊ Regarding M regions A １ , A ２ , ..., A Ｍ Mark Q ｊ Region A including ｉ（ｊ） (2) A process to identify the image data generated by the image sensor, and each of them outputs an image I ＯＵＴ M subimages I, which are part of a larger image. １ , I ２ , ..., I Ｍ Each of these has M regions A １ , A ２ , ..., A Ｍ M subimages I corresponding to any of the following １ , I ２ , ..., I Ｍ A synthesis process that generates a composite image I by superimposing the two marks q, (3) each mark q included in the composite image I ｊ Regarding the region A identified in the aforementioned specific processing ｉ（ｊ） Corresponding to Class C ｉ（ｊ） An optical computing device according to any one of embodiments 1 to 6, comprising: a processor that performs a superposition process of superimposing a label indicating onto a composite image I generated by the synthesis process.

[0068] [Aspect 8] The signal light that has been re-Fourier transformed by the second lens is detected, and the output image I ＯＵＴ An image sensor that generates image data representing the image, and an output image I that references the image data generated by the image sensor. ＯＵＴ Each region A included ｉ Regarding N marks q １ ,q ２ , ..., q Ｎ Area A ｉ An optical computing device according to any one of embodiments 1 to 6, comprising: a processor that performs a counting process for counting the number of marks contained in;

[0069] (Additional Notes) The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in each of the embodiments described above are also included in the technical scope of the present invention.

[0070] 1, 1A, 1B, 1C Optical computing unit 11 First lens 12 Second lens 13 Optical diffraction element

Claims

1. A first lens for Fourier-transforming a signal light, a second lens for re-Fourier-transforming the signal light Fourier-transformed by the first lens, and at least one optical diffraction element acting on the signal after being Fourier-transformed by the first lens and before being re-Fourier-transformed by the second lens, wherein the optical diffraction element: (a) when the input image I represented by the signal light input to the first lens includes N (N is an arbitrary natural number of 1 or more) objects p ｉ（ｊ） , ｊ ,..., p １ , p ２ ,..., p Ｎ as subjects, the output image I represented by the signal light output from the second lens includes marks q ＯＵＴ corresponding to each object p ｊ , and (b) for each mark q ｊ included in the output image I ＯＵＴ : (b1) among the predetermined M regions A ｊ , A １ , Az ２ ,..., Az Ｍ , the region A ｊ including the mark q ｉ（ｊ） represents the class C １ , C ２ ,..., C Ｍ to which the object p ｊ corresponding to the mark q ｊ belongs, and (b2) the position (x ｉ（ｊ） , y ｉ（ｊ） ) of the mark q ｊ in the region A ｊ represents the position (X ｊ , Y ＩＮ ) of the object p ｊ corresponding to the mark q ｊ in the input image I ｊ . An optical arithmetic device designed as such 2. The optical computing device according to claim 1, wherein the optical diffraction element is arranged such that, with the focal length of the first lens being f, it intersects with a section on the optical axis of the first lens where the distance from the principal point of the first lens is (3 / 4) × f or more and (5 / 4) × f or less.

3. The optical diffracting element is composed of a plurality of cells in which the phase modulation amount can be set independently of each other, and the optical computing device according to claim 1 or 2 performs a predetermined optical calculation on the signal light by (1) interfering with each other with the light transmitted through each cell, or (2) interfering with each other with the light reflected from the back surface of each cell.

4. The optical computing device according to any one of claims 1 to 3, wherein one or both of the first lens and the second lens are composed of a plurality of cells in which the phase modulation amount can be set independently of each other, and the signal light is focused by (1) interfering with each other with the light transmitted through each cell, or (2) interfering with each other with the light reflected from the back surface of each cell.

5. The optical computing device according to any one of claims 1 to 4, wherein the first lens and the second lens are configured to focus signal light that has passed through them, and the optical diffraction element is configured such that the signal light that has passed through it represents the result of the optical calculation.

6. The optical computing device according to any one of claims 1 to 4, wherein the first lens and the second lens are configured to concentrate signal light reflected from themselves, and the optical diffraction element is configured such that the signal light reflected from itself represents the result of the optical calculation.

7. The signal light, which has been re-Fourier transformed by the second lens, is detected, and output image I ＯＵＴ An image sensor that generates image data representing (1) an output image I ＯＵＴ Each mark q included ｊ Regarding M regions A １ , A ２ , ..., A Ｍ Mark Q ｊ Region A including ｉ（ｊ） (2) A process to identify the image data generated by the image sensor, and each of them outputs an image I ＯＵＴ M subimages I, which are part of a larger image. １ , I ２ , ..., I Ｍ Each of these has M regions A １ , A ２ , ..., A Ｍ M subimages I corresponding to any of the following １ , I ２ , ..., I Ｍ A synthesis process that generates a composite image I by superimposing the two marks q, (3) each mark q included in the composite image I ｊ Regarding the region A identified in the aforementioned specific processing ｉ（ｊ） Corresponding to Class C ｉ（ｊ） The optical computing apparatus according to any one of claims 1 to 6, comprising: a processor that performs a superposition process of superimposing a label indicating on the composite image I generated by the synthesis process.

8. The signal light, which has been re-Fourier transformed by the second lens, is detected, and output image I ＯＵＴ An image sensor that generates image data representing the image, and an output image I that references the image data generated by the image sensor. ＯＵＴ Each region A included ｉ Regarding N marks q １ ,q ２ , ..., q Ｎ Area A ｉ An optical computing device according to any one of claims 1 to 6, comprising: a processor that performs a counting process for counting the number of marks contained in;

Images