All-focus image generation method, all-focus image generation device, and all-focus image generation program

Abstract

A method (MT1) is provided with: an original image acquisition step (step ST1) for capturing an image of an object while shifting a focal position in a predetermined direction, and acquiring an original image for each focal position relating to the object; a frequency decomposition step (step ST2) for performing frequency decomposition on the original image of each focal position and generating an edge image of each frequency band; an all-focus edge image generation step (step ST3) in which the edge images for each frequency band are integrated in a prescribed direction and all-focus edge images for each frequency band are generated; and an all-focus image generation step (step ST4) for integrating the all-focus edge images of each frequency band and generating an all-focus image.

Description

Technical Field

[0001] This disclosure relates to a method, apparatus, and program for generating a full-focus image. Background Technology

[0002] A known method involves shifting the focus position and photographing the same object along its thickness direction (Z-axis direction) to obtain multiple images (Z-stacked images) with different focus positions, and generating an all-focused image with focus in each pixel based on the Z-stacked images (e.g., Patent Document 1).

[0003] In the method of Patent Document 1, an all-focus image is generated by aggregating pixels (representing focal positions) of the image with the maximum focus at each pixel position from the acquired Z-stacked image. Specifically, the focus calculation unit calculates the focus at all pixel positions of the two-dimensional image constituting the Z-stacked image and sends it to the all-focus image generation unit. The all-focus image generation unit selects the pixel value of any two-dimensional image in the Z direction with the highest focus (extracting the focal position) and generates an all-focus image by combining the pixel values ​​selected at all pixel positions.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: International Publication No. 2018 / 042629 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, in the method of Patent Document 1, the focus position is extracted at all pixel locations. Therefore, for example, for large objects spanning multiple pixels, it is impossible to evaluate the object as a whole, and the area near the center of the object may be blurred in a full-focus image. In contrast, expanding the region for extracting the focus position (e.g., each of multiple pixels) is also considered. However, in this case, the computation speed decreases, and it is difficult to calculate the focus with high accuracy compared to objects with smaller extraction regions, and smaller objects may be blurred in a full-focus image. Therefore, a method is desired that can generate full-focus images with high accuracy regardless of the size of the extraction region.

[0009] This disclosure provides a method, apparatus, and program for generating full-focus images with high precision without extracting regions from the original image.

[0010] means for solving problems

[0011] The main contents of this disclosure are as follows.

[0012] [1] A method for generating an all-focus image, comprising: an original image acquisition step, which shifts the focus position in a predetermined direction and captures an object, thereby acquiring an original image of each focus position related to the object; a frequency decomposition step, which performs frequency decomposition on the original image of each focus position to generate an edge image of each frequency band; an all-focus edge image generation step, which integrates the edge images of each frequency band in the predetermined direction to generate an all-focus edge image of each frequency band; and an all-focus image generation step, which integrates the all-focus edge images of each frequency band to generate an all-focus image.

[0013] In this all-focus image generation method, original images of each focal position offset from each other are acquired. Frequency components contained in the original images of each focal position are extracted for each frequency band, thereby generating an edge image for each frequency band. The frequency components contain information representing the focal position in each original image; therefore, by performing frequency decomposition on each original image, the focal position can be extracted with high precision in the edge image of each frequency band. The all-focus image generated by integrating the edge images of each frequency band and processing them separately for each frequency band reflects information representing different focal positions according to each frequency band, and focuses on all pixels. According to this method, all-focus images can be generated with high precision without extracting specific regions of the original images for each focal position.

[0014] [2] According to the full-focus image generation method described in [1], in the frequency decomposition step, the original image of each focal position is decomposed into multiple color component images, and frequency decomposition is performed on each of the multiple color component images, thereby generating an edge image of each frequency band corresponding to the multiple color component images. In this case, by performing frequency decomposition on each of the multiple color component images to generate an edge image of each frequency band corresponding to the multiple color component images, information representing different focal positions according to each color component image can be further reflected in the full-focus image. Thus, full-focus images can be generated with higher precision.

[0015] [3] According to the full-focus image generation method described in [1] or [2], in the frequency decomposition step, images that do not contain the focal position are excluded from the objects of frequency decomposition from the original image of each focal position. In this case, by excluding images that do not contain the focal position, the number of images that become objects of decomposition in the frequency decomposition step can be reduced, thereby improving the generation speed of full-focus images.

[0016] [4] The full-focus image generation method according to any one of [1] to [3] further includes a selection step, which selects a specific region in the original image of each focal position, and in the frequency decomposition step, performs frequency decomposition on the specific region in the original image of each focal position to generate an edge image of each frequency band in the specific region. In this case, the amount of data of the image that is the object of decomposition in the frequency decomposition step can be reduced, and the generation speed of the full-focus image can be improved. In addition, the specific region mentioned here is only a region selected to reduce the amount of data, and the selected specific region will not affect the accuracy of the full-focus image based on this method.

[0017] [5] The full-focus image generation method according to any one of [1] to [4] further includes a conversion step of selecting a specific region in the full-focus image and converting the full-focus image into an image with focus in that specific region. In this case, the full-focus image can be converted into an image that is focused only in the specific region selected by the user, thereby improving the usability of the full-focus image.

[0018] [6] According to any one of [1] to [5], in the frequency decomposition step, edge images of each frequency band are generated sequentially starting from the high-frequency band. In the edge images of each frequency band, the image similarity between edge images of each common focal position of the frequency band is calculated. If the image similarity satisfies a predetermined similarity condition, the subsequent frequency decomposition is stopped. If the image similarity satisfies the predetermined similarity condition, it is considered that there is no significant difference in pixel values ​​between an image containing pixels corresponding to the focal position and an image not containing pixels corresponding to the focal position. Therefore, the frequency band can be judged as a frequency band that is not needed in judging focus. Therefore, by stopping the subsequent frequency decomposition, the generation speed of the full-focus image can be improved without compromising accuracy.

[0019] [7] An all-focus image generation apparatus, comprising: an original image acquisition unit that shifts a focal position in a predetermined direction and captures an object, acquiring original images of each focal position related to the object; a frequency decomposition unit that performs frequency decomposition on the original images of each focal position to generate edge images of each frequency band; an all-focus edge image generation unit that integrates the edge images of each frequency band in the predetermined direction to generate all-focus edge images of each frequency band; and an all-focus image generation unit that integrates the all-focus edge images of each frequency band to generate an all-focus image.

[0020] In the aforementioned all-focus image generation apparatus, original images of each focal position offset from each other are acquired. Frequency components contained in the original images of each focal position are extracted for each frequency band, thereby generating an edge image for each frequency band. Since the frequency components contain information representing the focal position in each original image, by performing frequency decomposition on each original image, the focal position can be extracted with high precision in the edge image of each frequency band. The all-focus image generated by integrating the edge images of each frequency band and processing them separately for each frequency band reflects information representing different focal positions according to each frequency band, and focuses on all pixels. According to this method, all-focus images can be generated with high precision without extracting specific regions of the original images for each focal position.

[0021] [8] According to the all-focus image generation apparatus described in [7], in the frequency decomposition unit, the original image of each of the focal positions is decomposed into multiple color component images, and frequency decomposition is performed on each of the multiple color component images, thereby generating an edge image of each frequency band corresponding to the multiple color component images. In this case, by performing frequency decomposition on each of the multiple color component images and generating an edge image of each frequency band corresponding to the multiple color component images, information representing different focal positions according to each color component image can be further reflected in the all-focus image. Thus, it is possible to generate all-focus images with higher precision.

[0022] [9] The all-focus image generation apparatus according to claim [7] or [8], wherein, in the frequency decomposition unit, images that do not contain the focal position are excluded from the objects of frequency decomposition from the original image of each focal position. In this case, by excluding images that do not contain the focal position, the number of images that become objects of decomposition in the frequency decomposition unit can be reduced, and the generation speed of all-focus images can be improved.

[0023]

[10] The all-focus image generation apparatus according to any one of [7] to [9] further includes a selection unit that selects a specific region in the original image of each focal position, and in the frequency decomposition unit, performs frequency decomposition on the specific region in the original image of each focal position to generate an edge image of each frequency band in the specific region. In this case, the amount of data of the image that is the object of decomposition in the frequency decomposition unit can be reduced, and the generation speed of the all-focus image can be improved. In addition, the specific region mentioned here is only a region selected to reduce the amount of data, and the selected specific region will not affect the accuracy of the all-focus image based on this method.

[0024]

[11] The all-focus image generation apparatus according to any one of [7] to

[10] further includes a conversion unit that selects a specific region in the all-focus image and converts the all-focus image into an image with focus in that specific region. In this case, the all-focus image can be converted into an image that is focused only in the specific region selected by the user, thereby improving the usability of the all-focus image.

[0025]

[12] According to any one of [7] to

[11] , in the frequency decomposition unit, edge images of each frequency band are generated sequentially starting from the high-frequency band. In the edge images of each frequency band, the image similarity between edge images of each common focal position of the frequency band is calculated. If the image similarity satisfies a predetermined similarity condition, the subsequent frequency decomposition is stopped. If the image similarity satisfies the predetermined similarity condition, it is considered that there is no significant difference in pixel values ​​between an image containing pixels corresponding to the focal position and an image not containing pixels corresponding to the focal position. Therefore, the frequency band can be determined as a frequency band that is not needed in determining the focus. Therefore, by stopping the subsequent frequency decomposition, the generation speed of the full-focus image can be improved without compromising accuracy.

[0026]

[13] A full-focus image generation program, which causes a computer to perform the following steps: an original image acquisition step, which shifts the focus position in a predetermined direction and captures an object, and acquires an original image of each focus position related to the object; a frequency decomposition step, which performs frequency decomposition on the original image of each focus position, and generates an edge image of each frequency band; a full-focus edge image generation step, which integrates the edge images of each frequency band in the predetermined direction, and generates a full-focus edge image of each frequency band; and a full-focus image generation step, which integrates the full-focus edge images of each frequency band and generates a full-focus image.

[0027] In the above-described full-focus image generation procedure, original images of each focal position offset from each other are acquired. Frequency components contained in the original images of each focal position are extracted for each frequency band, thereby generating an edge image for each frequency band. The frequency components contain information representing the focal position in each original image; therefore, by performing frequency decomposition on each original image, the focal position can be extracted with high precision in the edge image of each frequency band. The full-focus image generated by integrating the edge images of each frequency band and processing them separately for each frequency band reflects information representing different focal positions according to each frequency band, and focuses on all pixels. According to this method, full-focus images can be generated with high precision without extracting specific regions of the original images for each focal position.

[0028] Invention Effects

[0029] According to this disclosure, it is possible to generate a full-focus image with high precision without extracting regions from the original image. Attached Figure Description

[0030] Figure 1 This is a block diagram illustrating a full-focus image generation apparatus according to one embodiment.

[0031] Figure 2 It means to obtain Figure 1 A diagram showing an example of the original image at each focal position of the original image acquisition unit.

[0032] Figure 3 This is a flowchart illustrating a full-focus image generation method according to one embodiment.

[0033] Figure 4 It is used for explanation Figure 3 The diagram shows the method for generating a full-focus image.

[0034] Figure 5 It means Figure 3 The flowchart shows an example of a frequency decomposition step.

[0035] Figure 6 It is used for explanation Figure 5 A diagram illustrating an example of the frequency decomposition steps.

[0036] Figure 7 This is a flowchart illustrating an example of a method for generating full-focus edge images and the lowest full-focus image for each frequency band.

[0037] Figure 8 This is a diagram used to illustrate the frequency decomposition steps involved in the first variation.

[0038] Figure 9 This is a flowchart illustrating the full-focus image generation method involved in the second variation.

[0039] Figure 10 It is used for explanation Figure 9 The diagram shows the selection steps.

[0040] Figure 11 This is a flowchart illustrating the full-focus image generation method involved in the third variation.

[0041] Figure 12 It is used for explanation Figure 11 The diagram shows the conversion steps.

[0042] Figure 13 This is a flowchart illustrating the frequency decomposition steps involved in the fourth variation.

[0043] Figure 14 It is used for explanation Figure 13 A diagram showing the frequency decomposition steps.

[0044] Figure 15 This is a diagram illustrating an application example of the image information judgment model involved in the fifth variation.

[0045] Figure 16 This is a block diagram representing an example of a full-focus image generation procedure and its recording medium.

[0046] Figure 17 This is a diagram used to illustrate the method for generating a full-focus image using the blocks involved in Comparative Example 1 and Comparative Example 2.

[0047] Figure 18 It is a graph representing the sample being evaluated.

[0048] Figure 19 This is a graph showing the results of calculating the first and second evaluation indicators.

[0049] Figure 20 It is a graph representing the results of measuring the processing time required from acquiring the raw image at each focal position to generating the all-focus image. Detailed Implementation

[0050] Hereinafter, with reference to the accompanying drawings, a preferred embodiment of the all-focus image generation method, all-focus image generation apparatus, and all-focus image generation program involved in one embodiment of the present disclosure will be described in detail.

[0051] [Hyperfocal Image Generation Device]

[0052] Figure 1 This is a block diagram illustrating a total focal length image generation apparatus according to one embodiment. The total focal length image generation apparatus 1 generates a total focal length image focused in each pixel based on the acquired original images (multiple original images) of each focal position. For example, the total focal length image generation apparatus 1 is configured to acquire original images of each focal position by photographing a cell sample placed on a glass slide, and generate a total focal length image of the cell sample. The generated total focal length image is used, for example, for cell diagnosis and pathological diagnosis.

[0053] The all-focus image generation apparatus 1 includes an original image acquisition unit 11, a frequency decomposition unit 12, an all-focus edge image generation unit 13, an all-focus image generation unit 14, and an all-focus image output unit 15. The frequency decomposition unit 12, the all-focus edge image generation unit 13, the all-focus image generation unit 14, and the all-focus image output unit 15 are physically, for example, computers equipped with processors such as CPUs and storage media such as RAM and ROM. The computer can also be a smart device such as a smartphone or tablet terminal that integrates a display unit and an input unit. The computer can also be constructed from a microcomputer or an FPGA (Field-Programmable Gate Array).

[0054] The raw image acquisition unit 11 acquires raw images at each focal point. The raw image acquisition unit 11 includes an imaging device. The imaging device is, for example, a microscope imaging device, which magnifies and photographs an object placed on a stage. Figure 2 This diagram illustrates an example of acquiring original images at each focal position of the original image acquisition unit 11. The original image acquisition unit 11 acquires original images 2 of an object having thickness. Here, for ease of explanation, the thickness direction of the object is defined as the Z-axis direction. Furthermore, the direction perpendicular to the Z-axis direction is defined as the X-axis direction, and the direction perpendicular to both the Z-axis and X-axis directions is defined as the Y-axis direction. The original image acquisition unit 11, for the same object, fixes the shooting position (position in the X-axis and Y-axis directions) other than the Z-axis direction, shifts the focal position in the Z-axis direction (the defined direction), and continuously captures multiple images of the object. In this way, the original image acquisition unit 11 acquires original images 2 at each focal position with a different focal position. The original images 2 at each focal position are continuous in the Z-axis direction, and are therefore referred to as Z-stacked images.

[0055] The frequency decomposition unit 12 generates edge images (multiple edge images) for each frequency band by performing frequency decomposition on each of the original images 2 at each focal position. In the frequency decomposition unit 12, the original image at each focal position is input from the original image acquisition unit 11. Each of the original images 2 at each focal position contains frequency components representing the intensity of the edge at the focal position. The frequency decomposition unit 12 generates edge images for each frequency band by sequentially extracting the frequency components contained in the original images 2, starting from the highest frequency band.

[0056] The full-focus edge image generation unit 13 integrates the edge images of each frequency band in the Z-axis direction to generate a full-focus edge image for each frequency band. In the full-focus edge image generation unit 13, the edge images of each frequency band are input from the frequency decomposition unit 12. Based on the edge images of each focal position (multiple edge images common to the frequency band) in the edge images of each frequency band, the full-focus edge image generation unit 13 integrates the pixels representing the focal positions in the Z-axis direction (a predetermined direction) to generate a full-focus edge image for each frequency band.

[0057] The full-focus image generation unit 14 integrates the full-focus edge images of each frequency band to generate a single full-focus image. In the full-focus image generation unit 14, the full-focus edge images of each frequency band are input from the full-focus edge image generation unit 13. The full-focus image generation unit 14 outputs the generated full-focus image to the full-focus image output unit 15.

[0058] The full-focus image output unit 15 may, for example, display the full-focus image generated by the full-focus image generation unit 14 on an external display (not shown). Alternatively, the full-focus image output unit 15 may, for example, transmit the generated full-focus image to an external recording medium or device.

[0059] [Methods for Generating Holofocal Images]

[0060] Secondly, refer to Figure 3 and Figure 4 The method for generating full-focus images is explained. Figure 3 This is a flowchart illustrating a full-focus image generation method (hereinafter, method MT1) according to one embodiment. Figure 4 This is a diagram used to illustrate the method of generating a full-focus image.

[0061] Method MT1 begins by capturing a raw image 2 at each focal position using the raw image acquisition unit 11. First, the raw image acquisition unit 11 acquires the raw image 2 at each focal position (raw image acquisition step: step ST1).

[0062] Next, the frequency decomposition unit 12 performs frequency decomposition on the original image 2 at each focal position, thereby generating an edge image for each frequency band (frequency decomposition step: step ST2). The frequency decomposition unit 12 sequentially extracts frequency components representing the intensity of the edges at the focal positions contained in each original image 2, starting from the high-frequency bands. In other words, the frequency decomposition unit 12 extracts high-frequency components sequentially by repeatedly filtering each original image 2. The edge image for each frequency band includes the edge image of each focal position common to all frequency bands.

[0063] In step ST2, the frequency decomposition unit 12 decomposes the original image 2 at each focal position into multiple color component images, and performs frequency decomposition on each of the multiple color component images. The multiple color component images are, for example, composed of a red image, a green image, and a blue image. The frequency decomposition unit 12 generates a frequency band edge image in each of the multiple color component images.

[0064] like Figure 4 As shown, step ST2 generates edge images 31 to 3 (N-1) for each frequency band and a bottom-level image 3N (multiple bottom-level images) for each focal position. Here, N represents an integer greater than 2.

[0065] exist Figure 4 In the example, the frequency decomposition unit 12 performs N frequency decompositions on each original image 2, thereby generating (N-1) edge images for each frequency band and one bottom-level image for each original image 2. The (N-1) edge images for each frequency band correspond to each frequency band. The (N-1) edge images for each frequency band are composed of edge image 31 generated by the first frequency decomposition, edge image 32 generated by the second frequency decomposition, edge image 33 generated by the third frequency decomposition, and edge image 3(N-1) generated by the (N-1)th frequency decomposition, in descending order of frequency band.

[0066] Each original image 2 may also contain multiple focus points at the focal position. In this case, the frequency decomposition unit 12 extracts multiple focus points in step ST2, for example, in order of focal density from high to low.

[0067] The edge images 31–3 (N-1) for each frequency band include edge images at each common focal position in each frequency band. The edge images 31, 32, 33, 3 (N-1), and the bottommost image 3N at each focal position generate the same number of images as the original image 2 for each focal position. The focal positions of the edge images 31–3 (N-1) and the bottommost image 3N at each focal position are different among the multiple images in each frequency band.

[0068] As described above, the frequency decomposition unit 12 generates an edge image for each frequency band in each of the multiple color component images in the original image 2 at each focal position. Thus, each edge image 31 at each focal position includes a red image 31r, a green image 31g, and a blue image 31b. Each edge image 32 at each focal position includes a red image 32r, a green image 32g, and a blue image 32b. Each edge image 33 at each focal position includes a red image 33r, a green image 33g, and a blue image 33b. Each edge image 3(N-1) at each focal position includes a red image 3(N-1)r, a green image 3(N-1)g, and a blue image 3(N-1)b.

[0069] In addition, Figure 4 In the image, the red image 3Nr, green image 3Ng, and blue image 3Nb are not illustrated in the bottommost image 3N at each focal position. This is because edge images are not generated from the bottommost image 3N at each focal position. Therefore, the bottommost image 3N is distinguished from the edge images 31 to 3(N-1) of each frequency band. The bottommost image 3N at each focal position contains the red image 3Nr, green image 3Ng, and blue image 3Nb.

[0070] Here, refer to Figure 5 and Figure 6 An example of step ST2 will be provided. Figure 5 as well as Figure 6 In the example, the frequency decomposition unit 12 performs frequency decomposition on each original image 2 using the Laplacian pyramid. First, the frequency decomposition unit 12 uses each original image 2 as the original image and compresses the resolution of each original image 2 to half (step ST21).

[0071] Next, the frequency decomposition unit 12 resizes the compressed image 21 (step ST22). The resized image 22 has the same resolution as image 21, and therefore is a state in which high-frequency components have been removed from each original image 2, becoming a blurred image compared to each original image 2. Next, the frequency decomposition unit 12 generates an edge image 31 by subtracting image 22 from each original image 2, which only extracts the high-frequency components of each original image 2 (step ST23).

[0072] Next, the frequency decomposition unit 12 determines whether the number of edge images for each frequency band generated from each original image 2 has reached a predetermined number (step ST24). In this embodiment, the predetermined number is (N-1) images. If the number of edge images for each frequency band has not reached the predetermined number (step ST24: No), the frequency decomposition unit 12 uses image 21 as the original image and starts frequency decomposition again from step ST21. The frequency decomposition unit 12 repeats step ST21 N times, and repeats steps ST22 and ST23 (N-1) times.

[0073] When the number of edge images for each frequency band reaches a predetermined number (step ST24: Yes), the frequency decomposition unit 12 ends the frequency decomposition (step ST25). In this embodiment, the frequency decomposition unit 12 ends the frequency decomposition during the stage of generating (N-1) edge images for each frequency band.

[0074] Refer again Figure 3 as well as Figure 4 After the frequency decomposition unit 12 generates edge images 31 to 3 (N-1) for each frequency band and the lowest layer image 3N for each focal position, the full-focus edge image generation unit 13 generates full-focus edge images 41 to 4 (N-1) (multiple full-focus edge images) and the lowest layer image 4N for each frequency band based on the edge images 31 to 3 (N-1) for each frequency band and the lowest layer image 3N for each focal position (full-focus edge image generation step: step ST3).

[0075] In step ST3, the full-focus edge image generation unit 13 generates a full-focus edge image for each frequency band in each of the multiple color component images. Thus, full-focus edge image 41 includes a full-focus red image 41r, a full-focus green image 41g, and a full-focus blue image 41b. Full-focus edge image 42 includes a full-focus red image 42r, a full-focus green image 42g, and a full-focus blue image 42b. Full-focus edge image 43 includes a full-focus red image 43r, a full-focus green image 43g, and a full-focus blue image 43b. Full-focus edge image 4(N-1) includes a full-focus red image 4(N-1)r, a full-focus green image 4(N-1)g, and a full-focus blue image 4(N-1)b.

[0076] The full-focus edge images 41–4 (N-1) for each frequency band consist of (N-1) images. Each of the (N-1) full-focus edge images for each frequency band corresponds to a specific frequency band. Each full-focus edge image 41–4 (N-1) is formed by integrating the pixels of the edge images in their respective frequency bands. The (N-1) full-focus edge images for each frequency band are composed of full-focus edge image 41, full-focus edge image 42, full-focus edge image 43, and full-focus edge image 4 (N-1) in descending order of frequency band.

[0077] The bottommost image 4N in full focus is composed of a single image. The bottommost image 4N in full focus is formed by integrating the pixels of the bottommost image 3N at each focal position.

[0078] Figure 7 This diagram illustrates an example of a method for generating full-focus edge images 41 to 4(N-1) and the lowest full-focus image 4N for each frequency band. In this embodiment, the maximum value extraction method is described. Step ST3 includes steps ST31 to ST33.

[0079] First, the full-focus edge image generation unit 13 extracts the pixel representing the maximum pixel value from the edge image of each focal position in each full-focus edge image 41-4 (N-1) and the bottommost full-focus image 4N at each pixel position (step ST31). For example, based on the generation of the full-focus edge image 41, the full-focus edge image generation unit 13 extracts the pixel with the highest pixel value in the edge image 31 at each pixel position of the full-focus edge image 41. The pixel with the highest pixel value becomes the focused pixel in the edge image 31 at each focal position.

[0080] Next, the full-focus edge image generation unit 13 generates a full-focus edge image by integrating pixels extracted from all pixel positions of each full-focus edge image 41 to 4 (N-1) and the lowest full-focus image 4N. (Step ST32). Each full-focus edge image 41 to 4 (N-1) becomes an image focused at all pixel positions. For example, the full-focus edge image generation unit 13 integrates the pixel values ​​of the edge image 31 at each focus position extracted from all pixel positions of the full-focus edge image 41 and generates a full-focus edge image 51.

[0081] Next, the full-focus image generation unit 14 generates a full-focus image based on each full-focus edge image 41 to 4(N-1) and the full-focus bottom layer image 4N (full-focus image generation step: step ST4). The full-focus image generation unit 14 generates the full-focus image 5, for example, through the following steps: The full-focus image generation unit 14 generates an image whose resolution is doubled for the full-focus bottom layer image 4N. Then, the full-focus image generation unit 14 adds the image with doubled resolution of the full-focus bottom layer image 4N to the full-focus edge images 4(N-1). The full-focus image generation unit 14 generates the full-focus image 5 by repeating these steps until the full-focus edge images 41 are added.

[0082] Finally, the full-focus image output unit 15 outputs a full-focus image 5 (ST5). The method MT ends with the output of the full-focus image 5.

[0083] As explained above, in the method MT1 of one aspect of this disclosure, an original image 2 of each focal position with a focal position offset is acquired, and the frequency components contained in each original image 2 of each focal position are extracted for each frequency band, thereby generating edge images 31-3 (N-1) of each frequency band and a bottom-level image 3N of each focal position. The frequency components contain information representing the focal position in each original image 2 (edge ​​intensity of the focal position), therefore, by performing frequency decomposition on each original image 2, the focal position can be extracted with high precision in the edge images 31-3 (N-1) of each frequency band and the bottom-level image 3N of each focal position. In the full-focus image 5 generated by integrating the edge images 31-3 (N-1) of each frequency band and the bottom-level image 3N of each focal position, information representing different focal positions in each frequency band is reflected, and focus is achieved in all pixels. According to this method, the full-focus image 5 can be generated with high precision without extracting specific regions of the original image 2.

[0084] In step ST2, each original image 2 at each focal position is decomposed into multiple color component images. Frequency decomposition is then performed on each of these color component images, thereby generating multiple edge images corresponding to each of the multiple color component images for each frequency band. In this case, by performing frequency decomposition on each of the multiple color component images and generating edge images for each frequency band corresponding to the multiple color component images, information representing different focal positions according to each color component image can be further reflected in the full-focus image 5. Therefore, the full-focus image 5 can be generated with higher precision.

[0085] [Variation Example]

[0086] The embodiments of this disclosure have been described above, but this disclosure is not limited to the above embodiments.

[0087] (First variation)

[0088] Step ST2 may also include excluding the original image that does not contain the focus position. Figure 8 This diagram illustrates the steps for excluding original images that do not contain focal positions. The frequency decomposition unit 12 excludes original images 2A that do not contain focal positions at any pixel location from the original images 2 acquired at each focal position. Figure 8 In the example, there are multiple original images 2A that do not contain a focal point. The topmost original image 2, the next level down from the topmost original image 2, the bottommost original image 2, and the level above the bottommost original image 2 are excluded as original images 2A that do not contain a focal point. Original image 2A that does not contain a focal point can also be a single image.

[0089] As an indicator for the frequency decomposition unit 12 to exclude original images 2A that do not contain focal positions, the original image acquisition unit 11 may also calculate the variance value. For example, the frequency decomposition unit 12 calculates the deviation of pixel values ​​among all pixels in each original image 2, and calculates the variance value of each original image 2 based on the calculated deviation. Original images 2A that do not contain focal positions tend to have small variance values. If the variance value of each original image 2 is lower than a variance threshold, the frequency decomposition unit 12 excludes the corresponding original image as an original image 2A that does not contain focal positions. Alternatively, as an indicator for excluding original images 2A that do not contain focal positions, the frequency decomposition unit 12 may also perform edge detection based on first or second derivative. The frequency decomposition unit 12 may also exclude original images 2 that do not detect edges in the edge detection results as original images 2A that do not contain focal positions.

[0090] In step ST2, the frequency decomposition unit 12 may also exclude original images 2A that do not contain focal positions from the objects of frequency decomposition from the original images 2 at each focal position. In this case, by excluding original images 2A that do not contain focal positions, the number of images that become objects of decomposition in step ST2 can be reduced, and the generation speed of the full-focus image 5 can be improved.

[0091] (Second variation)

[0092] Figure 9 This is a flowchart illustrating the full-focus image generation method (hereinafter, method MT2) involved in the second variation. The difference between method MT2 and method MT1 is that method MT2 includes a step of selecting a specific region (step ST6) between step ST1 and step ST2. Figure 10 This is a diagram used to illustrate step ST6. For example... Figure 10As shown, the original image acquisition unit 11 selects, for example, a specific region R to be focused on in each original image 2 at each acquired focal position, based on user input from the user interface. The specific region R can be arbitrarily selected by the user. The outer edge of the specific region R can be a curve or a straight line. The specific region R is common to all original images 2 at each focal position.

[0093] In method MT2, the steps following step ST2 are performed only on a specific region R. In other words, they are performed only on the pixels contained in the specific region R. In step ST2, by performing frequency decomposition on the specific region R of the original image 2 at each focal position, edge images 31 to 3(N-1) of each frequency band and the lowest layer image 3N of each focal position are generated in the specific region R of the original image 2 at each focal position.

[0094] In step ST3, the full-focus edge image generation unit 13 generates full-focus edge images 41-4(N-1) for each frequency band and a full-focus bottom layer image 4N in the specific region R (step ST3). In step ST4, the full-focus image generation unit 14 generates a full-focus image 5 that is focused only in the specific region R. In the full-focus image 5 that is focused only in the specific region R, the focus is on all pixels contained in the specific region R, while the focus is not on areas outside the specific region R.

[0095] In method MT2, step ST6 may not necessarily be performed by the original image acquisition unit 11. For example, the full-focus image generation apparatus 1 may also include a selection unit for selecting a specific region R, which selects the specific region R based, for example, on user input from the user interface.

[0096] Method MT2 may also include a step ST6 (selection step) of selecting a specific region R in the original image 2 for each focal position. In step ST2, frequency decomposition may also be performed on the specific region R in the original image 2 for each focal position, generating edge images 31 to 3 (N-1) of each frequency band in the specific region R and the lowest layer image 3N for each focal position. In this case, the amount of data of the image that is the object of decomposition in step ST2 can be reduced, and the generation speed of the full-focus image 5 can be improved. In addition, the specific region R mentioned here is only a region selected for reducing the amount of data, and the selected specific region R will not affect the accuracy of the full-focus image 5 based on this method.

[0097] (Third variation)

[0098] Figure 11This is a flowchart illustrating the full-focus image generation method (hereinafter, method MT3) involved in the third variation. The difference between method MT3 and method MT1 is that, after step ST4, it includes a step of converting the full-focus image 5 into an image focused in a specific area (step ST7). Figure 12 This is a diagram used to illustrate step ST7. In step ST7, as... Figure 12 As shown, the full-focus image generation unit 14 selects a specific region R in the generated full-focus image 5 that the user wants to focus on, for example, based on input from the user interface. The specific region R can be arbitrarily selected by the user. The outer edge of the specific region R can be a curve or a straight line.

[0099] Next, in step ST7, the full-focus image generation unit 14 converts the full-focus image 5 into an image 5R that is focused only on a selected specific region R. In image 5R, focus is on all pixels contained in the specific region R, while areas outside the specific region R are not in focus.

[0100] In step ST7, the full-focus image generation unit 14 can also convert the full-focus image 5 into image 5R by reducing the resolution of the region other than the specific region R in the full-focus image 5. Alternatively, the full-focus image generation unit 14 can also cut out the region other than the specific region R from any image of the original image 2 at each focal position or from an image after averaging the pixel values ​​of the original image 2 at each focal position, and apply the cut-out region to the full-focus image 5, thereby converting the full-focus image 5 into image 5R.

[0101] In method MT3, step ST7 may not necessarily be performed by the full-focus image generation unit 14. For example, the full-focus image generation device 1 may also include a conversion unit that can convert the full-focus image 5 into an image focused on a specific area.

[0102] Method MT3 may also include a step ST7 (conversion step) of selecting a specific region R in the full-focus image 5 and converting the full-focus image 5 into an image 5R focused only in the specific region R. In this case, the full-focus image 5 can be converted into an image 5R focused only in the specific region R selected by the user, thereby improving the usability of the full-focus image 5.

[0103] (Fourth variation)

[0104] Figure 13This is a flowchart illustrating step ST2A in the fourth variation. Step ST2A differs from step ST2 in that it includes step ST26 instead of step ST24. In step ST26, the frequency decomposition unit 12 determines whether the image similarity between edge images at each focal position common to the frequency bands in the edge images 31 to 3 (N-1) of each frequency band meets a similarity threshold. In step ST2A, the frequency decomposition unit 12 calculates the image similarity between edge images at each focal position, and stops subsequent frequency decomposition if the image similarity meets a predetermined similarity condition.

[0105] exist Figure 14 In the example, by performing three frequency decompositions, edge images 31, 32, and 33 are generated for each focal position. The frequency decomposition unit 12 calculates the image similarity in the edge image 33 at each focal position, and stops the frequency decomposition after the edge image 33 at each focal position if the image similarity meets the specified similarity condition (step ST26: Yes).

[0106] If the image similarity between edge images at each focal location within a common frequency band meets a specified similarity condition, then the significant difference in pixel values ​​between images containing pixels corresponding to the focal location and images not containing pixels corresponding to the focal location is considered to have disappeared. Therefore, this frequency band can be determined as a frequency band unnecessary for judging focus. In other words, image similarity meeting the specified similarity condition becomes a criterion for determining whether further frequency decomposition will improve the accuracy of full-focus images.

[0107] As an example of calculating image similarity between edge images at each focal location with common frequency bands, SSIM (Structural Similarity) can be cited. The formula for calculating similarity using SSIM is, for example, represented by Equation 1.

[0108] [Formula 1]

[0109]

[0110] In Equation 1, the number of edge images at each focal position is L (L is an integer greater than or equal to 1). With the Mth edge image (M is an integer greater than or equal to 1 and less than L) designated as variable x[M] and the (M+1)th edge image designated as variable y[M+1], the SSIM between the Mth and (M+1)th edge images is calculated, and the calculated results are summed to obtain the similarity DS.

[0111] SSIM consists of the product of comparison terms of the brightness (l(x,y)), contrast (c(x,y)), and structure (s(x,y)) of the edge image. The definition of SSIM is expressed by Equation 2.

[0112] [Formula 2]

[0113]

[0114] In Equation 2, α, β, and γ are parameters used to adjust the weights of the three comparison terms. Brightness (l(x,y)), contrast (c(x,y)), and structure (s(x,y)) are represented by Equations 3 to 5, respectively.

[0115] [Formula 3]

[0116]

[0117] [Formula 4]

[0118]

[0119] [Formula 5]

[0120]

[0121] μ x μ y It is the average of x[M] and y[M+1], σ x σ y σ is the standard deviation of x[M] and y[M+1]. x σ y It is the covariance of x[M] and y[M+1]. C1 to C3 are constants set in equations 3 to 5 to improve the instability of the output value when the denominator is close to 0.

[0122] With α, β, and γ set to 1 in Equation 2, and C3 set to C2 / 2 in Equations 4 and 5, SSIM is calculated using Equation 6 based on Equations 2 to 5.

[0123] [Formula 6]

[0124]

[0125] Other examples of calculating image similarity between edge images at each focal location in a common frequency band include PSNR (Peak Signal to Noise Ratio). If, for example, the sum of the differences between the PSNR of the Mth edge image and the PSNR of the (M+1)th edge image is less than a predetermined difference threshold, the frequency decomposition unit 12 stops frequency decomposition of the frequency band below the difference threshold.

[0126] As another example of image similarity calculation, variance value can be cited. When the sum of the differences between the variance values ​​of the Mth edge image and the (M+1)th edge image is lower than a predetermined difference threshold, the frequency decomposition unit 12 stops frequency decomposition of the frequency bands below the difference threshold.

[0127] In step ST2A, the image similarity between edge images of each focal position in the common frequency band can also be calculated. If the image similarity meets a predetermined similarity condition, subsequent frequency decomposition is stopped. If the image similarity meets the predetermined similarity condition, it is considered that there is no significant difference in pixel values ​​between images containing pixels corresponding to the focal position and images not containing pixels corresponding to the focal position. Therefore, this frequency band can be determined as unnecessary for determining focus. Thus, by stopping subsequent frequency decomposition, the generation speed of the full-focus image 5 can be improved without compromising accuracy.

[0128] (Fifth variation)

[0129] The full-focus image generation method described above, as well as its variations, can also be applied to machine learning as a fifth variation. For example, the full-focus image generation apparatus 1 further includes a model generation unit, in which an image information determination model can also be generated. Figure 15 This is a diagram illustrating an application example of the image information judgment model involved in the fifth variation. For example... Figure 15 As shown, the input data for the image information determination model 10 can be the original image 2 at each focal position, the edge images 31-3 (N-1) of each frequency band, and the lowest layer image 3N at each focal position, or the full-focus image 5. Figure 15 In the example, the output data of the image information determination model 10 is the image 5R that is focused only on a specific region R selected for the full-focus image 5. In this case, the specific region R may also be automatically selected by the image information determination model 10 instead of being selected by the user.

[0130] As another example of the output data of the image information determination model 10, the original image 2 or the all-focus image 5 for each focal position may also be output. Alternatively, if images for cell diagnosis are prepared as input data, image data reflecting the detection of specific cell locations, the estimation of cell species, and the results of cell staining according to cell species / state may also be output. Furthermore, the image information determination model 10 is not limited to outputting image data; for example, it may also determine whether to repeat method MT.

[0131] In the above-described embodiments and variations, in step ST2 or step ST2A, the frequency decomposition unit 12 may also perform frequency decomposition on each original image 2 using methods other than the Laplacian pyramid. Other methods for frequency decomposition include wavelet transform, discrete Fourier transform, high-speed Fourier transform, difference Gaussian filter or kernel-based filtering, Hilbert transform, etc.

[0132] In the above-described embodiments and variations, in step ST3, the full-focus edge image generation unit 13 may also generate full-focus edge images 41 to 4(N-1) for each frequency band and the lowest full-focus image 4N using methods other than the maximum value extraction method. The full-focus edge image generation unit 13 may also implement principal component analysis, for example. In this case, in the principal component analysis method, for example, when... Figure 4 When the edge image 31 at each focal position is used as a variable, each variable can be multiplied by a weighted constant to form a composite variable, and the value at which the variance of the composite variable is maximized can be used as the principal component. Alternatively, the principal components extracted at this point can be used to construct the full-focal edge image 41.

[0133] In addition to the maximum value extraction method, the full-focus edge image generation unit 13 can also perform weighted summation, average value calculation, outlier removal, and neighbor point calculation. Furthermore, the full-focus edge image generation unit 13 can also perform maximum value extraction based on waveform fitting. The full-focus edge image generation unit 13 can also perform maximum value extraction after converting the edge images 31 to 3(N-1) of each frequency band and the lowest layer image 3N of each focal position into grayscale.

[0134] Figure 16 This is a block diagram illustrating an example of a full-focus image generation program and its recording medium. In this example, the full-focus image generation program 6 includes a main module 61, a raw image acquisition module 62, a frequency decomposition module 63, a full-focus edge image generation module 64, a full-focus image generation module 65, and a full-focus image output module 66. The functions implemented by the computer through the execution of the full-focus image generation program 6 are the same as those of the full-focus image generation apparatus 1 described above. The full-focus image generation program 6 is provided, for example, by a computer-readable recording medium 7 such as a CD-ROM, DVD, or ROM. The full-focus image generation program 6 can be provided by a semiconductor memory or via a network as a computer data signal superimposed on a carrier wave.

[0135] [Example]

[0136] The embodiments of this disclosure will now be described. In this embodiment, full-focus images were generated using the full-focus image generation methods of Embodiment 1, Embodiment 2, Comparative Example 1, and Comparative Example 2. Then, for each generated full-focus image, the uniformity of focus across the pixels was evaluated using a first evaluation index and a second evaluation index. Furthermore, in Embodiment 1, Embodiment 2, Comparative Example 1, and Comparative Example 2, the processing time required from acquiring the image at each focus position to generating the full-focus image was measured.

[0137] In Example 1, according to Figure 3 The method MT1 shown generates a full-focus image. However, in step ST3, regarding the generation of each full-focus edge image, at each pixel position of each full-focus edge image, the pixel representing the maximum pixel value is extracted from the edge image at each focus position, and the extracted pixels are integrated to generate the full-focus edge image. On the other hand, regarding the generation of the bottom full-focus image, at each pixel position of the bottom full-focus image, the pixel representing the minimum pixel value is extracted from the bottom image at each focus position, and the extracted pixels are integrated to generate the bottom full-focus image.

[0138] In Example 2, Figure 3 In step ST3 of method MT1, the full-focus image is generated by principal component analysis instead of the maximum value extraction method.

[0139] In Comparative Examples 1 and 2, a full-focus image generation method using patches was used instead of method MT1. A patch is, for example, a region of an image consisting of 16 pixels × 16 pixels. Figure 17 As shown, in Comparative Example 1 and Comparative Example 2, for each original image, block P is scanned along a plane defined by the X-axis and Y-axis. In Comparative Example 1, the focal position is extracted for each location in block P using the first-order derivative. In Comparative Example 2, the focal position is extracted for each location in block P using the second-order derivative. The focal position is extracted from pixel units within block P. In Comparative Example 1 and Comparative Example 2, a full-focus image is generated by combining the pixels from which the focal positions have been extracted.

[0140] The primary evaluation metric is Q. S (Structural similarity based metrics). Based on Q... S In the evaluation, a full-focus image is divided into multiple images using a window, and the image similarity between the divided images is evaluated. Q S It can be calculated using Equation 7.

[0141] [Formula 7]

[0142]

[0143] Here, W and w represent sliding windows, Q(x, f|w) represents the local quality index between variables x and f within window w, and Q(y, f|w) represents the local quality index between variables y and f within window w. λ(w) represents the weighting coefficient, which determines the weights of Q(x, f|w) and Q(y, f|w). λ(w) is calculated using Equation 8.

[0144] [Formula 8]

[0145]

[0146] The second evaluation indicator is Q. CB (Human perception based metrics). Based on Q... CB In the evaluation, the full-focus image was divided into multiple images using windows. Furthermore, based on Q... CB In the evaluation, the similarity of contrast in each segmented image is calculated, and finally the similarity of contrast in the full-focus image is evaluated. First, the local contrast is calculated using Equation 9.

[0147] [Formula 9]

[0148]

[0149] Here φ k It is a Gaussian distribution function, calculated using Equation 10.

[0150] [Formula 10]

[0151]

[0152] Next, the local contrast calculated using Equation 4 is adjusted using a hyperparameter reflecting human visual recognition. The adjusted contrast C is then calculated using Equation 11. A ′.

[0153] [Formula 11]

[0154]

[0155] Here, t, h, p, q, and Z are actual scalar parameters that determine the nonlinear shape of the masking function. For example, t = 1, h = 1, p = 3, q ​​= 2, and Z = 0.0001.

[0156] Next, the weighting coefficient λ is calculated using Equation 12. A (x, y).

[0157] [Formula 12]

[0158]

[0159] Next, the contrast similarity Q in the segmented images is calculated using Equation 13. AF (x, y).

[0160] [Formula 13]

[0161]

[0162] The similarity Q of contrast in the other images that were segmented BF (x, y) is also calculated using Equation 8. Then, based on the similarity of the weight coefficients and contrast calculated using Equations 12 and 13, the global quality map Q is calculated using Equation 14. GQM (x, y).

[0163] [Formula 14]

[0164]

[0165] Finally, by using the global quality map Q GQM The similarity of contrast in the full-focus image is calculated using Equation 15 after averaging (x, y).

[0166] [Formula 15]

[0167]

[0168] The sample that becomes the object of evaluation is Figure 3 The original image 2 at each focal point in the image is used as a reference for multiple images. For example... Figure 18 As shown, samples A through G are multiple images used for cell diagnosis. The number of images and the size of each image differ between samples A through G.

[0169] Figure 19 This indicates that Q, calculated as the first evaluation index, was used in Examples 1, 2, Comparative Example 1, and 2. s and Q as the second evaluation indicator CB The result. Q s and Q CB The larger the value, the more uniform the focus across the pixels, and the better the quality of the all-focus image.

[0170] Regarding Q as the primary evaluation indicator s It can be seen that Q calculated in Example 2 s Compared with Q calculated in Comparative Example 1 and Comparative Example 2 s Compared to all samples, it is larger. Regarding Q as the second evaluation indicator... CB It can be seen that Q calculated in Example 1 CB Compared with Q calculated in Comparative Example 2 CBCompared to all samples, it is larger. Furthermore, it is known that Q calculated in Example 1... CB Compared with Q calculated in Comparative Example 1 CB In comparison, it is larger in samples A, B, and samples D-F. The Q calculated in Example 1... CB The Q calculated in sample C compared to Comparative Example 1 CB Same. Q calculated in Example 1 CB The value of Q in sample G is less than that calculated in Comparative Example 1. CB However, their differences can be said to be within the range of no significant difference.

[0171] Figure 20 This represents the results of measuring the processing time required from acquiring the image at each focal position to generating a full-focus image in Examples 1, 2, 1, and 2. It is evident that the processing time measured in Example 1 is shorter than that measured in Comparative Examples 1 and 2 for all samples. In particular, it is evident that the processing time measured in Example 1 can be reduced to 1 / 4 to 1 / 6 of the processing time measured in Comparative Examples 1 and 2 in samples E and G, which have a large number of images.

[0172] Explanation of reference numerals in the attached figures

[0173] 1…Full-focus image generation device, 11…Original image acquisition unit, 12…Frequency decomposition unit, 13…Full-focus edge image generation unit, 14…Full-focus image generation unit, 2…Original image, 2A…Image without focal position, 31, 32, 33, 3(N-1)…Edge image, 31r, 31g, 31b…Color component image, 41, 42, 43, 4(N-1)…Full-focus edge image, 5…Full-focus image, 5R…Image focused in a specific area, 6…Full-focus image generation procedure, MT1, MT2, MT3…Full-focus image generation method, R…Specific area, ST2, ST2A…Frequency decomposition step, ST3…Full-focus edge image generation step, ST4…Full-focus image generation step, ST6…Selection step, ST7…Conversion step.

Claims

1. A method for generating a full-focus image, wherein, have: The original image acquisition step involves shifting the focal position in a specified direction and photographing the object, thereby acquiring original images of each focal position related to the object. The frequency decomposition step performs frequency decomposition on the original image at each focal position to generate edge images for each frequency band. The full-focus edge image generation step integrates the edge images of each frequency band in the specified direction to generate a full-focus edge image for each frequency band. as well as The full-focus image generation step integrates the full-focus edge images of each frequency band to generate a full-focus image.

2. The method for generating a full-focus image according to claim 1, wherein, In the frequency decomposition step, the original image at each focal position is decomposed into multiple color component images, and frequency decomposition is performed on each of the multiple color component images to generate an edge image for each frequency band corresponding to the multiple color component images.

3. The method for generating a full-focus image according to claim 1 or 2, wherein, In the frequency decomposition step, images that do not contain the focal position are excluded from the frequency decomposition object from the original image of each focal position.

4. The method for generating a full-focus image according to any one of claims 1 to 3, wherein, It also includes a selection step, which selects a specific region in the original image for each said focal position. In the frequency decomposition step, frequency decomposition is performed on the specific region in the original image of each focal position to generate the edge image of each frequency band in the specific region.

5. The method for generating a full-focus image according to any one of claims 1 to 4, wherein, It also includes a conversion step of selecting a specific region in the full-focus image and converting the full-focus image into an image with focus aligned in that specific region.

6. The method for generating a full-focus image according to any one of claims 1 to 5, wherein, In the frequency decomposition step, edge images of each frequency band are generated sequentially, starting from the high-frequency bands. In the edge images of each frequency band, the image similarity between edge images at each focal position common to the frequency band is calculated. If the image similarity satisfies the specified similarity conditions, the subsequent frequency decomposition is stopped.

7. A panfocal image generation apparatus, wherein, have: The original image acquisition unit shifts the focal position in a predetermined direction and captures an image of the object, acquiring original images of each focal position related to the object. The frequency decomposition unit performs frequency decomposition on the original image at each focal position to generate edge images for each frequency band. The full-focus edge image generation unit integrates the edge images of each frequency band in the predetermined direction to generate a full-focus edge image for each frequency band. as well as The full-focus image generation unit integrates the full-focus edge images of each frequency band and generates a full-focus image.

8. The all-focus image generation apparatus according to claim 7, wherein, In the frequency decomposition unit, the original image at each focal position is decomposed into multiple color component images, and frequency decomposition is performed on each of the multiple color component images to generate an edge image of each frequency band corresponding to the multiple color component images.

9. The all-focus image generating apparatus according to claim 7 or 8, wherein, In the frequency decomposition section, images that do not contain the focal position are excluded from the frequency decomposition object from the original image of each focal position.

10. The all-focus image generating apparatus according to any one of claims 7 to 9, wherein, It also includes a selection unit that selects a specific region in the original image for each of the said focal positions. In the frequency decomposition unit, frequency decomposition is performed on the specific region in the original image of each focal position to generate the edge image of each frequency band in the specific region.

11. The all-focus image generating apparatus according to any one of claims 7 to 10, wherein, It also includes a conversion unit that selects a specific region in the full-focus image and converts the full-focus image into an image with focus aligned in that specific region.

12. The all-focus image generating apparatus according to any one of claims 7 to 11, wherein, In the frequency decomposition unit, edge images of each frequency band are generated sequentially, starting from the high-frequency bands. In the edge images of each frequency band, the image similarity between edge images at each focal position common to the frequency band is calculated. If the image similarity satisfies the specified similarity conditions, the subsequent frequency decomposition is stopped.

13. A full-focus image generation program that causes a computer to perform the following steps: The original image acquisition step involves shifting the focal position in a specified direction and photographing the object, thereby acquiring original images of each focal position related to the object. The frequency decomposition step performs frequency decomposition on the original image at each focal position to generate edge images for each frequency band. The full-focus edge image generation step integrates the edge images of each frequency band in the specified direction to generate a full-focus edge image for each frequency band. as well as The full-focus image generation step integrates the full-focus edge images of each frequency band to generate a full-focus image.

Images