Image acquisition device and image acquisition method

Abstract

To accurately combine multiple picked-up images.SOLUTION: An image acquisition device acquires multiple picked-up images 71 representing multiple divided regions, respectively, which are obtained by dividing a predetermined region on a target object, and in the picked-up images, each image pair 72 representing adjacent divided regions has a partly overlapping region 73. In the overlapping region 73 of each image pair 72, the amount of relative positional deviation in each image pair 72 is identified by generating a similarity map 74 indicating a distribution of the degree of similarity by template matching. Combination positions of the multiple picked-up images 71 are determined based on the amounts of positional deviation in the multiple image pairs 72 included in the multiple picked-up images 71 while it is assumed that an image pair 72 whose similarity map 74 has a directivity is a specific image pair 72 and a weight of the specific image pair 72 is set to be lower than those of other image pairs 72. Thus, the multiple picked-up images 71 can be accurately combined.SELECTED DRAWING: Figure 6B

Description

[Technical Field]

[0001] The present invention relates to an image acquisition device and an image acquisition method. [Background technology]

[0002] Conventionally, a method has been known in which an object is captured multiple times from different positions and then synthesized into a single image through image processing. For example, in Patent Document 1, multiple captured images are captured with adjacent captured images having overlapping regions. Corresponding point pairs are set in the overlapping regions between the adjacent captured images, and the amount of relative positional shift between the captured images is calculated based on the corresponding point pairs, etc. The relative positional shift between the captured images is corrected based on the calculated amount of positional shift, and the captured images are then stitched together to generate a single wide-field image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 053839 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, if the object to be imaged is contained in a container, and multiple images are acquired by providing overlapping areas between adjacent images as described above, the edge of the container may be reflected in the overlapping area. The edge of the container often has higher contrast than the object, and the misalignment amount calculated in such an overlapping area is less reliable, i.e., is more likely to be abnormal. Therefore, when combining multiple images taking all misalignment amounts into consideration, the influence of the abnormal misalignment amount makes it difficult to accurately combine the multiple images.

[0005] The present invention has been made in view of the above-mentioned problems, and has as its object to combine a plurality of captured images with high accuracy. [Means for solving the problem]

[0006] A first aspect of the present invention is an image acquisition device for acquiring images of an object, the image acquisition device including: a captured image acquisition unit for acquiring a plurality of captured images, each of which indicates a plurality of divided regions obtained by dividing a predetermined region on the object, and each of which indicates adjacent divided regions and has an overlapping region that partially overlaps in each image pair; a positional deviation amount specification unit for generating a similarity map indicating a distribution of similarity by template matching in the overlapping region of each of the image pairs, and specifying a relative positional deviation amount in each of the image pairs; a combination position determination unit for determining combination positions of the plurality of captured images based on the positional deviation amounts in the plurality of image pairs included in the plurality of captured images, while setting a weight of the specific image pair smaller than that of other image pairs; and a combined image generation unit for combining the plurality of captured images according to the combination positions and generating a combined image. The positional deviation amount specifying unit executes a determination process for determining whether or not there is directionality with respect to the similarity map of each of the image pairs. do.

[0007] A second aspect of the present invention is an image acquisition device of the first aspect, wherein the positional deviation amount determination unit obtains an evaluation value indicating the strength of directionality of the similarity map of the specific image pair, and the joining position determination unit determines the weight of the specific image pair using the evaluation value.

[0009] Aspects of the present invention 3 is the aspect 1 or 2 In the image acquisition device, for a given image pair, a planned angle of the directionality of the similarity map is set in advance based on the area on the object indicated by the image pair, and the positional deviation amount identification unit executes the judgment process only with respect to the planned angle and angles nearby the planned angle for the image pair.

[0010] Aspects of the present invention 4 is the aspect 1 or 2 (Aspect Any one of 1 to 3In the image acquisition device, specific image pair candidates are determined in advance based on a region on an object among the plurality of image pairs, and the positional deviation amount specifying unit executes the determination process only for the specific image pair candidates.

[0011] Aspects of the present invention 5 is embodiment 1 or 2 (embodiment 1 to 4 In the image acquisition device of any one of the above, the positional deviation amount specifying unit obtains a map angle indicating a direction of the similarity map of the specific image pair.

[0012] Aspects of the present invention 6 teeth, Acquire an image of the object 1. An image acquisition device, comprising: the plurality of captured images are provided with: a captured image acquisition unit that acquires a plurality of captured images, each of which indicates a plurality of divided regions obtained by dividing a predetermined region on an object, and in which overlapping regions are provided in each image pair indicating adjacent divided regions; a positional deviation amount specification unit that generates a similarity map indicating a distribution of similarity by template matching in the overlapping region of each of the image pairs, thereby specifying a relative positional deviation amount in each of the image pairs; a combining position determination unit that determines combining positions of the plurality of captured images based on the positional deviation amounts in the plurality of image pairs included in the plurality of captured images, while setting an image pair in which directionality occurs in the similarity map as a specific image pair and assigning a weight to the specific image pair smaller than that of other image pairs; and a combined image generation unit that combines the plurality of captured images in accordance with the combining positions, and generates a combined image, wherein the positional deviation amount specification unit obtains a map angle indicating the direction of directionality of the similarity map of the specific image pair, For a given image pair, a predetermined angle of the directionality of the similarity map is set in advance based on the area on the object represented by the image pair, and the joining position determination unit determines the weight of the image pair based on the difference between the map angle and the predetermined angle.

[0013] Aspects of the present invention 7 an image acquisition method for acquiring an image of an object, the method comprising: a) acquiring a plurality of captured images, each of which indicates a plurality of divided regions obtained by dividing a predetermined region on the object, and each of which indicates adjacent divided regions and has an overlapping region that partially overlaps each other in each of the image pairs; b) generating a similarity map that indicates a distribution of similarity by template matching in the overlapping region of each of the image pairs, thereby identifying a relative positional deviation amount in each of the image pairs; c) determining a specific image pair in which directionality occurs in the similarity map, and assigning a weight to the specific image pair less than that of other image pairs, while determining a joining position of the plurality of captured images based on the positional deviation amount in the plurality of image pairs included in the plurality of captured images; and d) joining the plurality of captured images according to the joining positions to generate a joined image. In the step b), a determination process is performed to determine whether or not there is directionality in the similarity map of each image pair. do. An eighth aspect of the present invention is an image acquisition method for acquiring an image of an object, comprising the steps of: a) acquiring a plurality of captured images, each of which indicates a plurality of divided regions obtained by dividing a predetermined region on the object, and each of which has an overlapping region that partially overlaps in each image pair indicating adjacent divided regions; b) generating a similarity map that indicates a distribution of similarity by template matching in the overlapping region of each of the image pairs, thereby identifying the relative amount of positional shift in each of the image pairs; and c) determining an image pair in which directionality occurs in the similarity map as a specific image pair, and increasing the weight of the specific image pair relative to other image pairs. and d) a step of combining the plurality of captured images in accordance with the combining positions, thereby generating a combined image. In the step b), a map angle indicating the orientation of the direction of the similarity map of the specific image pair is obtained, and for a predetermined image pair, a planned angle of the direction of the similarity map is set in advance based on the area on the object indicated by the image pair. In the step c), a weight of the image pair is determined based on the difference between the map angle and the planned angle. [Effects of the Invention]

[0014] According to the present invention, a plurality of captured images can be combined with high precision. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an image acquisition device. [Figure 2] FIG. 1 illustrates the configuration of a computer. [Figure 3] FIG. 10 is a diagram showing a processing flow for acquiring an image of an object. [Figure 4] FIG. [Figure 5] FIG. 2 is a diagram showing a plurality of captured images. [Figure 6A] FIG. 10 is a diagram for explaining a similarity map. [Figure 6B] FIG. 10 is a diagram for explaining a similarity map. [Figure 7] FIG. 10 illustrates a plurality of similarity maps. [Figure 8] FIG. 10 is a diagram showing the amount of misregistration determined from a plurality of image pairs. [Figure 9] FIG. DETAILED DESCRIPTION OF THE INVENTION

[0016] FIG. 1 is a diagram showing the configuration of an image acquisition device 1 according to one embodiment of the present invention. The image acquisition device 1 is a device that acquires an image of an object. In this embodiment, the object is, for example, cells in a container such as a petri dish. The image acquisition device 1 includes an image acquisition unit 2 and a computer 3. In FIG. 1, the functional configuration realized by the computer 3 (a positional deviation amount identification unit 41, a bonding position determination unit 42, and a combined image generation unit 43) is shown in blocks.

[0017] The captured image acquisition unit 2 includes an imaging unit 21 and a movement mechanism 22. The imaging unit 21 has an imaging element and the like, and captures an image of an object. The movement mechanism 22 has a motor, a ball screw and the like, and moves the imaging unit 21 relative to the object. The operation of the captured image acquisition unit 2 to capture an image of an object will be described later. The image captured by the captured image acquisition unit 2 (hereinafter referred to as the "captured image") is output to the computer 3.

[0018] FIG. 2 shows the configuration of the computer 3. The computer 3 has a typical computer system configuration including a CPU 31, a ROM 32, a RAM 33, a fixed disk 34, a display 35, an input unit 36, a reading device 37, a communication unit 38, a GPU 39, and a bus 30. The CPU 31 performs various arithmetic operations. The GPU 39 performs various arithmetic operations related to image processing. The ROM 32 stores basic programs. The RAM 33 and the fixed disk 34 store various types of information. The display 35 displays various types of information such as images. The input unit 36 ​​includes a keyboard 36a and a mouse 36b for receiving input from an operator. The reading device 37 reads information from a computer-readable recording medium 371 such as an optical disk, a magnetic disk, a magneto-optical disk, or a memory card. The communication unit 38 transmits and receives signals between the captured image acquisition unit 2 and external devices. The bus 30 is a signal circuit that connects the CPU 31, the GPU 39, the ROM 32, the RAM 33, the fixed disk 34, the display 35, the input unit 36, the reading device 37, and the communication unit 38.

[0019] In computer 3, program 372 is read in advance from recording medium 371 via reader 37 and stored on fixed disk 34. Program 372 may be stored on fixed disk 34 via a network. CPU 31 and GPU 39 execute arithmetic processing using RAM 33 and fixed disk 34 in accordance with program 372. CPU 31 and GPU 39 function as a computing unit in computer 3. Other components functioning as a computing unit may be employed in addition to CPU 31 and GPU 39.

[0020] In the image acquisition device 1, the computer 3 executes arithmetic processing and the like in accordance with the program 372, thereby realizing the functional configuration shown by the blocks in FIG. 1. That is, the CPU 31, GPU 39, ROM 32, RAM 33, fixed disk 34, and their peripheral components of the computer 3 realize a misalignment amount specifying unit 41, a joining position determining unit 42, and a combined image generating unit 43. All or part of these functions may be realized by dedicated electrical circuits. Details of the misalignment amount specifying unit 41, the joining position determining unit 42, and the combined image generating unit 43 will be described later.

[0021] FIG. 3 is a diagram showing a process flow in which the image acquisition device 1 acquires an image of an object. In acquiring an image of an object by the image acquisition device 1, first, a plurality of captured images are acquired by the captured image acquisition unit 2 (step S11). In the example of FIG. 4, the object 9 is a cell in a container 91, and a plurality of divided regions 81 are set by dividing a predetermined region (here, the entire object 9) from which an image is to be acquired in a planar view. In the captured image acquisition unit 2, the moving mechanism 22 sequentially positions the imaging unit 21 above the plurality of divided regions 81, thereby acquiring a plurality of captured images showing each of the plurality of divided regions 81. In this processing example, the captured image is assumed to be a grayscale image, but the captured image may also be a color image.

[0022] In this case, if two captured images showing two adjacent divided areas 81 in the vertical or horizontal direction of Fig. 4 are called an "image pair," each image pair has an overlapping area that partially overlaps with the other. For example, if there are four adjacent divided areas 81 in the vertical and horizontal directions for a divided area 81 shown in a certain captured image, the captured image forms an image pair with each of the four captured images (hereinafter referred to as "adjacent captured images") showing the four divided areas 81. Furthermore, the captured image has an overlapping area that partially overlaps with each of the four adjacent captured images.

[0023] FIG. 5 is a diagram showing four adjacent captured images 71, with the pixel arrangement directions (horizontal and vertical directions) in each captured image 71 indicated as the x and y directions. The captured image 71 at the top left of FIG. 5 forms an image pair 72 with the captured image 71 at the top right, and these two images show two divided regions 81 adjacent to each other in the horizontal direction. The captured image 71 at the top right also forms an image pair 72 with the captured image 71 at the bottom right, and these two images show two divided regions 81 adjacent to each other in the vertical direction. The captured image 71 at the bottom right also forms an image pair 72 with the captured image 71 at the bottom left, and these two images show two divided regions 81 adjacent to each other in the horizontal direction. The captured image 71 at the bottom left also forms an image pair 72 with the captured image 71 at the top left, and these two images show two divided regions 81 adjacent to each other in the vertical direction. In FIG. 5, the overlapping regions 73 in each image pair 72 are enclosed by a thick rectangle.

[0024] The overlapping regions 73 of the image pair 72 are regions of the same size, and when an ideal captured image 71 is acquired by the captured image acquisition unit 2 (i.e., when the captured image 71 is acquired without any positional deviation or the like occurring during imaging), the region on the object 9 indicated by the overlapping region 73 included in one captured image 71 matches the region on the object 9 indicated by the overlapping region 73 included in the other captured image 71. In reality, due to the influence of errors in the amount of movement of the imaging unit 21 by the movement mechanism 22, etc., the regions indicated by the two overlapping regions 73 in the image pair 72 do not match completely.

[0025] The misalignment amount specifying unit 41 generates a similarity map (also called a score map) by template matching in the overlapping region 73 of each image pair 72 (step S12). Figures 6A and 6B are diagrams for explaining the similarity map, showing the image pair 72 in the upper row and the similarity map 74 in the lower row. Each image pair 72 in Figures 6A and 6B shows two divided regions 81 adjacent in the left-right direction.

[0026] In generating the similarity map 74, for example, an image with the outer edges removed from the overlap region 73 included in one captured image 71 of the image pair 72 (i.e., an image showing the center of the overlap region 73) is extracted, and the center of this image is superimposed on the center of the overlap region 73 included in the other captured image 71. Then, the sum of squares of the differences in pixel values ​​between the overlapping pixels is calculated, and the smaller the sum of squares, the larger the value (e.g., the reciprocal of the sum of squares) is calculated as the similarity. This similarity becomes the value at the origin of the similarity map 74. The misalignment amount specifying unit 41 moves the image in one overlap region 73 relative to the other overlap region 73 in the x and y directions, and calculates the similarity at each position. This generates a multi-valued similarity map 74 indicating the similarity of the image at each relative position with respect to the other overlap region 73 (see the lower rows of FIGS. 6A and 6B ). The similarity map 74 indicates the distribution of similarity in the overlap region 73 of the image pair 72. Other values ​​indicating similarity, such as normalized correlation, may be used to generate similarity map 74. In similarity maps 74 in the lower parts of Figures 6A and 6B, the greater the similarity, the higher the density (blacker).

[0027] Once the similarity map 74 is generated, the relative amount of misalignment between each image pair 72 is identified (step S13). The amount of misalignment is represented, for example, by a vector from the origin to the position where the similarity is maximized in the similarity map 74. The amount of misalignment may be obtained from an image obtained by applying a smoothing filter or the like to the similarity map 74.

[0028] Here, the image pair 72 in FIG. 6A is compared with the image pair 72 in FIG. 6B. In the image pair 72 in FIG. 6A, the overlapping region 73 shows only the object 9. Therefore, in the similarity map 74, a dot-like region of high similarity appears based on the characteristics of the object 9 itself. On the other hand, in the image pair 72 in FIG. 6B, the overlapping region 73 shows not only the object 9 but also part of the edge of the container 91. Because the edge of the container 91 has higher contrast than the object 9, the similarity map 74 is likely to show a band-like region of high similarity extending toward the edge of the container 91. In this way, when a region of high similarity exists extending in a specific direction, the position where the similarity is maximum is likely to be included in that region, resulting in a misalignment amount that is likely to be abnormal, i.e., a misalignment amount with low reliability. Therefore, the following process by the image acquisition device 1 suppresses the influence of a misalignment amount with low reliability.

[0029] 7 is a diagram showing four similarity maps 74 generated from the four image pairs 72 of FIG. 5, and the positions of the four similarity maps 74 correspond to the positions of the four image pairs 72, respectively. FIG. 8 is a diagram showing the amount of misalignment determined from the four image pairs 72, and the relative amount of misalignment in each image pair 72 is indicated by arrows 79 (one arrow is denoted by reference numeral 79a) arranged between the two captured images 71 constituting the image pair 72. The amount of misalignment (see arrows 79a) in the image pair 72 of the captured images 71 at the top left and top right in FIG. 8 is abnormal due to the influence of the edge of the container 91.

[0030] Once the misalignment amount is identified, the misalignment amount identifying unit 41 executes a determination process to determine whether the similarity map 74 of each image pair 72 has directionality (step S14). In one example of the determination process, the similarity map 74 is binarized to generate a binarized image, thereby identifying a region of high similarity (hereinafter also referred to as a "region of interest"). Next, elliptical approximation is performed on the region of interest in the binarized image. If the ratio of the lengths of the major axis and minor axis of the approximated ellipse (major axis length / minor axis length) is equal to or greater than a predetermined value, it is determined that the similarity map 74 has directionality. On the other hand, if the ratio is less than a predetermined value, it is determined that the similarity map 74 does not have directionality. Before performing elliptical approximation on the region of interest, the area of ​​the region of interest may be calculated. If the area is less than the predetermined area, it may be determined that the similarity map 74 does not have directionality. The binarization threshold for the similarity map 74 may be determined by a known method or may be a fixed value. In the following description, an image pair 72 determined to have a direction in the similarity map 74 will be referred to as a "specific image pair 72."

[0031] Thereafter, a map angle indicating the direction of the directionality of the similarity map 74 is obtained for the specific image pair 72 (step S15). The map angle can be obtained, for example, by performing a Hough transform on the binarized image of the similarity map 74. The map angle may also be obtained by other well-known methods, such as determining the principal axes of inertia in the binarized image or performing elliptical approximation.

[0032] The above-described determination process and acquisition of the map angle (steps S14 and S15) can also be performed on the multi-value similarity map 74. For example, two-dimensional Gaussian fitting (Gaussian fitting at each angle) is performed on the similarity map 74 to check whether or not there is angular bias in the spread of the distribution (standard deviation). The presence or absence of bias can be determined, for example, by the ratio (maximum value / minimum value) of the maximum and minimum values ​​of the standard deviations at multiple angles. If this ratio is equal to or greater than a predetermined value, it is determined that the similarity map 74 has directionality, and the angle with the largest standard deviation is acquired as the map angle.

[0033] As described above, in the positional deviation amount specifying unit 41, when an area with high similarity in the similarity map 74 indicates directionality (i.e., the similarity map 74 indicates directionality), the image pair 72 of the similarity map 74 is determined to be a specific image pair 72, and a map angle is acquired. In the example of FIG. 7, the upper similarity map 74 indicates directionality, and the image pair 72 of the captured images 71 at the top left and top right in FIG. 8 corresponding to the similarity map 74 is specified as the specific image pair 72. Note that the map angle may only be acquired as necessary.

[0034] The combining position determination unit 42 determines combining positions, which are the final positions of the multiple captured images 71, based on the amount of positional deviation in the multiple image pairs 72 included in the multiple captured images 71 (step S16). Here, as described above, each captured image 71 forms image pairs 72 with captured images 71 of all divided areas 81 adjacent to the divided area 81 of the captured image 71 (i.e., adjacent captured images 71). Furthermore, each adjacent captured image 71 also forms image pairs 72 with other captured images 71. Therefore, the combining position of each captured image 71 is affected not only by the amount of positional deviation in the multiple image pairs 72 that the captured image 71 forms, but also by the amount of positional deviation in the other image pairs 72.

[0035] The combining position determination unit 42 treats as an error the difference between the final positional deviation amount between each captured image 71 arranged at the combining position and each adjacent captured image 71, i.e., the final positional deviation amount of each image pair 72 and the positional deviation amount specified for that image pair 72 in step S13 (hereinafter referred to as the "original positional deviation amount"). Then, the combining position is determined so that the sum of squares of the errors in multiple (all) image pairs 72 excluding the specific image pair 72 is minimized for each of the x and y directions.

[0036] As described above, in this processing example, the joining positions of the multiple captured images 71 are determined by the least squares method after excluding the specific image pair 72. In determining the joining positions, the original positional deviation amount in the specific image pair 72 may be changed to zero, and the joining positions may be determined so as to minimize the sum of squares of the errors in the multiple (all) image pairs 72 including the specific image pair 72. Alternatively, the joining positions may be determined by the weighted least squares method, with the error in the specific image pair 72 being set to a weight that is smaller than the errors in the other image pairs 72 (image pairs 72 other than the specific image pair 72). In determining the joining positions, both when the specific image pair 72 is excluded and when the original positional deviation amount in the specific image pair 72 is changed to zero, it can be said that the joining positions of the multiple captured images 71 are essentially determined by setting the weight (weight of the original positional deviation amount) of the specific image pair 72 smaller than that of the other image pairs 72 so as to reduce the influence of the specific image pair 72 on the determination of the joining positions. In determining the binding positions, in addition to the least squares method, the back propagation method or the like may be used, and various methods for minimizing errors can be used.

[0037] The combined image generating unit 43 combines the multiple captured images 71 according to the combining positions to generate the combined image 70 shown in FIG. 9 (step S17). Here, a comparative example process will be described in which the combining positions are determined taking into account the original positional deviation of the specific image pair 72, which is the upper-left and upper-right captured images 71, in the example of FIG. 8 . In the comparative example process, the relative positions of the upper-left and upper-right captured images 71 are significantly deviated from their ideal positions due to the influence of the positional deviation indicated by the arrow 79a. This influence makes it easier for deviations to occur in the other captured images 71 as well. In FIG. 9, the position where the upper-right captured image 71 is placed in the comparative example process is indicated by a two-dot chain rectangle. In contrast, in this processing example, the weight of the specific image pair 72 is reduced to determine the combining position, thereby suppressing positional deviations caused by the specific image pair 72 in the combined image 70 shown in FIG. 9 .

[0038] In the image acquisition device 1, an evaluation value indicating the strength of directionality of the similarity map 74 may be acquired for each specific image pair 72, and the evaluation value may be used in determining the joining position. In one example, the evaluation value is calculated when acquiring the map angle in step S15 (or when executing the determination process in step S14).

[0039] As described above, when a binarized image showing a region with high similarity (i.e., a region of interest) in the similarity map 74 is generated and a map angle is obtained by performing a Hough transform on the binarized image, the count number in the Hough transform, for example, is used as the evaluation value. When determining the principal axis of inertia in the binarized image, the evaluation value is, for example, the spread of the distribution of pixels included in the region of interest when viewed in the principal axis direction (when projected onto the principal axis) (e.g., the standard deviation of the coordinates of the pixels). When performing elliptical approximation of the region of interest in the binarized image, the evaluation value is, for example, the ratio of the lengths of the major axis and the minor axis of the approximated ellipse (major axis length / minor axis length). When performing two-dimensional Gaussian fitting on the similarity map 74, the evaluation value is, for example, the ratio of the maximum value to the minimum value of the standard deviation at multiple angles or the magnitude of the maximum value itself. The stronger the directionality of the similarity map 74, the larger the evaluation value is, and it can be considered as a score indicating the reliability of the map angle. The evaluation value can also be considered to indicate the low reliability of the amount of positional deviation in the specific image pair 72.

[0040] In step S16, for example, the reciprocal of the evaluation value for the error in the specific image pair 72 is set as a weight. The weight for the specific image pair 72 may be a value other than the reciprocal of the evaluation value, as long as it is smaller than the weights for other image pairs 72 and decreases as the evaluation value increases. Thereafter, the joining position is determined using the weighted least squares method. As a result, the lower the reliability of the positional deviation amount of a specific image pair 72, the smaller its influence on the determination of the joining position. Even when using another method such as backpropagation to determine the joining position, the weight of the specific image pair 72 may be determined using the evaluation value so that the influence of the specific image pair 72 on the determination of the joining position is reduced.

[0041] In the misalignment amount specifying unit 41, the determination process (step S14) for determining whether or not the similarity map 74 of each image pair 72 has directionality may be omitted, and evaluation values ​​may be acquired for all image pairs 72. In this case, the combining position determining unit 42 treats image pairs 72 whose evaluation values ​​are greater than a threshold as specific image pairs 72 and excludes them from determining the combining position. The threshold may be a predetermined value (a fixed value) or a value calculated from the evaluation values ​​of all image pairs 72 (for example, a value obtained by adding the standard deviation to the average of the evaluation values). In this way, image pairs 72 whose evaluation values ​​are absolutely or relatively large are treated as specific image pairs 72. In determining the combining position, weights based on the evaluation values ​​may be set for all image pairs 72. In this case, image pairs 72 to which a relatively small weight is set are essentially treated as specific image pairs 72.

[0042] As described above, in the image acquisition device 1, the captured image acquisition unit 2 acquires a plurality of captured images 71, each of which has an overlapping region 73 where the images in each image pair 72 representing adjacent divided regions 81 partially overlap. The misalignment amount identification unit 41 generates a similarity map 74 indicating a distribution of similarity by template matching in the overlapping region 73 of each image pair 72, thereby identifying the relative misalignment amount in each image pair 72. The combining position determination unit 42 determines the combining positions of the plurality of captured images 71 based on the misalignment amounts in the plurality of image pairs 72 included in the plurality of captured images 71, while setting a weight for the specific image pair 72 smaller than that for the other image pairs 72. The combined image generation unit 43 combines the plurality of captured images 71 according to the combining positions to generate a combined image 70. In this way, by reducing the influence of specific image pairs 72 with low reliability of positional deviation (high possibility of abnormality) when determining the joining position, multiple captured images 71 can be joined with high accuracy, and a joined image 70 with beautiful joints can be stably generated.

[0043] Preferably, the misalignment amount specifying unit 41 acquires an evaluation value indicating the strength of directionality of the similarity map 74 of the specific image pair 72, and the combining position determining unit 42 determines the weight of the specific image pair 72 using the evaluation value. This allows the weight of the specific image pair 72 to be appropriately determined based on the strength of directionality of the similarity map 74; specifically, the weight can be reduced for a specific image pair 72 with a lower reliability of the misalignment amount. As a result, the combining position can be determined with higher accuracy.

[0044] Preferably, the misalignment amount specifying unit 41 executes a determination process for determining whether or not there is directionality in the similarity map 74 of each image pair 72. This makes it possible to easily specify the specific image pair 72, and more reliably reduce the influence of the specific image pair 72 in determining the joining position.

[0045] Incidentally, when the shape of the container 91 that contains the object 9 is known, it is possible to predict which of the multiple captured images 71 will reflect the edge of the container 91 in the overlapping region 73, and it is also possible to predict the orientation of the edge portion in the overlapping region 73. In other words, it is possible to determine in advance, as a specific image pair candidate, an image pair 72 that includes the overlapping region 73 in which the edge of the container 91 is expected to be reflected, and to set in advance the orientation of the edge in the overlapping region 73 as a planned angle of directionality of the similarity map 74. Below, a processing example using the specific image pair candidate and the planned angle of directionality will be described.

[0046] In this processing example, after the misalignment amount specifying unit 41 specifies the misalignment amounts for the multiple image pairs 72 (FIG. 3: step S13), a determination process is performed to determine whether or not the similarity map 74 has directionality only for the specific image pair candidates (step S14). At this time, the determination process is performed only for the expected angle and its neighboring angles. Specifically, when two-dimensional Gaussian fitting is performed in the determination process, Gaussian fitting is performed only within a predetermined angle range (e.g., an angle range of 10 to 90 degrees) centered on the expected angle. Then, if the maximum value of the standard deviation for these angles is equal to or greater than a predetermined value, it is determined that the similarity map 74 has directionality. This allows the specific image pair 72 to be specified from the specific image pair candidates. In this case, the map angle is the angle at which the maximum standard deviation is obtained (step S15). When performing a determination process other than two-dimensional Gaussian fitting, the angle to be processed may be limited to within the above angle range. Determining the joining position and generating the joined image (steps S16 to S17) are similar to those described above.

[0047] As described above, in this processing example, specific image pair candidates are determined in advance for a plurality of image pairs 72 based on the regions on the object 9. The positional deviation amount specifying unit 41 executes the determination process only for the specific image pair candidates. This allows the specific image pair 72 to be specified in a shorter time than when the determination process is executed for all image pairs 72. It also makes it possible to prevent or suppress the specific image pair 72 from being erroneously specified, i.e., to stably specify the specific image pair 72.

[0048] Furthermore, for a predetermined image pair 72 (in the above example, a specific image pair candidate), a planned angle of the directionality of the similarity map 74 is set in advance based on the area on the object 9 indicated by the image pair 72. The positional deviation amount specifying unit 41 executes a determination process for the image pair 72 only with respect to the planned angle and angles nearby. This allows the determination process to be completed in a shorter time than when the determination process is executed for all angles. It also makes it possible to stably specify the specific image pair 72.

[0049] Information about the specific image pair candidates may be used in determining the combining position. In this case, the misalignment amount specifying unit 41 specifies the specific image pair 72 by, for example, performing a determination process on all image pairs 72. When determining the combining position, the combining position determining unit 42 assigns a lower weight to only the specific image pair 72 included in the specific image pair candidate than to the other image pairs 72 (only the specific image pair 72 included in the specific image pair candidate may be excluded). On the other hand, the specific image pair 72 not included in the specific image pair candidate is treated as a normal image pair 72 (an image pair 72 that is not a specific image pair 72), i.e., the weight is not reduced in determining the combining position. This makes it possible to prevent or suppress the influence of the image pair 72, which has directionality in the similarity map 74 due to the characteristics of the object 9 itself, from being reduced in determining the combining position, thereby enabling the multiple captured images 71 to be combined more accurately.

[0050] The information on the planned angle may be used in determining the joining position. For example, the joining position determination unit 42 calculates the difference (absolute value) between the map angle acquired in step S15 and the planned angle for each specific image pair 72 for which a planned angle is set (i.e., specific image pairs 72 included in the specific image pair candidates). Specific image pairs 72 for which the difference is equal to or less than a predetermined value are excluded in determining the joining position. On the other hand, specific image pairs 72 for which the difference is greater than the predetermined value are treated as normal image pairs 72 and used in determining the joining position. Specific image pairs 72 for which no planned angle is set may also be treated as normal image pairs 72. Furthermore, when determining the joining position using a weighted least squares method or the like, for each specific image pair 72 for which a planned angle is set, the weight of the specific image pair 72 is reduced as the difference between the map angle and the planned angle decreases. Specific image pairs 72 for which no planned angle is set may be treated as normal image pairs 72.

[0051] As described above, in this processing example, for a predetermined image pair 72 (in the above example, a specific image pair candidate), a planned angle of the directionality of the similarity map 74 is set in advance based on the area on the object 9 indicated by the image pair 72. In the combining position determination unit 42, the weight of the image pair 72 is determined based on the difference between the map angle and the planned angle. This reduces the influence (weight) of the image pair 72 that is likely to have an abnormal amount of misalignment, and appropriately reflects the influence of the image pair 72 that is unlikely to have an abnormal amount of misalignment, making it possible to accurately obtain the combining position.

[0052] The image acquisition device 1 and the image acquisition method can be modified in various ways.

[0053] The shape of the edge of the container 91 in plan view is not limited to a circle, and may be another shape such as a rectangle. Furthermore, the image acquisition device 1 may be used when directionality occurs in the similarity map 74 due to an influence other than that of the edge of the container 91. The image acquisition device 1 is particularly suitable for combining multiple captured images 71, including captured images 71 in which a member different from the target object 9 is reflected.

[0054] The object 9 is not limited to cells, and may be various substrates, mechanical parts, etc. The object 9 does not necessarily have to be contained in a container. The region of the object 9 from which the multiple captured images 71 are acquired, i.e., the region in which the multiple divided regions 81 are set, may be a region showing a part of the object 9.

[0055] The configurations in the above-described embodiment and each modification may be combined as appropriate as long as they are not mutually contradictory. [Explanation of symbols]

[0056] 1. Image acquisition device 2. Image acquisition unit 9 Objects 41 Position deviation amount determination unit 42 Binding position determination unit 43 Combined image generation unit 70 combined images 71 Captured images 72 image pairs 73 Overlapping area 74 Similarity Map 81 Split area Steps S11~S17

Claims

1. An image capture device for capturing an image of an object, an image acquisition unit that acquires a plurality of captured images, each of which represents a plurality of divided regions obtained by dividing a predetermined region on an object, and in which overlapping regions are provided in each image pair representing adjacent divided regions; a positional deviation amount specifying unit that specifies a relative positional deviation amount in each of the image pairs by generating a similarity map that indicates a distribution of similarity by template matching in the overlapping region of each of the image pairs; a combining position determining unit that determines combining positions of the plurality of captured images based on positional deviation amounts of a plurality of image pairs included in the plurality of captured images while determining a specific image pair in which a directionality occurs in the similarity map and weighting the specific image pair less than other image pairs; a combined image generating unit that combines the plurality of captured images according to the combining positions to generate a combined image; Equipped with The image acquisition device, wherein the positional deviation amount specifying unit executes a determination process for determining whether or not the similarity map of each of the image pairs has directionality.

2. 2. The image acquisition device according to claim 1, the positional deviation amount specifying unit obtains an evaluation value indicating the strength of directionality of the similarity map of the specific image pair; The image acquisition device is characterized in that the combining position determining unit determines a weight of the specific image pair using the evaluation value.

3. 3. The image acquisition device according to claim 1, a predetermined angle of the directionality of the similarity map is preset for a given pair of images based on an area on the object represented by the pair of images; The image acquisition device is characterized in that the positional deviation amount specifying unit executes the determination process only with respect to the expected angle and angles in the vicinity thereof for the image pair.

4. 3. The image acquisition device according to claim 1, In the plurality of image pairs, specific image pair candidates are determined in advance based on regions on an object; The image acquisition device is characterized in that the positional deviation amount specifying unit executes the determination process only for the specific image pair candidate.

5. 3. The image acquisition device according to claim 1, The image acquisition device, wherein the positional deviation amount specifying unit obtains a map angle indicating a direction of the similarity map of the specific image pair.

6. An image acquisition device for acquiring an image of an object, comprising: an image acquisition unit that acquires a plurality of captured images, each of which represents a plurality of divided regions obtained by dividing a predetermined region on an object, and in which overlapping regions are provided in each image pair representing adjacent divided regions; a positional deviation amount specifying unit that specifies a relative positional deviation amount in each of the image pairs by generating a similarity map that indicates a distribution of similarity by template matching in the overlapping region of each of the image pairs; a combining position determining unit that determines combining positions of the plurality of captured images based on positional deviation amounts of a plurality of image pairs included in the plurality of captured images while determining a specific image pair in which a directionality occurs in the similarity map and weighting the specific image pair less than other image pairs; a combined image generating unit that combines the plurality of captured images according to the combining positions to generate a combined image; Equipped with the positional deviation amount specifying unit obtains a map angle indicating a direction of the similarity map of the specific image pair; a predetermined angle of the directionality of the similarity map is preset for a given pair of images based on an area on the object represented by the pair of images; The image acquisition device, wherein the combining position determining unit determines a weight of the image pair based on a difference between the map angle and the expected angle.

7. 1. An image acquisition method for acquiring an image of an object, comprising: a) acquiring a plurality of captured images each showing a plurality of divided regions obtained by dividing a predetermined region on an object, and each pair of images showing adjacent divided regions having overlapping regions that partially overlap each other; b) generating a similarity map showing a distribution of similarities in the overlapping regions of each of the image pairs by template matching, thereby identifying a relative positional shift amount in each of the image pairs; c) determining a specific image pair that has directionality in the similarity map, and weighting the specific image pair less than other image pairs, while determining a joining position of the plurality of captured images based on the positional deviation amounts of the plurality of image pairs included in the plurality of captured images; d) combining the plurality of captured images according to the combining positions to generate a combined image; Equipped with In the step b), a determination process is executed to determine whether or not the similarity map of each image pair has directionality.

8. An image acquisition method for acquiring an image of an object, comprising: a) acquiring a plurality of captured images each showing a plurality of divided regions obtained by dividing a predetermined region on an object, and each pair of images showing adjacent divided regions having overlapping regions that partially overlap each other; b) generating a similarity map showing a distribution of similarities in the overlapping regions of each of the image pairs by template matching, thereby identifying a relative positional shift amount in each of the image pairs; c) determining a specific image pair that has directionality in the similarity map, and weighting the specific image pair less than other image pairs, while determining a joining position of the plurality of captured images based on the positional deviation amounts of the plurality of image pairs included in the plurality of captured images; d) combining the plurality of captured images according to the combining positions to generate a combined image; Equipped with In the step b), a map angle indicating a direction of the similarity map of the specific image pair is obtained; a predetermined angle of the directionality of the similarity map is preset for a given pair of images based on an area on the object represented by the pair of images; In the step c), a weight of the image pair is determined based on a difference between the map angle and the predetermined angle.

Images