Information processing method, information processing device, information processing system, and program

Abstract

This provides an information processing method that can accurately search for corresponding points in multiple images. [Solution] In a shape measurement method according to one aspect of the present disclosure, a first color vector is calculated based on two or more pixel values ​​indicating the color of a pixel in a first image, and a second color vector is calculated based on two or more pixel values ​​indicating the color of a pixel in a second image different from the first image (S10), the angle between the first color vector and the second color vector is calculated (S20), weight values ​​are calculated based on the length of the first color vector and the length of the second color vector (S30), and corresponding points in the first image and the second image are searched based on the calculated angle and weight values ​​(S40).

Description

[Technical Field]

[0001] This disclosure relates to information processing methods, information processing devices, information processing systems, and programs. [Background technology]

[0002] Conventionally, there is a technique for measuring the three-dimensional shape of an object based on images generated by two imaging devices capturing the object from different positions. Three-dimensional shape measurement is performed, for example, by triangulation using the image position shift (parallax) between the two images. For example, corresponding points indicating the same position in the two images are searched for, and the parallax of the two imaging devices is calculated based on the searched corresponding points.

[0003] Patent Document 1 discloses a three-dimensional shape measuring device that calculates information about the three-dimensional shape using triangulation based on the hue data of each pixel in an image obtained by capturing reflected light from a pattern light whose hue changes continuously from an object to be measured. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Patent No. 5743433 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] In cases where an object contains multiple components with different reflectivity, the color of the reflected light from the object may be the same even if the color of the reflected light differs. In such cases, corresponding points may not be found correctly. Also, for example, if the brightness environment in which the two images are captured is different, corresponding points may not be found correctly.

[0006] This disclosure provides an information processing method that can accurately search for corresponding points in multiple images. [Means for solving the problem]

[0007] An information processing method according to one aspect of the present disclosure calculates a first color vector based on two or more pixel values ​​indicating the color of a pixel in a first image, calculates a second color vector based on two or more pixel values ​​indicating the color of a pixel in a second image different from the first image, calculates the angle between the first color vector and the second color vector, calculates weight values ​​based on the length of the first color vector and the length of the second color vector, and searches for corresponding points in the first image and the second image based on the cost value calculated based on the angle and the weight values.

[0008] Furthermore, an information processing device according to one aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to calculate a first color vector based on two or more pixel values ​​indicating the color of pixels in a first image, and calculates a second color vector based on two or more pixel values ​​indicating the color of pixels in a second image different from the first image, calculates the angle between the first color vector and the second color vector, calculates weight values ​​based on the length of the first color vector and the length of the second color vector, and searches for corresponding points in the first image and the second image based on the calculated angle and cost values ​​based on the weight values.

[0009] Furthermore, an information processing system according to one aspect of the present disclosure comprises the information processing device described above, a first imaging device that generates the first image by imaging an object in two or more wavelength bands, and a second imaging device that generates the second image by imaging the object in the two or more wavelength bands from a position different from the first imaging device.

[0010] Furthermore, a program relating to one aspect of this disclosure is a program for a computer to execute an information processing method. [Effects of the Invention]

[0011] According to an information processing method and the like according to one aspect of the present disclosure, corresponding points in a plurality of images can be accurately searched for.

Brief Description of the Drawings

[0012] [Figure 1] FIG. 1 is a block diagram showing the configuration of a shape measurement system according to an embodiment. [Figure 2] FIG. 2 is a diagram for explaining a specific example of a calculation process of an angle formed by two color vectors according to an embodiment. [Figure 3] FIG. 3 is a diagram for explaining a cost value according to an embodiment. [Figure 4] FIG. 4 is a flowchart showing a shape measurement method according to an embodiment. [Figure 5] FIG. 5 is a block diagram showing an information processing apparatus according to an embodiment. [Figure 6] FIG. 6 is a flowchart showing an information processing method according to an embodiment.

Modes for Carrying Out the Invention

[0013] Hereinafter, embodiments will be specifically described with reference to the drawings.

[0014] In addition, all of the embodiments described below show comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Also, among the components in the following embodiments, components not described in the independent claims are described as optional components.

[0015] In this specification, ordinal numbers such as "first" and "second" do not mean the number or order of components, unless otherwise specified, and are used for the purpose of avoiding confusion and distinguishing between the same kind of components.

[0016] (Embodiment) [composition] Figure 1 is a block diagram showing the configuration of the shape measurement system 10 according to an embodiment.

[0017] The shape measurement system 10 is a system for measuring the shape of an object, such as a part. Specifically, the shape measurement system 10 images the object using multiple imaging devices, such as stereo cameras, which image the object from different positions, and measures the three-dimensional shape of the object based on the images of the object captured (generated) by each imaging device. The shape measurement system 10 is an example of an information processing system.

[0018] The shape measurement system 10 comprises a shape measurement device 100, a first imaging device 200, a second imaging device 210, and an illumination device 220.

[0019] The shape measuring device 100 is a computer that measures the shape of an object based on multiple images generated by the first imaging device 200 and the second imaging device 210 capturing images of the object. For example, the shape measuring device 100 calculates the three-dimensional shape of the object (information indicating the three-dimensional shape) from multiple images acquired from the first imaging device 200 and the second imaging device 210 using the triangulation method. The shape measuring device 100 is an example of an information processing device.

[0020] The shape measuring device 100 is implemented as a computer that includes, for example, a communication interface for communicating with the first imaging device 200, the second imaging device 210, and the illumination device 220 of the shape measuring system 10, a non-volatile memory for storing programs executed by each processing unit, a volatile memory which is a temporary storage area for executing programs, input / output ports for sending and receiving signals, and a processor such as a CPU (Central Processing Unit) for executing programs. The communication interface may be implemented by a connector to which a communication line is connected for wired communication, or by a wireless communication circuit for wireless communication.

[0021] The first imaging device 200 is a camera that generates a first image by imaging an object in two or more wavelength bands. In other words, the first imaging device 200 images an object by detecting reflected light from the object in two or more wavelength bands. The first image is an image generated by the first imaging device 200 imaging the object.

[0022] The second imaging device 210 is a camera that generates a second image by imaging the object in two or more wavelength bands. In other words, the second imaging device 210 images the object by detecting reflected light from the object in two or more wavelength bands. The second image is an image generated by the second imaging device 210 imaging the object.

[0023] The first imaging device 200 and the second imaging device 210, for example, capture images of the same object from different positions. As a result, parallax occurs between the first image and the second image. The shape measuring device 100 calculates this parallax and uses the calculated parallax to measure the shape of the object. Specifically, the shape measuring device 100 uses the first image, the second image, the parallax, and the distance between the first imaging device 200 and the second imaging device 210 to measure the three-dimensional shape of the object.

[0024] The first imaging device 200 and the second imaging device 210 image an object by detecting light in the same wavelength band. Specifically, the first imaging device 200 and the second imaging device 210 each image an object by detecting light in two or more different wavelength bands.

[0025] The first imaging device 200 and the second imaging device 210 function as, for example, a stereo camera. The first imaging device 200 and the second imaging device 210 are arranged side by side, for example, so that their respective imaging directions are parallel.

[0026] The two or more wavelength bands can be arbitrarily determined and are not particularly limited. For example, the two or more wavelength bands are the corresponding wavelength bands for each of the three colors RGB (Red, Green, Blue). For example, the wavelength band for red is 640nm to 770nm, the wavelength band for green is 490nm to 550nm, and the wavelength band for blue is 430nm to 490nm.

[0027] The two or more wavelength bands may correspond to RGB, or to any color such as Y (Yellow), M (Magenta), C (Cyan), UV (Ultraviolet), or IR (Infrared). The bandwidth of the wavelength bands may be arbitrarily determined. For example, the first imaging device 200 and the second imaging device 210 may detect two or more lights with different wavelength bands (e.g., peak wavelengths) and a bandwidth of 50 nm to 150 nm. Alternatively, the first imaging device 200 and the second imaging device 210 may detect two or more lights with different wavelengths and a single wavelength. Furthermore, the two or more wavelength bands may correspond to R and B, UV and IR, or UV, RGB, and IR, and each wavelength band may be arbitrarily combined to realize the system.

[0028] The illumination device 220 is a light source that irradiates light onto the object. Specifically, the illumination device 220 irradiates the object with light in two or more wavelength bands. More specifically, the illumination device 220 irradiates the object with light that includes two or more wavelength bands detected by the first imaging device 200 and the second imaging device 210. The first imaging device 200 and the second imaging device 210 image the object by detecting, for example, the reflected light that has been irradiated onto the object by the illumination device 220 and reflected by the object.

[0029] The light emitted by the illumination device 220 may be any light that includes two or more wavelength bands detected by the first imaging device 200 and the second imaging device 210. For example, the light emitted by the illumination device 220 may include only two or more wavelength bands detected by the first imaging device 200 and the second imaging device 210. Alternatively, for example, the light emitted by the illumination device 220 may be white light if the first imaging device 200 and the second imaging device 210 detect light corresponding to RGB.

[0030] Furthermore, for example, the irradiation device 220 may irradiate the object with patterned light. For example, the patterned light may be a dot pattern light.

[0031] Furthermore, the pattern of the patterned light does not have to be a dot pattern. For example, the pattern of the patterned light could be a random pattern or a stripe pattern.

[0032] The irradiation device 220 can be implemented using, for example, an LED (Light Emitting Diode). The irradiation device 220 can be implemented using any type of light source, such as an LD (Laser Diode), a halogen lamp, or a fluorescent lamp.

[0033] The shape measurement system 10 may have multiple imaging devices, and may have three or more.

[0034] Next, we will describe the specific configuration of the shape measuring device 100.

[0035] The shape measuring device 100 includes an acquisition unit 110, a first calculation unit 120, a search unit 130, a second calculation unit 140, a measurement unit 150, an output unit 160, and a storage unit 170.

[0036] The acquisition unit 110 is a processing unit that acquires a first image generated by the first imaging device 200 imaging the object in two or more wavelength bands, and a second image generated by the second imaging device 210 imaging the object in two or more wavelength bands from a position different from that of the first imaging device 200. For example, the acquisition unit 110 causes the illumination device 220 to irradiate the object with light, then causes the first imaging device 200 and the second imaging device 210 to image the object, and then acquires the first and second images from the first imaging device 200 and the second imaging device 210.

[0037] The first calculation unit 120 is a processing unit that calculates a color vector based on two or more pixel values ​​indicating the color of pixels in an image acquired by the acquisition unit 110. Specifically, the first calculation unit 120 calculates a first color vector based on two or more pixel values ​​indicating the color of pixels in a first image. The first calculation unit 120 also calculates a second color vector based on two or more pixel values ​​indicating the color of pixels in a second image. The first color vector is a color vector calculated from the pixel values ​​of pixels included in the first image. The second color vector is a color vector calculated from the pixel values ​​of pixels included in the second image.

[0038] A pixel value is a value that indicates the color of a pixel. Two or more pixel values ​​are, for example, values ​​that indicate the colors corresponding to two or more wavelength bands of light detected by the first imaging device 200 and the second imaging device 210, respectively. Two or more pixel values ​​are, for example, values ​​that indicate each of the three colors of RGB. In other words, the value indicating the R color, the value indicating the G color, and the value indicating the B color in a pixel are examples of pixel values.

[0039] A color vector is a vector used to represent the color of a pixel. For example, if two or more pixel values ​​are the respective values ​​of the three colors RGB, the color vector is represented as a three-dimensional vector, such as (pixel value for R, pixel value for G, pixel value for B). Thus, the color vector calculated by the first calculation unit 120 corresponds to two or more pixel values ​​and is a vector of two or more dimensions.

[0040] Furthermore, for two or more pixel values, any color values ​​such as Y, M, C, UV, or IR may be used, in addition to RGB, to correspond to two or more wavelength bands.

[0041] The first calculation unit 120 calculates, for example, a color vector corresponding to each of the multiple pixels included in the first image and the second image, based on the pixel value of the pixel.

[0042] Furthermore, the first calculation unit 120 calculates the angle between the first color vector and the second color vector. For example, the first calculation unit 120 combines each of the multiple first color vectors calculated from the first image with the multiple second color vectors calculated from the second image, and calculates the angle between the combined first color vector and the second color vector.

[0043] Furthermore, the first calculation unit 120 calculates weight values ​​based on the length of the first color vector and the length of the second color vector. A specific example of how the weight values ​​are calculated will be described later.

[0044] The combination of multiple first-color vectors and multiple second-color vectors can be arbitrary. For example, multiple first-color vectors and multiple second-color vectors may be combined in a brute-force manner. Alternatively, for example, the combination of first-color vectors and second-color vectors from which the angle is calculated may be determined based on the position of pixels in the image.

[0045] The search unit 130 is a processing unit that searches for corresponding points in the first and second images based on the calculated angle and weight values. Specifically, the search unit 130 searches for corresponding points in the first and second images based on a cost value derived from the calculated angle and weight values. More specifically, the search unit 130 calculates a cost value based on the angle between the first and second color vectors, the length of the first color vector, and a weight value calculated based on the length of the second color vector, and determines the corresponding points based on the calculated cost value.

[0046] Corresponding points are points (e.g., pixels) that indicate the same position (e.g., the same location on an object) in the first image and the second image. The search unit 130 identifies the corresponding pixels in the first image and the corresponding pixels in the second image, for example, based on a plurality of angles calculated by the first calculation unit 120.

[0047] For example, in the search for corresponding points, the search unit 130 searches for corresponding points using similarity based on angles and weight values ​​calculated by the first calculation unit 120. For example, the first calculation unit 120 further calculates cosine similarity based on the calculated angles. Also, for example, the first calculation unit 120 calculates cost values ​​based on the calculated cosine similarity and weight values. For example, in the search for corresponding points, the search unit 130 searches for corresponding points based on cost values ​​calculated by the first calculation unit 120. Specifically, in calculating cosine similarity, the first calculation unit 120 calculates multiple cosine similarities based on multiple first color vectors calculated from two or more pixel values ​​of multiple pixels located in a first range in the first image, and multiple second color vectors calculated from two or more pixel values ​​of multiple pixels located in a second range in the second image.

[0048] For example, the first calculation unit 120 combines multiple first color vectors and multiple second color vectors in a brute-force manner, calculates an angle for each combination, and calculates cosine similarity based on the calculated angle. Furthermore, the first calculation unit 120 calculates multiple weight values ​​corresponding to the calculated cosine similarity values. Also, the first calculation unit 120 weights the calculated cosine similarity values ​​using the weight values ​​corresponding to each cosine similarity. Finally, the first calculation unit 120 calculates a cost value by adding up the weighted cosine similarity values. In this way, the first calculation unit 120 calculates the cost value based on the calculated cosine similarity values ​​and weight values. For example, the search unit 130 searches for corresponding points based on the cost value calculated by the first calculation unit 120. For example, in calculating the cost value, the first calculation unit 120 repeatedly moves at least one of the positions of the first range in the first image and the second range in the second image, and repeatedly calculates the cost value. In other words, the first calculation unit 120 repeatedly changes at least one of the positions of the first range and the second range, and calculates the cost value for each combination of the positions of the first range and the second range.

[0049] Furthermore, for example, the first calculation unit 120 may repeatedly calculate the cost value by repeatedly moving either the position of the first range in the first image or the position of the second range in the second image along the epipolar line. For example, the first calculation unit 120 may repeatedly calculate the cost value by repeatedly moving the position of the second range along one of the two orthogonal axes in the image coordinate system.

[0050] For example, in the search for corresponding points, the search unit 130 identifies the first range and the second range that have the highest cost value calculated by the first calculation unit 120. In other words, for example, in the search for corresponding points, the search unit 130 identifies the combination of the first range and the second range that have the highest cost value among the multiple cost values ​​calculated by the first calculation unit 120.

[0051] Thus, for example, the first calculation unit 120 defines a rectangular area (also called a kernel) in the first image that includes a certain pixel and the pixels surrounding that pixel as the first region, similar to block matching, and identifies a second region from the second image that, when combined with the first region, yields the highest cost value. Furthermore, for example, the search unit 130 determines that there are corresponding points in multiple pixels located in the identified first region and multiple pixels located in the identified second region.

[0052] The first range and the second range each contain the same number of pixels, for example, four pixels in the row direction and four pixels in the column direction of the image. The search unit 130 determines, for example, that pixels located in the specified first range and pixels located in the specified second range correspond to each other if they are at the same position in each range.

[0053] The size of the first and second ranges (the number of pixels included in these ranges) can be determined arbitrarily and are not particularly limited. For example, the number of pixels included in the first and second ranges can be 3×3, 4×4, or 5×5, if the number of pixels in the row direction of the image is (number of pixels in the row direction of the image) × (number of pixels in the column direction of the image).

[0054] The second calculation unit 140 is a processing unit that calculates the disparity between the first image and the second image based on the corresponding points found by the search unit 130. For example, the second calculation unit 140 calculates the disparity by matching (stereo matching) each part (each pixel) of the first image and the second image. For example, the second calculation unit 140 calculates the difference (amount of positional shift) between the corresponding points, which are the positions of corresponding parts between the first image and the second image, as the disparity.

[0055] The measurement unit 150 is a processing unit that measures the shape of an object based on the calculated parallax. For example, the measurement unit 150 measures the three-dimensional shape of an object using triangulation based on the first image, the second image, the parallax calculated by the second calculation unit 140, and the distance between the first imaging device 200 and the second imaging device 210. For example, the measurement unit 150 generates a depth image showing the object, which indicates the measurement result of the object's shape.

[0056] The output unit 160 is a processing unit that outputs the measurement results from the measurement unit 150. The output unit 160 outputs the measurement results of the shape of the object to a display that does not show the shape, for example, and displays a depth image showing the shape of the object on the display.

[0057] The shape measuring device 100 may also be equipped with the display.

[0058] Each processing unit, such as the acquisition unit 110, the first calculation unit 120, the search unit 130, the second calculation unit 140, the measurement unit 150, and the output unit 160, is implemented, for example, by a processor and a memory in which a control program executed by the processor is stored.

[0059] The memory unit 170 is a storage device that stores various types of information. For example, the memory unit 170 stores information indicating the distance between the first imaging device 200 and the second imaging device 210, and information indicating the size of the kernel. The memory unit 170 is implemented by, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0060] [Specific example] Next, we will explain the specific processes for calculating color vectors and searching for corresponding points. In the specific examples described below, we will assume that the object 300 is imaged by the first imaging device 200 and the second imaging device 210 detecting light in wavelength bands corresponding to RGB, respectively.

[0061] Figure 2 is a diagram illustrating a specific example of the process for calculating the angle between two color vectors according to the embodiment.

[0062] As shown in Figure 2(a), for example, the first imaging device 200 and the second imaging device 210 are arranged side by side above the object 300, and the object 300 is imaged with the imaging direction of each device pointing downwards.

[0063] First, the first calculation unit 120 sets the first region in the first image and the second region in the second image. In this example, the first calculation unit 120 sets the first region and the second region, each containing 16 (4x4) pixels.

[0064] The i-th kernel shown in Figures 2(b) and 2(c) is a specific example of a defined region (first region or second region). i is, for example, an integer greater than or equal to 1. For example, the first region defined in the upper left of the first image is the first kernel, and the first region moved one pixel to the right from there is the second kernel. The first region moved one pixel further to the right from there is the third kernel. Also, for example, if the first region moved to the right edge of the first image is the k-th kernel, then the first region is moved back to the left edge, and then the first region is set by moving one pixel down from there. The first region set in this way is the k+1-th kernel. The second region is also positioned similarly to the first region, like the i-th kernel. For example, if the number of pixels in the first and second images is the same, and both the first and second regions are located at the i-th kernel, then the positions of the first and second regions in the image coordinate system are the same (coordinates).

[0065] In the above example, the kernel was moved one pixel at a time, but it may also be moved multiple pixels at a time. Furthermore, the kernel may be moved in a way that ensures no duplicate pixels are included.

[0066] For example, the first calculation unit 120 calculates 16 first color vectors corresponding to 16 pixels in the first kernel of the first image, and 16 second color vectors corresponding to 16 pixels in the first kernel of the second image. Next, the first calculation unit 120 combines pixels at the same position in the first and second images and calculates the angle between the first and second color vectors corresponding to each combined pixel. The first calculation unit 120 also calculates the cosine similarity corresponding to each of the 16 angles thus calculated. The first calculation unit 120 then calculates a cost value by adding up the 16 cosine similarities thus calculated.

[0067] Next, the first calculation unit 120 calculates 16 first color vectors corresponding to 16 pixels in the first kernel of the first image, and 16 second color vectors corresponding to 16 pixels in the second kernel of the second image. The first calculation unit 120 also calculates the angle, cosine similarity, and cost value as described above.

[0068] In this way, the first calculation unit 120 repeatedly moves the position of the kernel (second region) in the second image and repeatedly calculates the cost value. For example, the first calculation unit 120 repeatedly moves the position of the kernel in the second image from one end of the second image to the other end on the opposite side of the first end, such that any position in the kernel lies on the epipolar line. By doing this, the first calculation unit 120 calculates multiple cost values. The arbitrary position may be, for example, any pixel position in the kernel, or the center of the kernel, or any other arbitrary position. Alternatively, for example, the first calculation unit 120 may repeatedly move the position of the kernel in the second image from the left end to the right end of the second image in Figure 2(c). Of course, for example, the first calculation unit 120 may repeatedly move the kernel position in the second image to the left from the right edge to the left edge of the second image in Figure 2(c), or repeatedly move it downwards from the top edge to the bottom edge of the second image, or repeatedly move it upwards from the bottom edge to the top edge of the second image, or repeatedly move it in any direction.

[0069] Next, the search unit 130 identifies which kernel in the second image has the largest cost value among the multiple cost values ​​calculated by the first calculation unit 120. If the kernel number identified here in the second image is the mth kernel, then, for example, the search unit 130 determines that multiple pixels included in the first kernel in the first image correspond to multiple pixels included in the mth kernel in the second image.

[0070] Furthermore, for example, the first calculation unit 120 moves the kernel in the second image to the right edge of the second image (the right edge in the row direction of the image) and calculates the cost value, and then calculates 16 first color vectors corresponding to the 16 pixels contained in the second kernel in the first image, and 16 second color vectors corresponding to the 16 pixels contained in the first kernel in the second image.

[0071] In this way, the first calculation unit 120 repeatedly moves the position of the kernel (first region) in the first image and repeatedly calculates the cost value. The direction in which the kernel in the first image is moved is, for example, the same as the kernel in the second image, but may be different, or can be arbitrarily determined.

[0072] Similarly to the above, for example, the first calculation unit 120 fixes the position of the second kernel in the first image and repeatedly moves the position of the kernel in the second image, repeatedly calculating the cost value. Similarly to the above, the search unit 130 searches for corresponding points between multiple pixels included in the second kernel in the first image and pixels in the second image based on the multiple cost values ​​calculated.

[0073] For example, after moving the kernels in the first and second images to the rightmost edge of each image (the rightmost edge in the row direction of the image) and searching for corresponding points as described above, the kernels in each image are moved back to the leftmost edge of each image, and then moved down one pixel from there, and the search for corresponding points is performed in the same way as described above.

[0074] By repeatedly performing this process, corresponding points for each pixel in the first and second images are searched for.

[0075] Next, the above process will be explained using a general formula. For example, we search for the corresponding point in the second image to the v-th pixel (v: an integer greater than or equal to 1) contained in the i-th kernel of the first image. The v-th pixel is a number assigned to each of the multiple pixels contained in the kernel according to the order of the pixels. Also, the pixel value of the v-th pixel is (r1 pixel value, G pixel value, B pixel value) = (r1 i,v , g1 i,v , b1 i,v Assume that ). In this case, the first color vector c1 of the v-th pixel contained in the i-th kernel in the first image v In the coordinate system of the RGB color space shown in Figure 2(d), it is expressed as shown in equation (1) below.

[0076]

Number

[0077] The coordinate system of the RGB color space is, for example, a three-axis orthogonal coordinate system including an axis corresponding to the pixel value of R, an axis corresponding to the pixel value of G, and an axis corresponding to the pixel value of B.

[0078] Note that the unit vectors in the R-axis direction, the G-axis direction, and the B-axis direction are respectively

Number

[0079] Also, for example, assume that the pixel value of the v-th pixel included in the (i + d)-th kernel (where d is an integer such that i + d is 1 or more) in the second image is (pixel value of R, pixel value of G, pixel value of B) = (r2 i+d,v , g2 i+d,v , b2 i+d,v ). In this case, the second color vector c2 v of the v-th pixel included in the (i + d)-th kernel in the second image is expressed as in the following formula (2) in the coordinate system of the RGB color space shown in (d) of FIG. 2.

[0080]

Number

[0081] c1 v and c2 v and the angle formed by them is θ i,v Then, the cosine similarity cosθ i,v is expressed as in the following formula (3).

[0082]

Number

[0083] Furthermore, a cost value t is calculated from the pixel values ​​of multiple pixels included in the i-th kernel in the first image and the pixel values ​​of multiple pixels included in the i+d-th kernel in the second image. i,d This can be expressed as shown in equation (4) below.

[0084]

number

[0085] Furthermore, W i,v This is the weight value. Weight value W i,v This can be expressed, for example, as shown in equation (5) below.

[0086]

number

[0087] 'a' is, for example, the maximum possible length of the color vector (e.g., 255√3), but it can be arbitrarily determined and is not particularly limited.

[0088] Figure 3 is a diagram illustrating the cost values ​​according to the embodiment. Specifically, Figure 3 shows the cost values ​​t when the value of d is changed. i,d This graph schematically illustrates the situation.

[0089] From the graph shown in Figure 3, the value of d that maximizes the cost value (as shown in Figure 3) opti ) can be determined. The search unit 130 finds, for example, the v-th pixel included in the i-th kernel in the first image and i+d in the second image. opti Determine that the v-th pixel contained in the nth kernel corresponds to that point.

[0090] In equation (4) above, the cosine similarity is weighted by adding a weight value to the cosine similarity, but the cosine similarity may also be weighted by multiplying it by a weight value.

[0091] Furthermore, the weight values ​​only need to include the lengths of the color vectors (first color vector and second color vector), and may be calculated by methods other than the above equation (5). For example, when a = 255√3, according to the above equation (5), if the horizontal axis is the length of the first color vector and the vertical axis is the term containing the first color vector on the right side (the (first term) × (second term) on the right side of equation (5)), then the value on the vertical axis is 0 when the value on the horizontal axis is 0, increases linearly as the value on the horizontal axis increases from 0 to a / 2, becomes 1 when the value on the horizontal axis is a / 2, decreases linearly as the value on the horizontal axis increases from a / 2 to a = 255√3, and becomes 0 when the value on the horizontal axis is 255√3. For example, the weight values ​​may be set to draw a sine curve such that the value on the vertical axis is 0 when the value on the horizontal axis is 0, becomes 1 when the value on the horizontal axis is a / 2, and becomes 0 when the value on the horizontal axis is 255√3.

[0092] Furthermore, the method for calculating weight values ​​may be changed, for example, based on the length of the color vector.

[0093] For example, if the image is too bright, the color vector will be long, so weight values ​​that do not result in a very large cost may be adopted. In other words, the weight values ​​may be calculated using a method such that the longer the color vector, the smaller the weight value. For example, if at least one of the lengths of the first color vector and the second color vector is longer than a predetermined first length, the following equation (6) may be used to calculate the weight value.

[0094]

number

[0095] Furthermore, for example, if the image is too dark, the color vector will be shorter, so a weight value that results in a larger cost may be adopted. In other words, the weight value may be calculated using a method such that the longer the color vector, the larger the weight value. For example, if at least one of the lengths of the first color vector and the second color vector is shorter than a predetermined second length, the following equation (7) may be used to calculate the weight value.

[0096]

number

[0097] The predetermined first length and predetermined second length may be arbitrarily determined in advance and are not particularly limited.

[0098] [Processing Procedure] Figure 4 is a flowchart illustrating a shape measurement method according to an embodiment. For example, the shape measurement device 100 performs the process shown in Figure 4.

[0099] First, the shape measuring device 100 acquires a first image generated by the first imaging device 200 imaging the object in two or more wavelength bands, and a second image generated by the second imaging device 210 imaging the object in two or more wavelength bands from a position different from that of the first imaging device 200 (S110). For example, the shape measuring device 100 controls the first imaging device 200 and the second imaging device 210 to image the object, and acquires images of the object from the first imaging device 200 and the second imaging device 210, respectively.

[0100] Furthermore, when the shape measuring device 100 executes step S110, specifically when it causes the first imaging device 200 and the second imaging device 210 to image the object, it may control the illumination device 220 to irradiate the object with light (specifically, light in two or more wavelength bands) using the illumination device 220.

[0101] Next, the shape measuring device 100 calculates a first color vector based on two or more pixel values ​​indicating the color of a pixel in the first image, and calculates a second color vector based on two or more pixel values ​​indicating the color of a pixel in the second image (S120). Specifically, the shape measuring device 100 calculates a color vector for each pixel included in the first image and the second image based on the pixel value indicating the color of that pixel. For example, the shape measuring device 100 calculates multiple color vectors corresponding to each of the multiple pixels included in the first range in the first image and the multiple pixels included in the second range in the second image.

[0102] Next, the shape measuring device 100 calculates the angle between the first color vector and the second color vector (S130). The shape measuring device 100 calculates multiple angles, for example, by changing the combination of multiple first color vectors and multiple second color vectors.

[0103] Next, the shape measuring device 100 calculates weight values ​​based on the lengths of the first color vector and the second color vector (S140). The shape measuring device 100 calculates multiple weight values ​​by calculating a weight value for each combination of the first color vector and the second color vector.

[0104] Next, the shape measuring device 100 calculates a cost value based on the calculated angle and weight values ​​(S150). For example, the shape measuring device 100 calculates cosine similarity based on the calculated angle. Also, for example, in calculating cosine similarity, the shape measuring device 100 calculates multiple cosine similarities based on multiple first color vectors calculated from two or more pixel values ​​of multiple pixels located in a first range in the first image, and multiple second color vectors calculated from two or more pixel values ​​of multiple pixels located in a second range in the second image. Furthermore, the shape measuring device 100 weights each of the multiple cosine similarities by adding or multiplying them by a corresponding weight value. Furthermore, the shape measuring device 100 calculates a cost value by, for example, summing up the multiple weighted cosine similarities.

[0105] Next, the shape measuring device 100 determines whether it has calculated the cost for all points along the epipolar line (S160). Specifically, the shape measuring device 100 determines whether it has calculated the cost value for all positions on the movable epipolar line of the second range. For example, in calculating the cost value, the shape measuring device 100 repeatedly moves the position of the second range in the second image along the epipolar line and repeatedly calculates the cost value.

[0106] If the shape measuring device 100 determines that it has not calculated the entire length of the epipolar line (No in S160), it moves the second range along the epipolar line and repeats the process from step S120 onwards.

[0107] On the other hand, if the shape measuring device 100 determines that it has calculated all the points along the epipolar line (Yes in S160), it searches for corresponding points based on the calculated cost values ​​(S170). For example, in searching for corresponding points, the shape measuring device 100 identifies a first range and a second range in which the calculated cost value is the highest, and determines that the corresponding points are included in multiple pixels located in the identified first range and multiple pixels located in the identified second range.

[0108] Next, the shape measuring device 100 calculates the parallax between the first image and the second image based on the searched corresponding points (S180).

[0109] Next, the shape measuring device 100 measures (calculates) the distance between the first imaging device 200 and the object, and the distance between the second imaging device 210 and the object, based on the calculated parallax (S190). For example, the shape measuring device 100 further measures the shape of the object (e.g., three-dimensional shape) based on the distance between the first imaging device 200 and the object, and the distance between the second imaging device 210 and the object.

[0110] Next, the shape measuring device 100 outputs the measurement result (S200). For example, the shape measuring device 100 outputs shape information indicating the shape of the measured object as the measurement result. The shape measuring device 100 outputs the shape information to a display that does not show figures, for example, and displays a depth image or the like indicating the shape of the object on the display.

[0111] The measurement results may also be distance information indicating the distance between the first imaging device 200 and the object, and the distance between the second imaging device 210 and the object. Furthermore, the measurement results may also be position information indicating the location of corresponding points. The measurement results may include, for example, one or more arbitrary pieces of information from shape information, distance information, and position information.

[0112] [Representative example] Figure 5 is a block diagram showing an information processing device 400 according to an embodiment. Figure 6 is a flowchart showing an information processing method according to an embodiment.

[0113] The information processing device 400 comprises a processor 410 and a memory 420 connected to the processor 410. The processor 410 uses the memory 420 to execute the information processing method shown in Figure 6. The shape measuring device 100 is a specific example of the information processing device 400.

[0114] First, the information processing device 400 calculates a first color vector based on two or more pixel values ​​indicating the color of pixels in the first image, and then calculates a second color vector based on two or more pixel values ​​indicating the color of pixels in a second image which is different from the first image (S10).

[0115] Next, the information processing device 400 calculates the angle between the first color vector and the second color vector (S20).

[0116] Next, the information processing device 400 calculates weight values ​​based on the length of the first color vector and the length of the second color vector (S30).

[0117] Next, the information processing device 400 searches for corresponding points (e.g., the positions of the corresponding points) in the first and second images based on the cost values ​​derived from the calculated angles and weight values ​​(S40). The information processing device 400 outputs information indicating the searched corresponding points (e.g., the positions of the corresponding points) to a terminal used by the user.

[0118] In this way, the information processing device 400 searches for corresponding points in the first and second images based on the length of the first color vector based on the first image, the length of the second color vector based on the second image, and the angle between the first and second color vectors.

[0119] Here, for example, the reliability of the cost value decreases if either the first or second image is too bright or too dark. For example, consider the case where the first image is at standard brightness, and the second image is either too bright or too dark. That is, consider the case where the first image is generated when the object is captured at standard brightness, and the second image is generated when the object is captured in either an overly bright or underly dark state. In this case, the length of the second color vector based on the second image may be excessively long or short compared to the length of the first color vector.

[0120] If the second image is brighter than the first image, the length of the second color vector will be longer than that of the first color vector. Also, if the second image is too bright, it may be overexposed. Specifically, one of the RGB pixel values ​​in the second image may reach its maximum possible value. Each imaging device has predetermined ranges for the RGB pixel values, such as 0 to 255. Therefore, for example, even if the R pixel value should ideally be something like 400, it may be calculated as 255.

[0121] For example, consider the case where the correct ratio (true value) of the RGB pixel values ​​of corresponding points in the first and second images is (R pixel value, G pixel value, B pixel value) = (2:1:1).

[0122] For example, suppose the pixel values ​​of the corresponding points in the first image are (R pixel value, G pixel value, B pixel value) = (200:100:100). Also, suppose the pixel values ​​of the corresponding points in the second image should ideally be (R pixel value, G pixel value, B pixel value) = (400:200:200). However, since the R pixel value exceeds 255, it is calculated as (R pixel value, G pixel value, B pixel value) = (255:200:200). The corresponding points in the first image and the corresponding points in the second image should ideally be the same color. In other words, the ratio of the three pixel values ​​is the same for the corresponding points in the first image and the corresponding points in the second image. To put it another way, the direction of the first color vector and the second color vector at the corresponding points are the same for the corresponding points in the first image and the corresponding points in the second image. However, in this example, the pixel values ​​of R at the corresponding points in the second image are saturated, resulting in different orientations for the first and second color vectors.

[0123] In this case, if we let θ be the angle between the first and second color vectors, then normally cosθ = 1, meaning the angle θ = 0 degrees. However, in this example, cosθ = 0.976, meaning the angle θ = 12.7 degrees.

[0124] Furthermore, if the second image is darker than the first image, the length of the second color vector will be shorter than that of the first color vector. Also, because the values ​​of each RGB pixel (luminance value) in the second image are smaller, errors in each pixel value have a large effect on the angle θ. In other words, because the values ​​of each RGB pixel are small, even a small error in each pixel value will cause a large change in the angle θ.

[0125] For example, suppose the pixel values ​​of the corresponding points in the first image are (R pixel value, G pixel value, B pixel value) = (200:100:100). Also, suppose the pixel values ​​of the corresponding points in the second image should originally be (R pixel value, G pixel value, B pixel value) = (20:10:10). However, suppose that due to some influence, (R pixel value, G pixel value, B pixel value) is calculated to be (24:8:8).

[0126] In this case, if we let θ be the angle between the first and second color vectors, then normally cosθ = 1, meaning θ = 0 degrees. However, in this example, cosθ = 0.985, meaning the angle θ = 10.0 degrees.

[0127] In the second image, the pixel values ​​of the corresponding points show an error of only about 2 to 5 compared to the original pixel values. However, because the values ​​of each pixel are too small, the angle θ is calculated to be too large.

[0128] If the angle θ is not calculated correctly in this way, it can lead to mismatching, meaning that corresponding points will not be correctly searched (calculated), resulting in a decrease in reliability.

[0129] Therefore, the shape measuring device 100 calculates a new cost value by adding a term that includes the length of the color vector to the conventional CVS cost value, for example. For example, the shape measuring device 100 represents each pixel value of RGB as a three-dimensional vector (color vector) in the RGB color space, and calculates a cost value representing the similarity between the angle between each color vector obtained from each of the two images and the weight value (coefficient) of the length of each color vector, and searches for corresponding points.

[0130] This reduces computational costs and improves the matching rate (the probability of finding the correct corresponding point). In other words, the shape measuring device 100 is advantageous in color stereo matching. Furthermore, since the index for finding the corresponding point is angle, the corresponding point can be calculated accurately even when there is a large difference in brightness between the first image and the second image (i.e., when there is a large difference in the length of the color vectors).

[0131] The information processing device 400 may measure the shape of the object, similar to the shape measuring device 100. In this case, the information processing device 400 may output shape information as a measurement result without outputting position information. Furthermore, the information processing device 400 only needs to search for corresponding points, and if shape information is not output as a measurement result, it does not need to measure the shape of the object.

[0132] [Effects, etc.] The following describes examples of technologies that can be obtained from the disclosures in this specification, and explains the effects that can be obtained from those technologies.

[0133] Technology 1 is an information processing method that calculates a first color vector based on two or more pixel values ​​indicating the color of pixels in a first image, calculates a second color vector based on two or more pixel values ​​indicating the color of pixels in a second image different from the first image (S10), calculates the angle between the first color vector and the second color vector (S20), calculates weight values ​​based on the length of the first color vector and the length of the second color vector (S30), and searches for corresponding points in the first and second images based on cost values ​​based on the calculated angle and weight values ​​(S40).

[0134] According to this method, pixel values ​​are used to search for corresponding points. In other words, information indicating the color of a pixel is used to search for corresponding points. Therefore, for example, even if the locations on an object are the same color but have different brightness levels, the system can appropriately distinguish between them and search for corresponding points.

[0135] Depending on the shape and material of the object, the degree of reflected light from the object may differ between the first image, the second image, and the left and right images. Therefore, conventional correspondence point search methods such as NCC (Normalized Cross Correlation) have the problem of matching errors. In addition, in methods such as NCC, normalization is performed on multiple pixels within the kernel, so if there are differences in the brightness of some of the pixels within the kernel, matching errors are likely to occur. Furthermore, if the object contains multiple components and the reflectivity of these components differs from one another, the hue may be the same even if the color (e.g., RGB values) of the reflected light from the object is different. In such cases, conventional methods such as NCC cannot solve the problem of mismatching.

[0136] Therefore, the information processing method related to Technology 1 enables the search for corresponding points when performing stereoscopic imaging with an image (color image) without losing the high-density color information inherent in the color image, and without increasing the computational burden. In the information processing method related to Technology 1, for example, pixel values ​​such as the RGB values ​​of each pixel are represented by a three-dimensional vector (color vector) in the RGB color space. This makes it possible to improve the matching accuracy (in other words, the accuracy of searching for corresponding points) of occlusion parts such as the side parts of objects that appear in the image. Furthermore, it does not reduce the dimension of color information in the image (i.e., it does not reduce color information) compared to conventional stereo matching techniques that grayscale the image, so matching can be performed with high accuracy. In other words, mismatching can be reduced. In addition, according to the information processing method related to Technology 1, the computation time can be shortened compared to processing that simply extends the NCC method to color.

[0137] Based on the above, the information processing method related to Technology 1 can reduce computation time compared to stereo matching methods such as the NCC method. In other words, the information processing method related to Technology 1 can reduce the amount of processing required.

[0138] Furthermore, the information processing method related to Technology 1 searches for corresponding points while considering the length of the color vector. As described above, for example, when the brightness environment in which two images are captured is different, the pixel values ​​may saturate, or the pixel values ​​may become too small, causing the effect of measurement errors to become too large, and corresponding points may not be found correctly. On the other hand, the information processing method related to Technology 1 can accurately search for corresponding points in multiple images.

[0139] Technology 2 is an information processing method described in Technology 1, further comprising acquiring a first image generated by a first imaging device imaging an object in two or more wavelength bands, and a second image generated by a second imaging device imaging an object in two or more wavelength bands from a position different from the first imaging device, calculating the parallax between the first and second images based on the searched corresponding points, and measuring the shape of the object based on the calculated parallax.

[0140] The first imaging device is, for example, the first imaging device 200 described above. The second imaging device is, for example, the second imaging device 210 described above. The object is, for example, the object 300 described above.

[0141] According to this method, the shape is measured using the angle formed by the color vectors, so the shape of the object can be measured accurately even if there are differences in brightness in different parts of the object. Furthermore, because the shape is measured using color vectors, the number of calculation steps and the number of dimensions in the calculation process are fewer (i.e., less data / computation) than conventional color stereo matching techniques, resulting in faster computation time.

[0142] Based on the above, the information processing method related to Technology 2 allows for accurate measurement of the shape of an object.

[0143] The first and second images may be generated by different imaging devices, or by the same imaging device. For example, the first image may be generated by the imaging device capturing an object, and then the second image may be generated by the imaging device moving and capturing an object again. In other words, the first and second imaging devices may be the same device, or they may be different devices.

[0144] Technology 3 is an information processing method described in Technology 2, further comprising irradiating the object with light of two or more wavelength bands.

[0145] This method makes it easier to accurately reflect the colors of two or more wavelength bands in the first and second images. Therefore, it allows for accurate measurement of the shape of an object.

[0146] Technology 4 is an information processing method according to Technology 2 or 3, further comprising irradiating an object with patterned light.

[0147] According to this method, the shape of the object can be measured with even greater precision.

[0148] Technology 5 is the information processing method described in Technology 4, wherein the patterned light is a dot pattern light.

[0149] According to this method, the shape of the object can be measured with even greater precision.

[0150] Technology 6 is an information processing method described in any of Technologies 1 to 5, further comprising calculating cosine similarity based on the calculated angle, and calculating a cost value based on the calculated cosine similarity and weight values.

[0151] This allows for accurate searching of corresponding points.

[0152] Technology 7 is an information processing method described in Technology 6, wherein, in calculating cosine similarity, multiple cosine similarities are calculated based on multiple first color vectors calculated from two or more pixel values ​​of multiple pixels located in a first range in the first image and multiple second color vectors calculated from two or more pixel values ​​of multiple pixels located in a second range in the second image; in calculating weight values, multiple weight values ​​corresponding to the multiple cosine similarities are calculated; and in calculating cost values, a cost value is calculated based on the calculated multiple cosine similarities and multiple weight values.

[0153] This allows for accurate searching of corresponding points.

[0154] Technique 8 is an information processing method described in Technique 7, in which, in calculating the cost value, the position of the first range in the first image and the position of the second range in the second image are repeatedly moved along the epipolar line and the cost value is repeatedly calculated; and in searching for corresponding points, the first range and the second range that yield the highest calculated cost value are identified, and it is determined that the corresponding points are included in multiple pixels located in the identified first range and multiple pixels located in the identified second range.

[0155] This allows for accurate searching of corresponding points.

[0156] Technology 9 is an information processing device 400 comprising a processor 410 and a memory 420, wherein the processor 410 uses the memory 420 to calculate a first color vector based on two or more pixel values ​​indicating the color of pixels in a first image, calculates a second color vector based on two or more pixel values ​​indicating the color of pixels in a second image different from the first image, calculates the angle between the first color vector and the second color vector, calculates weight values ​​based on the length of the first color vector and the length of the second color vector, and searches for corresponding points in the first and second images based on the cost values ​​based on the calculated angle and weights.

[0157] According to this, it produces the same effect as the information processing method related to Technology 1.

[0158] Technology 10 is an information processing system comprising: an information processing device 400 described in Technology 9; a first imaging device that generates a first image by imaging an object in two or more wavelength bands; and a second imaging device that generates a second image by imaging the object in the two or more wavelength bands from a position different from the first imaging device.

[0159] The shape measurement system 10 is a specific example of an information processing system. The information processing system includes, for example, an information processing device 400, a first imaging device 200, and a second imaging device 210. The information processing device 400 acquires a first image and a second image by, for example, controlling the first imaging device 200 and the second imaging device 210.

[0160] According to this, the first imaging device and the second imaging device can be used to image an object, and the shape of the object can be measured with high accuracy.

[0161] Technology 11 is a program for a computer to execute one of the information processing methods described in Technology 1 to 8.

[0162] According to this, any of the information processing methods related to technologies 1 to 8 can be implemented by a computer.

[0163] (Other embodiments) Although embodiments have been described above, this disclosure is not limited to the embodiments described above.

[0164] For example, in the above embodiment, the shape measurement system 10 includes two imaging devices (a first imaging device 200 and a second imaging device 210). The number of imaging devices included in the shape measurement system 10 may be multiple, such as two or three or more. For example, the shape measurement system 10 may include three or more imaging devices arranged at different positions. Alternatively, for example, the shape measurement device 100 may acquire images of the object from each of the three or more imaging devices, calculate a color vector for each of the three or more acquired images as described above, search for corresponding points in each of the three or more images based on the calculated color vectors, and measure the shape of the object. The information processing device 400 and the information processing system are the same as those for the shape measurement device 100 and the shape measurement system 10.

[0165] Furthermore, for example, in the above embodiment, a process executed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0166] Furthermore, for example, in the above embodiment, each component of the processing unit may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0167] Furthermore, each component may be implemented by hardware. Each component may also be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or they may be separate circuits. Also, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0168] Furthermore, the method of communication between devices in the above embodiment is not particularly limited. In addition, relay devices (such as broadband routers) not shown may be involved in the communication between devices.

[0169] Furthermore, the general or specific aspects of this disclosure may be implemented as a system, apparatus, method, integrated circuit, computer program, or a computer-readable non-temporary recording medium such as a CD-ROM. They may also be implemented in any combination of systems, apparatus, methods, integrated circuits, computer programs, and recording media. For example, this disclosure may be implemented as a shape measurement method, as a program for causing a computer to execute a shape measurement method, or as a computer-readable non-temporary recording medium on which such a program is recorded.

[0170] Furthermore, this disclosure also includes forms obtained by applying various modifications to the embodiments that a person skilled in the art could conceive, or forms realized by arbitrarily combining the components and functions of the embodiments without departing from the spirit of this disclosure. [Industrial applicability]

[0171] This disclosure can be used in devices for measuring the three-dimensional shape of an object. [Explanation of Symbols]

[0172] 10 Shape Measurement System 100 Shape measuring device 110 Acquisition Department 120 First Calculation Unit 130 Search Department 140 Second Calculation Unit 150 Measurement Unit 160 Output section 170 Storage section 200 First Imaging Device 210 Second Imaging Device 220 Irradiation device 300 objects 400 Information Processing Devices 410 Processor 420 memory

Claims

1. A first color vector is calculated based on two or more pixel values ​​indicating the color of a pixel in the first image, and a second color vector is calculated based on two or more pixel values ​​indicating the color of a pixel in a second image different from the first image. The angle between the first color vector and the second color vector is calculated, A weight value is calculated based on the length of the first color vector and the length of the second color vector. Based on the calculated angle and cost value derived from the weight value, the system searches for corresponding points in the first and second images. Information processing methods.

2. moreover, The first image generated by the first imaging device imaging the object in two or more wavelength bands, and the second image generated by the second imaging device imaging the object in the two or more wavelength bands from a position different from the first imaging device, are acquired. Based on the searched corresponding points, the disparity between the first image and the second image is calculated. Based on the calculated parallax, the shape of the object is measured. The information processing method according to claim 1.

3. Furthermore, the object is irradiated with light of the two or more wavelength bands. The information processing method according to claim 2.

4. Furthermore, the object is irradiated with patterned light. The information processing method according to claim 2.

5. The aforementioned patterned light is a dot patterned light. The information processing method according to claim 4.

6. moreover, Based on the calculated angle, the cosine similarity is calculated, Based on the calculated cosine similarity and weight values, the cost value is calculated. The information processing method according to claim 1.

7. In calculating the cosine similarity, a plurality of cosine similarities are calculated based on a plurality of first color vectors calculated from two or more pixel values ​​of a plurality of pixels located in a first range in the first image, and a plurality of second color vectors calculated from two or more pixel values ​​of a plurality of pixels located in a second range in the second image. In calculating the weight values, a plurality of weight values ​​corresponding to a plurality of cosine similarities are calculated, In calculating the cost value, the cost value is calculated based on the calculated cosine similarity of multiple cosine similarities and weight values. The information processing method according to claim 6.

8. In calculating the cost value, the position of the first range in the first image and the position of the second range in the second image are repeatedly moved along the epipolar line, and the cost value is repeatedly calculated. In the search for the corresponding points, the first range and the second range are identified, which have the highest calculated cost value, and it is determined that the corresponding points are included in a plurality of pixels located in the identified first range and a plurality of pixels located in the identified second range. The information processing method according to claim 7.

9. Processor and Equipped with memory, The processor uses the memory to: A first color vector is calculated based on two or more pixel values ​​indicating the color of a pixel in the first image, and a second color vector is calculated based on two or more pixel values ​​indicating the color of a pixel in a second image different from the first image. The angle between the first color vector and the second color vector is calculated, A weight value is calculated based on the length of the first color vector and the length of the second color vector. Based on the calculated angle and cost value derived from the weight value, the system searches for corresponding points in the first and second images. Information processing device.

10. The information processing apparatus according to claim 9, A first imaging device that generates the first image by imaging an object in two or more wavelength bands, The system includes a second imaging device that generates a second image by imaging the object from a position different from the first imaging device in the two or more wavelength bands, Information processing system.

11. For a computer to execute the information processing method described in any one of claims 1 to 8, program.

Images