IMAGE PROCESSING METHOD, IMAGE PROCESSING APPARATUS AND PROGRAM

Abstract

To provide an image processing device, an image processing method, and a program for accurately analyzing how a pixel group such as an extracted segment, including broken pixel segment, corresponds to a line segment.SOLUTION: In an image processing device 1, a control part of an information processing unit includes: a pixel group setting part for setting a plurality of pixel groups, including a first pixel group and a second pixel group not connected to each other, from a plurality of pixels composing a first image; and a set setting part for setting a set of the first pixel group and the second pixel group including pixels within a predetermined range from the first pixel group in order to set a connection degree to the first pixel group.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to image processing method , Image Processing Device and regarding the program. [Background technology]

[0002] A method has been proposed for performing image processing on an image obtained by capturing an image of an object including linear parts appropriately equipped with branches, etc. (see Patent Document 1). In such image processing, it is necessary to analyze how the extracted pixel group corresponds to the linear parts. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent No. 6015113 Summary of the Invention

[0004] According to a first aspect of the present invention, an image processing method includes the steps of: inputting image data of an object; and, based on the image data of the object, selecting a first pixel group that is not connected to each other among a plurality of pixel groups corresponding to a plurality of the objects including a linear portion. and a calculation step of calculating a degree of connectivity between the first pixel group and the second pixel group, the degree of connectivity being determined based on whether the first pixel group and the second pixel group are in the same pixel group of the object. of line shape part Minutes Possibility to respond is an index that represents the second pixel group is a pixel group including pixels within a predetermined range from the first pixel group, The calculation step includes: Pixels in and before The second pixel group Pixels included in a first calculation step of calculating a first distance between Set to pass through at least two pixels that contain The first line segment and the second pixel group Set to pass through at least two pixels that contain The method includes a second calculation step of calculating a first angle formed by a second line segment, and calculates the connectivity degree using the first distance or the first angle. According to a second aspect of the present invention, an image processing device includes an input unit that inputs image data of an object, and a first pixel group that is not connected to each other among a plurality of pixel groups corresponding to a plurality of the objects including a linear portion based on the image data of the object. and a calculation unit for calculating a degree of connection between the first pixel group and the second pixel group, the degree of connection being determined based on whether the first pixel group and the second pixel group are the same pixel group of the object. of line shape part Minutes Possibility to respond It is an indicator of the second pixel group is a pixel group including pixels within a predetermined range from the first pixel group, The calculation unit calculates the first pixel group Pixels in and before The second pixel group Pixels included in a first calculation unit that calculates a first distance between Set to pass through at least two pixels that contain The first line segment and the second pixel group Set to pass through at least two pixels that contain The connection degree is calculated using the first distance or the first angle. According to a third aspect of the present invention, a program is a program for causing an image processing device to execute the following process, comprising: an acquisition process for acquiring image data of an object; and, based on the image data of the object, selecting a first pixel group that is not connected to each other among a plurality of pixel groups corresponding to a plurality of the objects including a linear portion. and and a calculation process for calculating a degree of connection between the first pixel group and the second pixel group, the degree of connection being determined based on whether the first pixel group and the second pixel group are the same pixel group of the object. of line shape part Minutes Possibility to respond It is an indicator of the second pixel group is a pixel group including pixels within a predetermined range from the first pixel group, The calculation process is Pixels in and before The second pixel group Pixels included in a first calculation process for calculating a first distance between the first pixel group and Set to pass through at least two pixels that contain The first line segment and the second pixel group Set to pass through at least two pixels that contain The method includes a second calculation process of calculating a first angle formed by a second line segment, and calculates the connectivity degree using the first distance or the first angle. [Brief description of the drawings]

[0005] [Figure 1]FIG. 1 is a conceptual diagram showing the configuration of an image processing apparatus according to the first embodiment. [Diagram 2] FIG. 2 is a conceptual diagram showing the configuration of the data processing unit in the first embodiment. [Diagram 3] FIG. 3 is a conceptual diagram showing a probability distribution image in the first embodiment. [Figure 4] FIG. 4 is a conceptual diagram showing a detection result image in the first embodiment. [Diagram 5] 5(A) and (B) are conceptual diagrams for explaining crossover detection in the first embodiment. [Figure 6] FIG. 6 is a conceptual diagram for explaining a search range in the first embodiment. [Figure 7] FIG. 7 is a conceptual diagram for explaining calculation of the connection degree in the first embodiment. [Figure 8] FIG. 8(A) is a conceptual diagram for explaining the grouping method in the first embodiment, and FIG. 8(B) is a connection graph corresponding to FIG. 8(A). [Figure 9] FIG. 9(A) is a conceptual diagram for explaining the grouping method in the first embodiment, and FIG. 9(B) is a connection graph corresponding to FIG. 9(A). [Figure 10] FIG. 10(A) is a conceptual diagram for explaining the grouping method in the first embodiment, FIG. 10(B) is a connection graph corresponding to FIG. 10(A), and FIG. 10(C) is a weighted connection graph after grouping. [Figure 11] FIG. 11 is a flowchart showing the flow of the image processing method according to the first embodiment. [Figure 12] FIG. 12 is a flowchart showing the flow of the image processing method according to the first embodiment. [Figure 13] FIG. 13 is a flowchart showing the flow of the image processing method according to the first embodiment. [Figure 14] 14(A) and 14(B) are conceptual diagrams for explaining the setting of pixel groups in Modification 1. In FIG. [Figure 15]15(A) and 15(B) are conceptual diagrams for explaining the setting of a search range in the second modification. [Figure 16] FIG. 16 is a graph showing the relationship between the distance and the degree of connectivity between pixel groups in Modification 3. [Figure 17] FIG. 17(A) is a conceptual diagram for explaining the setting of the connection degree at the crossover in the fourth modification, and FIG. 17(B) is a connected graph corresponding to FIG. 17(A). [Figure 18] FIG. 18 is a conceptual diagram showing the configuration of a data processing unit according to the seventh modification. [Figure 19] FIG. 19 is a conceptual diagram for explaining provision of a program in the eighth modification. [Figure 20] FIG. 20 is a conceptual diagram showing the configuration of a quality evaluation system according to the ninth modification. [Figure 21] FIG. 21(A) is a conceptual diagram showing the configuration of an information processing device according to the ninth modification, and FIG. 21(B) is a conceptual diagram showing the configuration of a terminal device according to the ninth modification. [Figure 22] FIG. 22 is a flowchart showing the flow of an image processing method according to the tenth modification. [Diagram 23] FIG. 23 is a flowchart showing the flow of an image processing method according to the tenth modification. [Figure 24] FIG. 24 is a conceptual diagram showing an example of a display screen in the tenth modification. [Diagram 25] FIG. 25 is a conceptual diagram showing an example of a display screen in the tenth modification. [Figure 26] FIG. 26 is a conceptual diagram showing an example of a display screen in the tenth modification. [Figure 27] FIG. 27 is a conceptual diagram showing an example of a display screen in the tenth modification. [Figure 28] FIG. 28 is a conceptual diagram showing an example of a display screen in the tenth modification. [Figure 29] FIG. 29 is a conceptual diagram showing an example of a display screen in the tenth modification. [Diagram 30]FIG. 30 is a conceptual diagram showing an example of a display screen in the tenth modification. [Diagram 31] FIG. 31 is a conceptual diagram showing an example of a display screen in the tenth modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0007] -First embodiment- 1 is a conceptual diagram showing the configuration of an image processing device of this embodiment. The image processing device 1 includes a culture unit 100 and an information processing unit 40. The culture unit 100 includes a culture room (culture chamber) 10, an observation sample stage (stage) 11, a drive unit 12, and an imaging unit 20. The information processing unit 40 includes an input unit 41, a communication unit 42, a storage unit 43, an output unit 44, and a control unit 50. The control unit 50 includes a data processing unit 51, an output control unit 52, and an apparatus control unit 53.

[0008] The image processing device 1 is configured as a culture device, and is configured such that captured image data, which is image data obtained by capturing an image of a sample (eg, cells) in a culture section 100, is input to a data processing section 51 and processed.

[0009] In this embodiment, linear parts including straight lines, curves, or intersections are extracted as extraction targets from a captured image corresponding to captured image data. In the following, an example will be described in which a cell Ce is a nerve cell and a part corresponding to a neurite Nr is extracted, but the extraction target is not limited to this example and can be any element constituting an image as long as it includes the linear parts described above. Note that extraction may be performed from an image other than a captured image, such as a drawn image.

[0010] The culture unit 100 includes a culture chamber 10 for culturing the cells Ce, and images of the cells Ce cultured in the culture chamber 10 are captured.

[0011] The culture chamber 10 houses therein a culture vessel C in which the cells Ce are cultured. The culture vessel C is, for example, a well plate or a dish. Inside the culture chamber 10, a temperature regulator (e.g., a heater) and a temperature sensor (not shown) controlled by the control unit 50 are arranged, and the temperature regulator and the temperature sensor are controlled so that the cells are cultured in a preset environment, such as maintaining a preset temperature. The driving unit 12 includes an actuator, and moves the culture vessel C at a preset time and places it on the observation specimen stage 11 inside the culture chamber 10. Furthermore, the driving unit 12 moves the imaging unit 20 or the observation specimen stage 11, etc. to an appropriate position (e.g., an observation position) so that the cells Ce are positioned on the focal plane of the imaging unit 20 in order to image the cells Ce.

[0012] The imaging unit 20 includes an imaging device including an imaging element such as a CMOS or a CCD, and images the cell Ce, particularly the neurite Nr of the cell Ce. The imaging method by the imaging unit 20 is not particularly limited as long as the pixel corresponding to the neurite Nr in the captured image including the cell Ce can be distinguished from other parts with a desired accuracy based on the luminance value of the pixel or a plurality of pixels around the pixel. For example, the imaging method by the imaging unit 20 can be a fluorescence observation method, a phase contrast observation method, or the like.

[0013] When the imaging unit 20 performs imaging by a fluorescence observation method, fluorescent staining can be performed by expressing a fluorescent protein such as GFP in the cells Ce by gene transfer, or by expressing a protein in which a protein localized in the neurite Nr is fused with a fluorescent protein. If there is no problem with the use of the cells Ce after imaging (e.g., culture, passage, pick-up), other labeling methods such as immunostaining may be performed.

[0014] The pixel signal obtained by the imaging unit 20 imaging the cell Ce is converted into a digital signal and input to the information processing unit 40 as imaged image data in which pixels correspond to brightness values ​​(arrow A1 in Figure 1), and stored in the memory unit 43.

[0015] The information processing unit 40 serves as an interface with a user of the image processing device 1 (hereinafter simply referred to as "user"), and also performs processes such as communication, storage, and calculation regarding various data. The information processing unit 40 may be configured as an information processing device that is physically separated from the culture unit 100. At least a part of the data used by the image processing device 1 may be stored in a remote server or the like.

[0016] The input unit 41 includes input devices such as a mouse, a keyboard, various buttons, a touch panel, etc. The input unit 41 receives data necessary for imaging by the culture unit 100 and data processing by the data processing unit 51 from a user.

[0017] The communication unit 42 includes a communication device capable of communicating via a wireless or wired connection such as the Internet, and transmits and receives data related to control and processing in the image processing device 1 as appropriate.

[0018] The storage unit 43 includes a non-volatile storage medium, and stores programs that cause the control unit 50 to perform processing, image data related to processing by the data processing unit 51, and the like.

[0019] The output unit 44 includes a display device such as a liquid crystal monitor, and outputs images and the like showing information obtained by processing in the data processing unit 51 .

[0020] The control unit 50 is configured with a processing device such as a CPU, and functions as a main body for controlling the image processing device 1, and executes programs stored in the storage unit 43 to perform various processes.

[0021] The data processing unit 51 of the control unit 50 processes the captured image data input from the imaging unit 20, extracts multiple pixel groups corresponding to the extraction target, and groups the multiple pixel groups based on which cell bodies the extracted multiple pixel groups are connected to.

[0022] 2 is a conceptual diagram showing the configuration of data processing unit 51. Data processing unit 51 includes a detection result image generation unit 511, a pixel group setting unit 512, a set setting unit 513, a calculation unit 514, a determination unit 600, and an image generation unit 700. The determination unit 600 includes an execution unit 601 and an end determination unit 602.

[0023] The detection result image generating unit 511 of the data processing unit 51 performs image processing on the captured image data stored in the storage unit 43, and generates and acquires detection result image data corresponding to a detection result image described later. In the following, an example will be described in which the detection result image generating unit 511 generates probability distribution image data corresponding to a probability distribution image described later from the captured image data, and generates detection result image data from the probability distribution image data. However, the present invention is not limited to this example, and the detection result image data may be generated from the captured image data without passing through the probability distribution image data.

[0024] FIG. 3 is a conceptual diagram showing a probability distribution image Gp. In the probability distribution image Gp, the brightness of each pixel Px of the probability distribution image Gp is ​​associated with the probability that the pixel Px corresponds to the neurite Nr to be extracted. In FIG. 3, pixels corresponding to one cell body So ​​and one neurite Nr extending from the cell body So ​​are shown, but the probability distribution image Gp preferably includes images of multiple cell bodies So and multiple neurites Nr, and the following will be described using this example. In the probability distribution image Gp, the value of the above probability or a value corresponding to the above probability is set for all pixels Px in the image. Such values ​​are also set for pixels other than the pixel corresponding to the neurite Nr shown in FIG. 3. In reality, pixels that do not correspond to the neurite Nr often have a positive value indicating the above probability, and the pixel Px in FIG. 3 shows such an example. In the probability distribution image Gp of FIG. 3, the darker the hatching in a part, the more likely it is that the pixel Px corresponding to that part is an image part corresponding to the neurite Nr. It is preferable that the probability distribution image Gp is ​​represented as a grayscale image in which the above probability corresponds to a one-dimensional brightness value for each pixel Px, but the form of the probability distribution image Gp is ​​not particularly limited as long as the above probability and brightness value are associated with each other.

[0025] The detection result image generating unit 511 processes the captured image data using a predetermined algorithm (hereinafter referred to as the first algorithm) to calculate the probability that each pixel Px corresponds to the neurite Nr to be extracted. The first algorithm is a trained machine learning. This machine learning is a deep learning in which a calculation device is trained by inputting a plurality of images obtained by capturing images of nerve cells and an image showing a portion corresponding to a neurite in the plurality of images. The first algorithm is not particularly limited as long as it can associate the above probability with the luminance value, and machine learning other than deep learning or an algorithm other than machine learning may be used.

[0026] The detection result image generation unit 511 processes the probability distribution image data using a predetermined algorithm (hereinafter referred to as the second algorithm) to generate detection result image data corresponding to a detection result image indicating whether or not each pixel Px corresponds to a nerve Nr.

[0027] 4 is a conceptual diagram showing a detection result image Gd. The detection result image Gd (for example, a first image) is an image that indicates whether or not each pixel Px corresponds to a neurite Nr by a luminance value. For example, as shown in FIG. 4, the detection result image Gd can be a binarized image, and a luminance value corresponding to black in the figure indicates that pixel Px1 corresponds to the neurite Nr, and a luminance value corresponding to white in the figure indicates that pixel Px2 does not correspond to the neurite Nr. In addition, in the detection result image Gd, whether or not each pixel Px corresponds to the cell body So ​​is preferably represented and determined by image data or the like other than the detection result image data. As long as it is possible to indicate whether or not each pixel Px corresponds to a neurite Nr using a luminance value, the method of expressing this in the detection result image Gd is not particularly limited.

[0028] In the following, in the detection result image Gd, a pixel group consisting of a plurality of pixels connected to each other and corresponding to the extraction target is called a fragment S. In the detection result image Gd of FIG. 4, two fragments Sa and Sb corresponding to one neurite Nr are shown. The fragment S is preferably uniform in width, but is not limited thereto, and can be a linear pixel group having any width depending on the location (position of the pixel in the detection result image Gd). In the following, a line with a width of one pixel as shown in FIG. 4 is appropriately called a thin line. The fragments Sa and Sb are separated by two or more pixels. Thus, in the detection result image Gd, unlike the subject cell Ce, the part corresponding to the neurite Nr is not necessarily connected. One of the reasons for this is that when imaging, the signal of the pixel corresponding to the linear part of the neurite Nr, etc. is weak, and imaging cannot be performed with sufficient accuracy. In particular, this tendency is also seen in fluorescence microscopes that are often used to image cells, etc. In such a case, unless it is determined which neurite Nr the fragment S corresponds to, it is not possible to calculate the exact length, number, etc. of the neurite Nr when analyzing the neurite Nr using the detection result image Gd, and it is not possible to properly obtain information from the neurite Nr in the acquired image.

[0029] When two pixels or two fragments S are adjacent only in the diagonal direction, there is no particular limitation on whether these pixels or fragments S are connected or disconnected, and this can be set appropriately in accordance with the definition of connectivity in the analysis algorithm when analyzing the neurite Nr as described above.

[0030] The second algorithm for generating the detection result image data is not particularly limited as long as the fragment S described below remains in the detection result image Gd, and the following algorithms can be used. For example, the second algorithm includes binarizing the brightness value of the probability distribution image Gp based on a predetermined threshold, determining pixels having a brightness value equal to or greater than the threshold as pixels corresponding to the neurite Nr, and determining pixels having a brightness value less than the threshold as pixels not corresponding to the neurite Nr. Alternatively, the second algorithm may include processing such as skeleton extraction for the fragment S. The detection result image generating unit 511 can perform skeleton extraction processing for each fragment S by the method shown in, for example, Lau et al.'s document ("Simultaneously Identifying All True Vessels From Segmented Retinal Images" IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, Vol. 60, No. 7, July 2013).

[0031] After acquiring the detection result image data, the detection result image generation unit 511 stores the detection result image data in a memory or storage unit 43 connected to the CPU of the control unit 50 so that it can be referenced by the CPU (hereinafter referred to as "storing in storage unit 43, etc."). Note that the image processing device 1 does not need to be configured to perform imaging or culturing as long as it can acquire detection result image data. The data processing unit 51 may acquire detection result image data from the communication unit 42 or the like, and store the detection result image in a memory or storage unit 43 or the like so as to be able to be referenced as described above.

[0032] The pixel group setting unit 512 sets and acquires a plurality of fragments S (e.g., a first pixel group, a second pixel group) that are not connected to each other from a plurality of pixels Px that constitute the detection result image Gd. At least two fragments S set by the pixel group setting unit 512 include fragments S that are not connected to each other and are not connected via other pixel groups in the detection result image Gd, such as a triangulation or crossing as described below. The pixel group setting unit 512 detects a triangulation or crossing from the detection result image Gd, performs a process of removing the triangulation or crossing, and then sets the multiple fragments S. Here, a crossing refers to a branch where four or more linear pixel groups are connected.

[0033] 5(A) and 5(B) are conceptual diagrams for explaining the process of detecting and removing crossings in a plurality of pixel groups. The pixel group setting unit 512 generates data corresponding to a first partial image Gs1, which is an image of a portion of the detection result image Gd. The first partial image Gs1 is an image corresponding to a square pixel region centered on the central pixel Pxc and having one side of 2N+1 pixels, where N is a natural number. The value of N is 5 in FIG. 5(A), but is appropriately set based on the characteristics of the object or the average value of the width or length of the fragments S in the detection result image Gd, etc.

[0034] The pixel group setting unit 512 generates data corresponding to the second partial image Gs2 from the first partial image Gs1, in which the pixel region corresponding to the circle of radius M centered on the central pixel Pxc is set to pixels that do not correspond to the neurite Nr, in other words, are not included in the fragment S. The pixel region corresponding to the circle of radius M is called the removed pixel region Re. M is a positive number smaller than N. In FIG. 5B, the removed pixel region Re is a 3×3 square pixel region consisting of pixels that include at least a part of the circle of radius M, with M being 1. At this time, if a triangulation or crossover is included in the part corresponding to the removed pixel region Re of the first partial image Gs1, the triangulation or crossover is removed, and the fragment Sc becomes a plurality of fragments S in the second partial image Gs2. As a result, the number of fragments S increases in the second partial image Gs2 compared to the second partial image Gs1. In FIG. 5B, the fragment Sc is separated into a plurality of fragments Sc1, Sc2, Sc3, and Sc4 by the removed pixel region Re.

[0035] If the difference obtained by subtracting the number of fragments S in the first partial image Gs2 from the number of fragments S in the second partial image Gs2 is 2 or more, the pixel group setting unit 512 determines that a crossover has been detected in the removed pixel region Re, and replaces the first partial image Gs1 of the detection result image Gd with the second partial image Gs2. The pixel group setting unit 512 moves the center pixel Pxc appropriately by one pixel or the like on the detection result image Gd to sequentially generate the first partial image Gs1 and the second partial image Gs2, and when a crossover is detected, replaces the first partial image Gs1 of the detection result image Gd with the second partial image Gs2. When the entire detection result image Gd is covered by the first partial image Gs1 or the second partial image Gs2 generated up to that point, the pixel group setting unit 512 stores data corresponding to the obtained detection result image Gd in the storage unit 43 or the like. As long as the removal of the trifurcations or crossovers can be performed, there are no particular limitations on the size, shape, etc. of the first partial image Gs1, the second partial image Gs2, and the removed pixel region Re.

[0036] The pixel group setting unit 512 generates data indicating which pixels Px each fragment S includes, based on the luminance value of each pixel in the detection result image Gd after removing the triangulations or crossovers. For example, the pixel group setting unit 512 groups multiple pixels Px1 included in the fragment S into groups of pixels that are connected to each other. The pixel group setting unit 512 stores the generated data in the storage unit 43 or the like.

[0037] The pair setting unit 513 sets a pair of a certain fragment S from the multiple fragments S set by the pixel group setting unit 512, and a fragment S including a pixel within the search range R from the certain fragment S. The pair set by the pair setting unit 513 includes at least one pair of fragments S that are not connected to each other, and furthermore, the pair is not connected via other pixel groups such as a triangulation or crossing in the detection result image Gd before the triangulation or crossing is removed. Of the multiple fragments S set by the pixel group setting unit 512, the fragment S for which a search range R described below is set (a fragment that is the reference for the search range R to be set) is set as the first fragment, and the other fragments (fragments that are not the reference for the search range R to be set) are set as the second fragment.

[0038] FIG. 6 is a conceptual diagram for explaining the setting of pairs. The pair setting unit 513 extracts pixels at both ends of the fragment S included in the detection result image Gd. Hereinafter, the pixels at both ends of the fragment S are referred to as edge pixels T. When the fragment S is a thin line, the pair setting unit 513 sets the pixels at both ends of the thin line corresponding to each fragment S as edge pixels T. When the fragment S is not a thin line, the pair setting unit 513 can set, for example, pixels close to both ends along a curve or line segment corresponding to the fragment S as edge pixels T. In FIG. 6, the pixels at both ends of the first fragment S1 are set as edge pixels T11 and T12.

[0039] The set setting unit 513 sets regions (pixel regions) of a predetermined range centered on the edge pixels T11 and T12, respectively, as search ranges R1 and R2 of the first fragment S1. The shape of the search range R is not particularly limited, and can be, for example, a pixel region corresponding to a circle or a rectangle. When the search range R is a pixel region corresponding to a circle, the radius of the circle can be, for example, about half the average, such as the arithmetic mean, of the lengths of the fragments S included in the detection result image Gd, but is not particularly limited.

[0040] The pair setting unit 513 detects edge pixel T of the second fragment (second fragments S2, S3, S4 in the example of Figure 6) that is included in the search range R of fragment S1. In Figure 6, edge pixel T21 of the second fragment S2 that is included in the search range R1 is detected. The pair setting unit 513 sets the pair of the first fragment S1 and the second fragment S2 as a fragment pair for which the connectivity degree is to be calculated. On the other hand, since the second fragments S3 and S4 do not have pixels that are included in the search range R, the pairs of the first fragment S1 and the second fragment S3 and the first fragment S1 and the second fragment S4 are not set as fragment pairs.

[0041] 6, when the search range R is a pixel region corresponding to a circle, the distance w (first distance) between the first fragment S1 and the second fragment S2 set as a pair is smaller than the radius r of the search range R. Therefore, the pair setting unit 513 sets the fragment pair based on the distance between the fragments S (e.g., the distance between two edge pixels T).

[0042] The pair setting unit 513 sets fragment pairs for each of the fragments S constituting the multiple fragments S set by the pixel group setting unit 512. In the example of FIG. 6, the pair setting unit 513 sets a search range R to the edge pixel T for each of the fragments S2, S3, and S4, and detects the edge pixel T of each of the other fragments S within the search range R. The pair setting unit 513 sets fragment pairs of the fragments S2, S3, and S4, and the fragment S including the detected edge pixel T. The pair setting unit 513 does not need to set an overlapping fragment pair when there is an overlap with an already set fragment pair. As a result, the calculation unit 514 does not perform a connection degree calculation process described later for pairs not detected based on the search range R, since the fragment pairs are set by the pair setting unit 513. The pair setting unit 513 stores the set fragment pairs in the storage unit 43 or the like.

[0043] The calculation unit 514 calculates the degree of connection between the two fragments S included in the fragment pair. In the following description, the two fragments included in the fragment pair are referred to as the first fragment S1 and the second fragment S2. In other words, the calculation unit 514 calculates the degree of connection of the second fragment S2 to the first fragment S1. Here, the degree of connection indicates the probability that the two fragments S correspond to the same linear part of the object, in other words, the same neurite Nr, and includes the ease of connection based on the shape and arrangement of the two fragments S. The degree of connection is expressed by a numerical value or the like, and may be expressed by a symbol or the like if the degrees of connection can be compared with each other. The calculation unit 514 stores the calculated degree of connection in the storage unit 43 or the like. Also, as described above, for example, when there is no fragment pair set by the pair setting unit 513, the calculation unit 514 does not perform the calculation process of the degree of connection.

[0044] Next, FIG. 7 is a conceptual diagram for explaining the calculation of the connectivity degree. The calculation unit 514 sets a first line segment L1 corresponding to the first fragment S1 and a second line segment L2 corresponding to the second fragment S2. For example, the calculation unit 514 can fix the edge pixel T1 of the first fragment S1 and change the inclination of the line segment passing through the edge pixel T1, and determine the line segment when the line segment passes through the most pixels included in the first fragment S1 as the first line segment L1. In this way, the first fragment S1 corresponding to the first pixel group has the first line segment L1. Similarly, the calculation unit 514 can fix the edge pixel T2 of the second fragment S2 and change the inclination of the line segment when the line segment passes through the most pixels included in the second fragment S2 as the second line segment L2. Alternatively, the calculation unit 514 may determine the line segment connecting the pixels at both ends of the first fragment S1 as the first line segment L1, and the line segment connecting the pixels at both ends of the second fragment S2 as the second line segment L2. In this way, the second fragment S2 corresponding to the second pixel group has the second line segment L2.

[0045] The calculation unit 514 sets a third line segment L3, which is a line segment connecting the edge pixel T1 of the first fragment S1 and the edge pixel T2 of the second fragment S2. The calculation unit 514 calculates the length of the third line segment L3, which is the distance between the edge pixel T1 and the edge pixel T2. Hereinafter, this length is referred to as the inter-fragment distance w, which is the distance between the first fragment S1 and the second fragment S2. In addition, the first line segment L1 does not have to pass through the edge pixel T1. For example, depending on the shape of the first fragment S1, passing through a pixel closer to the center of gravity of the first fragment S1 than the edge pixel T1 may better reflect the shape of the first fragment S1. Therefore, the first line segment L1 can be set to pass through two appropriate pixels included in the first fragment S1. The same applies to the second line segment L2. A line segment passing through the first line segment L1 corresponding to the first fragment S1 obtained in this manner and the second line segment L2 corresponding to the second fragment S2 may be set as the third line segment L3. Similarly, depending on the shapes of the first fragment S1 and the second fragment S2, the third line segment L3 can be set as a line segment connecting any point included in the first fragment S1 and any point included in the second fragment S2.

[0046] Next, the calculation unit 514 calculates a first angle θ1, which is the angle between the first line segment L1 and the second line segment L2. The calculation unit 514 calculates a second angle θ2, which is the angle between the first line segment L1 and the third line segment L3. The calculation unit 514 calculates a third angle θ3, which is the angle between the second line segment L2 and the third line segment L3. These angles can be calculated using the inclinations of the first line segment L1, the second line segment L2, and the third line segment L3, etc.

[0047] The calculation unit 514 calculates the connectivity degree DC of the second fragment S2 with respect to the first fragment S1 by the following formula (1). In the following, "*" indicates a product. DC=c0*|w|+c1*sinθ2+c2*sinθ3+c3*(1+cosθ1)…(1) Here, c0, c1, c2, and c3 are positive constants that are appropriately set based on the characteristics of the extraction target, etc. In addition, the connection degree DC of formula (1) is set to be positive so that the connection degree DC becomes the edge weight when an optimization method such as the Prim method is used for grouping described later. However, the connection degree DC can be appropriately set according to the algorithm for grouping described later by using at least one of the above-mentioned fragment distance w, first angle θ1, second angle θ2, and third angle θ3. In a non-limiting example, the connection degree DC may be only the first term (fragment distance w) or only the fourth term (first angle θ1) on the right side of formula (1), or only the first and fourth terms, or only the first, second (second angle θ2) and third (third angle θ3), or only the first, second and fourth terms, or only the first, third and fourth terms, or only the second, third and fourth terms.

[0048] The determination unit 600 determines the grouping of the multiple fragments S set by the pixel group setting unit 512 based on the degree of connectivity calculated by the calculation unit 514. In this grouping, the fragments S corresponding to the neurites Nr extending from the same cell body So ​​are set to belong to the same group. In this case, it is possible that all the fragments S belong to one group, and it is also possible that some of the fragments S do not belong to any of the groups corresponding to each pre-defined cell body So.

[0049] The algorithm for grouping by the determining unit 600 is not particularly limited as long as it uses the calculated connectivity degree. In the following, an example will be described in which grouping is performed by determining, using the Prim method, a minimum spanning tree of a weighted connected graph in which each fragment S is a node and the connectivity degree is the weight of the edge connecting the nodes. The minimum spanning tree may be found using the Kruskal method or the Brufka method.

[0050] The execution unit 601 of the determination unit 600 executes a grouping algorithm to obtain a minimum spanning tree. The execution unit 601 generates data corresponding to a weighted connected graph, with each fragment S as a node and the degree of connection between the two fragments S of a fragment pair as the weight of the edge between the two nodes corresponding to the two fragments S.

[0051] FIG. 8(A) is a conceptual diagram for explaining the initial stage of grouping, and FIG. 8(B) is a weighted connection graph G corresponding to FIG. 8(A). The weighted connection graphs described in the following figures are examples and do not limit the present invention. A fragment S5 is connected to the cell body So1. A fragment S6 is connected to the cell body So2. The execution unit 601 sets the nodes N5 and N6 corresponding to the fragments S5 and S6 connected to each cell body So ​​as root nodes, and sets the fragments S other than the fragment S corresponding to the root node as nodes. The execution unit 601 sets edges between the extracted nodes.

[0052] In FIG. 8(B), nodes N5, N6, N7, and N8 corresponding to fragments S5, S6, S7, and S8, respectively, are set. Fragment S5 and fragment S7, fragment S5 and fragment S8, fragment S6 and fragment S7, and fragment S6 and fragment S8 are fragment pairs, and edges E57, E58, E67, and E68 are set with the connection degree as weight. In the example of FIG. 8(B), the weight of edge E57 is set to 20, the weight of edge E58 is set to 15, the weight of edge 67 is set to 10, and the weight of edge 68 is set to 30. The virtual node NV is a node that is virtually set so that the entire graph becomes one connected graph. A virtual edge EV1 is set between the virtual node NV and the root node N5, and a virtual edge EV2 is set between the virtual node NV and the root node N6.

[0053] The execution unit 601 sets the root node as a determined node, and sets nodes other than the root node as undetermined nodes. Fragments S5 and S6 and nodes N5 and N6 corresponding to fragments S5 and S6, respectively, correspond to fragments connected to different cell bodies So, and therefore belong to different groups and are shown with different hatching. Nodes N7 and N8 corresponding to fragments S7 and S8, respectively, are undetermined nodes.

[0054] The execution unit 601 repeatedly sets the undetermined node connected to the edge with the smallest weight among the edges connecting the determined nodes and the undetermined nodes as the determined node. At the stage of Figures 8(A) and 8(B), the execution unit 601 selects the edge E67 with the smallest weight among the edges E57, E58, E67, and E68 connecting the determined node N5 or N6 and the undetermined node N7 or N8, and sets the undetermined node N7 connected to the edge E67 as the determined node.

[0055] FIG. 9(A) is a conceptual diagram for explaining the stage after node N7 is set as a decision node, and FIG. 9(B) is a weighted connected graph G corresponding to FIG. 9(A). After node N7 is set as a decision node, a weighted connected graph G including an edge E78 between node N7 and node N8 is created. In the example of FIG. 9(B), the weight of edge E78 is 50. In this way, every time a decision node is set, a weighted connected graph including an edge connecting a newly set decision node and an undetermined node is created. The execution unit 601 sets decision node N6 and decision node N7 connected by the selected edge E67 to belong to the same group. In FIGS. 9(A) and 9(B), the decision nodes N6 and N7, and the fragments S6 and S7 corresponding to decision nodes N6 and N7, respectively, are indicated by the same hatching to belong to the same group. Next, the execution unit 601 selects the edge E58 with the smallest weight from the edges E58, E68, and E78 connecting the determined nodes N5, N6, or N7 to the undetermined node N8, and sets the undetermined node N8 connected to the edge E58 as a determined node.

[0056] Fig. 10(A) is a conceptual diagram for explaining the stage after node N8 has been set as a decision node, and Fig. 10(B) is a weighted connected graph G corresponding to Fig. 10(A). The execution unit 601 sets decision nodes N5 and N8, which are connected by the selected edge E58, to belong to the same group. In Figs. 10(A) and 10(B), the decision nodes N5 and N8, and the fragments S5 and S8 corresponding to decision nodes N5 and N8, respectively, are shown by the same hatching to belong to the same group.

[0057] In FIG. 10C, when there are no more undetermined nodes in the weighted connected graph G, the execution unit 601 sets the weighted connected graph G1 including the edges E58 and E67 selected in the edge search so far, the decision node, and the virtual node NV as a minimum spanning tree. The weighted connected graph G1 can be regarded as connected because a virtual node NV is set by integrating multiple root nodes N5 and N6. In addition, the execution unit 601 generates grouping data, which is data indicating the grouping result of each fragment S, from the data corresponding to the weighted connected graph G1. The execution unit 601 stores the grouping data in the storage unit 43 or the like. The detection result image Gd is subjected to morphological analysis or the like as appropriate based on the grouping data by the data processing unit 51 or the like, and the neurite length is calculated or the like.

[0058] Returning to Fig. 2, the termination determination unit 602 of the determination unit 600 calculates the sum of the weights of the edges selected by the execution unit 601 (hereinafter referred to as the determination parameter), and terminates the process of obtaining the minimum spanning tree by the execution unit 601 when the determination parameter becomes equal to or greater than a predetermined threshold (hereinafter referred to as the determination threshold). The determination threshold can be set appropriately based on the average value of the connection degree, which is the edge weight, the characteristics of the object, the number of fragments S included in the detection result image Gd, etc. This makes it possible to prevent fragments S that do not need to be grouped, such as fragments S that are too far away from other fragments S or fragments to which a connected cell body So ​​does not exist in the detection result image Gd, from being grouped.

[0059] The image generating unit 700 generates image data (hereinafter referred to as output image data) corresponding to an output image which is an image showing the result of grouping each of the fragments S in the detection result image Gd. The form of the output image is not particularly limited, and the result of grouping can be shown by an image obtained by image processing the detection result image Gd or by characters. When generating an output image obtained by image processing the detection result image Gd, the fragments S can be shown separately for each group. For example, in the output image, at least one of the hue, brightness, and saturation of the fragments S can be different for each group, and for example, the brightness values ​​of RGB or the like can be different for each group.

[0060] Returning to FIG. 1, the output control unit 52 of the control unit 50 controls the output unit 44 to output an output image.

[0061] The device control unit 53 of the control unit 50 controls each unit of the culture unit 100 based on input from the input unit 41, etc. (arrow A2). The device control unit 53 executes control related to cell culture (e.g., control of the culture chamber 10 that manages temperature or humidity, control of the drive unit 12), and causes the imaging unit 20 to perform imaging.

[0062] 11 is a flowchart showing the flow of the image processing method according to this embodiment. In step S101, the detection result image generating unit 511 acquires captured image data obtained by imaging by the imaging unit 20. After step S101 ends, step S103 starts. In step S103, the detection result image generating unit 511 processes the captured image data to acquire probability distribution image data. After step S103 ends, step S105 starts.

[0063] In step S105, the detection result image generation unit 511 performs image processing on the probability distribution image data to obtain the detection result image data. After step S105 is completed, step S107 is started. In step S107, the pixel group setting unit 512 sets a plurality of fragments S. After step S107 is completed, step S109 is started.

[0064] In step S109, the pair setting unit 513 sets pairs (fragment pairs) and sets a connection degree corresponding to each pair. After step S109 is completed, step S111 is started. In step S111, the determination unit 600 determines the grouping of each fragment S. After step S111 is completed, step S113 is started. In step S113, the output control unit 52 outputs information indicating the grouping result. For example, the output control unit 52 outputs information such as a label using a name, number or symbol indicating fragments belonging to the same group, or an output image in which fragments belonging to the same group are represented by the same brightness value, etc. After step S113 is completed, the processing is terminated.

[0065] Fig. 12 is a flowchart showing the process of step S109 in the flowchart of Fig. 11. After step S107 is completed, step S1091 is started. In step S1091, the set setting unit 513 selects a fragment S. After step S1091 is completed, step S1093 is started. In step S1093, the set setting unit 513 sets a search range R for the selected fragment S. After step S1093 is completed, step S1095 is started.

[0066] In step S1095, the pair setting unit 513 sets a pair of the selected fragment S and a fragment S including a pixel included in the search range R of the selected fragment S. After step S1095 is completed, step S1097 is started. In step S1097, the calculation unit 514 calculates the connectivity degree, and associates the pair with the connectivity degree and stores them in the storage unit 43 or the like. After step S1097 is completed, step S1099 is started.

[0067] In step S1099, the group setting unit 513 determines whether or not all of the fragments S in the detection result image Gd have been selected. If all of the fragments S have been selected, the group setting unit 513 makes a positive decision in step S1099, and step S111 is started. If there is an unselected fragment S, the group setting unit 513 makes a negative decision in step S1099, and step S1091 is started.

[0068] Fig. 13 is a flowchart showing the process of step S111 in the flowchart of Fig. 11. After step S109 is completed, step S1111 is started. In step S1111, the execution unit 601 sets the root node as a decision node. After step S1111 is completed, step S1112 is started. In step S1112, the execution unit 601 sets nodes and edges other than the root node. After step S1112 is completed, step S1113 is started.

[0069] In step S1113, the execution unit 601 searches for an edge with the smallest weight connecting a determined node and an undetermined node, and sets the undetermined node connected to the edge obtained in the search as the determined node. After step S1113 ends, step S1114 is started. In step S1114, the execution unit 601 judges whether or not an undetermined node exists. If an undetermined node exists, the execution unit 601 makes an affirmative decision in step S1114, and step S1115 is started. If an undetermined node does not exist, the execution unit 601 makes a negative decision in step S1114, and step S1116 is started.

[0070] In step S1115, the termination determination unit 602 determines whether the determination parameter is less than the determination threshold. If the determination parameter is less than the determination threshold, the termination determination unit 602 makes an affirmative decision in step S1115, and step S1113 is started. If the determination parameter is equal to or greater than the determination threshold, the termination determination unit 602 makes a negative decision in step S1114, and step S1116 is started. In step S1116, the execution unit 601 sets grouping information by treating a plurality of nodes connected to each other by edges obtained by the search as one group. After step S1116 is finished, step S113 is started.

[0071] According to the above-described embodiment, the following advantageous effects can be obtained. (1) The image processing device 1 of this embodiment includes a pixel group setting unit 512 that acquires a plurality of fragments S including a first fragment S1 and second fragments S2, S3, and S4 that are not connected to one another from a plurality of pixels Px that constitute the detection result image Gd, and a pair setting unit 513 that sets a pair of the first fragment S1 and a second fragment S2 including a pixel that is within a search range R from the first fragment S1, among the plurality of fragments S, in order to set the degree of connection to the first fragment S1. This enables the image processing device 1 to accurately analyze how pixel groups such as the extracted fragments S, including disconnected pixel fragments, correspond to linear portions.

[0072] (2) The image processing device 1 of this embodiment includes a calculation unit 514 that calculates the inter-fragment distance w or the first angle θ1 between the first fragment S1 and the second fragment S2, and the calculation unit 514 calculates the connectivity degree using the inter-fragment distance w or the first angle θ1. This enables the image processing device 1 to accurately analyze how two pixel groups correspond to linear portions based on the positional relationship between these pixel groups.

[0073] (3) In the image processing device 1 of this embodiment, the calculation unit 514 calculates at least one of the second angle θ2 between the first line segment L1 and the third line segment L3 passing through a point included in the first fragment S1 and a point included in the second fragment S2, and the third angle θ3 between the third line segment L3 and the second line segment L2, and can calculate the degree of connectivity based on at least one of the second angle θ2 and the third angle θ3. This allows the image processing device 1 to more accurately analyze how the two pixel groups correspond to the linear portions based on a more detailed positional relationship between these two pixel groups.

[0074] (4) In the image processing device 1 of this embodiment, the calculation unit 514 can calculate at least one of the angles between the shortest line segment among the third line segments L3 and the first line segment L1 and between the line segment L3 and the second line segment L2, and calculate the degree of connectivity based on the at least one angle. This allows the image processing device 1 to more accurately analyze how the two pixel groups correspond to the linear portions based on a more detailed positional relationship between these two pixel groups.

[0075] (5) In the image processing device 1 of this embodiment, the set setting unit 513 sets sets for each of the fragments S constituting the multiple fragments S, the calculation unit 514 calculates the connectivity degree for each set, and the determination unit 600 determines how the multiple fragments S are grouped based on the calculated connectivity degree. This enables the image processing device 1 to accurately analyze how extracted pixel groups correspond to linear portions across the entire image, including disconnected pixel fragments.

[0076] (6) In the image processing device 1 of this embodiment, the determination unit 600 determines which group each node belongs to based on a minimum spanning tree obtained by the Prim's method, Kruskal's method, or Brufka's method, with each of the multiple fragments S as a node and the degree of connection as an edge weight. This enables the image processing device 1 to efficiently and optimally group pixels.

[0077] (7) In the image processing device 1 of this embodiment, the determination unit 600 includes a termination determination unit 602 that, when selecting edges in the Prim method, the Kruskal method, or the Brufka method, determines whether or not to terminate the process of obtaining a minimum spanning tree based on the sum of the numerical values ​​corresponding to the selected edges. This enables the image processing device 1 to prevent pixel groups that do not need to be grouped from being grouped.

[0078] (8) The image processing device 1 of this embodiment includes an image generation unit 700 that generates an output image showing the grouping result. This enables the image processing device 1 to provide information on how the extracted pixel groups correspond to linear portions.

[0079] (9) In the image processing device 1 of this embodiment, before acquiring the multiple fragments S, the pixel group setting unit 512 removes pixels corresponding to a triangulation or crossover from the pixels constituting the detection result image Gd, and sets these as pixels not included in the multiple fragments S. This allows the image processing device 1 to accurately analyze how the extracted pixel groups correspond to linear portions, including disconnected pixel fragments, even for images including a triangulation or crossover.

[0080] (10) In the image processing device 1 of this embodiment, the detection result image Gd is an image obtained by performing image processing on an image obtained by capturing an image of an object, and is a binarized image in which, for each pixel of the detection result image Gd, a pixel value indicates whether or not the pixel corresponds to a linear structure, and the pixel group setting unit 512 obtains a plurality of fragments S based on the pixel values. This allows the image processing device 1 to accurately analyze how the pixel groups extracted from the captured image, including disconnected pixel fragments, correspond to the linear portions of the subject.

[0081] (11) In the image processing device 1 of this embodiment, the detection result image Gd is an image showing a plurality of pixels Px1 corresponding to the neurite Nr, and the connectivity degree is a numerical value indicating the possibility that the first fragment S1 and the second fragment S2 correspond to parts of a neurite extending from the same nerve cell. This allows the image processing device 1 to accurately analyze how the group of pixels extracted from the captured image, including disconnected pixel fragments, correspond to the neurite Nr.

[0082] (12) The image processing device 1 of this embodiment includes a pixel group setting unit 512 that acquires the first fragment S1 and the second fragment S2 from the detection result image Gd, a calculation unit 514 that calculates the second angle θ2 between the third line segment L3 and the first line segment L1 that pass through the first fragment S1 and the second fragment S2, and a determination unit 600 that determines whether the first fragment S1 and the second fragment S2 correspond to a linear region where the first fragment S1 and the second fragment S2 are connected based on the second angle θ2. This allows the image processing device 1 to accurately analyze how the extracted pixel group corresponds to the linear portion based on the positional relationship between the first fragment S1 and the second fragment S2. In addition, by using the second angle θ2, the image processing device 1 can accurately group the first fragment S1 and the second fragment S2 even when the thin lines constituting the first fragment S1 and the second fragment S2 are shifted in parallel, when one thin line is extremely short, when the first fragment S1 and the second fragment S2 are curved fragments, and the like.

[0083] (13) In the image processing device 1 of this embodiment, the calculation unit 514 calculates the third angle θ3 between the third line segment L3 and the second line segment L2, and the determination unit 600 determines whether or not the first fragment S1 and the second fragment S2 correspond to a linear region where the fragment S1 and the fragment S2 are connected, based on the second angle θ2 and the third angle θ3. This allows the image processing device 1 to accurately analyze how the extracted pixel group corresponds to the linear portion, based on a more detailed positional relationship between the two fragments S.

[0084] (14) In the image processing device 1 of this embodiment, the determination unit 600 can determine whether the first fragment S1 and the second fragment S2 correspond to a linear region where the first fragment S1 and the second fragment S2 are connected, based on the first angle θ1 and the second angle θ2 formed by the first line segment L1 corresponding to the first fragment S1 and the second line segment L2 corresponding to the second fragment S2. This allows the image processing device 1 to accurately analyze how the extracted pixel group corresponds to the linear portion, based on a more detailed positional relationship between the two fragments S.

[0085] (15) In the image processing device 1 of this embodiment, the determination unit 600 can determine whether or not the first fragment S1 and the second fragment S2 correspond to a linear region where the fragment S1 and the fragment S2 are connected, based on the first angle θ1. This allows the image processing device 1 to accurately analyze how the extracted pixel group corresponds to the linear portion, based on the direction in which the two fragments S are arranged.

[0086] (16) In the image processing device 1 of this embodiment, the determination unit 600 can determine whether the first fragment S1 and the second fragment S2 correspond to a linear region where the fragments S are connected, based on the first angle θ1, the second angle θ2, and the third angle θ3. This allows the image processing device 1 to accurately analyze how the extracted pixel group corresponds to the linear portion, based on a more detailed positional relationship between the two fragments S.

[0087] (17) The imaging device according to this embodiment includes the above-described image processing device 1 and an imaging unit 20. This allows the imaging device to accurately analyze how a group of pixels extracted from a captured image, including disconnected pixel fragments, corresponds to a linear portion.

[0088] (18) The above-described image processing device 1, which is the culture device according to this embodiment, includes a culture unit 100 that cultures cells. This allows the culture device to accurately analyze how a group of pixels extracted from an image of cultured cells, including disconnected pixel fragments, corresponds to a linear portion.

[0089] (19) The image processing method according to this embodiment includes obtaining a plurality of fragments S including a first fragment S1 and second fragments S2, S3, and S4 that are not connected to one another from a plurality of pixels Px constituting the detection result image Gd, and setting, among the plurality of fragments S, a pair of the first fragment S1 and a second fragment S2 including a pixel within a search range R from the first fragment S1 in order to set a degree of connectivity to the first fragment S1. This makes it possible for the image processing method to accurately analyze how the extracted pixel group, including the disconnected pixel fragments, corresponds to the linear portion.

[0090] (20) The image processing method according to this embodiment includes obtaining a first fragment S1 and a second fragment S2 from the detection result image Gd, calculating a second angle θ2 between the first fragment L1 and a third line segment L3 passing through a point included in the first fragment S1 and a point included in the second fragment S2, and determining whether or not the first fragment S1 and the second fragment S2 correspond to a linear region connected thereto based on the second angle θ2. This makes it possible for the image processing method to accurately analyze how the extracted pixel group corresponds to the linear portion based on the positional relationship between the two fragments S.

[0091] The following modifications are also within the scope of the present invention and can be combined with the above-described embodiment.

[0092] (Variation 1) In the above embodiment, the pixel group setting section 512 may set, as one fragment S, a pixel group made up of a plurality of pixel fragments that are not connected to each other, from the plurality of pixels Px included in the detection result image Gd.

[0093] 14(A) and 14(B) are diagrams showing examples of pixel groups (e.g., multiple pixel groups) consisting of multiple pixel fragments. The pixel group Sg1 in FIG. 14(A) includes pixel fragments SS1, SS2, and SS3. The pixel fragment SS2 corresponds to, for example, a circular pixel to which it is difficult to assign a thin line. Even when such a pixel fragment SS2 exists, the pixel group setting unit 512 can set these pixel fragments SS1, SS2, and SS3 as one pixel group Sg1 if they are arranged linearly. For example, when the pixel fragments are within a predetermined distance from each other and the pixel fragment SS2 is located at a distance shorter than a predetermined distance from the line segment corresponding to the pixel fragment SS1 or SS3, the pixel fragment SS2 can be set as a pixel fragment constituting the pixel group Sg1. The above-mentioned predetermined distance can be set in advance so that the pixels constituting the pixel group Sg1 are arranged linearly.

[0094] 14(B) includes pixel fragments SS4, SS5, SS6, and SS7. Even when pixel fragments are arranged in this manner, if, for example, the pixel fragments are within a predetermined distance from each other and each of the pixel fragments SS4 to SS7 is within a predetermined distance from a line segment corresponding to at least one of the pixel fragments SS4 to SS7, the pixel group setting unit 512 can set these as pixel fragments that make up the pixel group Sg2.

[0095] The data processing unit 51 can process the pixel group Sg1 or Sg2 consisting of a plurality of pixel fragments as one fragment S. For example, the calculation unit 514 can set the pixels at both ends of the pixel group Sg1 as the edge pixels T1 and T2 in the above-mentioned embodiment, and can set the first line segment L1, the second line segment L2, the first angle θ1, the second angle θ2, the third angle θ3, and the inter-fragment distance w in accordance with the definitions in the above-mentioned embodiment. Therefore, even if at least a portion of the fragment S in the above-mentioned embodiment is a pixel group each including a plurality of pixel fragments, processing such as setting sets and grouping can be performed in a similar manner.

[0096] In the image processing device of this modified example, the pixel group setting unit 512 obtains, from the multiple pixels Px that make up the detection result image Gd, a pixel group made up of multiple connected pixels Px1, or a pixel group Sg1 or Sg2 made up of multiple unconnected pixel groups SS1-3, or 4-7, as a pixel group that makes up the multiple fragments S. This makes it possible to efficiently analyze how the extracted pixel groups correspond to the linear portions when the detection result image Gd includes many pixel fragments made up of a small number of pixels.

[0097] (Variation 2) The set setting unit 513 may set the search range R in a manner different from that of the above-described embodiment. In the following modified examples 2-1 and 2-2, the length and width of the search range may be an average, such as an arithmetic mean, of the lengths of the line segments corresponding to the fragments S included in the detection result image Gd, or a value based on the average.

[0098] (Variation 2-1) 15(A) is a conceptual diagram for explaining a method for setting a search range R3 in this modified example. The pair setting unit 513 sets a rectangular or circular pixel area centered on the center of gravity C1 of the pixels of the fragment S101 as the search range R3. The pair setting unit 513 can set pairs of the fragment S101 and each of the fragments S102 and S103 having pixels included in the search range R3 as fragment pairs.

[0099] (Variation 2-2) 15(B) is a conceptual diagram for explaining a method for setting search range R4 in this modified example. The pair setting unit 513 sets a rectangular or circular pixel area centered on center C2 of a rectangle Rt circumscribing the pixels of fragment S201 as search range R4. The pair setting unit 513 can set pairs of fragment S201 and each of fragments S202 and S203 having pixels included in search range R4 as fragment pairs.

[0100] In the image processing device 1 of the above-mentioned modified example 2, the pair setting unit 513 sets, among the multiple fragments S, a pair of fragment S101 and fragments S102 and S103 including pixels within a search range R3 set corresponding to the center of gravity of fragment S101, or a pair of fragment S201 and fragments S202 and S203 including pixels within a search range R4 set corresponding to the center of a rectangle Rt circumscribing fragment S201. This enables the image processing device 1 to suppress variation in the position of the search range due to the shape of fragment S101 or S201, and to more accurately analyze how the extracted pixel group corresponds to the linear portion.

[0101] (Variation 3) In the above embodiment, the calculation unit 514 may change the method of calculating the degree of connection when the distance in the detection result image between the first fragment S1 and the second fragment S2 included in the fragment pair is longer than a predetermined distance. Hereinafter, this predetermined distance is referred to as the distance threshold DIST. The distance between the first fragment S1 and the second fragment S2 defined here may be the inter-fragment distance w, or may be any distance defined based on the first fragment S1 and the second fragment S2, and may be defined by the number of pixels corresponding to the any distance in the detection result image.

[0102] When the distance between the first fragment S1 and the second fragment S2 is smaller than the distance threshold DIST, the pair setting unit 513 sets the connectivity degree DC of the second fragment S2 relative to the first fragment S1 by the above-mentioned formula (1). DC=c0*|w|+c1*sinθ2+c2*sinθ3+c3*(1+cosθ1)…(1) If the distance between the first fragment S1 and the second fragment S2 is equal to or greater than DIST, the pair setting unit 513 sets the connectivity degree DC of the second fragment S2 relative to the first fragment S1 by the following formula (2). DC = c0 * |w| + c1 + c2 + 2 * c3 … (2)

[0103] FIG. 16 is a graph showing the relationship between the distance (horizontal axis) between the first fragment S1 and the second fragment S2 and the connection degree (vertical axis) of the second fragment S2 to the first fragment S1 in this modified example. In FIG. 16, an area V1 corresponding to the connection degree calculated by the above formula (1) and an area V2 corresponding to the connection degree calculated by the above formula (2) are shown by hatching. When comparing the case where the distance between the first fragment S1 and the second fragment S2 is less than the distance threshold DIST with the case where the distance is equal to or greater than the distance threshold DIST, there is no overlap in the connection degree values ​​except for the boundary value DC1=c0*DIST+c1+c2+2*c3. Therefore, when the distance between the first fragment S1 and the second fragment S2 is equal to or greater than the distance threshold DIST, the connection degree is equal to or greater than the connection degree when the distance is less than the distance threshold DIST. This makes it possible to increase the possibility that the first fragment S1 and the second fragment S2 belong to different groups in the former case than in the latter case.

[0104] The calculation unit 514 may also change the method of calculating the degree of connectivity when a pixel included in another fragment S exists between the first fragment S1 and the second fragment S2 included in the fragment pair. When there is no pixel included in another fragment S between the first fragment S1 and the second fragment S2, for example on the third line segment L3, the set setting unit 513 sets the degree of connectivity DC of the second fragment S2 to the first fragment S1 by the above formula (1). When there is a pixel included in another fragment S between the first fragment S1 and the second fragment S2, the set setting unit 513 sets the degree of connectivity DC of the second fragment S2 to the first fragment S1 by the above formula (2). This makes it possible to increase the possibility that the first fragment S1 and the second fragment S2 belong to different groups when a pixel included in another fragment S exists between the first fragment S1 and the second fragment S2.

[0105] In the image processing device of this modified example, when a first condition is satisfied that there is another pixel group acquired by the pixel group setting unit 512 between the first fragment S1 and the second fragment S2, or when a second condition is satisfied that the distance or the number of pixels between the first fragment S1 and the second fragment S2 is determined based on a distance threshold DIST, the calculation unit 514 calculates the degree of connectivity in a manner different from that when neither the first condition nor the second condition is satisfied. This enables the image processing device 1 to more accurately analyze how the extracted pixel groups correspond to the linear portions in accordance with the arrangement of the two pixel groups.

[0106] (Variation 4) In the above embodiment, when the pixel group setting section 512 detects a crossover, the calculation section 514 can set the degree of connectivity so that the fragments S facing each other across the crossover belong to the same group.

[0107] 17(A) and (B) are conceptual diagrams for explaining the setting of the connection degree of this modified example. FIG. 17(A) is a diagram showing four fragments S301, S302, S303, and S304 that were connected to the crossover after the crossover was detected and removed. After removing the crossover, the pixel group setting unit 512 stores information indicating that the fragments S301, S302, S303, and S304 are fragments that were connected to the removed crossover in the storage unit 43 or the like. For example, the pixel group setting unit 512 includes a crossover pixel group setting unit that sets the fragments that were connected to the crossover as one crossover pixel group, and the set setting unit 513 and the calculation unit 514 may process the crossover pixel group as one fragment S with respect to other fragments S. After referring to the information, the calculation unit 514 sets the degree of connection so that the probability that two opposing fragments S301 and S302 are connected, and the probability that two opposing fragments S303 and S304 are connected are smaller than the probability that other two fragments of the fragments that were connected at the crossover are connected.

[0108] FIG. 17B shows a weighted connected graph G2 corresponding to the fragments S301 to S304. The nodes corresponding to the fragments S301, S302, S303, and S304 are N301, N302, N303, and N304, respectively. The calculation unit 514 sets the weight of the edge connecting the nodes N301 and N302 and the edge connecting the nodes N303 and N304 to 0. The calculation unit 514 sets the weight of the other edges E connecting the nodes N301 to N304, excluding these two edges, to the maximum (MAXCOST). Before calculating the connection degrees for the other fragments S, the calculation unit 514 calculates which fragments S face each other in the four regions corresponding to the crossing, and sets the connection degrees so that the probability that two fragments S determined to face each other is connected is high. By performing grouping after setting the connection degrees in this way, two fragments S determined to face each other at the time of grouping can be reliably connected. Note that there are no particular limitations on the specific value of the edge weight as long as it makes it easier for two opposing fragments S to be grouped into the same group. The intersection pixel group setting unit may set a fragment that includes a pixel within a certain range from any of the fragments S301 to S304 as a fragment that constitutes an intersection pixel group.

[0109] In the image processing device according to this modification, the calculation unit 514 calculates the degree of connectivity so that a pair of fragments S that are generated by excluding pixels corresponding to crossovers and that face each other across a crossover correspond to a part of a connected linear region. This enables the image processing device to more accurately analyze how the extracted pixel group including the crossover corresponds to the linear portion.

[0110] In the image processing device according to this modification, pixel group setter 512 includes a crossing pixel group setter that, when three other pixel groups S302, S303, and S304 are present within a certain range from fragment S301, sets four fragments S including fragment S301 and the three pixel groups S302, S303, and S304 as a crossing pixel group, and calculation unit 514 can calculate the degree of connectivity so that a pair of fragments S facing each other across a crossing corresponds to part of a connected linear region. This enables the image processing device to more accurately analyze how the extracted pixel groups including the crossing correspond to the linear portion.

[0111] (Variation 5) In the above-described embodiment, the calculation unit 514 can further calculate the connectivity degree based on at least one characteristic of the segment S shown in the following modified examples 5-1 to 5-4. (Variation 5-1) The calculation unit 514 may calculate the degree of connectivity based on an average value such as an arithmetic average of the luminance of pixels included in each of the first fragment S1 and the second fragment S2. In this case, the luminance of the detection result image Gd, as well as the luminance of the captured image or the probability distribution image Gp, may be used. The smaller the difference between the average luminance of the pixels of the first fragment S1 and the average luminance of the pixels of the second fragment S2, the smaller the degree of connectivity is set, so that the first fragment S1 and the second fragment S2 are more likely to be in the same group.

[0112] (Variation 5-2) The calculation unit 514 may calculate the degree of connectivity based on the width of the first fragment S1 and the width of the second fragment S2. In this case, the width can be set, for example, by the number of pixels through which a straight line perpendicular to the line segments corresponding to the first fragment S1 and the second fragment S2 passes. The calculation unit 514 can set the degree of connectivity to be smaller as the difference between the width of the first fragment S1 and the width of the second fragment S2 becomes smaller, so that the first fragment S1 and the second fragment S2 are more likely to be in the same group.

[0113] (Variation 5-3) The calculation unit 514 may calculate the degree of connection based on the curvature of the curve corresponding to the first fragment S1 and the curvature of the curve corresponding to the second fragment S2. In this case, the curvature may be a curve calculated by a known fitting method or the like for each of the first fragment S1 and the second fragment S2. The calculation unit 514 can set the degree of connection to be smaller as the difference between the curvature of the curve corresponding to the first fragment S1 and the curvature of the curve corresponding to the second fragment S2 is smaller, so that the first fragment S1 and the second fragment S2 are more likely to be in the same group.

[0114] (Variation 5-4) The calculation unit 514 may calculate the degree of connection based on the length of at least a part of the straight line portion of the broken line corresponding to the first fragment S1 and the length of at least a part of the straight line portion of the broken line corresponding to the second fragment S2. In this case, it is preferable to use the length of the straight line portion of the first fragment S1 closest to the second fragment and the straight line portion of the second fragment S2 closest to the first fragment. The longer the length of the straight line portion, the more accurately the first angle θ1 and the like can be calculated, and the more accurately the grouping can be performed. The broken line may be a broken line calculated by a known fitting method or the like for each of the first fragment S1 and the second fragment S2. The calculation unit 514 may set the degree of connection to be smaller as the difference between the length of the straight line portion corresponding to the first fragment S1 and the length of the straight line portion corresponding to the second fragment S2 becomes smaller, so that the first fragment S1 and the second fragment S2 are more likely to be in the same group.

[0115] In the image processing device of this modified example, the calculation unit 514 calculates the degree of connectivity based on at least one of the average luminance of the pixels Px constituting the first fragment S1 and the second fragment S2, the width of the first fragment S1 and the second fragment S2, the curvature of the curve when the first fragment S1 and the second fragment S2 correspond to the curve, and the length of the line segment constituting the broken line when the first fragment S1 or the second fragment S2 corresponds to the broken line. This allows the image processing device to more accurately analyze how the extracted pixel group corresponds to the linear portion based on these characteristics (e.g., width, curvature, etc.).

[0116] (Variation 6) In the above-described embodiment, the subject of the captured image is a cell Ce, and the linear portion to be extracted is a neurite Nr. However, the subject to be extracted is not limited to this example as long as it includes a linear portion, and it is also preferable to extract a blood vessel, for example.

[0117] Furthermore, it is more preferable that the subject of the captured image is the fundus, and the linear portion to be extracted is a blood vessel in the fundus, and it is even more preferable that the linear portion is a blood vessel in the fundus, and in particular, a blood vessel in the retina or a blood vessel in the choroid. In the fundus image, the linear portion to be extracted is a blood vessel, and the detection result image generating unit 511 generates a probability distribution image Gp from the captured image in which the probability that each pixel corresponds to a blood vessel corresponds to a luminance value, and generates a detection result image Gd from this probability distribution image Gp. The pixel group setting unit 511, the set setting unit 512, the calculation unit 514, and the determination unit 600 perform the above-mentioned processing on the generated detection result image Gd, so that blood vessels connected to each other can be appropriately extracted. In this case, the execution unit 601 can set the type of fragmented blood vessel, etc., for some pixels of the detection result image Gd as a root node that is not a virtual node of the minimum spanning tree. Here, the type of blood vessel can include an artery, a vein, etc. When grouping fragments S corresponding to choroidal blood vessels, multiple vortex veins can also be set as root nodes, and the fragments S can be grouped based on whether or not they are connected to the vortex vein corresponding to each root node.

[0118] In the image processing device of this modified example, the detection result image Gd is an image showing a plurality of pixels Px1 corresponding to blood vessels, and the connectivity degree is a numerical value indicating the possibility that the first fragment S1 and the second fragment S2 correspond to a part of the same blood vessel. This allows the image processing device to accurately analyze how the group of pixels extracted from the captured image, including the disconnected pixel fragments, correspond to the parts of the blood vessels divided for each root node.

[0119] (Variation 7) In the above-described embodiment, the data processing unit may calculate at least one of the second angle θ2 and the third angle θ3 for the two fragments S set by the pixel group setting unit 512, and determine whether the two fragments S correspond to the same linear portion of the object based on whether the calculated second angle θ2 and third angle θ3 are within a predetermined range.

[0120] FIG. 18 is a conceptual diagram showing the configuration of the data processing unit 51a in the image processing device of this modified example. The data processing unit 51a includes a detection result image generating unit 511, a pixel group setting unit 512, a calculation unit 514a, a determination unit 600a, and an image generating unit 700. The calculation unit 514a calculates at least one of the second angle θ2 and the third angle θ3 for two fragments S among the fragments S set by the pixel group setting unit 512. The determination unit 600a determines that the two fragments S correspond to the same neurite Nr when at least one of the calculated second angle θ2 and the third angle θ3 is included in the numerical range indicating the above-mentioned predetermined range stored in the storage unit 43 or the like. The above-mentioned predetermined range is appropriately set so that the two fragments S are arranged in a line. The method of this modified example also makes it possible to accurately analyze how the extracted pixel group corresponds to the linear part.

[0121] (Variation 8) A program for realizing the information processing function of the information processing unit 40 of the above-mentioned embodiment may be recorded on a computer-readable recording medium, and the program recorded on the recording medium, which is related to the processing by the data processing unit 51, such as the setting of the above-mentioned fragment S, fragment pairs, and connection degree, may be read into the computer system and executed. The term "computer system" as used herein includes an OS (Operating System) and hardware of peripheral devices. The term "computer-readable recording medium" refers to portable recording media such as flexible disks, magneto-optical disks, optical disks, and memory cards, and storage devices such as hard disks built into the computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing part of the above-mentioned functions, or may be a program that realizes the above-mentioned functions by combining with a program already recorded in the computer system.

[0122] When applied to a personal computer (hereinafter referred to as PC), the program relating to the above-mentioned control can be provided through a recording medium such as a CD-ROM or a data signal such as the Internet. FIG. 19 is a diagram showing this state. A PC 950 is provided with the program through a CD-ROM 953. The PC 950 also has a function of connecting to a communication line 951. A computer 952 is a server computer that provides the above-mentioned program, and stores the program in a recording medium such as a hard disk. The communication line 951 is a communication line such as the Internet or personal computer communication, or a dedicated communication line. The computer 952 reads out the program using the hard disk, and transmits the program to the PC 950 through the communication line 951. That is, the program is carried as a data signal by a carrier wave and transmitted through the communication line 951. In this way, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium or a carrier wave.

[0123] As a program for realizing the above-mentioned information processing function, a program for making the processing device perform a pixel group setting process (corresponding to step S107 in the flowchart of FIG. 11) for setting a plurality of fragments S including a first fragment S1 and a second fragment S2 that are not connected to each other from a plurality of pixels Px constituting the detection result image Gd, and a pair setting process (corresponding to step S1095 in the flowchart of FIG. 12) for setting a pair of the first fragment S1 and a second fragment S2 including a pixel within a predetermined range from the first fragment S1, among the plurality of fragments S, in order to set the degree of connection to the first fragment S1. This makes it possible to accurately analyze how the extracted pixel groups, including the disconnected pixel fragments, correspond to the linear portions.

[0124] The program for realizing the above-mentioned information processing function includes a program for making the processing device perform a pixel group setting process (corresponding to step S107 in the flowchart of FIG. 11) for acquiring a first fragment S1 and a second fragment S2 that are not connected to each other from the detection result image Gd, a calculation process (corresponding to step S1097 in the flowchart of FIG. 12) for calculating a second angle θ2 formed by the first fragment L1 and a third fragment L3 passing through the first fragment S1 and the second fragment S2, and a determination process (corresponding to step S111 in the flowchart of FIG. 11) for determining whether or not the first fragment S1 and the second fragment S3 correspond to a linear region where they are connected based on the second angle θ2. This makes it possible to accurately analyze how the extracted pixel group corresponds to the linear portion based on the positional relationship between the first fragment S1 and the second fragment S2.

[0125] (Variation 9) In the above-described embodiment, the processing by the data processing unit 51 may be provided by cloud computing via a network.

[0126] 20 is a conceptual diagram showing the configuration of a quality evaluation system 300 according to this modified example. The quality evaluation system 300 includes an information processing device 310, a terminal device 320, and a gateway terminal 330. The quality evaluation system 300 evaluates the quality of an observation object, such as a cell to be cultured, by performing data processing on an image of the observation object observed by an observation device 340. Such an observation object is preferably one that includes a linear portion, such as a nerve cell having a neurite.

[0127] In this modification, an information processing device 310, a terminal device 320, and a gateway terminal 330 are communicably connected to each other via a network 9. The network 9 may be the Internet, a mobile communication network, a local area network, or the like, or may be a network in which a plurality of these types of networks are combined.

[0128] The gateway terminal 330 transmits the captured image captured by the observation device 340 to the information processing device 310 via the network 9. Although three gateway terminals 331, 332, and 333 are shown in FIG. 20 as the gateway terminals 330, the number of gateway terminals 330 in the quality evaluation system is not particularly limited. The gateway terminals 331, 332, and 333 are connected to the observation devices 341, 342, and 343 wirelessly or by wire so as to be able to communicate with each other. The observation device 340 is handled by a manager of the observation target, which is, for example, a cell. Although the number of observation devices 340 is three in FIG. 20, it can be any number equal to or greater than one.

[0129] The configuration of the observation device 340 is not particularly limited as long as it can capture an image of the observation target. The observation device 340 can include, for example, the above-mentioned culture unit 100. The observation device 340 can include a communication device that transmits captured image data obtained by imaging to the gateway terminal 330, or the captured image data can be transmitted to the gateway terminal 330 by a control PC for the observation device 340 or the like that is configured separately from the observation device. The observation device 340 can include, for example, a detector that detects the temperature, air pressure, illuminance of the light source, oxygen concentration, and the like during cultivation, and can transmit data obtained by detection to the gateway terminal 330.

[0130] The gateway terminal 330 is typically a general-purpose gateway configured to be capable of mutual conversion of protocols or address systems, but is not particularly limited thereto. The gateway terminal 330 may be a PC or the like having a gateway function. From the viewpoint of improving the analysis accuracy of the information processing device 310, the gateway terminal 330 is preferably configured by a plurality of gateway terminals 330, but may be configured by a single gateway terminal. In this case, one gateway terminal 330 and a plurality of observation devices 340 are connected so as to be able to communicate with each other.

[0131] 21(A) is a conceptual diagram showing the configuration of an information processing device 310. The information processing device 310 includes a control unit 311, a storage unit 312, and a communication unit 313. The information processing device 310 of this modified example is a web server for performing quality evaluation of an observation target of an observation device 340 by cloud computing, but is not particularly limited to this. The information processing device 310 may be configured by any computer such as a PC.

[0132] The control unit 311 is configured with a processing device such as a CPU, functions as a main unit of operations for controlling the information processing device 310, and performs various processes by executing a program installed in the storage unit 312. The control unit 311 includes the above-mentioned data processing unit 51, and performs processes such as setting the above-mentioned fragments S, fragment pairs, and connection degrees (e.g., analysis process, first algorithm process, and connection degree calculation process) on the captured image data received and acquired from the observation device 340 via the network 9. The control unit 311 transmits to the terminal device 320 via the communication unit 313 information indicating the results of grouping the fragments S in the detection result image Gd obtained by the process, or output image data of an output image that displays the fragments S determined to be in the same group in the same color or the same brightness, etc.

[0133] The storage unit 312 of the information processing device 310 includes a non-volatile storage medium, and stores data and programs used for processing in the information processing device 310, as well as data obtained by processing by the data processing unit 51. The communication unit 313 of the information processing device 310 includes a communication device capable of communicating via the network 9, and receives captured image data and the like transmitted from the observation device 340, and transmits information generated by the control unit 311 and the like to the terminal device 320.

[0134] 21(B) is a conceptual diagram showing the configuration of the terminal device 320. The terminal device 320 is handled by a quality evaluator of an object to be observed by the observation device 340. The terminal device 320 can be configured to include any computer such as a PC, a smartphone, or a tablet. The terminal device 320 includes a communication unit 321, an output unit 322, and an input unit 323.

[0135] The communication unit 321 of the terminal device 320 includes a communication device capable of communicating via the network 9, and receives information obtained by the processing of the data processing unit 51, etc., transmitted from the information processing device 310. This information can include position information of fragments S belonging to the same group and an output image in which fragments belonging to the same group are colored in the same color. In addition, the information may include a detection result image before removing the triangulation or crossover, if the triangulation or crossover has been removed in advance by a triangulation or crossover detection process. The output unit 322 of the image terminal device 320 includes a display device such as a liquid crystal monitor, and outputs images showing information obtained by the processing of the data processing unit 51. A user (e.g., a quality evaluator) can evaluate the observation target of the observation device 340 by viewing the images displayed on the output unit 322 or by performing data analysis on the images. For example, if the observation target is a nerve cell, the length and number of neurites can be quantified in the above data analysis. The evaluation by the quality evaluator may be input from an input unit (not shown) of the terminal device 320, and the input data may be transmitted to the information processing device 310 via the communication unit 321 and stored in the storage unit 312 of the information processing device 310. Note that the captured image data received by the control unit 311 may be transmitted to the information processing device 310 by a user using an input unit 323 provided in the terminal device 320.

[0136] (Variation 10) In the above-described embodiment, the user can be configured to input or change parameters for the detection and removal of triangulation and crossover, the setting of fragment S, the setting of fragment pairs and connection degrees, and the determination of grouping by viewing the display screen of the output unit 44 as appropriate. Hereinafter, triangulation or crossover is referred to as an intersection, and the detection and removal of an intersection is referred to as an intersection removal process. Also, the setting of fragment S, the setting of fragment pairs and connection degrees, and the determination of grouping are referred to as an allocation process.

[0137] Fig. 22 is a flowchart showing the flow of the image processing method of this modified example. Steps 201 to S205 are similar to steps S101 to S105 in the flowchart of Fig. 11, and therefore a description thereof will be omitted. After step S205 ends, step S207 is started.

[0138] In step S207, the pixel group setting unit 512 removes intersections in the detection result image Gd.

[0139] FIG. 23 is a flowchart showing the process of removing the intersections in step S207. When step S205 is completed, step S2071 is started. In step S2071, the output control unit 52 displays the detection result image Gd on the display screen of the output unit 44. When step S2071 is completed, step S2072 is started. In step S2072, the pixel group setting unit 512 sets parameters for the intersection removal process. The parameters for this intersection removal process are called intersection detection parameters. The intersection detection parameters may include a numerical value indicating the size of the removal pixel region Re (FIG. 5(B)), in particular the number of pixels (width) in the horizontal or vertical direction of the removal pixel region Re.

[0140] The user can input the intersection detection parameters via the input unit 41 by viewing the captured image or the detection result image Gd, and further viewing the intersection removal result image that has already been obtained (described later) when performing the intersection removal process again after performing the intersection removal process. For example, the user can set the width of the removed pixel region Re to be approximately the same as the fragment width of the fragment S in the detection result image Gd, or set the width of the removed pixel region Re to be longer when the shape of the fragment S in the detection result image Gd has many irregularities. Alternatively, the user can set the width of the removed pixel region Re to be smaller when there are many false detections of intersections in the intersection removal result image, and set the width of the removed pixel region Re to be larger when there are many intersections not detected in the intersection removal process. The pixel group setting unit 512 sets the intersection detection parameters based on the user's input via the input unit 41. When step S2072 is completed, step S2073 is started.

[0141] In step S2073, the pixel group setting unit 512 performs the above-mentioned intersection removal process in a manner similar to that of the above-mentioned embodiment, using the intersection detection parameters set in step S2072. When step S2073 is completed, step S2074 is started. In step S2074, the output control unit 52 displays the detection result image after being subjected to the intersection removal process on the display screen of the output unit 44. This detection result image after being subjected to the intersection removal process is called the intersection removal result image. When step S2074 is completed, step S2075 is started.

[0142] In step S2075, the data processing unit 51 determines whether or not an instruction to perform another intersection removal process has been input by the user via the input unit 41. If the instruction has been input, the data processing unit 51 makes an affirmative decision in step S2075, and step S2072 is started. If the instruction has not been input, the data processing unit 51 makes a negative decision in step S2075, and step S209 is started.

[0143] Returning to FIG. 22, in step S209, the data processing unit 51 sets parameters in the allocation process. The parameters in this allocation process are called allocation parameters. The allocation parameters can include a weight c0 of a term including the inter-fragment distance w in formula (1) of the above embodiment, a weight c1 of a term including the second angle θ2, a weight c2 of a term including the third angle θ3, and a weight c3 of a term including the first angle θ1. Furthermore, the allocation parameters can indicate, by a binary value or the like, whether or not a process is performed to set the degree of connection so that the fragments S facing each other are connected to each other for the four fragments S corresponding to the crossover as in the above-mentioned modified example 4. The variable indicated by this binary value is f. In addition, the allocation parameters can include any parameter used in the allocation process. For example, as in the above-mentioned modified example 5, when the difference in width of the fragments S or the difference in brightness between the fragments is used to calculate the degree of connection, the weight c4 of a term including the difference in width of the fragments S in the formula for calculating the degree of connection and the weight c5 of a term including the difference in brightness between the fragments can be included in the allocation parameters. When the inter-fragment distance w is equal to or greater than a predetermined value in the calculation of the connectivity degree, the calculation unit 514 can calculate the connectivity degree without including the term including the inter-fragment distance w. This predetermined value D can be included in the allocation parameter.

[0144] The user can input the allocation parameters via the input unit 41 by viewing the captured image, the detection result image Gd, or the intersection removal result image, and when performing the allocation process again after performing the allocation process, viewing the already obtained allocation result image described later. For example, the user can set the above-mentioned predetermined value D to about twice the value of the intersection detection parameter. When the curvature of the fragment S in the detection result image Gd is small, the user can set the values ​​of the weights c1, c2, and c3 high, and when the curvature of the fragment S is large or there is a lot of noise, the user can lower the values ​​of the weights c1, c2, and c3 and set the value of the weight c0 high. Alternatively, when the variation in the width of the fragment S is large, the user sets the value of the weight c4 of the term including the width of the fragment S high, and when the variation is small, the user sets the value of the weight c4 low, for example, to 0. The data processing unit 51 sets the allocation parameters based on the user's input via the input unit 41. When step S209 is completed, step S211 is started.

[0145] In step S211, the data processing unit 51 performs the above-mentioned allocation process in the same manner as in the above-mentioned embodiment, using the allocation parameters set in step S209. When step S211 ends, step S213 is started. In step S213, the output control unit 52 displays the above-mentioned output image obtained by the allocation process on the display screen of the output unit 44. Hereinafter, this output image is an image in which at least one of the color, lightness, and saturation of the fragments S differs for each group, and is referred to as an allocation result image. When step S213 ends, step S215 is started.

[0146] In step S215, the data processing unit 51 judges whether or not an instruction to perform re-allocation processing has been input by the user via the input unit 41. If the instruction has been input, the data processing unit 51 judges step S215 as positive, and step S207 is started. If the instruction has not been input, the data processing unit 51 judges step S215 as negative, and the processing is terminated. Note that step S209 may be started after the data processing unit 51 makes an affirmative decision in step S215.

[0147] Below, examples of display screens displayed on the output unit 44 for the above-mentioned intersection removal process are shown, but the image format on each display screen, as well as the shapes of buttons and icons, etc., are not limited to those shown in the following figures.

[0148] FIG. 24 is an example of a screen displayed on the output unit 44 before performing the intersection removal process, and this screen is referred to as a first basic screen 701. The first basic screen 701 includes an image display unit 800, a first input element display unit 710, and a video operation unit 810. The first input element display unit 710 includes a first display setting button 711, an intersection detection parameter selection button 712, and an intersection removal execution button 713. The video operation unit 810 includes a rewind button 811 for rewinding a video displayed on the image display unit 800, a play button 812 for playing the video, a pause button 813 for pausing the video, and a fast-forward button 814 for fast-forwarding the video. Although a detection result image Gd is displayed on the image display unit 800 of the first basic screen 701 in FIG. 24, the image displayed on the image display unit 800 can be changed as appropriate.

[0149] The first input element display section 710 includes buttons 711, 712, and 713 which are input elements for causing the image processing device 1 to perform a predetermined operation when the user positions a cursor using a mouse or a touch pad and clicks or the like.

[0150] The first display setting button 711 is an input element for transitioning the display screen to a first display setting screen which is a screen for setting an image to be displayed on the image display unit 800. When the first display setting button 711 is clicked, the output control unit 52 causes the output unit 44 to display the first display setting screen 702 of FIG.

[0151] The intersection detection parameter selection button 712 is an input element for transitioning the display screen to an intersection detection parameter selection screen which is a screen for inputting intersection detection parameters. When the intersection detection parameter selection button 712 is clicked, the output control unit 52 causes the output unit 44 to display the intersection detection parameter selection screen 703 of FIG. 26.

[0152] The intersection removal execute button 713 is an input element for causing the pixel group setting unit 512 to execute the intersection removal process and transition the display screen to a first result display screen which is a screen for displaying the results of the intersection removal process. When the intersection removal execute button 713 is clicked, the output control unit 52 causes the output unit 44 to display the first result display screen 704 of FIG.

[0153] 25 is a diagram showing an example of the first display setting screen 702. The first display setting screen 702 includes an image display section 800, a first image selection section 720, and a video operation section 810. The first image selection section 720 includes a first captured image display button 721, a first detection result image display button 722, a first intersection removal result image display button 723, and a first confirmation button 724. Before these buttons are clicked, the output control section 52 can display, in the image display section 800, an image that was previously displayed on the first basic screen 701 or the like.

[0154] First image selection portion 710 includes buttons 721, 722, 723, and 724 which are input elements for causing image processing portion 800 to select an image to be displayed when the user positions a cursor with a mouse or a touch pad and clicks or the like.

[0155] The first captured image display button 721 is an input element for displaying a captured image on the image display section 800 by clicking the first captured image display button 721. In the following, a fluorescent image of a cell Ce captured by a fluorescent microscope is displayed as the captured image. When the first captured image display button 721 is clicked, the output control section 52 causes the image display section 800 to display the captured image.

[0156] The first detection result image display button 722 is an input element for displaying the detection result image Gd in the image display unit 800 by clicking the first detection result image display button 722. When the first detection result image display button 722 is clicked, the output control unit 52 causes the image display unit 800 to display the detection result image.

[0157] The first intersection removal result image display button 723 is an input element for displaying the intersection removal result image Gk in the image display unit 800 by clicking the first intersection removal result image display button 723. When the first intersection removal result image display button 722 is clicked, the output control unit 52 displays the intersection removal result image Gk in the image display unit 800. The example in Fig. 25 shows the intersection result image Gk displayed after the first intersection removal result image display button 722 is clicked.

[0158] The first confirm button 724 is an input element for transitioning the display screen to the first basic screen 701 by clicking the first confirm button 724. When the first confirm button 724 is clicked, the output control unit 52 transitions the display screen to the first basic screen 701, and then displays the image that was displayed in the image display area 800 of the first display setting screen 702 in the screen display area 800 of the first basic screen 701 as is.

[0159] Fig. 26 is a diagram showing an example of the intersection detection parameter selection screen 703. The intersection detection parameter selection screen 703 includes an image display section 800, a first parameter input section 730, and a video operation section 810. The first parameter input section 730 includes a text box 731 and a second confirmation button 732. The output control section 52 can display, in the image display section 800 of the intersection detection parameter selection screen 703, an image that has been displayed in the image display section 800 of the first basic screen 701 up to that point. The example of Fig. 26 shows a fluorescent image Gf that is a captured image of a cell Ce.

[0160] The text box 731 is an input element for inputting the intersection detection parameters. The user can input a numerical value using a keyboard or the like into the text box 731. The letter "R" indicating the type of intersection detection parameter is displayed above the text box 731. Any input element other than a text box can be placed in the first parameter input section 731 as long as it is possible to input an intersection detection parameter.

[0161] The second confirm button 732 is an input element for transitioning the display screen to the first basic screen 701 by clicking the second confirm button 732. When the second confirm button 732 is clicked, the pixel group setting unit 512 sets the numerical value input in the text box 731 as an intersection detection parameter. In addition, the output control unit 52 transitions the display screen to the first basic screen 701.

[0162] 27 is a diagram showing an example of the first result display screen 704. The first result display screen 704 includes an image display section 800, a first judgment input section 740, and a video operation section 810. The first judgment input section 740 includes a first end button 741 and a first re-execute button 742. The output control section 52 displays an intersection removal result image Gk obtained by the intersection removal process in the image display section 800 of the first result display screen 704. The user can view the intersection removal result image Gk and input whether to end the intersection removal process or to perform the intersection removal process again.

[0163] The first determination input section 740 includes buttons 741 and 742 which are input elements for causing the image processing device 1 to perform a predetermined operation when the user positions the cursor with a mouse, a touch pad or the like and clicks or the like.

[0164] The first end button 741 is a button for ending the intersection removal process and proceeding to setting and execution of the allocation process when the first end button 741 is clicked. When the first end button 741 is clicked, the output control unit 52 displays on the output unit 44 the second basic screen 705 of Fig. 28, which is a screen for setting the allocation process.

[0165] The first re-execution button 742 is a button for performing the intersection removal process again by clicking the first re-execution button 742. When the first re-execution button 742 is clicked, the output control unit 52 displays the first basic screen 701 on the output unit 44.

[0166] Below, examples of display screens displayed on the output unit 44 for the above-mentioned allocation process are shown, but the image format on each display screen, as well as the shapes of buttons and icons, etc., are not limited to those shown in the following figures.

[0167] Fig. 28 is an example of a screen displayed on the output unit 44 before the allocation process is performed, and this screen is referred to as a second basic screen 705. The second basic screen 705 includes an image display section 800, a second input element display section 750, and a video operation section 810. The second input element display section 750 includes a second display setting button 751, an allocation parameter selection button 752, and an allocation execution button 753. Although the detection result image Gd is displayed in the image display section 800 of Fig. 28, the image displayed in the image display section 800 of the second basic screen 705 can be changed as appropriate.

[0168] The second input element display section 750 includes buttons 751, 752, and 753 which are input elements for causing the image processing device 1 to perform a predetermined operation when the user places a cursor on the button using a mouse or a touch pad and clicks on the button.

[0169] The second display setting button 751 is an input element for transitioning the display screen to a second display setting screen which is a screen for setting an image to be displayed on the image display unit 800. When the second display setting button 751 is clicked, the output control unit 52 causes the output unit 44 to display the second display setting screen 706 of FIG.

[0170] The allocation parameter selection button 752 is an input element for transitioning the display screen to an allocation parameter selection screen, which is a screen for inputting allocation parameters. When the allocation parameter selection button 752 is clicked, the output control unit 52 causes the output unit 44 to display the allocation parameter selection screen 707 of FIG.

[0171] The execute allocation button 753 is an input element for causing the data processing unit 51 to execute the allocation process and transition the display screen to a second result display screen which is a screen for displaying the results of the allocation process. When the execute allocation button 753 is clicked, the output control unit 52 causes the output unit 44 to display the second result display screen 708 of FIG.

[0172] 29 is a diagram showing an example of the second display setting screen 706. The second display setting screen 706 includes an image display section 800, a second image selection section 760, and a video operation section 810. The second image selection section 760 includes a second captured image display button 761, a second detection result image display button 762, a second intersection removal result image display button 763, an allocation result image display button 764, and a third confirmation button 765. Before these buttons are clicked, the output control section 52 can display, in the image display section 800, an image that was previously displayed on the second basic screen 705 or the like.

[0173] Second image selection portion 760 includes buttons 761, 762, 763, 764, and 765 which are input elements for causing image processing portion 800 to select an image to be displayed when the user positions a cursor with a mouse or a touch pad and clicks or the like.

[0174] The second captured image display button 761 is an input element for displaying a captured image in the image display section 800 by clicking the second captured image display button 761. When the second captured image display button 761 is clicked, the output control section 52 causes the image display section 800 to display the captured image.

[0175] The second detection result image display button 762 is an input element for displaying the detection result image Gd in the image display unit 800 by clicking the second detection result image display button 762. When the second detection result image display button 762 is clicked, the output control unit 52 causes the image display unit 800 to display the detection result image Gd.

[0176] The second intersection removal result image display button 763 is an input element for displaying the intersection removal result image Gk in the image display unit 800 by clicking the second intersection removal result image display button 763. When the second intersection removal result image display button 763 is clicked, the output control unit 52 causes the image display unit 800 to display the intersection removal result image Gk.

[0177] The allocation result image display button 764 is an input element for displaying the above-mentioned allocation result image Gr in the image display unit 800 by clicking the allocation result image display button 764. When the allocation result image display button 764 is clicked, the output control unit 52 displays the allocation result image Gr in the image display unit 800. In the allocation result image Gr in Fig. 29, neurites of the same group are shown with the same hue, etc., as diagrammed by dashed lines.

[0178] The third confirm button 765 is an input element for transitioning the display screen to the second basic screen 705 by clicking the third confirm button 765. When the third confirm button 765 is clicked, the output control unit 52 transitions the display screen to the second basic screen 705, and then displays the image that was displayed in the image display unit 800 of the second display setting screen 706 in the image display unit 800 of the second basic screen 705 as is.

[0179] FIG. 30 is a diagram showing an example of the allocation parameter selection screen 707. The allocation parameter selection screen 707 includes an image display section 800, a second parameter input section 770, and a video operation section 810. The second parameter input section 770 includes a first text box 771, a second text box 772, a third text box 773, a fourth text box 774, a fifth text box 775, a sixth text box 776, a seventh text box 777, an eighth text box 778, and a fourth confirmation button 779. The output control section 52 can display, in the image display section 800 of the allocation parameter selection screen 707, an image that has been displayed in the image display section 800 of the second basic screen 705 up until that point. In the example of FIG. 30, a fluorescence image Gf is shown.

[0180] A first text box 771, a second text box 772, a third text box 773, a fourth text box 774, a fifth text box 775, a sixth text box 776, a seventh text box 777 and an eighth text box 778 are input elements for inputting allocation parameters c0, c1, c2, c3, c4, c5, D and f, respectively. A user can input numerical values ​​into each of the text boxes 771-778 of the second parameter input section 770 using a keyboard or the like. Characters indicating the type of the allocation parameter are displayed above each of the text boxes 771-778 of the second parameter input section 770. Any input element other than a text box can be placed in the parameter input section 731 as long as it can input an allocation parameter.

[0181] The fourth confirm button 779 is an input element for transitioning the display screen to the second basic screen 705 by clicking the fourth confirm button 779. When the fourth confirm button 779 is clicked, the data processing unit 51 sets the numerical values ​​input in each of the text boxes 771 to 778 as allocation parameters. In addition, the output control unit 52 transitions the display screen to the second basic screen 705.

[0182] 31 is a diagram showing an example of the second result display screen 708. The second result display screen 708 includes an image display unit 800, a second judgment input unit 780, and a video operation unit 810. The second judgment input unit 780 includes a second end button 781, a second re-execution button 782, and a third re-execution button 783. The output control unit 52 displays an allocation result image Gr obtained by the allocation process in the image display unit 800 of the second result display screen 708. The user can input whether to end the allocation process or to perform the intersection removal process or the allocation process again by looking at the allocation result image Gr.

[0183] The second determination input section 780 includes buttons 781, 782, and 783 which are input elements for causing the image processing device 1 to perform a predetermined operation when the user positions the cursor with a mouse or a touch pad and clicks or the like.

[0184] The second end button 781 is a button for ending the allocation process by clicking the second end button 781. When the second end button 781 is clicked, the data processing unit 52 ends the process as appropriate.

[0185] The second re-execution button 782 is a button for performing the intersection removal process again by clicking the second re-execution button 782. When the second re-execution button 782 is clicked, the output control unit 52 displays the first basic screen 701 on the output unit 44.

[0186] The third retry button 782 is a button for performing the allocation process again by clicking the third retry button 783. When the third retry button 783 is clicked, the output control unit 52 displays the second basic screen 705 on the output unit 44.

[0187] In the image processing method of this modified example, the user can adjust the parameters for the intersection removal process and the allocation process while checking the results of these processes. This makes it possible to accurately analyze how the pixel group such as the fragment S corresponds to the linear portion for each specific captured image. In addition, by the above-mentioned screen transition, the parameters can be input while viewing the images showing the results of the intersection removal process and the allocation process as necessary, so that the above analysis can be performed efficiently.

[0188] The present invention is not limited to the above-described embodiment, and other aspects that are conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

[0189] The disclosures of the following priority applications are incorporated herein by reference: Japan Patent Application No. 2019-164505 (filed September 10, 2019) [Explanation of symbols]

[0190] 1...image processing device, 9...network, 10...culture chamber, 20...imaging section, 40...information processing section, 43...storage section, 44...output section, 50...control section, 51, 51a...data processing section, 100...culture section, 300...quality evaluation system, 310...information processing device, 320...terminal device, 330, 331, 332, 333...gateway terminal, 340, 341, 342, 343...observation device, 511...detection result image generation section, 512...pixel group setting section, 513...group Setting unit, 514, 514a...calculation unit, 600, 600a...decision unit, 601...execution unit, 602...termination judgment unit, 700...image generation unit, 701...first basic screen, 702...first display setting screen, 703...intersection detection parameter selection screen, 704...first result display screen, 705...second basic screen, 706...second display setting screen, 707...assignment parameter selection screen, 708...second result display screen, C1...centre of gravity of fragment, C2...centre of rectangle, Ce...cell, D ...Distance between fragments, E, E12, E34, E57, E58, E67, E68...Edges, N5, N6, N7, N8, N301, N302, N303, N304...Nodes, G, G1, G2...Weighted connected graph, Gd...Detection result image, Gp...Probability distribution image, L1...First line segment, L2...Second line segment, L3...Third line segment, Nr...Neural process, Px, Px1, Px2...Pixels, R, R1, R2, R3, R4...Search range, Rt...Rectangle, S, Sa, Sb, Sc, S1, S2, S3, S4, S5, S6, S7, S8, S101, S102, S103, S201, S202, S203, S301, S302, S303, S304...fragments, Sg1, Sg2...pixel group, SS1, SS2, SS3, SS4, SS5, SS6, SS7...pixel fragments, So, So1, So2...cell body, T, T1, T2, T11, T12, T21...edge pixel, w...distance between fragments, θ1...first angle, θ2...second angle, θ3...third angle.

Claims

1. inputting image data of an object; a calculation step of calculating a degree of connectivity between a first pixel group and a second pixel group that are not connected to each other among a plurality of pixel groups corresponding to a plurality of the objects including linear portions, based on the image data of the object; the connectivity degree is an index representing a possibility that the first pixel group and the second pixel group correspond to the same linear portion of the object, the second pixel group is a pixel group including pixels within a predetermined range from the first pixel group, the calculation step includes either a first calculation step of calculating a first distance between a pixel included in the first pixel group and a pixel included in the second pixel group, or a second calculation step of calculating a first angle between a first line segment set to pass through at least two pixels included in the first pixel group and a second line segment set to pass through at least two pixels included in the second pixel group, and calculates the degree of connectivity using the first distance or the first angle. Image processing methods.

2. 2. The image processing method according to claim 1, the second calculation step calculates at least one of a second angle between the first line segment and a third line segment passing through a point included in the first pixel group and a point included in the second pixel group, and a third angle between the third line segment and the second line segment; the calculating step calculates the connectivity degree based on at least one of the first angle, the second angle, and the third angle, or a combination of the angle and the first distance. Image processing methods.

3. 2. The image processing method according to claim 1, further comprising: a pair setting step of setting a pair of the first pixel group and the second pixel group that are not connected to each other based on the image data for each pixel group constituting the plurality of pixel groups; a determination step of determining grouping of the plurality of pixel groups based on the degree of connectivity calculated in the calculation step for each of the groups set in the group setting step; and an output step of outputting the objects determined to be in the same group as a result of the grouping in the determination step as information or an output image indicating the same group. Image processing methods.

4. 2. The image processing method according to claim 1, further comprising: the calculation step includes comparing the first distance with a predetermined distance, and changing a formula for calculating the degree of connectivity in accordance with a comparison result. Image processing methods.

5. 4. The image processing method according to claim 3, the pair setting step sets the pair of the first pixel group and the second pixel group including a pixel located at an end of the first pixel group, a center of gravity of the first pixel group, or a pixel located within the predetermined range set corresponding to a center of a rectangle circumscribing the first pixel group, among the plurality of pixel groups; Image processing methods.

6. 6. The image processing method according to claim 1, The image represented by the image data is an image representing a plurality of pixels corresponding to neurites or blood vessels, the connectivity is a numerical value indicating a possibility that the first pixel group and the second pixel group correspond to a part of the neurite extending from the same nerve cell or to a part of the same blood vessel; Image processing methods.

7. An input unit for inputting image data of an object; a calculation unit that calculates a degree of connectivity between a first pixel group and a second pixel group that are not connected to each other among a plurality of pixel groups corresponding to a plurality of the objects including linear portions, based on the image data of the object, the connectivity degree is an index indicating a possibility that the first pixel group and the second pixel group correspond to the same linear portion of the object, the second pixel group is a pixel group including pixels within a predetermined range from the first pixel group, the calculation unit includes either a first calculation unit that calculates a first distance between a pixel included in the first pixel group and a pixel included in the second pixel group, or a second calculation unit that calculates a first angle between a first line segment that is set to pass through at least two pixels included in the first pixel group and a second line segment that is set to pass through at least two pixels included in the second pixel group, and calculates the degree of connectivity using the first distance or the first angle. Image processing device.

8. 8. The image processing device according to claim 7, the second calculation unit calculates at least one of a second angle between the first line segment and a third line segment passing through a point included in the first pixel group and a point included in the second pixel group, and a third angle between the third line segment and the second line segment; the calculation unit calculates the degree of connection based on at least one of the first angle, the second angle, and the third angle, or a combination of the angle and the first distance. Image processing device.

9. A program for causing an image processing device to execute the following process, An acquisition process for acquiring image data of the object; a calculation process for calculating a degree of connectivity between a first pixel group and a second pixel group that are not connected to each other among a plurality of pixel groups corresponding to a plurality of the objects including linear portions, based on the image data of the object; the connectivity degree is an index indicating a possibility that the first pixel group and the second pixel group correspond to the same linear portion of the object, the second pixel group is a pixel group including pixels within a predetermined range from the first pixel group, the calculation process includes either a first calculation process of calculating a first distance between a pixel included in the first pixel group and a pixel included in the second pixel group, or a second calculation process of calculating a first angle between a first line segment set to pass through at least two pixels included in the first pixel group and a second line segment set to pass through at least two pixels included in the second pixel group, and calculates the degree of connectivity using the first distance or the first angle. program.

10. The program according to claim 9, the second calculation process calculates at least one of a second angle between the first line segment and a third line segment passing through a point included in the first pixel group and a point included in the second pixel group, and a third angle between the third line segment and the second line segment; the calculation process calculates the connectivity degree based on at least one of the first angle, the second angle, and the third angle, or a combination of the angle and the first distance. program.

Images