Image processing device, image processing method, and program

Abstract

The present invention provides an image processing device, an image processing method and a program. The image processing device comprises: a pixel group setting unit that sets a plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other from a plurality of pixels constituting a first image; and a group setting unit that sets a group of the first pixel group and the second pixel group among the plurality of pixel groups in order to set a degree of connection with the first pixel group, wherein the second pixel group includes pixels within a predetermined range from the first pixel group.

Description

Technical Field

[0001] The present invention relates to an image processing device, an image processing method, and a program. Background Art

[0002] A method of performing image processing on an image obtained by capturing an object including a linear portion having branches or the like as appropriate has been proposed (see Patent Document 1). In such image processing, it is necessary to analyze how the extracted pixel group corresponds to the linear portion.

[0003] Patent Document 1: Japanese Patent No. 6015113 Summary of the invention

[0004] According to a first embodiment of the present invention, an image processing device comprises: a pixel group setting unit, which sets a plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other from a plurality of pixels constituting a first image; and a group setting unit, which sets a group of the first pixel group and the second pixel group among the plurality of pixel groups in order to set a degree of connection with the first pixel group, wherein the second pixel group includes pixels that are within a specified range from the first pixel group.

[0005] According to a second embodiment of the present invention, an image processing method includes: obtaining a plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other from a plurality of pixels constituting a first image; and setting a group of the first pixel group and the second pixel group among the plurality of pixel groups in order to set a degree of connection with the first pixel group, wherein the second pixel group includes pixels within a specified range from the first pixel group.

[0006] According to a third embodiment of the present invention, a program is used to cause a processing device to perform the following processing: a pixel group setting processing, obtaining a plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other from a plurality of pixels constituting a first image; and a group setting processing, setting a group of the first pixel group and the second pixel group among the plurality of pixel groups in order to set a degree of connection with the first pixel group, wherein the second pixel group includes pixels within a specified range from the first pixel group. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a conceptual diagram showing the structure of the image processing device according to the first embodiment.

[0008] Figure 2 This is a conceptual diagram showing the configuration of a data processing unit in the first embodiment.

[0009] Figure 3 It is a conceptual diagram showing a probability distribution image in the first embodiment.

[0010] Figure 4 It is a conceptual diagram showing a detection result image in the first embodiment.

[0011] Figure 5 (A) and (B) are conceptual diagrams for explaining the detection of intersection in the first embodiment.

[0012] Figure 6 This is a conceptual diagram for explaining the search range in the first embodiment.

[0013] Figure 7 This is a conceptual diagram for explaining the calculation of the degree of connection in the first embodiment.

[0014] Figure 8 (A) is a conceptual diagram for explaining the grouping method in the first embodiment. Figure 8 (B) corresponds to Figure 8 (A) is a connection diagram.

[0015] Fig. 9 (A) is a conceptual diagram for explaining the grouping method in the first embodiment. Fig. 9 (B) corresponds to Fig. 9 (A) is a connection diagram.

[0016] Fig.10 (A) is a conceptual diagram for explaining the grouping method in the first embodiment. Fig.10 (B) corresponds to Fig.10 The connection diagram of (A) Fig.10 (C) is the weighted connected graph after grouping.

[0017] Fig.11 This is a flowchart showing the flow of the image processing method according to the first embodiment.

[0018] Fig.12 This is a flowchart showing the flow of the image processing method according to the first embodiment.

[0019] Fig.13 This is a flowchart showing the flow of the image processing method according to the first embodiment.

[0020] Fig.14 (A) and Fig.14 (B) is a conceptual diagram for explaining the setting of the pixel group in Modification Example 1.

[0021] Fig.15 (A) and Fig.15 (B) is a conceptual diagram for explaining the setting of the search range in Modification Example 2.

[0022] Fig.16 This is a graph showing the relationship between the distance between pixel groups and the degree of connection in Modification Example 3.

[0023] Fig.17 (A) is a conceptual diagram for explaining the setting of the degree of connection when crossing in Modification Example 4. Fig.17 (B) corresponds to Fig.17 (A) is a connection diagram.

[0024] Fig.18 This is a conceptual diagram showing the structure of the data processing unit of Modification Example 7.

[0025] Fig.19 This is a conceptual diagram for explaining the provision of the program in Modification Example 8.

[0026] Fig. 20 This is a conceptual diagram showing the structure of a quality evaluation system according to Modification Example 9.

[0027] Fig.21 (A) is a conceptual diagram showing the structure of the information processing device of Modification Example 9, Fig.21 (B) is a conceptual diagram showing the structure of the terminal device of variant example 9.

[0028] Fig. 22 This is a flowchart showing the flow of the image processing method according to Modification 10.

[0029] Fig.23 This is a flowchart showing the flow of the image processing method according to Modification 10.

[0030] Fig.24 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0031] Fig.25 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0032] Fig.26 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0033] Fig. 27 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0034] Fig.28 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0035] Fig.29 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0036] Fig.30 This is a conceptual diagram showing an example of a display screen in Modification 10.

[0037] Fig.31 This is a conceptual diagram showing an example of a display screen in Modification 10. DETAILED DESCRIPTION

[0038] Hereinafter, a mode for carrying out the present invention will be described with reference to the drawings.

[0039] - First Embodiment -

[0040] Figure 1 1 is a conceptual diagram showing the structure of the image processing device of the present embodiment. The image processing device 1 includes a culture unit 100 and an information processing unit 40. The culture unit 100 includes a culture chamber (culture chamber) 10, an observation sample stage (workbench) 11, a drive unit 12, and a photographing unit 20. The information processing unit 40 includes an input unit 41, a communication unit 42, a storage unit 43, an output unit 44, and a control unit 50. The control unit 50 includes a data processing unit 51, an output control unit 52, and a device control unit 53.

[0041] The image processing device 1 is configured as a culture device, and has a configuration in which image data obtained by imaging a sample (eg, cells) in the culture unit 100 , namely, captured image data, is input to a data processing unit 51 and processed.

[0042] In this embodiment, a linear portion including a straight line, a curve, or a cross is extracted from the captured image corresponding to the captured image data as an extraction object. In the following, an example is used in which the cell Ce is a nerve cell and a portion corresponding to the neurite Nr is extracted for explanation, but the extraction object is not limited to this example and can be any element constituting the image as long as it includes the above-mentioned linear portion.

[0043] Furthermore, extraction may be performed from images other than captured images, such as drawn images.

[0044] The culture unit 100 includes a culture chamber 10 for culturing cells Ce, and images of the cells Ce cultured in the culture chamber 10 are captured.

[0045] The culture chamber 10 stores the culture container C in which the cells Ce are cultured. The culture container C is, for example, a well plate or a culture dish. Inside the culture chamber 10, a temperature regulator (e.g., a heater) and a temperature sensor not shown in the figure and controlled by the control unit 50 are arranged to control the culture of cells in a predetermined environment by maintaining a temperature preset by the temperature regulator and the temperature sensor. The drive unit 12 has an actuator to move the culture container C at a predetermined time and place it on the observation sample stage 11 inside the culture chamber 10. In addition, in order to photograph the cells Ce, the drive unit 12 moves the photographing unit 20 or the observation sample stage 11 to an appropriate position (e.g., an observation position) so that the cells Ce are arranged on the focal plane of the photographing unit 20.

[0046] The imaging unit 20 includes an imaging device including an imaging element such as a CMOS or a CCD, and images the cell Ce, particularly the neurite Nr of the cell Ce. The imaging method of the imaging unit 20 is not particularly limited as long as the pixel corresponding to the neurite Nr in the captured image including the cell Ce can be distinguished from other parts with a desired accuracy by the brightness value of the pixel or a plurality of pixels around the pixel. For example, the imaging method of the imaging unit 20 can use a fluorescence observation method, a phase difference observation method, etc.

[0047] When the photographing unit 20 is photographed by fluorescence observation, fluorescent proteins such as GFP can be found in the cells Ce by gene transfer, or proteins formed by fusion of proteins located in the neurites Nr with fluorescent proteins can be found for fluorescent staining. If there is no problem with the use of the photographed cells Ce (e.g., culture, passage, picking), other marking methods such as immunostaining can also be performed.

[0048] The pixel signal obtained by the imaging unit 20 when imaging the cell Ce is converted into a digital signal and input to the information processing unit 40 ( Figure 1 ) and stored in the storage unit 43.

[0049] The information processing unit 40 serves as an interface with a user of the image processing apparatus 1 (hereinafter simply referred to as “user”) and also performs processes such as communication, storage, and calculation related to various data.

[0050] In addition, the information processing unit 40 may be configured as an information processing device physically separated from the culture unit 100. In addition, at least a part of the data used by the image processing device 1 may be stored in a remote server or the like.

[0051] The input unit 41 includes an input device such as a mouse, a keyboard, various buttons, or a touch panel. The input unit 41 receives data required for imaging of the culture unit 100 and data processing by the data processing unit 51 from a user.

[0052] The communication unit 42 includes a communication device capable of communicating via wireless or wired connection such as the Internet, and appropriately transmits and receives data related to control and processing in the image processing device 1 .

[0053] The storage unit 43 includes a nonvolatile storage medium, and stores programs for causing the control unit 50 to perform processes, image data related to processes of the data processing unit 51 , and the like.

[0054] The output unit 44 includes a display device such as a liquid crystal monitor, and outputs an image or the like representing information obtained through processing by the data processing unit 51 .

[0055] The control unit 50 is constituted by a processing device such as a CPU, functions as a main body for controlling the operation of the image processing device 1 , and performs various processes by executing a program loaded in the storage unit 43 .

[0056] The data processing unit 51 of the control unit 50 processes the captured image data input from the imaging unit 20 , extracts a plurality of pixel groups corresponding to the extraction target, and groups the plurality of pixel groups based on which cell bodies the extracted plurality of pixel groups are connected to.

[0057] Figure 2 2 is a conceptual diagram showing the structure of the data processing unit 51. The data processing unit 51 includes a detection result image generating unit 511, a pixel group setting unit 512, a group setting unit 513, a calculating unit 514, a determining unit 600, and an image generating unit 700. The determining unit 600 includes an executing unit 601 and an end determining unit 602.

[0058] The detection result image generating unit 511 of the data processing unit 51 performs image processing on the captured image data stored in the storage unit 43, and generates and acquires detection result image data corresponding to the detection result image described later. Hereinafter, an example is described in which the detection result image generating unit 511 generates probability distribution image data corresponding to the probability distribution image described later based on the captured image data, and generates the detection result image data based on the probability distribution image data. However, the present invention is not limited to this example, and the detection result image data may be generated from the captured image data without passing through the probability distribution image data.

[0059] Figure 3 : is a conceptual diagram showing a probability distribution image Gp. In the probability distribution image Gp, the brightness of each pixel Px of the probability distribution image Gp is ​​associated with the probability that the pixel Px corresponds to the neurite Nr to be extracted. Figure 3In FIG. 1 , a cell body So ​​and a pixel corresponding to a neurite Nr extending from the cell body So ​​are shown, but the probability distribution image Gp preferably includes images of multiple cell bodies So and multiple neurites Nr, and this example is also used for the following description. In the probability distribution image Gp, the above-mentioned probability value or the value corresponding to the above-mentioned probability is set for all pixels Px in the image. Figure 3 Such values ​​are also set for pixels other than the pixels corresponding to the neurite Nr shown. In fact, there are many cases where pixels not corresponding to the neurite Nr have positive values ​​indicating the above probability. Figure 3 The pixel Px in FIG. 1 shows such an example. Figure 3 In the probability distribution image Gp, the thicker the hatching, the higher the probability that the pixel Px corresponding to the portion is the image portion corresponding to the neurite Nr. The probability distribution image Gp preferably corresponds the above probability to a one-dimensional brightness value for each pixel Px and is expressed as a grayscale image, but the method of the probability distribution image Gp is ​​not particularly limited as long as the above probability and the brightness value are associated.

[0060] The detection result image generation unit 511 processes the captured image data using a predetermined algorithm (hereinafter referred to as the first algorithm) to calculate the probability of each pixel Px corresponding to the neurite Nr to be extracted. The first algorithm is a machine learning that has been learned. The machine learning is a deep learning that inputs multiple images obtained by photographing nerve cells and images of the parts of the multiple images corresponding to the neurites into a computing device for learning.

[0061] In addition, the first algorithm is not particularly limited as long as it establishes a correspondence between the above-mentioned probability and the brightness value, and machine learning other than deep learning and algorithms other than machine learning may also be used.

[0062] The detection result image generation unit 511 generates detection result image data corresponding to the detection result image indicating whether each pixel Px corresponds to the nerve Nr by processing the probability distribution image data using a predetermined algorithm (hereinafter referred to as a second algorithm).

[0063] Figure 4 is a conceptual diagram showing the detection result image Gd. The detection result image Gd (e.g., the first image) is an image that indicates whether each pixel Px corresponds to a neurite Nr by a brightness value. Figure 4As shown, the detection result image Gd can be used as a binary image, and the brightness value corresponding to the black in the figure indicates that the pixel Px1 corresponds to the neurite Nr, and the brightness value corresponding to the white in the figure indicates that the pixel Px2 does not correspond to the neurite Nr. In addition, in the detection result image Gd, it is preferable to express and judge whether each pixel Px corresponds to the cell body So ​​by image data different from the detection result image data.

[0064] Furthermore, as long as it is possible to indicate whether each pixel Px corresponds to the neurite Nr by the luminance value, the method of expressing the detection result image Gd is not particularly limited.

[0065] Hereinafter, in the detection result image Gd, a pixel group consisting of a plurality of mutually connected pixels corresponding to the extraction object is referred to as a segment S. Figure 4 In the detection result image Gd, two segments Sa and Sb corresponding to one neurite Nr are shown. The segment S preferably has a uniform width, but is not limited thereto, and can form a linear pixel group with an arbitrary width according to the location (the position of the pixel in the detection result image Gd). Figure 4 The line with a width of 1 pixel shown in is appropriately called a thin line. Segment Sa and segment Sb are separated by more than two pixels. As such, in the detection result image Gd, unlike the cell Ce of the subject, the portion corresponding to the neurite Nr is not necessarily connected. One of the reasons is that when shooting, the signal of the pixel corresponding to the linear portion of the neurite Nr, etc. is weak, so it is impossible to shoot with sufficient accuracy. In particular, such a trend can also be seen in the fluorescence microscope that is often used in the shooting of cells, etc. In such a case, if it is not determined which neurite Nr the segment S corresponds to, it is impossible to calculate the exact length, number, etc. of the neurite Nr when using the detection result image Gd to analyze the neurite Nr, and it is impossible to properly obtain information from the neurite Nr in the acquired image.

[0066] When two pixels or two segments S are adjacent only in the oblique direction, there is no particular limitation on whether these pixels or segments S are connected or not, and they can be appropriately set based on the definition of connectivity in the analysis algorithm when analyzing the neurite Nr as described above.

[0067] The second algorithm for generating the detection result image data is not particularly limited as long as the segment S described later remains in the detection result image Gd, and the following algorithm can be used. For example, the second algorithm includes binarizing the brightness value of the probability distribution image Gp based on a specified threshold value, treating pixels with brightness values ​​above the threshold value as pixels corresponding to the neurite Nr, and treating pixels with brightness values ​​less than the threshold value as pixels not corresponding to the neurite Nr. Alternatively, the second algorithm may also include processing such as skeleton extraction for the segment S. The detection result image generation unit 511 can, for example, use the method shown in the document of Lau et al. ("Simultaneously Identifying All True Vessels From Segmented Retinal Images" IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, Vol. 60, No. 7, July 2013) to perform skeleton extraction processing on each segment S.

[0068] After acquiring the detection result image data, the detection result image generating unit 511 stores the detection result image data in a memory connected to the CPU of the control unit 50 or the storage unit 43 (hereinafter referred to as "stored in the storage unit 43, etc.") so that it can be referenced from the above-mentioned CPU.

[0069] In addition, as long as the image processing device 1 can obtain the detection result image data, it is not necessary to have a structure in which the image processing device 1 performs shooting and cultivation. Alternatively, the data processing unit 51 may obtain the detection result image data from the communication unit 42, etc., and store the detection result image in the memory or storage unit 43, etc. so that it can be referenced as described above.

[0070] The pixel group setting unit 512 sets and acquires a plurality of segments S (e.g., a first pixel group and a second pixel group) that are not connected to each other from a plurality of pixels Px constituting the detection result image Gd. At least two segments S set by the pixel group setting unit 512 include segments S that are not connected to each other and are not connected via other pixel groups such as a trifurcation or a cross described below in the detection result image Gd. The pixel group setting unit 512 sets the plurality of segments S after detecting a trifurcation or a cross from the detection result image Gd and removing the trifurcation or the cross. Here, the so-called cross refers to a branch that connects four or more linear pixel groups.

[0071] Figure 5(A) and (B) of FIG. 5 are conceptual diagrams for explaining the process of detecting and removing intersections in a plurality of pixel groups. The pixel group setting unit 512 generates data corresponding to the first partial image Gs1 which is a partial image of the detection result image Gd. The first partial image Gs1 is an image corresponding to a square pixel area centered on the central pixel Pxc, with N being a natural number and with a side of 2N+1 pixels. Figure 5 In (A), the value of N is 5, but the value of N can be appropriately set based on the characteristics of the object or the average value of the width or length of the segment S located in the detection result image Gd.

[0072] The pixel group setting unit 512 generates data corresponding to the second partial image Gs2 based on the first partial image Gs1. The second partial image Gs2 includes a pixel area corresponding to a circle with a radius M centered on the central pixel Pxc as a pixel that does not correspond to the neurite Nr, in other words, is not included in the segment S. The pixel area corresponding to the circle with a radius M is called a removed pixel area Re. M is a positive number less than N. Figure 5 In (B), M is set to 1, and the removed pixel region Re is a 3×3 square pixel region composed of pixels that include at least a portion of a circle with a radius of M. At this time, if the portion of the first partial image Gs1 corresponding to the removed pixel region Re includes a trident or a cross, the segment Sc becomes a plurality of segments S in the second partial image Gs2 by removing the trident or the cross. As a result, the number of segments S in the second partial image Gs2 increases compared to the first partial image Gs1. Figure 5 In (B), the segment Sc is separated into a plurality of segments Sc1, Sc2, Sc3, and Sc4 by removing the pixel region Re.

[0073] When the difference obtained by subtracting the number of segments S in the first partial image Gs1 from the number of segments S in the second partial image Gs2 is 2 or more, the pixel group setting unit 512 regards that an intersection is detected in the removed pixel area Re, and replaces part of the first partial image Gs1 of the detection result image Gd with the second partial image Gs2. The pixel group setting unit 512 moves the center pixel Pxc on the detection result image Gd by one pixel at a time as appropriate, and sequentially generates the first partial image Gs1 and the second partial image Gs2. If an intersection is detected, the pixel group setting unit 512 replaces part of the first partial image Gs1 of the detection result image Gd with the second partial image Gs2 and repeats the process. If the entire detection result image Gd is covered by the first partial image Gs1 or the second partial image Gs2 generated previously, the pixel group setting unit 512 stores data corresponding to the obtained detection result image Gd in the storage unit 43, etc.

[0074] Furthermore, as long as the removal of the trident or cross can be performed, the size and shape of the first partial image Gs1 , the second partial image Gs2 , and the removal pixel region Re are not particularly limited.

[0075] The pixel group setting unit 512 generates data indicating which pixel Px is included in each segment S based on the brightness value of each pixel in the detection result image Gd after removing the tridents or crosses. For example, the pixel group setting unit 512 groups the plurality of pixels Px1 included in the segment S according to the mutually connected pixels. The pixel group setting unit 512 stores the generated data in the storage unit 43 or the like.

[0076] The group setting unit 513 sets a group of a certain segment S among the plurality of segments S set by the pixel group setting unit 512 and a segment S including pixels within the search range R from the segment S. The group set by the group setting unit 513 includes at least one group of segments S that are not connected to each other, and the group is not connected via other pixel groups such as tridents or intersections in the detection result image Gd before removing tridents or intersections. Among the plurality of segments S set by the pixel group setting unit 512, the segment S (segment that serves as a reference for the set search range R) in which the search range R described later is set is set as the first segment, and the other segments (segments that are not references for the set search range R) are set as the second segment.

[0077] Figure 6 It is a conceptual diagram for explaining the setting of the group. The group setting unit 513 extracts the pixels at both ends of the segment S included in the detection result image Gd. Hereinafter, the pixels at both ends of the segment S are referred to as end pixels T. When the segment S is a thin line, the group setting unit 513 sets the pixels at both ends of the thin line corresponding to each segment S as end pixels T, respectively. When the segment S is not a thin line, the group setting unit 513 can, for example, set the pixels close to both ends along the curve or line segment corresponding to the segment S as end pixels T. Figure 6 In the example, pixels at both ends of the first segment S1 are set as end pixels T11 and T12.

[0078] The group setting unit 513 sets the areas (pixel areas) of the predetermined ranges centered on the end pixels T11 and T12 as the search ranges R1 and R2 of the first segment S1. The shape of the search range R is not particularly limited, and can be, for example, a pixel area corresponding to a circle or a rectangle. When the search range R is set to a pixel area corresponding to a circle, the radius of the circle can be, for example, about half of the average of the arithmetic mean of the lengths of the segments S included in the detection result image Gd, but is not particularly limited.

[0079] The group setting unit 513 detects the second segment (in Figure 6In the example of , it is the end pixel T of the second segment (S2, S3, S4). Figure 6 , the end pixel T21 of the second segment S2 included in the search range R1 is detected. The group setting unit 513 sets the group of the first segment S1 and the second segment S2 as a segment pair to be calculated for the degree of connection. On the other hand, since the second segments S3 and S4 do not have pixels included in the search range R, the group of the first segment S1 and the second segment S3 and the group of the first segment S1 and the second segment S4 are not set as segment pairs.

[0080] like Figure 6 As shown, when the search range R is set to a pixel region corresponding to a circle, the distance w (first distance) between the first segment S1 and the second segment S2 set as a group becomes smaller than the radius r of the search range R. Therefore, the group setting unit 513 sets a segment pair based on the distance between the segments S (e.g., the distance between the two end pixels T).

[0081] The group setting unit 513 sets a segment pair for each segment S constituting the plurality of segments S set by the pixel group setting unit 512. Figure 6 In the example of , the group setting unit 513 sets a search range R at an end pixel T for each of the segments S2, S3, and S4, and detects the end pixels T of the other segments S within the search range R. The group setting unit 513 sets segment pairs of the segments S2, S3, and S4 and the segment S including the detected end pixel T. In the case where there is overlap with an already set segment pair, the group setting unit 513 does not need to set the overlapped segment pair. Therefore, since the segment pairs are set by the group setting unit 513, the calculation unit 514 does not perform the calculation process of the degree of connection described later for the group that is not detected based on the search range R. The group setting unit 513 stores the set segment pairs in the storage unit 43, etc.

[0082] The calculation unit 514 calculates the degree of connection between the two segments S included in the segment pair. In the following, the two segments included in the segment pair are referred to as the first segment S1 and the second segment S2 for explanation. In other words, the calculation unit 514 calculates the degree of connection of the second segment S2 relative to the first segment S1. Here, the degree of connection indicates the probability that the two segments S correspond to the same linear portion of the object, in other words, the same neurite Nr, and includes the ease of connection based on the shape and configuration of the two segments S. The degree of connection is represented by a numerical value, etc., and can also be represented by a symbol, etc. as long as the degrees of connection can be compared with each other. The calculation unit 514 stores the calculated degree of connection in the storage unit 43, etc. In addition, as described above, for example, in the case where there is no segment pair set by the group setting unit 513, the calculation unit 514 does not perform the calculation process of the degree of connection.

[0083] Next, Figure 7: is a conceptual diagram for explaining the calculation of the degree of connection. The calculation unit 514 sets the first line segment L1 corresponding to the first segment S1 and the second line segment L2 corresponding to the second segment S2. For example, when the end pixel T1 of the first segment S1 is fixed and the slope of the line segment passing through the end pixel T1 is changed, the calculation unit 514 sets the line segment when the line segment passes through the most pixels included in the first segment S1 as the first line segment L1. In this way, the first segment S1 corresponding to the first pixel group has the first line segment L1. Similarly, when the end pixel T2 of the second segment S2 is fixed and the slope of the line segment passing through the end pixel T2 is changed, the calculation unit 514 sets the line segment when the line segment passes through the most pixels included in the second segment S2 as the second line segment L2. Alternatively, the calculation unit 514 may set the line segment connecting the pixels at both ends of the first segment S1 as the first line segment L1 and the line segment connecting the pixels at both ends of the second segment S2 as the second line segment L2. In this way, the second segment S2 corresponding to the second pixel group has the second line segment L2.

[0084] The calculation unit 514 sets a third line segment L3 that connects the end pixel T1 of the first segment S1 and the end pixel T2 of the second segment S2. The calculation unit 514 calculates the distance between the end pixel T1 and the end pixel T2, that is, the length of the third line segment L3. Hereinafter, this length is set as the distance between the first segment S1 and the second segment S2, that is, the inter-segment distance w.

[0085] In addition, the first line segment L1 may not pass through the end pixel T1. For example, depending on the shape of the first segment S1, a line segment passing through a pixel closer to the center of gravity of the first segment S1 than the end pixel T1 may better reflect the shape of the first segment S1. Therefore, the first line segment L1 may be set to pass through two appropriate pixels included in the first segment S1. The same is true for the second line segment L2. The line segment obtained in this way, which passes through the first line segment L1 corresponding to the first segment S1 and the second line segment L2 corresponding to the second segment S2, may be set as the third line segment L3. In addition, similarly, depending on the shapes of the first segment S1 and the second segment S2, the third line segment L3 may be set as a line segment connecting an arbitrary point included in the first segment S1 and an arbitrary point included in the second segment S2.

[0086] Next, the calculation unit 514 calculates the first angle θ1, which is the angle between the first line segment L1 and the second line segment L2. The calculation unit 514 calculates the second angle θ2, which is the angle between the first line segment L1 and the third line segment L3. The calculation unit 514 calculates the third angle θ3, which is the angle between the second line segment L2 and the third line segment L3. These angles can be calculated using the slopes of the first line segment L1, the second line segment L2, and the third line segment L3.

[0087] The calculation unit 514 calculates the degree of connection DC of the second segment S2 with respect to the first segment S1 by the following equation (1). Hereinafter, “*” represents a product.

[0088] DC=c0*|w|+c1*sinθ2+c2*sinθ3+c3*(1+cosθ1)…(1)

[0089] Here, c0, c1, c2, and c3 are positive constants and can be appropriately set based on the characteristics of the extraction object and the like.

[0090] It should be noted that formula (1) is set so that when an optimization method such as Prim's algorithm is used in the grouping described later, the degree of connectivity DC is positive, so that the degree of connectivity DC becomes the weight of the edge. However, if the degree of connectivity DC uses at least one of the above-mentioned inter-segment distance w, the first angle θ1, the second angle θ2, and the third angle θ3, it can be appropriately set according to the algorithm for grouping described later. In a non-limiting example, the degree of connectivity DC may be only the first term (inter-segment distance w) or only the fourth term (first angle θ1) on the right side of formula (1), or only the first and fourth terms, or only the first, second (second angle θ2), and third (third angle θ3), or only the first, second, and fourth terms, or only the first, third, and fourth terms, or only the second, third, and fourth terms.

[0091] The determination unit 600 determines the grouping of the plurality of segments S set by the pixel group setting unit 512 based on the degree of connection calculated by the calculation unit 514. In this grouping, the segments S corresponding to the neurites Nr extending from the same cell body So ​​are set to belong to the same group. At this time, all the segments S may belong to one group, or some of the segments S may not belong to any group corresponding to each predetermined cell body So.

[0092] The algorithm for grouping by the decision unit 600 is not particularly limited as long as the calculated degree of connectivity is used. Hereinafter, an example of grouping by using the Prim algorithm to find a minimum spanning tree of a weighted connected graph having each segment S as a node and the degree of connectivity as the weight of the edge connecting the nodes will be described.

[0093] Alternatively, the minimum spanning tree may be obtained using Kruskal's method or Boruvka's method.

[0094] The execution unit 601 of the decision unit 600 executes the grouping algorithm to obtain the minimum spanning tree. The execution unit 601 uses each segment S as a node and the degree of connection between two segments S of a segment pair as the weight of the edge between two nodes corresponding to the two segments S to generate data corresponding to the weighted connected graph.

[0095] Figure 8 (A) is a conceptual diagram for explaining the initial stage of grouping. Figure 8 (B) is related to Figure 8 The weighted connected graph G corresponding to (A) is obtained. The weighted connected graphs described in the following figures are examples and do not limit the present invention. Segment S5 is connected to cell body So1. Segment S6 is connected to cell body So2. The execution unit 601 uses nodes N5 and N6 corresponding to segments S5 and S6 connected to each cell body So ​​as root nodes, and uses segments S other than the segment S corresponding to the root node as nodes. The execution unit 601 sets edges between the extracted nodes.

[0096] exist Figure 8 In (B), nodes N5, N6, N7, and N8 corresponding to each of the segments S5, S6, S7, and S8 are set. Segments S5 and S7, S5 and S8, S6 and S7, and S6 and S8 are segment pairs, and edges E57, E58, E67, and E68 are set with the degree of connection as the weight. Figure 8 In the example (B), the weight of edge E57 is set to 20, the weight of edge E58 is set to 15, the weight of edge 67 is set to 10, and the weight of edge 68 is set to 30. The virtual node NV is a node that is virtually set to form a connected graph as a whole. A virtual edge EV1 is set between the virtual node NV and the root node N5, and a virtual edge EV2 is set between the virtual node NV and the root node N6.

[0097] The execution unit 601 sets the root node as a determined node and sets nodes other than the root node as undetermined nodes. Segments S5 and S6 and nodes N5 and N6 corresponding to segments S5 and S6, respectively, correspond to segments connected to different cell bodies So, and therefore belong to different groups and are indicated by different hatching lines. Nodes N7 and N8 corresponding to segments S7 and S8, respectively, are undetermined nodes.

[0098] The execution unit 601 repeatedly sets the undecided node connected to the edge with the smallest weight among the edges connecting the determined node and the undecided node as the determined node. Figure 8 In stages (A) and 8 (B), the execution unit 601 selects the edge E67 with the smallest weight from the edges E57, E58, E67 and E68 connecting the determined node N5 or N6 and the undetermined node N7 or N8, and sets the undetermined node N7 connected to the edge E67 as the determined node.

[0099] Fig. 9(A) is a conceptual diagram for explaining the stage after the node N7 is set as the decision node. Fig. 9 (B) is related to Fig. 9 After setting the node N7 as the decision node, a weighted connected graph G including an edge E78 between the node N7 and the node N8 is created. Fig. 9 In the example of (B), the weight of edge E78 is 50. In this way, each time a decision node is set, a weighted connected graph including edges connecting the newly set decision node and the undecided node is created. The execution unit 601 sets the decision node N6 and the decision node N7 connected by the selected edge E67 to belong to the same group. Fig. 9 In (A) and (B) of 9, the same hatching is used to indicate that the decision nodes N6 and N7, and the segments S6 and S7 corresponding to the decision nodes N6 and N7, respectively, belong to the same group. Next, the execution unit 601 selects the edge E58 with the smallest weight from the edges E58, E68, and E78 connecting the decision node N5, N6, or N7 and the undecided node N8, and sets the undecided node N8 connected to the edge E58 as the decision node.

[0100] Fig.10 (A) is a conceptual diagram for explaining the stage after the node N8 is set as the decision node. Fig.10 (B) is related to Fig.10 The execution unit 601 sets the decision node N5 and the decision node N8 connected by the selected edge E58 to belong to the same group. Fig.10 In (A) of FIG. 10 and (B) of FIG. 10 , the same hatching indicates that the decision nodes N5 and N8 and the segments S5 and S8 corresponding to the decision nodes N5 and N8 respectively belong to the same group.

[0101] exist Fig.10 In (C), if there are no undetermined nodes in the weighted connected graph G, the execution unit 601 sets the weighted connected graph G1 including the edges E58 and E67 selected in the previous edge search and the decision node and the virtual node NV as a minimum spanning tree. The weighted connected graph G1 can be regarded as connected because a virtual node NV integrating multiple root nodes N5 and N6 is set. In addition, the execution unit 601 generates data representing the result of grouping each fragment S, that is, grouping data, based on the data corresponding to the weighted connected graph G1. The execution unit 601 stores the grouping data in the storage unit 43, etc. Based on the grouping data, the detection result image Gd is appropriately provided to morphological analysis, etc. through the data processing unit 51, etc., and the calculation of the length of the neurite is performed.

[0102] Back to Figure 2The end judgment unit 602 of the decision unit 600 calculates the sum of the weights of the edges selected by the execution unit 601 (hereinafter referred to as the judgment parameter), and when the judgment parameter is equal to or greater than a predetermined threshold value (hereinafter referred to as the judgment threshold value), the processing of obtaining the minimum spanning tree by the execution unit 601 is terminated. The judgment threshold value can be appropriately set based on the average value of the degree of connection as the weight of the edge, the characteristics of the object, or the number of segments S included in the detection result image Gd. In this way, it is possible to suppress the grouping of segments S that do not need to be grouped, such as segments S that are too separated from other segments S or segments to which the cell body So ​​connected does not exist in the detection result image Gd.

[0103] The image generation unit 700 generates image data corresponding to an output image that represents the result of grouping the segments S in the detection result image Gd (hereinafter referred to as output image data). The method of outputting the image is not particularly limited, and the result of grouping can be represented by an image or text after image processing of the detection result image Gd. When generating the output image after image processing of the detection result image Gd, the segments S can be distinguished and represented by groups. For example, in the output image, the segments S can be made different in at least one of hue, brightness, and saturation by group, for example, the brightness values ​​of RGB, etc. can be made different by group.

[0104] Back to Figure 1 The output control unit 52 of the control unit 50 controls the output unit 44 and outputs the output image.

[0105] The device control unit 53 of the control unit 50 controls each unit of the culture unit 100 (arrow A2) based on input from the input unit 41. The device control unit 53 performs control related to cell culture (for example, control of the culture chamber 10 for managing temperature or humidity, control of the drive unit 12), or causes the imaging unit 20 to perform imaging.

[0106] Fig.11 : is a flowchart showing the flow of the image processing method of this embodiment. In step S101, the detection result image generation unit 511 obtains the captured image data obtained by the shooting of the shooting unit 20. After step S101 is completed, step S103 is started. In step S103, the detection result image generation unit 511 performs image processing on the captured image data and obtains probability distribution image data. After step S103 is completed, step S105 is started.

[0107] In step S105, the detection result image generation unit 511 performs image processing on the probability distribution image data and obtains the detection result image data. After step S105 is completed, step S107 is started. In step S107, the pixel group setting unit 512 sets a plurality of segments S. After step S107 is completed, step S109 is started.

[0108] In step S109, the group setting unit 513 sets the group (segment pair) and sets the connection degree corresponding to each group. After step S109 is completed, step S111 starts. In step S111, the decision unit 600 determines the grouping of each segment S. After step S111 is completed, step S113 is started. In step S113, the output control unit 52 outputs information indicating the result of the grouping. For example, the output control unit 52 outputs information indicating the name of the segments belonging to the same group, labels based on numbers or symbols, etc., or output images indicating the segments belonging to the same group with the same brightness value, etc. After step S113 is completed, the processing ends.

[0109] Fig.12 Yes means Fig.11 Flowchart of the process of step S109 in the flowchart of . After step S107 is completed, step S1091 is started. In step S1091, the group setting unit 513 selects a segment S. After step S1091 is completed, step S1093 is started. In step S1093, the group setting unit 513 sets a search range R for the selected segment S. After step S1093 is completed, step S1095 is started.

[0110] In step S1095, the group setting unit 513 sets the selected segment S and the group of the segment S including the pixels included in the search range R of the selected segment S. After step S1095 is completed, step S1097 is started. In step S1097, the calculation unit 514 calculates the degree of connection, associates the group with the degree of connection, and stores it in the storage unit 43, etc. After step S1097 is completed, step S1099 is started.

[0111] In step S1099, the group setting unit 513 determines whether all the segments S of the detection result image Gd are selected. If all the segments S are selected, the group setting unit 513 makes an affirmative determination in step S1099 and starts step S111. If there are unselected segments S, the group setting unit 513 makes a negative determination in step S1099 and starts step S1091.

[0112] Fig.13 Yes means Fig.11 Flowchart of the process of step S111 in the flowchart of . After step S109 ends, step S1111 starts. In step S1111, the execution unit 601 sets the root node as the decision node. If step S1111 ends, step S1112 starts. In step S1112, the execution unit 601 sets nodes and edges other than the root node. If step S1112 ends, step S1113 starts.

[0113] In step S1113, the execution unit 601 searches for the edge with the smallest weight connecting the decision node and the undecided node, and sets the undecided node connected to the edge obtained by the search as the decision node. After step S1113 is completed, step S1114 is started. In step S1114, the execution unit 601 determines whether there is an undecided node. In the case of an undecided node, the execution unit 601 makes a positive determination on step S1114 and starts step S1115. In the case of no undecided node, the execution unit 601 makes a negative determination on step S1114 and starts step S1116.

[0114] In step S1115, the end determination unit 602 determines whether the determination parameter is less than the determination threshold. When the determination parameter is less than the determination threshold, the end determination unit 602 makes a positive determination on step S1115 and starts step S1113. When the determination parameter is greater than the determination threshold, the end determination unit 602 makes a negative determination on step S1114 and starts step S1116. In step S1116, the execution unit 601 treats the multiple nodes connected to each other by the edges obtained by the search as a group and sets the grouping information. After step S1116 is completed, step S113 is started.

[0115] According to the above-mentioned embodiment, the following effects are obtained.

[0116] (1) The image processing device 1 of this embodiment includes: a pixel group setting unit 512 that obtains a plurality of segments S including a first segment S1 and second segments S2, S3, and S4 that are not connected to each other from a plurality of pixels Px constituting a detection result image Gd; and a group setting unit 513 that sets a group of a first segment S1 and a second segment S2 among the plurality of segments S in order to set the degree of connection with respect to the first segment S1, wherein the second segment S2 includes pixels within a search range R from the first segment S1. Thus, the image processing device 1 can correctly analyze how a pixel group of the extracted segment S including a discontinuous pixel segment corresponds to a linear portion.

[0117] (2) The image processing device 1 of this embodiment includes a calculation unit 514 that calculates the inter-segment distance w or the first angle θ1 between the first segment S1 and the second segment S2, and the calculation unit 514 calculates the degree of connection using the inter-segment distance w or the first angle θ1. Thus, the image processing device 1 can correctly analyze how the two pixel groups correspond to the linear portion based on the positional relationship between the two pixel groups.

[0118] (3) In the image processing device 1 of the present embodiment, the calculation unit 514 can calculate at least one of the second angle θ2 formed between the third line segment L3 passing through the points included in the first segment S1 and the points included in the second segment S2 and the first line segment L1, and the third angle θ3 formed between the third line segment L3 and the second line segment L2, and calculate the degree of connection based on at least one of the second angle θ2 and the third angle θ3. Thus, the image processing device 1 can more accurately analyze how the two pixel groups correspond to the linear portion based on a more detailed positional relationship between the two pixel groups.

[0119] (4) In the image processing device 1 of this embodiment, the calculation unit 514 can calculate at least one of the angles formed between the shortest line segment in the third line segment L3 and the first line segment L1 and the angle formed between the line segment and the second line segment L2, and calculate the degree of connection based on the at least one angle. Thus, the image processing device 1 can further correctly analyze how the two pixel groups correspond to the linear portion based on a more detailed positional relationship between the two pixel groups.

[0120] (5) In the image processing device 1 of this embodiment, the group setting unit 513 sets groups for each of the segments S constituting the plurality of segments S, and the calculation unit 514 includes a determination unit 600 that calculates the degree of connection for each group and determines the grouping of the plurality of segments S based on the calculated degree of connection. Thus, the image processing device 1 can accurately analyze how the extracted pixel groups including the interrupted pixel segments correspond to the linear portions throughout the entire image.

[0121] (6) In the image processing device 1 of this embodiment, the decision unit 600 uses each of the plurality of segments S as a node and the degree of connection as the weight of the edge, and sequentially determines which group the node belongs to based on the minimum spanning tree obtained by Primm's algorithm, Kruskal's method, or Boruvka's method. Thus, the image processing device 1 can efficiently group the pixels into the optimal group.

[0122] (7) In the image processing device 1 of the present embodiment, the decision unit 600 includes an end determination unit 602. When selecting edges using the Prim's algorithm, the Kruskal method, or the Boruvka method, the end determination unit 602 determines whether to end the process of finding the minimum spanning tree based on the sum of the numerical values ​​corresponding to the selected plurality of edges. Thus, the image processing device 1 can suppress grouping of pixel groups that do not need to be grouped.

[0123] (8) The image processing device 1 of this embodiment includes the image generating unit 700 that generates an output image indicating the result of grouping. Thus, the image processing device 1 can provide information on how the extracted pixel group corresponds to the linear portion.

[0124] (9) The pixel group setting unit 512 of the image processing device 1 of this embodiment removes pixels corresponding to a trifurcation or a cross from pixels constituting the detection result image Gd as pixels not included in the multiple segments S before acquiring the multiple segments S. Thus, the image processing device 1 can correctly analyze how the extracted pixel group including the interrupted pixel segments corresponds to the linear portion even in an image including a trifurcation or a cross.

[0125] (10) In the image processing device 1 of the present embodiment, the detection result image Gd is an image obtained by performing image processing on an image obtained by photographing the object, and is a binary image in which each pixel of the detection result image Gd indicates whether or not the pixel corresponds to a linear structure using a pixel value, and the pixel group setting unit 512 acquires a plurality of segments S based on the pixel values. Thus, the image processing device 1 can correctly analyze how the pixel group extracted from the photographed image, including the interrupted pixel segments, corresponds to the linear portion of the object.

[0126] (11) In the image processing device 1 of this embodiment, the detection result image Gd is an image representing a plurality of pixels Px1 corresponding to the neurite Nr, and the degree of connection is a numerical value representing the possibility that the first segment S1 and the second segment S2 correspond to a portion of the neurite extending from the same nerve cell. Thus, the image processing device 1 can correctly analyze how a pixel group extracted from the captured image, including interrupted pixel segments, corresponds to the neurite Nr.

[0127] (12) The image processing device 1 of this embodiment includes: a pixel group setting unit 512 for acquiring the first segment S1 and the second segment S2 from the detection result image Gd; a calculation unit 514 for calculating the second angle θ2 formed by the third line segment L3 passing through the first segment S1 and the second segment S2 and the first line segment L1; and a determination unit 600 for determining whether the pixel group corresponds to the linear region connecting the first segment S1 and the second segment S2 based on the second angle θ2. Thus, the image processing device 1 can correctly analyze how the extracted pixel group corresponds to the linear portion based on the positional relationship between the first segment S1 and the second segment S2. In addition, by using the second angle θ2, the image processing device 1 can also correctly group the first segment S1 and the second segment S2 even when the thin lines constituting the first segment S1 and the second segment S2 are offset in parallel, when one thin line is very short, when the first segment S1 and the second segment S2 are curved segments, and the like.

[0128] (13) In the image processing device 1 of this embodiment, the calculation unit 514 calculates the third angle θ3 formed between the third line segment L3 and the second line segment L2, and the determination unit 600 determines whether the third line segment L3 corresponds to the linear region connecting the first segment S1 and the second segment S2 based on the second angle θ2 and the third angle θ3. Thus, the image processing device 1 can correctly analyze how the extracted pixel group corresponds to the linear portion based on the more detailed positional relationship between the two segments S.

[0129] (14) In the image processing device 1 of this embodiment, the determination unit 600 can determine whether or not the pixel corresponds to the linear region connecting the first segment S1 and the second segment S2 based on the first angle θ1 and the second angle θ2 formed by the first line segment L1 corresponding to the first segment S1 and the second line segment L2 corresponding to the second segment S2. Thus, the image processing device 1 can correctly analyze how the extracted pixel group corresponds to the linear portion based on the more detailed positional relationship between the two segments S.

[0130] (15) In the image processing device 1 of this embodiment, the determination unit 600 can determine whether the pixel corresponds to the linear region connecting the first segment S1 and the second segment S2 based on the first angle θ1. Thus, the image processing device 1 can correctly analyze how the extracted pixel group corresponds to the linear portion based on the direction in which the two segments S are arranged.

[0131] (16) In the image processing device 1 of this embodiment, the determination unit 600 can determine whether or not the pixel group corresponds to the linear region connecting the first segment S1 and the second segment S2 based on the first angle θ1, the second angle θ2, and the third angle θ3. Thus, the image processing device 1 can correctly analyze how the extracted pixel group corresponds to the linear region based on the more detailed positional relationship between the two segments S.

[0132] (17) The imaging device of the present embodiment includes the above-described image processing device 1 and imaging unit 20. Thus, the imaging device can accurately analyze how a pixel group including interrupted pixel segments extracted from a captured image corresponds to a linear portion.

[0133] (18) The above-mentioned image processing device 1 as the culture device of this embodiment includes a culture section 100 for culturing cells. Thus, the culture device can accurately analyze how a pixel group extracted from an image of cultured cells including interrupted pixel segments corresponds to a linear portion.

[0134] (19) The image processing method of this embodiment includes: obtaining a plurality of segments S including a first segment S1 and second segments S2, S3, and S4 that are not connected to each other from a plurality of pixels Px constituting a detection result image Gd; and setting a group of a first segment S1 and a second segment S2 in the plurality of segments S in order to set the degree of connection with respect to the first segment S1, wherein the second segment S2 includes pixels within a search range R from the first segment S1. Thus, the image processing method can correctly analyze how the extracted pixel group including the interrupted pixel segment corresponds to the linear portion.

[0135] (20) The image processing method of this embodiment includes: obtaining a first segment S1 and a second segment S2 from the detection result image Gd; calculating a second angle θ2 formed between a third line segment L3 passing through points included in the first segment S1 and points included in the second segment S2 and the first line segment L1; and determining whether the third line segment L3 corresponds to a linear region connecting the first segment S1 and the second segment S2 based on the second angle θ2. Thus, the image processing method can correctly analyze how the extracted pixel group corresponds to the linear portion based on the positional relationship between the two segments S.

[0136] The following modifications are also within the scope of the present invention and can be combined with the above-described embodiment.

[0137] (Variant 1)

[0138] In the above-described embodiment, the pixel group setting unit 512 may set a pixel group consisting of a plurality of pixel segments that are not connected to each other as one segment S from the plurality of pixels Px included in the detection result image Gd.

[0139] Fig.14 14(A) and 14(B) are diagrams showing examples of a pixel group (eg, a plurality of pixel groups) composed of a plurality of pixel segments. Fig.14 The pixel group Sg1 of (A) includes a pixel segment SS1, a pixel segment SS2, and a pixel segment SS3. The pixel segment SS2 corresponds to, for example, a circular pixel that is difficult to be assigned a thin line. Even when such a pixel segment SS2 exists, the pixel group setting unit 512 can set these segments as one pixel group Sg1 when these pixel segments SS1, SS2, and SS3 are arranged in a line. For example, when the pixel segments are within a prescribed distance from each other and the pixel segment SS2 is present at a distance shorter than a prescribed distance from a line segment corresponding to the pixel segment SS1 or SS3, the pixel segment SS2 can be used as a pixel segment constituting the pixel group Sg1. The prescribed distance can be pre-set so that the pixels constituting the pixel group Sg1 are arranged in a line.

[0140] Fig.14The pixel group Sg2 of (B) includes pixel segments SS4, SS5, SS6, and SS7. Even when the pixel segments are arranged like this, for example, when the pixel segments are within a predetermined distance from each other and each of the pixel segments SS4 to SS7 is a predetermined distance from a line segment corresponding to at least one of the pixel segments SS4 to SS7, the pixel group setting unit 512 can set these pixel segments as pixel segments constituting the pixel group Sg2.

[0141] The data processing unit 51 can process the pixel group Sg1 or Sg2 composed of a plurality of pixel segments as one segment S. For example, the calculation unit 514 can treat the pixels at both ends of the pixel group Sg1 as the end pixels T1 and T2 in the above-mentioned embodiment, and can set the first line segment L1, the second line segment L2, the first angle θ1, the second angle θ2, the third angle θ3, and the inter-segment distance w according to the definition of the above-mentioned embodiment. Therefore, in the above-mentioned embodiment, even if at least a part of the segment S is treated as a pixel group including a plurality of pixel segments, the group setting and grouping can be performed in the same manner.

[0142] In the image processing device of this modified example, the pixel group setting unit 512 obtains a pixel group consisting of a plurality of pixels Px1 connected to each other, or a pixel group Sg1 or Sg2 consisting of a plurality of pixel groups SS1 to SS3 or SS4 to SS7 not connected to each other, from a plurality of pixels Px constituting the detection result image Gd, as pixel groups constituting a plurality of segments S. Thus, in a case where the detection result image Gd includes a large number of pixel segments consisting of a small number of pixels, it is possible to efficiently analyze how the extracted pixel groups correspond to the linear portions.

[0143] (Variant 2)

[0144] The group setting unit 513 may set the search range R in a method different from the above embodiment. In the following modifications 2-1 and 2-2, the vertical and horizontal lengths of the search range may be an average such as the arithmetic mean of the lengths of the line segments corresponding to the segments S included in the detection result image Gd, or a value based on the average.

[0145] (Variation 2-1)

[0146] Fig.15 (A) is a conceptual diagram for explaining a method for setting the search range R3 of this modification. The group setting unit 513 sets a rectangular or circular pixel region centered on the centroid C1 of the pixels of the segment S101 as the search range R3. The group setting unit 513 can set the segment S101 and the segments S102 and S103 having pixels included in the search range R3 as a segment pair.

[0147] (Variant 2-2)

[0148] Fig.15 (B) is a conceptual diagram for explaining a method for setting the search range R4 of this modification. The group setting unit 513 sets a rectangular or circular pixel region centered on the center C2 of the rectangle Rt circumscribing the pixels of the segment S201 as the search range R4. The group setting unit 513 can set the segment S201 and the segments S202 and S203 having pixels included in the search range R4 as a segment pair.

[0149] In the image processing device 1 of the second modification, the group setting unit 513 sets a group of the segment S101 and the segments S102 and S103 including pixels within the search range R3 set corresponding to the centroid of the segment S101, or a group of the segment S201 and the segments S202 and S203 including pixels within the search range R4 set corresponding to the center of the rectangle Rt circumscribing the segment S201. Thus, the image processing device 1 can suppress the position of the search range from being dispersed due to the shape of the segment S101 or S201, and can more accurately analyze how the extracted pixel group corresponds to the linear portion.

[0150] (Variant 3)

[0151] In the above-mentioned embodiment, the calculation unit 514 may also change the calculation method of the degree of connection when the distance in the detection result image between the first segment S1 and the second segment S2 included in the segment pair is longer than a predetermined distance. Hereinafter, the predetermined distance is referred to as the distance threshold DIST. The distance between the first segment S1 and the second segment S2 defined here may be the inter-segment distance w, or may be an arbitrary distance defined based on the first segment S1 and the second segment S2, or may be defined according to the number of pixels corresponding to the arbitrary distance in the detection result image.

[0152] When the distance between the first segment S1 and the second segment S2 is smaller than the distance threshold DIST, the group setting unit 513 sets the degree of connection DC of the second segment S2 with respect to the first segment S1 according to the above-mentioned equation (1).

[0153] DC=c0*|w|+c1*sinθ2+c2*sinθ3+c3*(1+cosθ1)…(1)

[0154] When the distance between the first segment S1 and the second segment S2 is greater than or equal to DIST, the group setting unit 513 sets the degree of connection DC of the second segment S2 with respect to the first segment S1 according to the following equation (2).

[0155] DC=c0*|w|+c1+c2+2*c3…(2)

[0156] Fig.16 : is a graph showing the relationship between the distance between the first segment S1 and the second segment S2 (horizontal axis) and the degree of connection of the second segment S2 to the first segment S1 (vertical axis) in this modification. Fig.16 In FIG. 1 , the area V1 corresponding to the degree of connection calculated by the above-mentioned formula (1) and the area V2 corresponding to the degree of connection calculated by the above-mentioned formula (2) are shown by hatching. When the case where the distance between the first segment S1 and the second segment S2 is less than the distance threshold DIST and the case where the distance between the first segment S1 and the second segment S2 is greater than the distance threshold DIST are compared, there is no overlap in the value of the degree of connection except for the boundary value DC1=c0*DIST+c1+c2+2*c3. Therefore, the degree of connection when the distance between the first segment S1 and the second segment S2 is greater than the distance threshold DIST is a value greater than the degree of connection when the distance between the first segment S1 and the second segment S2 is less than the distance threshold DIST. Therefore, in the former case, the possibility that the first segment S1 and the second segment S2 belong to different groups can be increased compared with the latter case.

[0157] In addition, when there are pixels included in other segments S between the first segment S1 and the second segment S2 included in the segment pair, the calculation unit 514 may also change the calculation method of the degree of connection. When there are no pixels included in other segments S between the first segment S1 and the second segment S2, for example, on the third line segment L3, the group setting unit 513 sets the degree of connection DC of the second segment S2 with respect to the first segment S1 by the above-mentioned formula (1). When there are pixels included in other segments S between the first segment S1 and the second segment S2, the group setting unit 513 sets the degree of connection DC of the second segment S2 with respect to the first segment S1 by the above-mentioned formula (2). Thus, when there are pixels included in other segments S between the first segment S1 and the second segment S2, the possibility that the first segment S1 and the second segment S2 belong to different groups can be increased.

[0158] In the image processing device of this modified example, when the first condition that there is another pixel group acquired by the pixel group setting unit 512 between the first segment S1 and the second segment S2 is satisfied, or when the distance or the number of pixels between the first segment S1 and the second segment S2 satisfies the second condition defined based on the distance threshold DIST, the calculation unit 514 calculates the degree of connection using a method different from the method used when both the first condition and the second condition are not satisfied. Thus, the image processing device 1 can more accurately analyze how the extracted pixel group corresponds to the linear portion in accordance with the arrangement of the two pixel groups.

[0159] (Variant 4)

[0160] In the above-described embodiment, when the pixel group setting unit 512 detects an intersection, the calculation unit 514 can set the degree of connection so that the segments S facing each other across the intersection belong to the same group.

[0161] Fig.17 (A) and (B) are conceptual diagrams for explaining the setting of the degree of connection in this modification example. Fig.17 (A) is a diagram showing four segments S301, S302, S303, and S304 connected to form a cross after the cross is detected and removed. After the cross is removed, the pixel group setting unit 512 stores information indicating that the segments S301, S302, S303, and S304 are segments connected to form the removed cross in the storage unit 43, etc. For example, the pixel group setting unit 512 includes a cross pixel group setting unit that sets the segments connected to form a cross as one cross pixel group, and the group setting unit 513 and the calculation unit 514, etc. may also treat the cross pixel group as one segment S relative to other segments S. The calculation unit 514 sets the degree of connection after referring to the information so that the probability of connecting the two opposite segments S301 and S302 and the probability of connecting the two opposite segments S303 and S304 are smaller than the probability of connecting the other two segments that are connected to form a cross.

[0162] Fig.17 (B) is a weighted connected graph G2 corresponding to the segments S301 to 304. The nodes corresponding to the segments S301, S302, S303, and S304 are set to N301, N302, N303, and N304, respectively. The calculation unit 514 sets the weights of the edge connecting the nodes N301 and N302, and the edge connecting the nodes N303 and N304 to 0. The calculation unit 514 sets the weights of the other edges E between the connecting nodes N301 to N304 other than these two edges to the maximum (MAXCOST). Before calculating the degree of connection related to other segments S, the calculation unit 514 calculates which segment S is opposite to the four areas corresponding to the intersection, and sets the degree of connection to increase the probability of connecting the two segments S that are determined to be opposite. By performing grouping after setting such a degree of connection, the two segments S that are determined to be opposite during grouping can be reliably connected.

[0163] Furthermore, as long as two opposing segments S can be easily grouped into the same group, the specific value of the edge weight is not particularly limited. In addition, the intersection pixel group setting unit may set a segment including pixels within a certain range from any of the segments S301 to S304 as a segment constituting the intersection pixel group.

[0164] In the image processing device of this modification, the calculation unit 514 calculates the degree of connection in such a manner that the group of segments S facing each other across the intersection generated by removing the pixels corresponding to the intersection corresponds to a part of the connected linear region. Thus, the image processing device can more accurately analyze how the extracted pixel group including the intersection corresponds to the linear portion.

[0165] In the image processing device of this modification, the pixel group setting unit 512 includes an intersection pixel group setting unit, and when there are three other pixel groups S302, S303, and S304 within a certain range from the segment S301, the intersection pixel group setting unit collects four segments S including the segment S301 and the three pixel groups S302, S303, and S304 as an intersection pixel group, and the calculation unit 514 can calculate the degree of connection in a manner corresponding to a part of a linear region connecting the groups of segments S facing each other across the intersection. Thus, the image processing device can more accurately analyze how the extracted pixel group including the intersection corresponds to the linear portion.

[0166] (Variant 5)

[0167] In the above-described embodiment, the calculation unit 514 can further calculate the degree of connection based on the characteristics of at least one segment S as shown in the following Modifications 5-1 to 5-4.

[0168] (Variant 5-1)

[0169] The calculation unit 514 may also calculate the degree of connection based on an average value such as the arithmetic average of the brightness of the pixels included in each of the first segment S1 and the second segment S2. In this case, the brightness may be the brightness of the captured image or the probability distribution image Gp in addition to the detection result image Gd. The smaller the difference between the average brightness of the pixels of the first segment S1 and the average brightness of the pixels of the second segment S2 is, the smaller the degree of connection is, and the first segment S1 and the second segment S2 are likely to become the same group.

[0170] (Variant 5-2)

[0171] The calculation unit 514 may also calculate the degree of connection based on the width of the first segment S1 and the width of the second segment S2. The width in this case may be set, for example, based on the number of pixels through which a straight line perpendicular to the line segments corresponding to the first segment S1 and the second segment S2 passes. The calculation unit 514 may set the degree of connection to be smaller as the difference between the width of the first segment S1 and the width of the second segment S2 is smaller, so that the first segment S1 and the second segment S2 are more likely to become the same group.

[0172] (Variant 5-3)

[0173] The calculation unit 514 may calculate the degree of connection based on the curvature of the curve corresponding to the first segment S1 and the curvature of the curve corresponding to the second segment S2. In this case, the curvatures of the curves calculated for the first segment S1 and the second segment S2 respectively by a known fitting method or the like can be used. The calculation unit 514 may set the degree of connection to be reduced as the difference between the curvature of the curve corresponding to the first segment S1 and the curvature of the curve corresponding to the second segment S2 is smaller, so that the first segment S1 and the second segment S2 are more likely to become the same group.

[0174] (Variant 5-4)

[0175] The calculation unit 514 may calculate the degree of connection based on the length of at least a portion of the straight line portion in the broken line corresponding to the first segment S1 and the length of at least a portion of the straight line portion in the broken line corresponding to the second segment S2. In this case, it is preferable to use the length of the straight line portion closest to the second segment in the first segment S1 and the length of the straight line portion closest to the first segment in the second segment S2. The side with the longer length of the straight line portion can more accurately calculate the first angle θ1 and the like, and can more accurately perform grouping. The broken line can use the broken line calculated for the first segment S1 and the second segment S2 respectively by a known fitting method or the like. The calculation unit 514 can be set so that the smaller the difference between the length of the straight line portion corresponding to the first segment S1 and the length of the straight line portion corresponding to the second segment S2 is, the smaller the degree of connection is, and the first segment S1 and the second segment S2 are more likely to be in the same group.

[0176] In the image processing device of this modified example, the calculation unit 514 can calculate the degree of connection based on at least one of the average value of the brightness of the plurality of pixels Px constituting the first segment S1 and the second segment S2, the width of the first segment S1 and the second segment S2, the curvature of the curve when the first segment S1 and the second segment S2 correspond to the curve, and the length of the line segment constituting the broken line when the first segment S1 or the second segment S2 corresponds to the broken line. Thus, the image processing device can more accurately analyze how the extracted pixel group corresponds to the linear portion based on these characteristics (e.g., width, curvature, etc.).

[0177] (Variant 6)

[0178] In the above embodiment, the subject of the captured image is the cell Ce and the linear portion of the extraction target is the neurite Nr. However, the extraction target is not limited to this example as long as it includes the linear portion, and is preferably a blood vessel, for example.

[0179] Furthermore, it is more preferable to use the object of the captured image as the fundus, and the linear part of the extraction object as the blood vessel in the fundus, and particularly preferably the blood vessel of the retina or the choroidal blood vessel. In the fundus image, the linear part of the extraction object is used as the blood vessel, and the detection result image generation unit 511 generates a probability distribution image Gp that makes the probability of each pixel corresponding to the blood vessel correspond to the brightness value based on the captured image, and generates the detection result image Gd based on the probability distribution image Gp. The generated detection result image Gd is processed as described above by the pixel group setting unit 511, the group setting unit 512, the calculation unit 514 and the determination unit 600, so that the blood vessels connected to each other can be appropriately extracted. In this case, the execution unit 601 can set the type of segmented blood vessels, etc. for a part of the pixels of the detection result image Gd as the root node of the virtual node that is not the minimum spanning tree. Here, the type of blood vessel can include arteries or veins. In the case of grouping the segments S corresponding to the choroidal blood vessels, multiple vortex veins can also be set as root nodes, and the segments S can be grouped based on whether they are connected to the vortex veins corresponding to each root node.

[0180] In the image processing device of this modified example, the detection result image Gd is an image representing a plurality of pixels Px1 corresponding to blood vessels, and the degree of connection is a numerical value representing the possibility that the first segment S1 and the second segment S2 correspond to a portion of the same blood vessel. Thus, the image processing device can correctly analyze how a pixel group extracted from a captured image, including interrupted pixel segments, corresponds to a portion of a blood vessel divided for each root node.

[0181] (Variant 7)

[0182] In the above-mentioned embodiment, the data processing unit may also calculate at least one of the second angle θ2 and the third angle θ3 for the two segments S set by the pixel group setting unit 512, and determine whether the two segments S correspond to the same linear portion in the object based on whether the calculated second angle θ2 and third angle θ3 are within a predetermined range.

[0183] Fig.18: This is a conceptual diagram showing the structure of a data processing unit 51a in the image processing device of this variant. The data processing unit 51a includes a detection result image generating unit 511, a pixel group setting unit 512, a calculating unit 514a, a determining unit 600a, and an image generating unit 700. The calculating unit 514a calculates at least one of the second angle θ2 and the third angle θ3 for two segments S among the segments S set by the pixel group setting unit 512. When at least one of the calculated second angle θ2 and the third angle θ3 is included in the numerical range representing the above-mentioned prescribed range stored in the storage unit 43, etc., the determining unit 600a determines that the above-mentioned two segments S correspond to the same neurite Nr. The above-mentioned prescribed range is appropriately set in such a manner that the two segments S are arranged in a linear shape. According to the method of this variant, it is also possible to correctly analyze how the extracted pixel group corresponds to the linear portion.

[0184] (Variant 8)

[0185] It is also possible to record a program for realizing the information processing function of the information processing unit 40 of the above-mentioned embodiment in a computer-readable recording medium, so that the computer system reads and executes the program recorded in the recording medium, which is related to the processing performed by the data processing unit 51, such as the setting of the above-mentioned segment S, segment pairs, and connection degree. In addition, the "computer system" mentioned here refers to the hardware structure including OS (Operating System) and peripheral devices. In addition, the so-called "computer-readable recording medium" refers to a portable recording medium such as a floppy disk, a magneto-optical disk, an optical disk, a memory card, and a storage device such as a hard disk built into the computer system. And the so-called "computer-readable recording medium" may also include a structure that dynamically maintains the program for a short time, such as a communication line in the case of sending a program via a network such as the Internet or a communication line such as a telephone line, and a structure that maintains the program for a certain period of time, such as a volatile memory inside a computer system that becomes a server or client in this case. In addition, the above-mentioned program may be a structure for realizing a part of the above-mentioned function, and further, it may be a structure that realizes the above-mentioned function by combining with a program already recorded in the computer system.

[0186] In addition, when applied to a personal computer (hereinafter referred to as a PC) or the like, the program related to the above-mentioned control can be provided via a recording medium such as a CD-ROM or a data signal via the Internet or the like. Fig.19950 is a diagram showing this situation. PC950 receives the program via CD-ROM953. In addition, PC950 has a connection function with communication line 951. Computer 952 is a server computer that provides the above-mentioned program, and stores the program on a recording medium such as a hard disk. Communication line 951 is a communication line such as the Internet, personal computer communication, or a dedicated communication line. Computer 952 uses a hard disk to read the program and sends the program to PC950 via communication line 951. That is, the program is transmitted as a data signal through a carrier wave and sent via communication line 951. In this way, the program can be provided as a computer-readable computer program product in various forms such as a recording medium and a carrier wave.

[0187] The program for realizing the above-mentioned information processing function includes a program for causing the processing device to perform the following processing: pixel group setting processing, in which a plurality of segments S (corresponding to the pixel group) including a first segment S1 and a second segment S2 which are not connected to each other are set from a plurality of pixels Px constituting the detection result image Gd. Fig.11 and group setting processing, in order to set the degree of connection with the first segment S1, a group of the first segment S1 and the second segment S2 among the plurality of segments S is set, wherein the second segment S2 includes pixels (corresponding to pixels) within a predetermined range from the first segment S1. Fig.12 Thus, it is possible to correctly analyze how the extracted pixel group including the interrupted pixel segments corresponds to the linear portion.

[0188] The program for realizing the above-mentioned information processing function includes a program for causing the processing device to perform the following processing: pixel group setting processing, obtaining the first segment S1 and the second segment S2 (corresponding to the pixel group) which are not connected to each other from the detection result image Gd. Fig.11 107 of the flowchart); calculating a second angle θ2 (corresponding to Fig.12 and determining whether the linear region (corresponding to the linear region connecting the first segment S1 and the second segment S3) corresponds to the linear region connecting the first segment S1 and the second segment S3 based on the second angle θ2. Fig.11 Thus, it is possible to correctly analyze how the extracted pixel group corresponds to the linear portion based on the positional relationship between the first segment S1 and the second segment S2.

[0189] (Variant 9)

[0190] In the above-described embodiment, the processing performed by the data processing unit 51 may be provided by cloud computing via a network.

[0191] Fig. 20 3 is a conceptual diagram showing the structure of a quality evaluation system 300 of this modification. The quality evaluation system 300 includes an information processing device 310, a terminal device 320, and a gateway terminal 330. The quality evaluation system 300 evaluates the quality of the observation object by performing data processing on an image of the observation object of the observation device 340 such as cultured cells. As such an observation object, a structure including a linear portion such as a nerve cell having a neurite is preferred.

[0192] In this modification, information processing device 310, terminal device 320, and gateway terminal 330 are connected to each other so as to be communicable via network 9. Network 9 may be the Internet, a mobile communication network, or a local area network, or may be a combination of these multiple networks.

[0193] The gateway terminal 330 transmits the captured image captured by the observation device 340 to the information processing device 310 via the network 9. Fig. 20 330 are shown in FIG. 330 , but the number of gateway terminals 330 in the quality evaluation system is not particularly limited. The gateway terminals 331 , 332 , and 333 are connected to observation devices 341 , 342 , and 343 by wireless or wired communication. The observation device 340 is managed by a manager of the observation object, such as a cell. Fig. 20 Although the number of the observation devices 340 is three, the number of the observation devices 340 may be any number greater than one.

[0194] The structure of the observation device 340 is not particularly limited as long as it can capture an image of the observation object. The observation device 340 can include, for example, the above-mentioned culture unit 100. The observation device 340 can include a communication device that sends the captured image data obtained by capturing to the gateway terminal 330, or the captured image data can be sent to the gateway terminal 330 by a control PC for the observation device 340 that is not integrally formed with the observation device. The observation device 340 can also include, for example, a detector that detects the temperature, air pressure, illumination of the light source, oxygen concentration, etc. during the culture, and sends the data obtained by the detection to the gateway terminal 330.

[0195] The gateway terminal 330 is typically a general gateway configured to convert protocols or address systems, but is not particularly limited thereto. The gateway terminal 330 may also be a PC or the like having a gateway function. From the perspective of improving the resolution accuracy of the information processing device 310, the gateway terminal 330 is preferably composed of a plurality of gateway terminals 330, but may also be composed of a single gateway terminal. In this case, one gateway terminal 330 is communicatively connected to a plurality of observation devices 340.

[0196] Fig.21 (A) is a conceptual diagram showing the structure of the information processing device 310. The information processing device 310 includes a control unit 311, a storage unit 312, and a communication unit 313. The information processing device 310 of this modification is a network server for performing quality evaluation of the observation object of the observation device 340 based on cloud computing, but is not particularly limited to this. The information processing device 310 can also be composed of any computer such as a PC.

[0197] The control unit 311 is composed of a processing device such as a CPU, and functions as a main body for controlling the operation of the information processing device 310, and performs various processes by executing the program loaded in the storage unit 312. The control unit 311 includes the above-mentioned data processing unit 51, and performs the above-mentioned processing such as setting of the segments S, segment pairs, and connection degree (for example, analysis processing, processing of the first algorithm, and calculation processing of the connection degree) on the captured image data received and acquired from the observation device 340 via the network 9. The control unit 311 transmits information indicating the result of grouping of each segment S in the detection result image Gd obtained by the processing, or output image data of an output image showing the segments S determined to be in the same group in the same color or the same brightness, etc., to the terminal device 320 via the communication unit 313.

[0198] The storage unit 312 of the information processing device 310 includes a nonvolatile storage medium, and stores data and programs used for processing in the information processing device 310, and data obtained by processing by the data processing unit 51. The communication unit 313 of the information processing device 310 includes a communication device capable of communicating via the network 9, and receives captured image data and the like sent from the observation device 340, or sends information and the like generated by the control unit 311 to the terminal device 320.

[0199] Fig.21 (B) is a conceptual diagram showing the structure of the terminal device 320. The terminal device 320 is handled by the quality evaluator of the observation object of the observation device 340. The terminal device 320 can be composed of any computer such as a PC, a smartphone or a tablet. The terminal device 320 includes a communication unit 321, an output unit 322 and an input unit 323.

[0200] The communication unit 321 of the terminal device 320 has a communication device that can communicate via the network 9, and receives information processed by the data processing unit 51 and sent from the information processing device 310. The information can include position information of the segments S belonging to the same group and an output image colored with the same color for the segments belonging to the same group. In addition, in the case where the trifurcation or cross is removed in advance by the detection process of the trifurcation or cross, the information can also include the detection result image before the trifurcation or cross is removed. The output unit 322 of the image terminal device 320 has a display device such as a liquid crystal monitor, and outputs an image representing the information obtained by the processing of the data processing unit 51. The user (for example, a quality evaluator) can evaluate the observation object of the observation device 340 by observing the image displayed on the output unit 322, or performing data analysis on the image. For example, when the observation object is a nerve cell, the length and number of neurites can be quantified in the above-mentioned data analysis. The evaluation of the quality evaluator may be input from an input unit (not shown) of the terminal device 320, and the input data may be sent to the information processing device 310 via the communication unit 321 and stored in the storage unit 312 of the information processing device 310. In addition, the captured image data received by the control unit 311 may be sent to the information processing device 310 by the user using the input unit 323 provided in the terminal device 320.

[0201] (Variant 10)

[0202] In the above-mentioned embodiment, the user can appropriately input or change the parameters of the detection and removal of trigeminals and intersections, the setting of the segments S, the setting of the segment pairs and the degree of connection, and the determination of the groups by viewing the display screen of the output unit 44. Hereinafter, trigeminals and intersections are referred to as intersections, and the detection and removal of intersections are referred to as intersection removal processing. In addition, the setting of the segments S, the setting of the segment pairs and the degree of connection, and the determination of the groups are referred to as allocation processing.

[0203] Fig. 22 is a flowchart showing the flow of the image processing method of this variation. Steps 201 to S205 are related to Fig.11 Steps S101 to S105 of the flowchart are the same, so the description is omitted. When step S205 is completed, step S207 is started.

[0204] In step S207 , the pixel group setting unit 512 removes intersection points in the detection result image Gd.

[0205] Fig.234 is a flowchart showing the process of removing the intersection in step S207. If step S205 is completed, step S2071 is started. In step S2071, the output control unit 52 displays the detection result image Gd on the display screen of the output unit 44. If step S2071 is completed, step S2072 is started. In step S2072, the pixel group setting unit 512 sets the parameters in the intersection removal process. The parameters in the intersection removal process are called intersection detection parameters. The intersection detection parameters can include the parameters indicating the pixel area Re ( Figure 5 (B)), in particular, the number of pixels (width) in the horizontal or vertical direction excluding the pixel area Re.

[0206] The user can observe the captured image or the detection result image Gd, and observe the intersection removal result image to be described later obtained when the intersection removal process is performed again after the intersection removal process is performed, and input the intersection detection parameters via the input unit 41. For example, the user can set the width of the removed pixel area Re to be the same as the width of the segment S in the detection result image Gd, or set the width of the removed pixel area Re to be longer when there are many concavities and convexities in the shape of the segment S in the detection result image Gd. Alternatively, the user can set the width of the removed pixel area Re to be smaller when there are many false detections of intersections in the intersection removal result image, and to be larger when there are many intersections that are not detected in the intersection removal process. The pixel group setting unit 512 sets the intersection detection parameters based on the input via the user's input unit 41. If step S2072 ends, step S2073 starts.

[0207] In step S2073, the pixel group setting unit 512 uses the intersection detection parameters set in step S2072 to perform the above-mentioned intersection removal process in the same manner as in the above-mentioned embodiment. If step S2073 is completed, step S2074 is started. In step S2074, the output control unit 52 displays the detection result image supplied to the intersection removal process on the display screen of the output unit 44. The detection result image supplied to the intersection removal process is called the intersection removal result image. If step S2074 is completed, step S2075 is started.

[0208] In step S2075, the data processing unit 51 determines whether the user has input an instruction to perform the intersection removal process again via the input unit 41. If the instruction has been input, the data processing unit 51 makes an affirmative determination in step S2075 and starts step S2072. If the instruction has not been input, the data processing unit 51 makes a negative determination in step S2075 and starts step S209.

[0209] Back to Fig. 22In step S209, the data processing unit 51 sets the parameters in the allocation process. The parameters in the allocation process are referred to as allocation parameters. The allocation parameters can include the weight c0 of the term including the distance w between segments, the weight c1 of the term including the second angle θ2, the weight c2 of the term including the third angle θ3, and the weight c3 of the term including the first angle θ1 in the formula (1) of the above-mentioned embodiment. In addition, the allocation parameter can be represented by a binary value or the like to indicate whether to set the degree of connection so that the segments S corresponding to the four segments S corresponding to the intersection are connected to each other as in the above-mentioned modification example 4. The variable represented by the binary value is set to f. In addition, the allocation parameter can include any parameter used for the allocation process. For example, as in the above-mentioned modification example 5, when the difference in width of the segment S and the difference in brightness between the segments are used to calculate the degree of connection, the weight c4 of the term including the difference in width of the segment S and the weight c5 of the term including the difference in brightness between the segments in the formula for calculating the degree of connection can be included in the allocation parameter. When the inter-segment distance w is equal to or greater than a predetermined value in the calculation of the degree of connectivity, the calculation unit 514 can eliminate the term including the inter-segment distance w and perform the calculation of the degree of connectivity. The predetermined value D can be included in the allocation parameter.

[0210] The user can observe the captured image, the detection result image Gd, or the intersection removal result image, and observe the obtained allocation result image described later when the allocation process is performed again after the allocation process is performed, and input the allocation parameter through the input unit 41. For example, the user can set the above-mentioned predetermined value D to about twice the value of the intersection detection parameter. When the curvature of the segment S in the detection result image Gd is small, the user can set the values ​​of the weights c1, c2, and c3 to be high, and when the curvature of the segment S is large and the noise is large, the user can reduce the values ​​of the weights c1, c2, and c3, and set the value of the weight c0 to be high. Alternatively, when the deviation of the width of the segment S is large, the user sets the value of the weight c4 of the item including the width of the segment S to be high, and when the deviation is small, the user reduces the value of the weight c4, for example, to 0. The data processing unit 51 sets the allocation parameter based on the input through the user's input unit 41. When step S209 is completed, step S211 is started.

[0211] In step S211, the data processing unit 51 uses the allocation parameters set in step S209 to perform the above allocation processing in the same manner as in the above embodiment. If step S211 is completed, step S213 is started. In step S213, the output control unit 52 displays the above output image obtained by the allocation processing on the display screen of the output unit 44. Hereinafter, the output image is an image in which the segments S are grouped so that at least one of color, brightness and saturation is different, and is referred to as an allocation result image. If step S213 is completed, step S215 is started.

[0212] In step S215, the data processing unit 51 determines whether an instruction to perform reallocation processing is input by the user via the input unit 41. If the instruction is input, the data processing unit 51 makes an affirmative determination in step S215 and starts step S207. If the instruction is not output, the data processing unit 51 makes a negative determination in step S215 and performs the counting process.

[0213] Alternatively, the data processing unit 51 may start step S209 after making an affirmative determination in step S215 .

[0214] In the following, an example of a display screen displayed on the output unit 44 is shown for the above-mentioned intersection removal processing, but the form of the image in each display screen and the shape of buttons and icons are not limited to those shown in the following figures.

[0215] Fig.24 This is an example of a screen displayed on the output unit 44 before the intersection removal process is performed, and this screen is referred to as the first basic screen 701. The first basic screen 701 includes an image display unit 800, a first input element display unit 710, and an animation operation unit 810. The first input element display unit 710 includes a first display setting button 711, an intersection detection parameter selection button 712, and an intersection removal execution button 713. The animation operation unit 810 includes a rewind button 811 for rewinding when the image display unit 800 displays an animation, a play button 812 for playing, a temporary stop button 813 for temporarily stopping, and a fast forward button 814 for fast forwarding. Fig.24 The image display unit 800 of the first basic screen 701 shows the detection result image Gd, but the image displayed on the image display unit 800 can be changed as appropriate.

[0216] The first input element display unit 710 includes buttons 711 , 712 , and 713 which are input elements for causing the image processing device 1 to perform a predetermined operation by the user pointing a cursor with a mouse or a touch pad and clicking.

[0217] The first display setting button 711 is an input element for causing the display screen to transition to a first display setting screen, which is a screen for setting an image displayed on the image display unit 800. When the first display setting button 711 is clicked, the output control unit 52 causes the display screen to transition to the first display setting screen. Fig.25 The first display setting screen 702 is displayed on the output unit 44 .

[0218] The intersection detection parameter selection button 712 is an input element for causing the display screen to transition to the screen for inputting intersection detection parameters, that is, the intersection detection parameter selection screen. When the intersection detection parameter selection button 712 is clicked, the output control unit 52 causes the display screen to transition to the screen for inputting intersection detection parameters, that is, the intersection detection parameter selection screen. Fig.26 The intersection detection parameter selection screen 703 is displayed on the output unit 44.

[0219] The intersection removal execution button 713 is an input element for the pixel group setting unit 512 to execute the intersection removal process and to transfer the display screen to the screen for displaying the result of the intersection removal process, that is, the first result display screen. Fig. 27 The first result display screen 704 is displayed on the output unit 44 .

[0220] Fig.25 702. The first display setting screen 702 includes an image display unit 800, a first image selection unit 720, and an animation operation unit 810. The first image selection unit 720 includes a first captured image display button 721, a first detection result image display button 722, a first intersection removal result image display button 723, and a first confirmation button 724. Before clicking these buttons, the output control unit 52 can display the image that was previously displayed on the first basic screen 701, etc., on the image display unit 800.

[0221] The first image selection unit 710 includes buttons 721 , 722 , 723 , and 724 which are input elements for the user to select an image displayed in the image processing unit 800 by pointing a cursor at the button using a mouse or a touch pad, and clicking the button.

[0222] The first captured image display button 721 is an input element for displaying a captured image on the image display unit 800 by clicking the first captured image display button 721. The following is a configuration for displaying a fluorescent image of a cell Ce captured by a fluorescent microscope as a captured image. When the first captured image display button 721 is clicked, the output control unit 52 displays the captured image on the image display unit 800.

[0223] The first detection result image display button 722 is an input element for displaying the detection result image Gd on the image display unit 800 by clicking the first detection result image display button 722. When the first detection result image display button 722 is clicked, the output control unit 52 displays the detection result image on the image display unit 800.

[0224] The first intersection removal result image display button 723 is an input element for displaying the intersection removal result image Gk on the image display unit 800 by clicking the first intersection removal result image display button 723. When the first intersection removal result image display button 722 is clicked, the output control unit 52 displays the intersection removal result image Gk on the image display unit 800. Fig.25 In the example of FIG. 7 , the intersection result image Gk displayed after the first intersection removal result image display button 722 is clicked is shown.

[0225] The first confirmation button 724 is an input element for causing the display screen to transition to the first basic screen 701 by clicking the first confirmation button 724. When the first confirmation button 724 is clicked, the output control unit 52 causes the display screen to transition to the first basic screen 701, and then displays the image displayed on the image display unit 800 of the first display setting screen 702 on the screen display unit 800 of the first basic screen 701 as it is.

[0226] Fig.26 703. The intersection detection parameter selection screen 703 includes an image display unit 800, a first parameter input unit 730, and an animation operation unit 810. The first parameter input unit 730 includes a text box 731 and a second confirmation button 732. The output control unit 52 can display the image previously displayed on the image display unit 800 of the first basic screen 701 on the image display unit 800 of the intersection detection parameter selection screen 703. Fig.26 In the example of FIG. 1 , a fluorescent image Gf is shown as a captured image of a cell Ce.

[0227] The text box 731 is an input element for inputting an intersection detection parameter. The user can input a numerical value in the text box 731 using a keyboard, etc. Above the text box 731, a character "R" indicating the type of the intersection detection parameter is displayed.

[0228] Furthermore, any input element other than a text box may be arranged in the first parameter input unit 731 as long as the intersection detection parameter can be input.

[0229] The second confirmation button 732 is an input element for causing the display screen to transition to the first basic screen 701 by clicking the second confirmation button 732. When the second confirmation button 732 is clicked, the pixel group setting unit 512 sets the value input to the text box 731 as the intersection detection parameter. In addition, the output control unit 52 causes the display screen to transition to the first basic screen 701.

[0230] Fig. 277 is a diagram showing an example of the first result display screen 704. The first result display screen 704 includes an image display unit 800, a first determination input unit 740, and an animation operation unit 810. The first determination input unit 740 includes a first end button 741 and a first re-execution button 742. The output control unit 52 displays the intersection removal result image Gk obtained by the intersection removal process on the image display unit 800 of the first result display screen 704. The user can observe the intersection removal result image Gk and input whether to end the intersection removal process or to perform the intersection removal process again.

[0231] The first determination input unit 740 includes buttons 741 and 742 which are input elements for causing the image processing apparatus 1 to perform a predetermined operation by the user pointing a cursor with a mouse, a touch pad, or the like and clicking the buttons.

[0232] The first end button 741 is a button for ending the intersection removal process and entering the setting and execution of the allocation process by clicking the first end button 741. When the first end button 741 is clicked, the output control unit 52 displays the screen for setting the allocation process, that is, Fig.28 The second basic screen 705 is displayed on the output unit 44.

[0233] The first re-execution button 742 is a button for performing the intersection point removal process again by clicking the first re-execution button 742. When the first re-execution button 742 is clicked, the output control unit 52 displays the first basic screen 701 on the output unit 44.

[0234] In the following, an example of a display screen displayed on the output unit 44 is shown for the above-mentioned allocation processing, but the form of images and the shapes of buttons and icons in each display screen are not limited to those shown in the following figures.

[0235] Fig.28 705. The second basic screen 705 includes an image display unit 800, a second input element display unit 750, and an animation operation unit 810. The second input element display unit 750 includes a second display setting button 751, an allocation parameter selection button 752, and an allocation execution button 753. Fig.28 The image display unit 800 shows the detection result image Gd, but the image displayed on the image display unit 800 of the second basic screen 705 can be changed as appropriate.

[0236] The second input element display unit 750 includes buttons 751 , 752 , and 753 which are input elements for causing the image processing device 1 to perform a predetermined operation by the user pointing a cursor with a mouse, a touch pad, or the like and clicking the buttons.

[0237] The second display setting button 751 is an input element for causing the display screen to transition to a second display setting screen, which is a screen for setting an image to be displayed on the image display unit 800. When the second display setting button 751 is clicked, the output control unit 52 causes the output unit 44 to display Fig.29 The second display setting screen 706 is displayed.

[0238] The distribution parameter selection button 752 is an input element for causing the display screen to transition to a distribution parameter selection screen, which is a screen for inputting distribution parameters. Fig.30 The allocation parameter selection screen 707 is displayed on the output unit 44.

[0239] The allocation execution button 753 is an input element for the data processing unit 51 to execute the allocation processing and to transfer the display screen to the screen for displaying the result of the allocation processing, that is, the second result display screen. Fig.31 The second result display screen 708 is displayed on the output unit 44 .

[0240] Fig.29 706. The second display setting screen 706 includes an image display unit 800, a second image selection unit 760, and an animation operation unit 810. The second image selection unit 760 includes a second captured image display button 761, a second detection result image display button 762, a second intersection removal result image display button 763, an allocation result image display button 764, and a third confirmation button 765. Before clicking these buttons, the output control unit 52 can display the image that was previously displayed on the second basic screen 705, etc., on the image display unit 800.

[0241] The second image selection unit 760 includes buttons 761 , 762 , 763 , 764 , and 765 , which are input elements for the user to select an image displayed in the image processing unit 800 by pointing a cursor and clicking with a mouse or a touch pad.

[0242] The second captured image display button 761 is an input element for displaying a captured image on the image display unit 800 by clicking the second captured image display button 761. When the second captured image display button 761 is clicked, the output control unit 52 displays the captured image on the image display unit 800.

[0243] The second detection result image display button 762 is an input element for displaying the detection result image Gd on the image display unit 800 by clicking the second detection result image display button 762. When the second detection result image display button 762 is clicked, the output control unit 52 displays the detection result image Gd on the image display unit 800.

[0244] The second intersection removal result image display button 763 is an input element for displaying the intersection removal result image Gk on the image display unit 800 by clicking the second intersection removal result image display button 763. When the second intersection removal result image display button 763 is clicked, the output control unit 52 displays the intersection removal result image Gk on the image display unit 800.

[0245] The allocation result image display button 764 is an input element for displaying the above-mentioned allocation result image Gr on the image display unit 800 by clicking the allocation result image display button 764. When the allocation result image display button 764 is clicked, the output control unit 52 displays the allocation result image Gr on the image display unit 800. Fig.29 In the allocation result image Gr, the neurites of the same group are schematically shown by dot-dash lines and are represented by the same color tone or the like.

[0246] The third confirmation button 765 is an input element for causing the display screen to transition to the second basic screen 705 by clicking the third confirmation button 765. When the third confirmation button 765 is clicked, the output control unit 52 causes the display screen to transition to the second basic screen 705, and then displays the image displayed on the image display unit 800 of the second display setting screen 706 on the image display unit 800 of the second basic screen 705 as it is.

[0247] Fig.307 is a diagram showing an example of the allocation parameter selection screen 707. The allocation parameter selection screen 707 includes an image display unit 800, a second parameter input unit 770, and an animation operation unit 810. The second parameter input unit 770 includes a first text box 771, a second text box 772, a third text box 773, a fourth text box 774, a fifth text box 775, a sixth text box 776, a seventh text box 777, an eighth text box 778, and a fourth confirmation button 779. The output control unit 52 can display the image displayed previously on the image display unit 800 of the second basic screen 705 on the image display unit 800 of the allocation parameter selection screen 707. Fig.30 In the example of , a fluorescent image Gf is shown.

[0248] The first text box 771, the second text box 772, the third text box 773, the fourth text box 774, the fifth text box 775, the sixth text box 776, the seventh text box 777, and the eighth text box 778 are input elements for inputting allocation parameters c0, c1, c2, c3, c4, c5, D, and f, respectively. The user can input numerical values ​​in each text box 771 to 778 of the second parameter input unit 770 using a keyboard or the like. Above each text box 771 to 778 of the second parameter input unit 770, a text indicating the type of allocation parameter is displayed.

[0249] Furthermore, any input element other than the text box may be arranged in the parameter input unit 731 as long as the allocation parameter can be input.

[0250] The fourth confirmation button 779 is an input element for causing the display screen to transition to the second basic screen 705 by clicking the fourth confirmation button 779. When the fourth confirmation button 779 is clicked, the data processing unit 51 sets the numerical values ​​input to the text boxes 771 to 778 as the allocation parameters. In addition, the output control unit 52 causes the display screen to transition to the second basic screen 705.

[0251] Fig.31 7 is a diagram showing an example of the second result display screen 708. The second result display screen 708 includes an image display unit 800, a second determination input unit 780, and an animation operation unit 810. The second determination input unit 780 includes a second end button 781, a second re-execution button 782, and a third re-execution button 783. The output control unit 52 displays the allocation result image Gr obtained by the allocation process on the image display unit 800 of the second result display screen 708. The user observes the allocation result image Gr and can input whether to end the allocation process or to perform the intersection removal process or the allocation process again.

[0252] The second determination input unit 780 includes buttons 781 , 782 , and 783 which are input elements for causing the image processing device 1 to perform a predetermined operation by the user pointing a cursor with a mouse or a touch pad and clicking the buttons.

[0253] The second end button 781 is a button for ending the allocation process by clicking the second end button 781. When the second end button 781 is clicked, the data processing unit 52 ends the process as appropriate.

[0254] The second re-execution button 782 is a button for performing the intersection point removal process again by clicking the second re-execution button 782. When the second re-execution button 782 is clicked, the output control unit 52 displays the first basic screen 701 on the output unit 44.

[0255] The third re-execution button 782 is a button for performing the allocation process again by clicking the third re-execution button 783. When the third re-execution button 783 is clicked, the output control unit 52 displays the second basic screen 705 on the output unit 44.

[0256] In the image processing method of this modified example, the user can confirm the results of the intersection removal process and the results of the allocation process and adjust the parameters of these processes. As a result, it is possible to accurately analyze how the pixel groups of the segment S and the like correspond to the linear portions for each specific captured image. In addition, since the above parameters can be input as needed while viewing the images showing the results of the intersection removal process and the results of the allocation process through the screen transition as described above, the above analysis can be performed efficiently.

[0257] The present invention is not limited to the contents of the above-mentioned embodiment, and other aspects that can be conceived within the scope of the technical concept of the present invention are also included in the scope of the present invention.

[0258] The disclosed contents of the following priority basic applications are incorporated into this specification as references.

[0259] Japan Special Application No. 2019-164505 (applied on September 10, 2019)

[0260] Description of Reference Numerals

[0261] 1…image processing device, 9…network, 10…culture room, 20…shooting unit, 40…information processing unit, 43…storage unit, 44…output unit, 50…control unit, 51, 51a…data processing unit, 100…culture unit, 300…quality evaluation system, 310…information processing device, 320…terminal device, 330, 331, 332, 333…gateway terminal, 340, 341, 342, 343…observation device, 511…detection result image generation unit, 512…pixel group setting unit, 513…group setting unit, 514, 514a…calculation unit, 600, 600a…determination unit, 601…execution unit, 602…end determination unit, 700…image generation unit, 701…first basic screen, 702…first display setting screen, 703…intersection detection parameter selection screen, 704…first result display screen, 705…second basic screen, 706…second display setting screen, 707…allocation parameter selection screen, 708…second result display screen, C1…centroid of segment, C2…center of rectangle, Ce…cell, D…distance between segments , E, E12, E34, E57, E58, E67, E68…edge, N5, N6, N7, N8, N301, N302, N303, N304…node, G, G1, G2…weighted connected graph, Gd…detection result image, Gp…probability distribution image, L1…first line segment, L2…second line segment, L3…third line segment, Nr…neurite, Px, Px1, Px2…pixel, R, R1, R2, R3, R4…search range, Rt…rectangle, S, Sa, Sb, Sc, S1, S2, S3, S4, S5, S6, S7, S8, S101, S102, S103, S201, S202, S203, S301, S302, S303, S304…segments, Sg1, Sg2…pixel groups, SS1, SS2, SS3, SS4, SS5, SS6, SS7…pixel segments, So, So1, So2…cell bodies, T, T1, T2, T11, T12, T21…end pixels, w…distance between segments, θ1…first angle, θ2…second angle, θ3…third angle.

Claims

1. An image processing device comprising: a pixel group setting unit that acquires image data of a first image representing an object including a linear portion from the imaging unit, and sets the plurality of pixel groups corresponding to the first image after removing pixels corresponding to intersections from pixels constituting the first image, thereby setting the plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other; A group setting unit sets a group of the first pixel group and the second pixel group among the plurality of pixel groups in order to calculate a degree of connection between the second pixel group and the first pixel group, wherein: The second pixel group includes pixels within a specified range from the first pixel group; and a calculation unit that calculates the connection degree of the group set by the group setting unit, The prescribed range is a search range set based on a first distance between the first pixel group and the second pixel group, or a search range set based on a first angle formed by a first line segment corresponding to the first pixel group and a second line segment corresponding to the second pixel group. The calculation unit calculates the first distance or the first angle, and calculates the degree of connection using the first distance or the first angle.

2. The image processing device according to claim 1, wherein: The calculation unit calculates at least one of a second angle between a third line segment passing through points included in the first pixel group and points included in the second pixel group and the first line segment, and a third angle between the third line segment and the second line segment, and calculates the degree of connection based on at least one of the second angle and the third angle.

3. The image processing device according to claim 2, wherein: The calculation unit calculates at least one of a fourth angle between the shortest fourth line segment among the third line segments and the first line segment, and a fifth angle between the fourth line segment and the second line segment, and calculates the degree of connection based on at least one of the fourth angle and the fifth angle.

4. The image processing device according to claim 1, wherein: The group setting unit sets the group for each pixel group constituting the plurality of pixel groups. The calculation unit calculates the connection degree for each of the groups. A determination unit is provided for determining grouping of the plurality of pixel groups based on the calculated degree of connection.

5. The image processing device according to claim 4, wherein: The decision unit uses each pixel group in the plurality of pixel groups as a node and the degree of connection as an edge weight, and sequentially determines which group the node belongs to based on a minimum spanning tree obtained using Prim's algorithm, Kruskal's method or Boruvka's method.

6. The image processing device according to claim 5, wherein: The decision unit includes a termination determination unit that determines whether to terminate the process of finding the minimum spanning tree based on a sum of numerical values ​​corresponding to a plurality of selected edges when an edge is selected in the Prim's algorithm, the Kruskal method, or the Boruvka method.

7. The image processing device according to claim 4, wherein: An image generating unit is provided, the image generating unit generating a second image indicating a result of the grouping.

8. The image processing device according to claim 2, wherein: The pixel group setting unit removes pixels corresponding to intersections from pixels constituting the first image before setting the plurality of pixel groups, and sets the opposite segments across the intersection as intersection pixel groups. Furthermore, the calculation unit calculates the degree of connection so that a group of segments of the intersection pixel group corresponds to a part of the connected linear region.

9. The image processing apparatus according to claim 2, wherein: The pixel group setting unit includes a cross pixel group setting unit. When there are three other pixel groups within a certain range from the first pixel group, the cross pixel group setting unit collects four pixel groups including the first pixel group and the three pixel groups as the cross pixel group. The calculation unit calculates the degree of connection such that a group of the intersection pixel groups facing each other across the intersection corresponds to a part of the connected linear region.

10. The image processing apparatus according to claim 2, wherein: The calculation unit calculates the degree of connection based on at least one of an average value of brightness of a plurality of pixels constituting the first pixel group or the second pixel group, a width of the first pixel group or the second pixel group, a curvature of the curve when the first pixel group or the second pixel group corresponds to the curve, and a length of a line segment constituting the broken line when the first pixel group or the second pixel group corresponds to the broken line.

11. The image processing apparatus according to claim 1, wherein: The group setting unit sets a group of the first pixel group and the second pixel group among the multiple pixel groups, wherein the second pixel group includes pixels located at the end of the first pixel group or pixels within the specified range set corresponding to the centroid of the first pixel group or the center of a rectangle circumscribed to the first pixel group.

12. The image processing apparatus according to claim 1, wherein: The first image is an image obtained by performing image processing on an image obtained by photographing the object, and is a binary image in which each pixel of the first image is represented by a pixel value indicating whether the pixel corresponds to a linear structure. The pixel group setting unit sets the plurality of pixel groups based on the pixel values.

13. The image processing apparatus according to claim 1, wherein: The first image is an image representing a plurality of pixels corresponding to neurites. The degree of connection is a numerical value indicating the possibility that the first pixel group and the second pixel group correspond to a part of a neurite extending from the same nerve cell.

14. The image processing apparatus according to claim 1, wherein: The first image is an image showing a plurality of pixels corresponding to blood vessels. The degree of connection is a numerical value indicating the possibility that the first pixel group and the second pixel group correspond to a part of the same blood vessel.

15. An image processing method, comprising: A step of acquiring image data of a first image representing an object including a linear portion from an imaging unit, and setting the plurality of pixel groups corresponding to the first image after removing pixels corresponding to intersections from pixels constituting the first image, thereby setting the plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other; In order to calculate the degree of connection between the second pixel group and the first pixel group, a step of setting a group of the first pixel group and the second pixel group among the plurality of pixel groups is performed, wherein: The second pixel group includes pixels within a specified range from the first pixel group; and a step of calculating the degree of connection of the group set by the setting step, The prescribed range is a search range set based on a first distance between the first pixel group and the second pixel group, or a search range set based on a first angle formed by a first line segment corresponding to the first pixel group and a second line segment corresponding to the second pixel group. In the calculation step, the first distance or the first angle is calculated, and the degree of connection is calculated using the first distance or the first angle.

16. A storage medium storing a program for causing an image processing device to perform the following processing: A pixel group setting process of acquiring image data of a first image representing an object including a linear portion from an imaging unit, and setting a plurality of pixel groups corresponding to the first image by removing pixels corresponding to intersections from pixels constituting the first image, and then setting the plurality of pixel groups including a first pixel group and a second pixel group that are not connected to each other; The group setting process is to set the first pixel group and the second pixel group in the plurality of pixel groups in order to calculate the degree of connection between the second pixel group and the first pixel group, wherein: The second pixel group includes pixels within a specified range from the first pixel group; and The calculation process calculates the connection degree of the set group.

Images