Inspection equipment and program

Abstract

To provide an inspection apparatus which can identify each of test objects to inspect them, on the basis of an image that represents two or more test objects.SOLUTION: An inspection apparatus includes: feature point information output means which outputs information for identifying positions of multiple feature points in an entire image, the information obtained by inputting the entire image to a trained model equipped in advance, the entire image being an image that represents two or more test objects; feature point set extraction means which extracts, from among the feature points output by the feature point information output means, a feature-point set which is a set of two or more feature points each representing two or more sections of the test objects, and satisfying requirements of positional relationship based on a tendency of skeleton shape of the test objects; and inspection means which performs, based on the feature-point set, inspection of the test objects represented in the entire image, specified by positions of the feature points constituting the feature-point set.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an inspection device, a learning model generation device, and a program for inspecting two or more objects to be inspected based on an image in which each of the objects is represented. [Background technology]

[0002] Patent documents 1 and 2 disclose an inspection device that extracts an image of each inspection object from an overall image of multiple inspection objects being transported by a transport means, inputs the image into a trained model, and determines whether the inspection objects are normal or not. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6754155 [Patent Document 2] International Publication No. 2020 / 189043 Summary of the Invention [Problem to be solved by the invention]

[0004] To extract an image of each test object from an image that shows two or more test objects, it is necessary to be able to identify each test object in the image. Identifying each test object in the image is easy for granular objects such as almonds or beans, which are unlikely to overlap with other test objects during transport, because the individual test objects are photographed separately. However, for test objects that are prone to overlapping or contacting with each other during transport, multiple test objects may be photographed while overlapping or touching each other. In this case, the images of each test object are connected at the overlapping or contacting parts, making it difficult to identify each test object in the image.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an inspection device, a learning model generation device, and a program that are capable of identifying and inspecting two or more objects based on an image depicting each object. [Means for solving the problem]

[0006] The inspection device of the present invention comprises: a feature point information output means for outputting information capable of identifying the positions of multiple feature points in the overall image, obtained by inputting an image depicting two or more objects to be inspected as an overall image into a pre-prepared trained model; a feature point set extraction means for extracting, from the multiple feature points, feature point sets, which are sets of feature points representing two or more predetermined parts of the object to be inspected and which satisfy positional relationship requirements based on the tendency of the skeletal shape of the object to be inspected; and an inspection means for inspecting the object to be inspected depicted in the overall image, identified from the positions of each feature point constituting the feature point set, based on the feature point set.

[0007] The requirement for the positional relationship may be a requirement for an arc formed by sequentially connecting a predetermined number of feature points extracted as a provisional feature point set from among a plurality of feature points.

[0008] The requirement on the arc may be a requirement on the radius of curvature of the arc.

[0009] The requirement for the arc may also be a requirement for the length of the arc.

[0010] The feature point information output means may further output information on the parts represented by each of the plurality of feature points in the overall image, obtained by inputting the overall image into the trained model, and the feature point set extraction means may extract a provisional feature point set consisting of a predetermined number of feature points from the plurality of feature points, taking into account the part information, and then extract the feature point set by determining whether the provisional feature point set satisfies a positional relationship requirement.

[0011] The feature points set representing the object to be inspected may be a set of three feature points representing one end, a central portion, and the other end portion of the object to be inspected, and the feature points set extraction means may extract, as provisional feature points sets from the plurality of feature points, a feature point whose representative portion is the central portion and two feature points whose representative portions are the end portions, and then extract the feature points set by determining whether or not the positional relationship requirements are satisfied.

[0012] The requirement for the positional relationship may be that the distance between a feature point whose representative part is the center and a feature point whose representative part is an end part is within a predetermined range.

[0013] The apparatus may further include a cutting-out means for cutting out a predetermined range, based on the position of a feature point belonging to the feature point set, from the overall image as an image of a portion of the object to be inspected that is represented by the feature point, and the inspection means may inspect the object to be inspected based on the image cut out by the cutting-out means.

[0014] In the feature point information output means, the information that can identify the position of the feature points obtained by inputting the entire image into the trained model may be a probability distribution in which the position of the feature points is expressed as a median and a mode, and the probability density follows a normal distribution.

[0015] The inspection device may further include a transport means for transporting the object to be inspected, an irradiation means for irradiating a predetermined electromagnetic wave onto the object to be inspected transported by the transport means, a detection means that is arranged at a position where it can detect the electromagnetic wave irradiated by the irradiation means and transmitted through the object to be inspected, and that detects the electromagnetic wave and outputs detection data, and an image generation means that generates an overall image based on the detection data and provides it to the feature point information output means.

[0016] The learning model generation device of the present invention performs machine learning using training data including training images that are images depicting two or more test objects and correct labels that are information on the positions of each of a plurality of feature points in the training images, and generates a learning model that outputs information that can identify the positions of a plurality of feature points in the entire image when an entire image that is an image depicting two or more test objects and in which a plurality of feature points are unknown is input.

[0017] The correct answer label may further include information on the areas represented by each of the multiple feature points in the training image, and the training model may further output information on the areas represented by each of the multiple feature points in the entire image.

[0018] The learning model outputs, as information capable of identifying the positions of feature points, a probability distribution in which the positions of feature points are expressed as a median and a mode, and the probability density follows a normal distribution. The correct label is a probability distribution in which the positions of feature points set in the learning image as the correct label are the median and a mode, and the probability density follows a normal distribution. In machine learning, the parameters of the learning model may be adjusted so as to increase the degree of agreement between the probability distribution output by the learning model and the probability distribution that is the correct label.

[0019] The inspection device of the present invention may be realized by executing a program in which the functions of the inspection device of the present invention are described on a computer.

[0020] The learning model generation device of the present invention may be realized by executing a program in which the functions of the learning model generation device of the present invention are described on a computer. [Effects of the Invention]

[0021] The inspection device, learning model generation device, and program of the present invention make it possible to identify and inspect each object even in an image that shows two or more objects being inspected, even if they are photographed in an overlapping or touching state. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a configuration diagram of an inspection device 100 and a learning model generation device 200. [Figure 2] FIG. 10 is a diagram showing a state in which an object to be inspected W1 and an object to be inspected W2 are partially overlapped. [Figure 3] FIG. 10 is a diagram showing an electromagnetic wave transmission image P1 of objects W1 and W2 to be inspected. [Figure 4] FIG. 10 is a diagram showing an example in which the positions of feature points themselves are output from a trained model. [Figure 5] FIG. 10 is a diagram illustrating an example in which a probability distribution corresponding to a feature point is output from a trained model. [Figure 6] This figure explains that the position of a feature point can be identified from the probability distribution output from a trained model. [Figure 7] FIG. 10 is a diagram showing a state in which an object to be inspected W11 and an object to be inspected W12 are partially overlapped. [Figure 8] FIG. 10 is a diagram showing an electromagnetic wave transmission image P11 of objects W11 and W12 to be inspected. [Figure 9] FIG. 10 is a diagram showing an example in which feature points themselves are set on inspection objects W11 and W12. [Figure 10] FIG. 10 is a diagram showing an example in which feature points set on inspection objects W11 and W12 are converted into a probability distribution. [Figure 11] FIG. 10 is a diagram showing another example in which a probability distribution corresponding to a feature point is output from a learning model. [Figure 12] 10A and 10B are diagrams illustrating deviations between feature points based on a probability distribution output from a learning model and feature points that are correct labels. [Figure 13] FIG. 10 is a diagram showing an example of an arc formed by sequentially connecting feature points belonging to a feature point set. [Figure 14] FIG. 10 is a diagram showing another example of an arc formed by sequentially connecting feature points belonging to a feature point set. [Figure 15] FIG. 10 is a diagram showing a state in which a feature point set is superimposed on an electromagnetic wave transmission image P1. [Figure 16] FIG. 10 is a diagram showing an example of cutting out a partial image from an entire image. [Figure 17]FIG. 10 is a diagram illustrating another example of cutting out a partial image from an entire image. DETAILED DESCRIPTION OF THE INVENTION

[0023] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same components will be designated by the same reference numerals, and the description of components that have already been described will be omitted as appropriate.

[0024] FIG. 1 shows a configuration diagram of an inspection device 100 and a learning model generation device 200 according to the present invention.

[0025] The inspection device 100 includes a conveying means 110 , an irradiating means 120 , a detecting means 130 , an image generating means 140 , a feature point information output means 150 , a feature point set extracting means 160 , and an inspecting means 170 .

[0026] The transport means 110 is a conveyor of any type that transports the placed inspection object W in the Y-axis direction at a predetermined speed. The transport means 110 has a depth in the X-axis direction, which is perpendicular to the Y-axis direction, which is the transport direction, when viewed from the front.

[0027] The irradiation means 120 irradiates a predetermined electromagnetic wave toward the inspection object W transported by the transport means 110. The type of electromagnetic wave to be irradiated may be appropriately selected depending on the content of the inspection, such as X-rays, visible light, or infrared light.

[0028] The detection means 130 is disposed at a position where it can detect the electromagnetic waves irradiated from the irradiation means 120 and transmitted through the object under test W, detects the electromagnetic waves that have passed through the object under test W or have reached it directly, and outputs detection data at predetermined intervals. FIG. 1 shows an example in which the detection means 130 is disposed opposite the irradiation means 120 so as to be able to detect the electromagnetic waves that have passed through the object under test W. The detection means 130 may be disposed inside the transport means 110 as shown in FIG. 1 and configured to detect the electromagnetic waves that have further passed through the conveyor belt or the like of the transport means 110, or may be configured to directly detect the electromagnetic waves that have passed through the object under test W by providing a small gap between two transport means 110 arranged in the Y-axis direction. When detecting the electromagnetic waves that have passed through the conveyor belt or the like of the transport means 110, it is preferable to use a conveyor belt or the like made of a material that is highly permeable to the electromagnetic waves irradiated from the irradiation means 120.

[0029] The detecting means 130 includes a plurality of detecting elements arranged in a line in a direction perpendicular to the conveying direction, and each detecting element detects the electromagnetic waves that reach it and outputs detection data. In the example of Fig. 1, the plurality of detecting elements are arranged in the X-axis direction perpendicular to the Y-axis direction.

[0030] The irradiation means 120 and the detection means 130 are fixed in position relative to the conveying means 110, and the entire object W to be inspected, being conveyed in the Y-axis direction by the conveying means 110, crosses the detection means 130 arranged in a line in the X-axis direction, thereby obtaining detection data of the transmitted electromagnetic waves when the object W to be inspected is viewed in the XY plane.

[0031] The image generating means 140 generates an electromagnetic wave transmission image that expresses the shape and internal state of the object W by shading or the like, based on the detection data of the electromagnetic waves transmitted through the object W under inspection obtained by the detecting means 130. At this time, when two or more objects under inspection are transported overlapping each other, an integrated electromagnetic wave transmission image that expresses the overlapping state in an XY plane view is generated. For example, when two objects under inspection W1 and W2 are transported with a partial overlap as shown in FIG. 2, an electromagnetic wave transmission image P1 with a contour as shown in FIG. 3 is generated.

[0032] The electromagnetic wave transmission image P1 thus generated, which expresses the state in which two or more objects to be inspected are overlapped, is provided to the feature point information output means 150 as the entire image.

[0033] The feature point information output means 150 outputs information that can identify the positions of each of multiple feature points in the overall image, obtained by inputting the overall image provided by the image generation means 140 into a pre-trained model.

[0034] Examples of information that can identify the positions of feature points include information on the positions of feature points themselves, and a probability distribution in which the positions of feature points are expressed as a median and a mode and the probability density follows a normal distribution.

[0035] Specifically, for example, when outputting the positions of three feature points for each inspection object, by inputting an electromagnetic wave transmission image P1 shown in Figure 3 for two inspection objects W1 and W2, the information on the positions of feature points C1 to C6 themselves may be output as shown in Figure 4, or probability distributions H1 to H6, which are heat maps of probability densities in which the positions of feature points C1 to C6 are expressed as medians and modes, may be output as shown in Figure 5. In this case, the centers of the probability distributions H1 to H6 can be identified as feature points C1 to C6, as shown in Figure 6. A specific description of the output of probability distributions from a trained model will be given later.

[0036] The trained model is generated in advance by the learning model generation device 200. The learning model generation device 200 repeatedly performs machine learning using a large amount of training data including training images, which are images depicting two or more objects to be inspected, and information on the positions of each of a plurality of feature points in the training images, which are correct labels, to generate a trained model that outputs information capable of identifying the positions of each of a plurality of feature points in an input entire image. The machine learning may be general machine learning in which a person defines features, or deep learning in which a machine defines features.

[0037] The device user or the like sets a predetermined number of feature points as correct labels for portions of the training image corresponding to each object to be inspected. A feature point is one point constituting a set of feature points that characterize the skeletal shape of the object to be inspected represented in the entire image, and also represents the portion of the object to be inspected corresponding to the set position, and is a base point used to cut out the image of that portion. Therefore, the setting positions and number of feature points may be determined depending on the tendency of the skeletal shape of the object to be inspected, the portion to be inspected, etc. For example, if the skeletal shape is curved, feature points may be set in three locations: one end, a center, and the other end.

[0038] When the electromagnetic wave transmission image P11 shown in FIG. 8, which depicts two objects W11 and W12 partially overlapping as shown in FIG. 7, is used as a learning image, an example is shown in FIG. 9 in which feature points C11, C12, and C13 are set at one end, center, and other end of the portion corresponding to the object W1, and feature points C14, C15, and C16 are set at one end, center, and other end of the portion corresponding to the object W2.

[0039] The generation and training of a learning model is generally performed by inputting training data into the learning model, determining the degree of match between the output obtained and the correct label of the training data, and repeatedly adjusting the parameters of the learning model to increase the degree of match. The learning model generated by the learning model generation device 200, which receives an entire image as input and outputs information capable of identifying the positions of each of a plurality of feature points in the entire image, may be a learning model that directly outputs the positions of each of a plurality of feature points. However, when training a learning model that directly outputs the positions of a plurality of feature points, the positions of the feature points output from the learning model and the positions of the feature points set in the training image, which are the correct label, are both points, and the probability that the two match is almost zero. Therefore, it is not possible to determine the degree of match, making it difficult to train using the above-mentioned general learning method.

[0040] Therefore, for example, the positions in the training image are treated as random variables, and the positions of feature points set in the training image as correct labels are converted into a probability distribution whose probability density follows a normal distribution, with the median and mode being the median and mode, and the converted correct labels are used for training. The conversion of the correct labels may be performed, for example, using an appropriately created application. By performing training using the converted correct labels, a training model is generated that outputs, as an output for an input of a training image, a probability distribution whose probability density follows a normal distribution, with the feature point positions expressed as the median and mode. Since the output from the training model and the converted correct labels are both probability distributions of positions, it is possible to determine the degree of match. For example, the degree of match of the medians can be determined, and parameters can be adjusted to the training model to increase the degree of match. Furthermore, since the probability distribution output by inputting an entire image into a trained model generated by repeatedly training the training model expresses the feature point positions as the median and mode, the trained model corresponds to a trained model that outputs information that can identify the positions of feature points when the entire image is input.

[0041] Because the probability density of both the probability distribution output from the learning model and the probability distribution of the correct label is greatest at the center, which is the position of the feature point, and decreases according to a normal distribution as you move away from the center, they can be represented as an image in the form of a heat map, for example, where the density is highest at the center and decreases concentrically as you move away from the center. Therefore, by creating heat map images of the probability distribution output from the learning model and the probability distribution of the correct label, the degree of agreement of the medians can be determined on the image from the degree of separation between the positions of highest density in both probability distributions.

[0042] For example, when training a learning model using the electromagnetic wave transmission image P11 shown in Fig. 8 and the feature points C11 to C16 which are the correct labels corresponding to the electromagnetic wave transmission image P11 shown in Fig. 9 as training data, first, the feature points C11 to C16 which are the correct labels are converted into a probability distribution H11 to H16 expressed in a heat map as shown in Fig. 10, and the electromagnetic wave transmission image P11 is input to the learning model to obtain an output of the probability distribution H21 to H26 expressed in a heat map as shown in Fig. 11, and the feature points C21 to C26 corresponding to the probability distribution H21 to H26 are identified. As a result, positional deviations D1 to D6 between the feature points C21 to C26 based on the probability distribution output from the learning model and the feature points C11 to C16 which are the correct labels can be obtained as shown in Fig. 12, and based on this, parameters can be adjusted in the learning model to improve the degree of match.

[0043] The feature point set extraction means 160 extracts, from the plurality of feature points output by the feature point information output means 150, feature point sets that are sets of feature points that respectively represent two or more predetermined number of parts of the object to be inspected and that satisfy the requirements of the positional relationship based on the tendency of the skeletal shape of the object to be inspected.

[0044] Each feature point belonging to the multiple feature points output by the trained model is located at a position that represents one of the parts of one of the objects shown in the overall image, but it is not clear which part of the object the feature point represents. Therefore, the feature point set extraction means 160 groups the feature points of each object based on the tendency of the skeletal shape of the object.

[0045] A specific method for extracting a feature point set may be, for example, to extract a predetermined number of feature points corresponding to a predetermined number of parts from among a plurality of feature points as a provisional feature point set, define a requirement for the positional relationship based on an arc formed by sequentially connecting the extracted feature points, and extract a feature point set that satisfies this requirement.

[0046] 2 are sausages, the skeleton shape of a sausage generally meets certain product standards, such as not being too curved, having a length within a certain range, etc. Therefore, it is possible to define requirements for arcs that correspond to these standards, such as whether the radius of curvature is equal to or greater than a certain value, whether the length of the arc is within a certain range, etc.

[0047] When an electromagnetic wave transmission image P1 of objects W1 and W2 shown in Fig. 3 is input to a trained model, and six feature points C1 to C6 in total, three for each object, are output in the positional relationship shown in Fig. 4, two provisional sets of three feature points are extracted, and each set of three feature points is sequentially connected to form an arc. For example, as shown in Fig. 13, when arc A1 is formed by sequentially connecting feature points C1, C2, and C6, and arc A2 is formed by sequentially connecting feature points C3, C4, and C5, it is possible to determine that the length of arc A1 is too long and the radius of curvature of arc A2 is too small. This type of determination is repeated while changing the combination of feature points that make up the set of tentative feature points. Finally, as shown in FIG. 14, an arc A3 that sequentially connects feature points C1, C2, and C3, and an arc A4 that sequentially connects feature points C4, C5, and C6, which satisfy each of the requirements, are identified. This makes it possible to extract a feature point set G1 consisting of the feature points that make up arc A3, and a feature point set G2 consisting of the feature points that make up arc A4.

[0048] Instead of forming a circular arc, the curve connecting each feature point in sequence may be expressed by a piecewise polynomial, and the requirements may be defined by providing a certain relational expression or range for the coefficients of the piecewise polynomial instead of the radius of the circular arc.

[0049] The extraction of a provisional feature point set prior to the extraction of a feature point set by the feature point set extraction means 160 may be performed by taking into account information about the regions represented by the feature points. Specifically, when generating and learning a learning model, information about the region represented by each feature point is added to the correct label for each feature point, and the learning model is configured so that information about the region represented by each feature point is obtained as the output of the trained model. For example, if three feature points, one at each end and one at the center, are set for each object to be inspected, the output of the trained model can provide information about whether each feature point represents the center or the end. Therefore, the feature point set extraction means 160 extracts a provisional feature point set consisting of a feature point representing the center and two feature points representing the end portions. By adopting an extraction method that takes into account information about the regions in this way, it is possible to avoid determining a provisional feature point set based on a combination of feature points in an unnatural positional relationship, thereby shortening the extraction processing time and improving extraction accuracy. When three feature points, one at each end and one at the center, are set for each object to be inspected, the provisional feature point set may be determined by further considering the requirement that the distance between the feature point representing the center and the feature point representing the end is within a predetermined range. This can further improve the extraction accuracy.

[0050] Alternatively, a method may be employed in which, in addition to information about the region represented by a feature point, information about the orientation of the feature point toward adjacent feature points is also taken into account. Specifically, when generating and training a learning model, information about the region represented by each feature point and information about the orientation of the feature point toward adjacent feature points are added to the correct label for each feature point, and the learning model is configured so that information about the region represented by each feature point and the orientation of the feature point toward adjacent feature points is obtained as the output of the trained model. Then, the feature point set extraction means 160 takes into account information about the region and the orientation of the feature point toward adjacent feature points when selecting feature points to form a provisional feature point set. For example, if three feature points are set for each object to be inspected, one at each end and one at the center, the output of the trained model can obtain information about whether each feature point represents the center or an end. In addition, if the feature point is in the center, information about the orientation of the feature point toward each end, and if the feature point is at the end, information about the orientation of the feature point toward the center can also be obtained. Therefore, a provisional feature point set is configured from a feature point representing the center and two feature points representing the end points, whose orientation information is roughly the same but in opposite directions. This allows the number of provisional feature point sets to be determined to be reduced compared to when only part information is taken into account, thereby further shortening the extraction processing time and further increasing the extraction accuracy.

[0051] In addition to the above-described method, the feature point set extraction means 160 may extract a feature point set from a plurality of feature points using a trained model that outputs a feature point set in response to input of a plurality of feature points. In this case, the trained model used in the feature point set extraction means 160 may be provided by, for example, providing the learning model generation device 200 with a function for generating a learning model that outputs a feature point set in response to input of a plurality of feature points, in addition to a function for generating a learning model that outputs a feature point set in response to input of a whole image, or by providing a trained model generated by another learning model generation device (not shown). The learning model that outputs a feature point set in response to input of a plurality of feature points can be generated by repeatedly performing machine learning using a large amount of training data that includes information on the positional relationships between the plurality of feature points and information on the feature points that constitute the feature point set, which is the correct label.

[0052] The inspection means 170 inspects the object to be inspected, which is represented in the entire image and is identified from the positions of each feature point constituting the feature point set, based on the feature point set. For example, it is possible to determine the normality of the curvature and length of the object to be inspected from the shape and length of the arc connecting each feature point constituting the feature point set.

[0053] Furthermore, a cutting-out means 180 may be provided between the feature point set extraction means 160 and the inspection means 170, which cuts out from the entire image a predetermined range based on the position of a feature point belonging to the feature point set extracted by the feature point set extraction means 160 as an origin, as an image of a portion of the object to be inspected represented by the feature point, and the inspection means 170 may be configured to inspect the object to be inspected based on the image cut out by the cutting-out means 180.

[0054] The predetermined range, specifically the shape, size, orientation, etc., of the image of the part of the object to be inspected, which is extracted from the entire image by the extraction means 180, may be determined appropriately depending on the shape, size, orientation, etc. of the object to be inspected. Furthermore, the shape, size, and orientation of the part to be inspected may be different for each part.

[0055] For example, when two feature point sets G1 and G2 are extracted as shown in Fig. 14 from the electromagnetic transmission image P1 for two objects W1 and W2 shown in Fig. 3, the feature point sets G1 and G2 are superimposed at corresponding positions on the electromagnetic transmission image P1, and predetermined ranges are cut out from the electromagnetic transmission image P1 using each of the feature points C1 to C6 as a base point, as shown in Fig. 15. At this time, the images cut out using the feature points C1 to C3 belonging to the feature point set G1 as a base point are partial images of one end, center, and other end of the object W1, and the images cut out using the feature points C4 to C6 belonging to the feature point set G2 as a base point are partial images of one end, center, and other end of the object W2. FIG. 16 shows an example of cutting out partial images T1 to T3 of one end, center, and other end of the object to be inspected W1 using feature points C1 to C3 that make up feature point set G1 as base points, and FIG. 17 shows an example of cutting out partial images T4 to T6 of one end, center, and other end of the object to be inspected W2 using feature points C4 to C6 that make up feature point set G2 as base points.

[0056] The inspection performed by the inspection means 170 based on the image cut out by the cut-out means 180 may be performed by image processing, or may be performed by inputting the image into a trained model for inspection prepared in advance. Note that the images cut out by the cut-out means 180 are partial images for each part of the object to be inspected, and therefore the inspection results are inspection results for each part of the object to be inspected, but by comprehensively judging the inspection results for each part, the inspection results for the entire object to be inspected can be obtained.

[0057] In addition, when inspection of a part of the region is omitted, for example, inspection is performed on both ends but not on the center, there is no need for the extraction means 180 to extract an image of that part of the region.

[0058] The inspection device and learning model generation device of the present invention may be realized by having a computer execute a program describing the functions of each device. That is, in the computer, the functions of each device may be realized by having a CPU read and execute a program stored in any storage means.

[0059] According to the inspection device and learning model generation device of the present invention described above, it is possible to identify and inspect each object even in an image that shows two or more objects being inspected, even if they are photographed in an overlapping or touching state.

[0060] The present invention is not limited to the above-described embodiments. Each embodiment is merely an example, and any embodiment that has substantially the same configuration as the technical idea described in the claims of the present invention and exhibits similar effects is included within the technical scope of the present invention. In other words, appropriate modifications are possible within the scope of the technical idea expressed in the present invention, and forms incorporating such modifications and improvements are also included within the technical scope of the present invention. [Explanation of symbols]

[0061] 100 Inspection equipment 110 Means of transport 120 Irradiation means 130 Detection means 140 Image generation means 150 Feature point information output means 160 Feature point set extraction means 170 Inspection methods 180 Cutting Method A1~A4 Arc C1~C6, C11~C16, C21~C26 feature points D1~D6 Position deviation G1, G2 feature point set H1~H6, H11~H16 H21~H26 Probability distribution P1, P11 full image T1~T6 Partial images W1, W2, W11, W12 Inspected object

Claims

1. a feature point information output means for outputting information capable of identifying positions of a plurality of feature points in an entire image, the information being obtained by inputting an image representing two or more objects to be inspected into a trained model provided in advance; and a feature point set extraction means for extracting, from the plurality of feature points, feature point sets which are sets of feature points representative of two or more predetermined number of parts of the object to be inspected and which satisfy requirements for a positional relationship based on a tendency of a skeletal shape of the object to be inspected; an inspection means for inspecting the object to be inspected represented in the entire image based on the feature point set, the inspection means being specified from the positions of the feature points constituting the feature point set; a conveying means for conveying the object to be inspected; an irradiation means for irradiating the object to be inspected conveyed by the conveying means with a predetermined electromagnetic wave; a detection means, which is disposed at a position where it can detect the electromagnetic waves irradiated from the irradiation means and transmitted through the object to be inspected, and which detects the electromagnetic waves and outputs detection data; an image generating means for generating the entire image based on the detection data and providing the image to the feature point information output means; An inspection device comprising:

2. 2. The inspection device according to claim 1, wherein the requirement for the positional relationship is a requirement for an arc formed by sequentially connecting the predetermined number of feature points extracted as a provisional feature point set from among the plurality of feature points.

3. 3. The inspection apparatus according to claim 2, wherein the requirement for the arc is a requirement regarding the radius of curvature of the arc.

4. 3. The inspection device according to claim 2, wherein the requirement for the arc is a requirement regarding the length of the arc.

5. the feature point information output means further outputs information on parts represented by each of the plurality of feature points in the entire image, obtained by inputting the entire image to the trained model; The feature point set extraction means extracts a tentative feature point set consisting of the predetermined number of feature points from the plurality of feature points, taking into account information about the body part, and then extracts the feature point set by determining whether the tentative feature point set satisfies the requirement for the positional relationship.

5. The inspection device according to claim 1, wherein the inspection device is a semiconductor laser.

6. the set of feature points representing the object to be inspected is a set of three feature points representing one end portion, a central portion, and the other end portion of the object to be inspected, The feature point set extraction means extracts, as a provisional feature point set from the plurality of feature points, a feature point whose representative portion is the center and two feature points whose representative portions are the end portions, and then extracts the feature point set by determining whether or not the requirement of the positional relationship is satisfied.

6. The inspection device according to claim 5.

7. 7. The inspection device according to claim 6, wherein the requirement for the positional relationship is that the distance between a feature point whose representative portion is a central portion and a feature point whose representative portion is an end portion is within a predetermined range.

8. a cutout means for cutting out a predetermined range from the entire image, the predetermined range being based on the position of the feature point belonging to the feature point set, as an image of a portion of the object to be inspected represented by the feature point; The inspection means inspects the object to be inspected based on the image extracted by the extraction means.

8. The inspection device according to claim 1, wherein the inspection device is a semiconductor laser.

9. The inspection device according to any one of claims 1 to 8, characterized in that, in the feature point information output means, the information capable of identifying the positions of the feature points obtained by inputting the entire image to the trained model is a probability distribution in which the positions of the feature points are expressed as a median and a mode, and the probability density follows a normal distribution.

10. A program for causing a computer to function as the inspection device according to any one of claims 1 to 9.

Images