Measurement system and measurement program

Abstract

To provide a measurement system and a measurement program capable of evaluating a three-dimensional shape in detail in a short time with a simple configuration.SOLUTION: A measurement system includes a region segmentation unit, a restoration unit, and a computing unit. The region segmentation unit divides at least a color image out of the color image and a depth image including a plurality of measurement targets, into regions of the measurement targets to obtain position information of the segmented regions. The restoration unit generates point cloud data from the depth image, divides the generated point cloud data into first point cloud data, which is point cloud data of the measurement targets based on the position information of the segmented regions, and restores three-dimensional shapes of the measurement targets from each first point cloud data. The computing unit calculates a parameter for evaluating the three-dimensional shapes of the measurement targets based on the three-dimensional shapes of the measurement targets.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments relate to a measurement system and a measurement program. [Background technology]

[0002] There is a demand for the ability to measure the shape and distribution of irregularly shaped, large quantities, and randomly arranged measurement targets, such as rocks, quickly and in detail. In the case of rocks, a measurement method such as a sieving test is known. In addition, a measurement method that combines a belt conveyor and an image sensor is also known. While these measurement methods can measure a large number of targets at once, they require large-scale equipment. Another highly accurate shape measurement method is to pick up individual measurement targets one by one and measure their three-dimensional shape. However, while the measurement method of picking up individual measurement targets one by one allows for detailed measurements, it tends to take a long time to measure. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6672823 Summary of the Invention [Problem to be solved by the invention]

[0004] The embodiments provide a measurement system and a measurement program that can perform detailed evaluation of a three-dimensional shape in a short time with a simple configuration. [Means for solving the problem]

[0005] A measurement system according to one embodiment includes an area division unit, a restoration unit, and a calculation unit. The area division unit divides at least a color image, which includes a color image and a depth image including multiple measurement targets, into areas for each measurement target, thereby obtaining position information for each divided area. The restoration unit generates point cloud data from the depth image, divides the generated point cloud data into first point cloud data representing point cloud data for each measurement target based on the position information for each divided area, and restores the three-dimensional shape of each measurement target from each first point cloud data. The calculation unit calculates parameters for evaluating the three-dimensional shape of each measurement target based on the three-dimensional shape of each measurement target. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a measurement system according to an embodiment. [Figure 2A] FIG. 2A is a diagram showing an example of a mask output from the region division model. [Figure 2B] FIG. 2B is a diagram showing an example of a mask output from the region segmentation model. [Figure 3] FIG. 3 is a diagram illustrating an example of a data set. [Figure 4] FIG. 4 is a diagram illustrating an example of a hardware configuration of the measurement system. [Figure 5] FIG. 5 is a flowchart showing the learning operation of the area division model. [Figure 6] FIG. 6 is a flowchart showing the measurement process of the measurement system using the trained area division model. [Figure 7A] FIG. 7A is a diagram showing an example of point cloud data for the i-th measurement target. [Figure 7B] FIG. 7B is a diagram showing a first example of the restoration process. [Figure 7C] FIG. 7C is a diagram showing a second example of the restoration process. [Figure 8] FIG. 8 is a diagram showing an example of a summary displayed on a display device. [Figure 9] FIG. 9 is a flowchart showing the evaluation process. [Figure 10] FIG. 10 is a diagram showing an example of the display of the evaluation results on the display device. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, an embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of a measurement system according to an embodiment. As shown in FIG. 1, a measurement system 1 has an area dividing unit 11, a restoration unit 12, a calculation unit 13, and a display control unit 14. The measurement system 1 is configured to be able to communicate with a camera 2. The communication between the measurement system 1 and the camera 2 may be wireless or wired. The measurement system 1 is also configured to be able to communicate with a display device 3. The communication between the measurement system 1 and the display device 3 may be wireless or wired.

[0008] Camera 2 is a camera configured to measure information related to the three-dimensional shape of a measurement object O. In this embodiment, the measurement object is an object with an irregular shape, such as a large number of loosely piled rocks, which are randomly arranged in large quantities. The following description will continue assuming that the measurement object is a rock. Camera 2 is, for example, an RGB-D camera. An RGB-D camera is a camera configured to measure an RGB-D image. An RGB-D image includes a depth image and a color image (RGB color image). A depth image is an image in which the depth of each point on the measurement object is used as a pixel value. A color image is an image in which the RGB value of each point on the measurement object is used as a pixel value. In this embodiment, camera 2 performs measurements so that multiple measurement objects O are included within its angle of view. Therefore, the color and depth images measured by camera 2 also include multiple measurement objects O. Camera 2 does not have to be an RGB-D camera as long as it can measure information related to the three-dimensional shape of the measurement object O. For example, camera 2 may be a stereo camera, etc.

[0009] The display device 3 is a display device such as a liquid crystal display or an organic EL display. The display device 3 displays various images based on data transferred from the measurement system 1. For example, the display device 3 displays analytical information on the three-dimensional shape of the measurement target.

[0010] The region division unit 11 acquires a color image and a depth image from the camera 2, divides the acquired color image into real regions, and outputs the position information of each real region. The region division unit 11 divides the color image into real regions using a region division model 151 stored in a database (DB) 15. A real region is a region of each measurement target appearing in the input color image. The region division model 151 may be a mask inference model using deep learning such as SOLO (Segmenting Objects by Locations) or Mask R-CNN (Region-based Convolutional Neural Network). The mask inference model is configured to infer a mask representing the position information of each real region in the input color image. A mask for one real region is a mask that designates the region of one measurement target appearing in the input color image as a white region and the remaining regions as black regions. FIGS. 2A and 2B show examples of masks output from the region division model 151. FIG. 2A shows a mask that leaves only the measurement target O1 appearing in a certain color image as a white region. 2B shows a mask that leaves only a measurement object O2, which is different from a measurement object O1 in the same color image, as a white region. The region segmentation model 151 can output such a mask for the number of substantial regions of the input color image.

[0011] The region segmentation model 151 is trained using a dataset 152. FIG. 3 is a diagram showing an example of the dataset 152. The dataset 152 includes a large number of color images 1521 of measurement targets and a large number of teaching images 1522 of the measurement targets. The color images 1521 are color images of a large number of measurement targets that are used to train the region segmentation model 151. The teaching images 1522 are images in which teaching is given for the boundaries of each measurement target shown in each color image 1521. For example, in the teaching image 1522, teaching for the boundaries of each measurement target is given by color-coding the measurement targets. Here, FIG. 3 shows that the measurement targets are color-coded using hatching. The color coding may be performed randomly or according to a specific rule, such as not using the same color for adjacent measurement targets. The learning of the region segmentation model 151 is performed by repeatedly updating the model parameters until a functional representing the error between the result of the real region segmentation of the color image 1521 input to the region segmentation model 151 and the input color image 1521 and the corresponding teaching image 1522 converges.

[0012] The restoration unit 12 receives a depth image from the camera 2 and generates point cloud data from the depth image. The restoration unit 12 then receives the division result from the area division unit 11 and divides the point cloud data into point cloud data for each measurement object according to the received division result. The restoration unit 12 then restores point cloud data representing the three-dimensional shape of each measurement object from the point cloud data for each measurement object.

[0013] The calculation unit 13 calculates analytical information of the three-dimensional shape of each measurement object from the point cloud data representing the three-dimensional shape of each measurement object reconstructed by the reconstruction unit 12. The analytical information of the three-dimensional shape of the measurement object may include the three-axis diameter, surface area, volume, sphericity, etc. of each measurement object.

[0014] The display control unit 14 displays on the display device 3 the analytical information of the three-dimensional shape of the measurement object calculated by the calculation unit 13.

[0015] The DB 15 stores an area division model 151 and a data set 152. As described above, the data set 152 is used for learning the area division model 151. Therefore, if the area division model 151 has been sufficiently learned, the DB 15 does not need to store the data set 152. The DB 15 may also be provided outside the measurement system 1. In this case, the area division unit 11 of the measurement system 1 acquires information from the DB 15 as necessary.

[0016] Fig. 4 is a diagram showing an example of the hardware configuration of the measurement system 1. The measurement system 1 can be various types of terminal devices such as a personal computer (PC) or a tablet terminal. As shown in Fig. 4, the measurement system 1 has a processor 201, a ROM 202, a RAM 203, a storage 204, an input interface 205, and a communication device 206 as hardware.

[0017] The processor 201 controls the overall operation of the measurement system 1. The processor 201 operates as the area dividing unit 11, the restoration unit 12, the calculation unit 13, and the display control unit 14 by executing a program stored in the storage 204, for example. The processor 201 is, for example, a CPU (Central Processing Unit). The processor 201 may be an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. The processor 201 may be a single CPU or the like, or may be multiple CPUs or the like.

[0018] The ROM (Read Only Memory) 202 is a non-volatile memory. The ROM 202 stores a startup program and the like for the measurement system 1. The RAM (Random Access Memory) 203 is a volatile memory. The RAM 203 is used as a working memory for processing in the processor 201, for example.

[0019] The storage 204 is, for example, a storage such as a hard disk drive or a solid state drive. The storage 204 stores various programs executed by the processor 201, such as a measurement program. The storage 204 may also store the DB 15. The DB 15 does not necessarily have to be stored in the storage 204.

[0020] The input interface 205 includes input devices such as a touch panel, a keyboard, a mouse, etc. When an input device of the input interface 205 is operated, a signal corresponding to the operation content is input to the processor 201. The processor 201 performs various processes according to this signal.

[0021] The communication device 206 is a communication device that enables the measurement system 1 to communicate with external devices such as the camera 2 and the display device 3. The communication device 206 may be a communication device for wired communication or a communication device for wireless communication.

[0022] Next, a description will be given of the operation of the measurement system 1 in this embodiment. Fig. 5 is a flowchart showing the learning operation of the area division model 151 of the measurement system 1. The processing of Fig. 5 is executed by the processor 201.

[0023] In step S1, the processor 201 acquires one or more color images 1521 and teaching images 1522 corresponding to each color image 1521 from, for example, the DB 15.

[0024] In step S2, the processor 201 performs substance region segmentation by inputting the color image 1521 to the region segmentation model 151. The color images 1521 may be input to the region segmentation model 151 one by one, or multiple images may be input to the region segmentation model 151 together.

[0025] In step S3, the processor 201 calculates a mask resulting from the solid region segmentation of the color image 1521 and a functional that represents the error between this color image 1521 and the corresponding teaching image 1522. The functional may be any function that can represent an error, such as a mean square error.

[0026] In step S4, the processor 201 determines whether the value of the functional has converged. For example, if the functional is a function whose value decreases as the error decreases, it is determined that the value of the functional has converged when the value of the functional becomes smaller than a certain threshold. If it is determined in step S4 that the value of the functional has not converged, the process proceeds to step S5. If it is determined in step S4 that the value of the functional has converged, the process of FIG. 5 ends. This means that learning of the domain decomposition model 151 has been completed.

[0027] In step S5, the processor 201 updates the model parameters of the domain decomposition model 151 by, for example, backpropagation based on the value of the functional. Then, the process returns to step S2. That is, learning of the domain decomposition model 151 continues until the value of the functional converges.

[0028] 6 is a flowchart showing the measurement process of the measurement system 1 using the trained area division model 151. The process of FIG.

[0029] In step S101, the processor 201 controls the camera 2 to collectively measure a large number of randomly arranged measurement targets O. The processor 201 then acquires an RGB-D image from the camera 2. Depending on the angle of view of the camera 2, it may not be possible to measure all of the measurement targets O at once. In this case, multiple measurements may be performed. The measurement of the measurement targets O may also be performed by a user. In this case, the user holds the camera 2 in their hand and measures the measurement targets O. Furthermore, prior to processing the RGB-D image obtained by the camera 2, preprocessing such as color correction and angle of view correction may be performed. Preprocessing of the RGB-D image is expected to improve the accuracy of entity region segmentation and point cloud data generation, which will be described later.

[0030] In step S102, the processor 201 performs entity region segmentation by inputting the acquired color image to the trained region segmentation model 151. The region segmentation model 151 outputs a mask representing the position of each entity region in response to the input color image.

[0031] In step S103, the processor 201 resizes the size of each mask to match the size of the point cloud data to be generated later.

[0032] In step S104, the processor 201 generates point cloud data from the acquired depth image.

[0033] In step S105, the processor 201 divides the point cloud data generated from the depth image into point cloud data for each measurement target using the mask output from the area division model 151. Specifically, the processor 201 aligns the point cloud data generated from the depth image with the mask, and then applies the mask to the point cloud data to extract point cloud data for each measurement target.

[0034] In step S106, the processor 201 initializes i to 0. i is a parameter representing the point cloud data currently being processed. The maximum value imax of i is equal to the number of masks generated by the area division model 151, i.e., the number of measurement targets appearing in the measured image.

[0035] In step S107, the processor 201 determines whether i is less than imax. If it is determined in step S107 that i is less than imax, the process proceeds to step S108. If it is determined in step S107 that i has reached imax, the process proceeds to step S112.

[0036] In step S108, the processor 201 restores, from the ith point cloud data, point cloud data that represents the three-dimensional shape of the ith measurement object O. The restoration process will be described below.

[0037] FIG. 7A is a diagram illustrating an example of point cloud data for the i-th measurement target. In FIG. 7A, the point cloud data P1 is depicted as a plane. This is for simplicity; the actual point cloud data P1 is data containing three-dimensional coordinate information. In this embodiment, a large number of measurement targets O, each of which has an irregular shape, are measured collectively by the camera 2. In this case, information about the measurement targets O outside the angle of view of the camera 2 is typically not obtained. For example, as shown in FIG. 1, when the camera 2 measures the measurement targets O from above, the camera 2 captures only the surface of each measurement target O. The resulting depth image represents the depth of the surface of the measurement target O. Therefore, the point cloud data P1 generated from the depth image only captures point cloud data corresponding to the surface of the measurement target O, and point cloud data corresponding to the back surface is missing. In this embodiment, the three-dimensional shape of each measurement target O is evaluated. Therefore, in this embodiment, a restoration process is performed to restore the point cloud data of the missing portion in order to evaluate the detailed three-dimensional shape.

[0038] FIG. 7B illustrates a first example of the restoration process. Generally, rocks and other objects serving as the measurement object O in this embodiment have an irregular shape but are generally symmetrical. Therefore, when only point cloud data P1 of the surface of the measurement object O is obtained, point cloud data representing the three-dimensional shape of the measurement object O can be restored by combining point cloud data P2, which is a mirror image of the point cloud data P1 with respect to the symmetry plane S, with the back side of the point cloud data P1. Here, the symmetry plane S is determined by a normal vector starting from the symmetry point G. The symmetry point G can be, for example, the midpoint between the center of gravity of the surface point cloud data P1 and the center of a least-squares sphere of the point cloud data P1. The least-squares sphere is a sphere that minimizes the sum of the distances between each point in the point cloud data P1 and the symmetry plane S. The point cloud data P2 can be obtained, for example, by calculating a mapping of each point constituting the point cloud data P1 to the symmetry plane S.

[0039] 7C is a diagram showing a second example of the restoration process. In particular, the back surface of a rock or other object to be measured O placed on flat ground such as the ground often becomes flat along the ground. For this reason, when only point cloud data P1 of the front surface of the object to be measured O has been obtained, point cloud data representing the three-dimensional shape of the object to be measured O can be restored by combining planar point cloud data P2 with the back side of point cloud data P1.

[0040] Here, the embodiment shows a first example and a second example of the restoration process. Only one of the restoration processes of the first example and the second example may be performed, or both may be performed. Furthermore, the restoration processes of the first example and the second example may be used depending on the environment in which a rock or the like serving as the measurement object O is placed. For example, if a rock or the like serving as the measurement object O is placed on the ground, the restoration process of the second example may be performed, and if not, the restoration process of the first example may be performed.

[0041] Furthermore, the restoration process is not limited to the first and second examples. For example, the restoration process may be performed using a point cloud restoration model. This point cloud restoration model may include, for example, a point cloud autoencoder. The point cloud autoencoder is a machine learning model that includes an encoder that extracts features of an input point cloud and a decoder that restores a point cloud from the features, and is configured to train the decoder based on the features extracted by the encoder. The processor 201 as the restoration unit 12 inputs point cloud data of only the front surface into the point cloud restoration model, for example, and obtains point cloud data representing the three-dimensional shape of the measurement object O, including point cloud data of the back side.

[0042] Returning now to the description of FIG. 6, in step S109 after the restoration process, the processor 201 generates 3D mesh data from the restored point cloud data. The 3D mesh data is, for example, data in an STL (stereolithography) format. In this case, the processor 201 may generate the 3D mesh data by attaching a triangular mesh to the point cloud data. The 3D mesh data is not limited to data in the STL format.

[0043] In step S110, the processor 201 stores the generated three-dimensional mesh data as three-dimensional mesh data of the i-th measurement object O in, for example, the storage 204.

[0044] In step S111, the processor 201 increments i by 1. After that, the process returns to step S107. In this case, the restoration process and the generation process of 3D mesh data for the next measurement object O are performed.

[0045] In step S112, when it is determined that i has reached imax, the processor 201 displays a summary of each measurement object based on the generated three-dimensional mesh data on the display device 3. After the summary display is completed, the processing in FIG. 6 ends.

[0046] FIG. 8 is a diagram showing an example of a summary display on the display device 3. In the summary display, a 3D model M representing, for example, the 3D shape of each measurement object is displayed based on the 3D mesh data of each measurement object. The 3D model M may be displayed while being rotated to emphasize that it is a 3D image. As described above, the measurement object in this embodiment is a large object such as rocks. For this reason, it is usually difficult to display all of the 3D models M on a single screen. When it is difficult to display all of the 3D models M on a single screen, a predetermined number of the 3D models M may be displayed at a time. FIG. 8 shows an example in which three 3D models M are displayed simultaneously. When a predetermined number of the 3D models M are displayed at a time, a back button 31a and a forward button 31b are displayed. The back button 31a is a button selected by the user to display three 3D models M before the currently displayed 3D model M. The forward button 31b is a button selected by the user to display three 3D models M after the currently displayed 3D model M. The back button 31a and the forward button 31b allow the user to check the desired three-dimensional model M. Instead of the back button 31a and the forward button 31b, the three-dimensional model to be displayed may be switched in response to a specific operation such as a swipe operation.

[0047] In addition, the summary display may also display an end button 32 and an evaluation button 33. The end button 32 is a button that is operated by the user when ending the summary display. The evaluation button 33 is a button that is selected by the user when displaying the evaluation results of the three-dimensional shape of the measurement object O. When the evaluation button 33 is selected, an evaluation process is performed.

[0048] 6, the summary is displayed when the generation of 3D mesh data for all measurement targets O is completed. However, the summary may be displayed at any timing after the generation of 3D mesh data for each measurement target O is completed. For example, the summary may be displayed in response to a user instruction to display the summary on a specific screen.

[0049] The evaluation process will be described below. Fig. 9 is a flowchart showing the evaluation process. The process of Fig. 9 is executed by the processor 201.

[0050] In step S201, the processor 201 acquires three-dimensional mesh data of each measurement target from the storage 204, for example.

[0051] In step S202, the processor 201 initializes i to 0. i is a parameter that represents the 3D mesh data currently being processed.

[0052] In step S203, the processor 201 determines whether i is less than imax. If it is determined in step S203 that i is less than imax, the process proceeds to step S204. If it is determined in step S203 that i has reached imax, the process proceeds to step S207.

[0053] In step S204, the processor 201 calculates, from the ith 3D mesh data, shape parameters for evaluating the 3D shape of the ith measurement object O. The shape parameters include, for example, the three-axis diameters, surface area, volume, and sphericity of the measurement object O.

[0054] The three-axis diameters can be obtained, for example, by performing principal component analysis on the data of each vertex in the three-dimensional mesh data of the measurement object O. Specifically, when principal component analysis is performed on the coordinate data of the vertices of the three-dimensional mesh data of the measurement object O, the first principal component faces the direction in which the measurement object O is most stretched, and the third principal component faces the direction in which the measurement object O is most compressed. The three-axis diameters are the differences between the maximum and minimum values of each coordinate axis when the coordinate system is shifted in the direction of the first principal component and the direction of the third principal component.

[0055] The surface area is obtained by calculating the value of the planar integral of the surface data of the 3D mesh data, and the volume is obtained by calculating the value of the volume integral of the surface data of the 3D mesh data.

[0056] Sphericity is calculated from the square of the ratio of the diameter of an equivalent sphere with an equal surface area to the diameter of an equivalent sphere with an equal volume of the measurement object O. Here, the equivalent diameter of an equivalent sphere with an equal surface area of the measurement object O is the diameter of a sphere with the same surface area as the surface area of the measurement object O. Also, the equivalent diameter of an equivalent sphere with an equal volume of the measurement object O is the diameter of a sphere with the same volume as the volume of the measurement object O.

[0057] In step S205, the processor 201 stores the calculated shape parameters as shape parameters of the i-th measurement object O in, for example, the storage 204.

[0058] In step S206, the processor 201 increments i by 1. After that, the process returns to step S203. In this case, calculation of the shape parameters for the next measurement object O is performed.

[0059] In step S207, when it is determined that i has reached imax, the processor 201 calculates statistical data based on the calculated shape parameters. The statistical data is, for example, a histogram representing the distribution of the three-axis diameter, the equivalent surface area sphere diameter, the equivalent volume sphere diameter, and the sphericity of each measurement object O. The statistical data may also include other statistical values, such as the average value of these values.

[0060] In step S208, the processor 201 stores the calculated statistical data in the storage 204, for example.

[0061] In step S209, the processor 201 displays the evaluation result based on the calculated statistical data on the display device 3. After the evaluation result has been displayed, the processing in FIG.

[0062] FIG. 10 is a diagram showing an example of the display of the evaluation results on the display device 3. When displaying the evaluation results, a histogram 34 is displayed that shows, for example, the distribution of the three-dimensional shapes of each measurement object based on statistical data. FIG. 10 shows an example in which a histogram 34 showing the distribution of equivalent spherical diameters with equal surface areas is displayed. The histogram in FIG. 10 allows the user to intuitively grasp the shapes of the rocks contained therein.

[0063] In addition, the display of the evaluation results may also display an end button 35, an equivalent volume sphere diameter button 36, and an individual data button 37. The end button 35 is a button operated by the user to end the display of the evaluation results. The equivalent volume sphere diameter button 36 is a button selected by the user to switch the display of the evaluation results to a histogram of equivalent volume sphere diameters. Although not shown in FIG. 10 , it goes without saying that a three-axis diameter button for displaying a histogram representing the distribution of three-axis diameters and a sphericity button for displaying a histogram representing the distribution of sphericity may also be displayed. Furthermore, instead of or in addition to the buttons, the display of the histogram may be switched in response to a specific operation such as a swipe operation. The individual data button 37 is a button selected by the user to display information on the shape parameters of each measurement object O. When the individual data button 37 is selected, for example, the summary display shown in FIG. 8 is performed, and the shape parameters of the measurement object O selected in the summary display are displayed on the display device 3.

[0064] 9, the evaluation results are displayed when the calculation of the statistical data is completed. However, the evaluation results may be displayed at any time after the calculation of the statistical data is completed. For example, the evaluation results may be displayed in response to a user's instruction to display the evaluation results on a specific screen.

[0065] As described above, in the embodiment, an RGB-D image including irregularly shaped, numerous, and randomly arranged measurement targets, such as rocks, is measured. The measured color image is input to the area division model 151, whereby the areas of each of the irregularly shaped, numerous, and randomly arranged measurement targets are identified. Point cloud data is divided using information on the areas of each measurement target identified by the area division model 151, thereby obtaining point cloud data for each measurement target. Furthermore, a restoration process is performed on the point cloud data for each measurement target, whereby point cloud data representing the three-dimensional shape of each measurement target is restored. Three-dimensional mesh data is generated from the restored point cloud data, and the three-dimensional shape of the measurement target is evaluated based on this three-dimensional mesh data.

[0066] As described above, in the embodiment, there is no need to extract the measurement objects one by one to determine their 3D shapes, and measurement can be completed with a simple configuration and in a short time. Furthermore, the restoration process allows highly accurate 3D mesh data to be obtained for each measurement object without extracting the measurement objects one by one. Therefore, the 3D shapes of the measurement objects can be evaluated with high accuracy.

[0067] In addition, in the embodiment, the region segmentation model 151 performs entity region segmentation using a color image. For example, if the measurement object is a rock, the rock in the color image often has a different color from the ground, etc. Therefore, if the image is a color image, it is expected that the measurement object and the background can be correctly distinguished. Therefore, it is expected that the accuracy of entity region segmentation will be improved.

[0068] [Variations] A modified example of the embodiment will be described. In the embodiment, the region segmentation model 151 performs entity region segmentation using a color image. Here, in the embodiment, a depth image is also measured to generate point cloud data. Therefore, the region segmentation model 151 may perform entity region segmentation by including a depth image as an input in addition to the color image. In this case, a dataset 152 for training the region segmentation model 151 includes color images of the measurement target, teaching images for each color image, and depth images for each measurement target. The region segmentation model 151 then performs entity region segmentation on the input color image and depth image, and performs training by repeatedly updating model parameters until a functional representing the error between the result of the entity region segmentation and the input image and the corresponding teaching image converges.

[0069] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0070] 1 Measurement system, 2 Camera, 3 Display device, 11 Region segmentation unit, 12 Restoration unit, 13 Calculation unit, 14 Display control unit, 15 Database (DB), 151 Region segmentation model, 152 Data set, 201 Processor, 202 ROM, 203 RAM, 204 Storage, 205 Input interface, 206 Communication device.

Claims

1. an area dividing unit that divides at least a color image of a color image and a depth image containing a plurality of measurement targets into areas of the respective measurement targets, thereby obtaining position information of each divided area; a restoration unit that generates point cloud data from the depth image, divides the generated point cloud data into first point cloud data that are point cloud data of each measurement object based on position information of each divided area, and restores a three-dimensional shape of each measurement object from each of the first point cloud data; a calculation unit that calculates parameters for evaluating the three-dimensional shape of each of the measurement objects based on the three-dimensional shape of each of the measurement objects; A measurement system comprising:

2. the region dividing unit further divides the depth image into regions of measurement targets to obtain position information of each divided region; The measurement system of claim 1 .

3. The measurement system according to claim 1 , further comprising a display control unit that displays an evaluation result of the three-dimensional shape of the measurement object based on a parameter for evaluating the three-dimensional shape of the measurement object.

4. The evaluation results of the three-dimensional shape of the measurement objects include a histogram representing the distribution of three-axis diameters, surface area, volume, and sphericity of each measurement object. The measurement system according to claim 3 .

5. Dividing at least the color image of a color image and a depth image containing a plurality of measurement targets into regions of the respective measurement targets to obtain position information of each divided region; generating point cloud data from the depth image; Dividing the generated point cloud data into first point cloud data, which are point cloud data of each measurement target, based on position information of each divided area; Reconstructing a three-dimensional shape of each measurement object from each of the first point cloud data; calculating parameters for evaluating the three-dimensional shapes of the respective measurement objects based on the three-dimensional shapes of the respective measurement objects; A measurement program for causing a processor to execute the above.

Images