Shape recognition device and program

Abstract

To automatically determine an outer shape of a moving three-dimensional article without contact.SOLUTION: A shape recognition device comprises: obtaining means which includes a mounting surface of a moving article in an imaging range, and obtains point group data which is a group of three-dimensional coordinate values of the mounting surface and a surface of the article; coordinate system setting means which coordinate-converts xy axes of the obtaining means into an xy coordinate system along the mounting surface; and determining means which extracts point group data of the article from the point group data obtained by the obtaining means, virtualizes a convex polygon forming a contour of the article on an xy plane along the xy axes from the point group data, and determines an outer shape of the article at a ratio of an area of the convex polygon to an area of a square circumscribed so that an area of the convex polygon on the xy plane is minimized.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a shape recognition device and a program. [Background technology]

[0002] A recognition system that performs position and orientation recognition to measure the shape of a measurement object and recognize the position and orientation of the measurement object is known, and includes a measurement unit that captures a surface image of the measurement object and calculates a three-dimensional point cloud corresponding to the surface image, a model plane data acquisition unit that acquires model plane data by extracting data of planar portions from model data indicating the three-dimensional shape of the measurement object, a measurement data acquisition unit that acquires measurement data including the surface image captured by the measurement unit and the three-dimensional point cloud calculated by the measurement unit, a measurement plane data extraction unit that extracts three-dimensional coordinates of the planar portions from the three-dimensional point cloud as measurement plane data based on the surface image acquired by the measurement data acquisition unit, and a matching unit that matches the model plane data and the measurement plane data to recognize the position and orientation of the measurement object (Patent Document 1).

[0003] There is also known an identification device for identifying parts that includes a surface shape acquisition unit that acquires point cloud data of the surface of the part, a shape deviation evaluation unit that evaluates the shape deviation between the point cloud data acquired by the surface shape acquisition unit, and a shape matching unit that determines whether the part is a specific part based on the shape deviation evaluated by the shape deviation evaluation unit (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-80188 [Patent Document 2] Patent Publication No. 2021-174299 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention automatically determines the outer shape of a moving three-dimensional object in a non-contact manner. [Means for solving the problem]

[0006] In order to solve the above problem, the shape recognition device according to claim 1 comprises: an acquisition means for continuously acquiring point cloud data that includes a placement surface of a moving object in an imaging range and is a set of three-dimensional coordinate values ​​of the placement surface and a surface of the object; a coordinate system setting means for converting the x and y axes of the acquisition means into an x ​​and y coordinate system along the placement surface; A discrimination means for extracting point cloud data of the article from the point cloud data acquired by the acquisition means and discriminating an outer shape of the article, It is characterized by:

[0007] The invention described in claim 2 is the shape recognition device described in claim 1, The acquisition means projects invisible light from an array light source consisting of a plurality of light-emitting elements through a diffractive optical element as a regular dot pattern that does not overlap with each other, and receives reflected light reflected from the surface of the object and the placement surface to acquire the point cloud data. It is characterized by:

[0008] The invention described in claim 3 is the shape recognition device described in claim 1, the discrimination means detects noise by calculating variance at neighboring points of any one point of the acquired point cloud data, and removes the noise from the acquired point cloud data to extract point cloud data of the article; It is characterized by:

[0009] The invention described in claim 4 is the shape recognition device described in claim 3, The discrimination means extracts the point cloud data that is smaller than a pre-registered distance from the acquisition means to the placement surface as the point cloud data of the article. It is characterized by:

[0010] The invention described in claim 5 is the shape recognition device described in claim 4, The discrimination means extracts point cloud data of the item by removing point cloud data indicating a tray that accommodates the item on the placement surface from the point cloud data acquired by the acquisition means. It is characterized by:

[0011] The invention described in claim 6 is the shape recognition device described in claim 5, the discrimination means detects a tray containing the item by calculating a variance of the point cloud data in a z-axis direction perpendicular to the x-axis and the y-axis directions, calculates an area of ​​the tray from a size of the tray that is stored in advance, and extracts the point cloud data of an area where the item exists based on a distance to the placement surface and the area of ​​the tray. It is characterized by:

[0012] The invention described in claim 7 is the shape recognition device described in claim 4 or 5, the discrimination means imagines a convex polygon that is the contour of the article on an xy plane along the xy axes from the extracted point cloud data, estimates a circumscribing rectangular planar frame that minimizes the area of ​​the convex polygon on the xy plane, and stretches the planar frame in the z-axis direction to estimate a cubic frame that surrounds the article at a height in the z-axis direction, and discriminates the outer shape of the article based on the ratio of the area of ​​the convex polygon to the area of ​​the rectangle. It is characterized by:

[0013] The invention described in claim 8 is the shape recognition device described in claim 7, the discrimination means discriminates the outer shape of the article based on a ratio of a total height obtained by multiplying the height from the bottom surface of the cubic frame by the number of the point cloud data within the cubic frame to a total height value of the height from the bottom surface of the cubic frame of the point cloud data within the cubic frame; 8. The shape recognition device according to claim 7.

[0014] The invention described in claim 9 is the shape recognition device described in claim 4 or 5, the discrimination means discriminates the external shape of the article using a shape discrimination model generated by machine learning in which the point cloud data, which is a set of three-dimensional coordinate values ​​of the surface of the article acquired by imaging the article, is used as input data, and external shape discrimination information for discriminating the external shape of the article is used as output data; It is characterized by:

[0015] The invention described in claim 10 is the shape recognition device described in claim 4 or 5, A storage means for storing specific shape data indicating an outer shape of at least a part of the article, The discrimination means discriminates an outer shape of the article based on whether or not the specific shape data is included in the extracted point cloud data. 6. The shape recognition device according to claim 4 or 5.

[0016] The invention described in claim 11 is the shape recognition device described in claim 1, The discrimination means discriminates whether the article is of a regular shape or an irregular shape based on an external shape of the article. It is characterized by:

[0017] The invention described in claim 12 is the shape recognition device described in claim 1, the acquiring means acquires the point cloud data by capturing an image of the article placed on the moving placement surface from directly above the article. It is characterized by:

[0018] In order to solve the above problem, the program according to claim 13 comprises: On the computer, an acquisition step of continuously acquiring point cloud data that includes a placement surface of a moving object in an imaging range and is a set of three-dimensional coordinate values ​​of the placement surface and a surface of the object; a coordinate system setting step of converting the x and y axes of the acquiring step into an x ​​and y coordinate system along the placement surface; a determination step of extracting point cloud data of the article from the point cloud data acquired by the acquisition step and determining an outer shape of the article; It is characterized by: Effect of the Invention

[0019] According to the invention as set forth in claims 1 and 13, the external shape of a three-dimensional object can be determined in a non-contact manner.

[0020] According to the invention as recited in claim 2, the external shape of a three-dimensional object can be acquired at high speed as point cloud data.

[0021] According to the invention as recited in claim 3, noise can be removed from the acquired point cloud data.

[0022] According to the invention as recited in claim 4, it is possible to extract point cloud data of only the article from the acquired point cloud data.

[0023] According to the invention as recited in claim 5, it is possible to extract point cloud data of only the item to be identified from the acquired point cloud data.

[0024] According to the invention as recited in claim 6, it is possible to extract point cloud data of the article to be discriminated from the acquired point cloud data.

[0025] According to the seventh aspect of the invention, the external shape of a three-dimensional object can be determined in a non-contact manner.

[0026] According to the invention as recited in claim 8, the external shape of a three-dimensional object can be determined including the height direction.

[0027] According to the invention as recited in claims 9 and 10, the external shape of a three-dimensional object can be directly determined from the acquired point cloud data.

[0028] According to the invention described in claim 11, articles can be sorted into shaped articles and shapeless articles.

[0029] According to the invention as recited in claim 12, the external shape of an article moving on a conveyor can be determined in a non-contact manner. [Brief description of the drawings]

[0030] [Figure 1] 1 is a block diagram showing a functional configuration of a shape recognition device according to a first embodiment. [Diagram 2] FIG. 2A is a schematic plan view showing an example of the configuration of a three-dimensional sensor, and FIG. 2B is a schematic cross-sectional view. [Diagram 3] FIG. 13 is a schematic diagram showing dot pattern irradiation of a three-dimensional sensor. [Figure 4] 1A and 1B are diagrams illustrating the observation of a laser dot in a three-dimensional sensor. [Diagram 5] FIG. 1 is a schematic diagram illustrating an example of the configuration of a shape recognition device. [Figure 6] 10 is a flowchart showing the flow of a process for determining the external shape of an object in the shape recognition device. FIG. [Figure 7] FIG. 13 is a conceptual diagram illustrating a tray data removal process. [Figure 8] FIG. 1 is a conceptual diagram illustrating a process for removing noise from point cloud data; [Figure 9] 10A and 10B are schematic diagrams illustrating a process flow for determining the outer shape of an object from point cloud data of only an area in which the object exists. [Figure 10] FIG. 2 is a diagram showing the relationship between a convex polygon that forms the outline of an article and a circumscribing rectangle. [Figure 11] 11 is a diagram illustrating an example of calculating the ratio of the area of ​​a convex polygon to the area of ​​a circumscribing rectangle. FIG. [Figure 12] FIG. 11 is a flowchart showing the flow of a process for determining the external shape of an object in a shape recognition device according to a modified example. [Figure 13] 13A to 13C are schematic diagrams illustrating the flow of processing in a modified example in which the outer shape of an article is determined from point cloud data of only an area in which the article exists. [Figure 14] FIG. 11 is a block diagram showing a functional configuration of a shape recognition device according to a second embodiment. [Figure 15]13A to 13C are diagrams for explaining a method for determining the external shape of an object in the shape recognition device according to the second embodiment. [Figure 16] FIG. 11 is a block diagram showing a functional configuration of a shape recognition device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Next, specific examples of the embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. In the following explanation using the drawings, it should be noted that the drawings are schematic and the ratios of the dimensions, etc. may differ from the actual ones, and in order to facilitate understanding, illustrations other than those of components necessary for the explanation have been omitted as appropriate.

[0032] "First embodiment" (1) Overall configuration of the measuring device Figure 1 is a block diagram showing the functional configuration of the shape recognition device 1 in this embodiment, Figure 2(a) is a schematic plan view showing an example of the configuration of a three-dimensional sensor 10, (b) is a schematic cross-sectional view, Figure 3 is a schematic view showing the irradiation of a dot pattern by the three-dimensional sensor 10, Figure 4 is a diagram explaining the observation of laser dots in the three-dimensional sensor 10, and Figure 5 is a schematic view showing an example of the configuration of the shape recognition device 1. The overall configuration of the shape recognition device 1 will be described below with reference to the drawings.

[0033] As shown in FIG. 1, the shape recognition device 1 has a three-dimensional sensor 10 as an example of an acquisition means for continuously acquiring point cloud data P, which is a collection of three-dimensional coordinate values ​​of the surface of a three-dimensional object (hereinafter simply referred to as object) G, which is the object to be identified, and a shape discrimination unit 20 as an example of a discrimination means for extracting point cloud data P of the object G from the point cloud data P acquired by the three-dimensional sensor 10 and discriminating the external shape of the object G.

[0034] The three-dimensional sensor 10 includes a projector 11 consisting of a light-emitting element that emits light, an image sensor module 12 consisting of an image sensor that captures light reflected from the surface of an object G, and a control unit 13 that controls the operation of the projector 11 and the image sensor module 12 and acquires point cloud data P. In this embodiment, the control unit 13 can be, for example, a program processing device implemented as a Raspberry Pi (registered trademark), which is a single-board computer equipped with an ARM processor, or a small computer device, but is not particularly limited thereto.

[0035] As shown in FIG. 2, the projector 11 is composed of a VCSEL (Vertical Cavity Surface Emitting LASER) 111, which is an array light source consisting of multiple light-emitting elements, and a transmissive diffractive optical element (DOE) 112 that controls the distribution of laser light above the VCSEL 111. The diffractive optical element 112 converts the light emitted by the VCSEL 111 into a regular dot pattern of light distribution, and irradiates the light as multiple beams of distance measuring light onto the article G, which is the object to be identified, without overlapping each other.

[0036] 3 shows an example in which light emitted from the projector 11 is converted into distance measurement light L1 to Ln having a spread angle θ1 so as to be distributed as a regular dot pattern on the object G, which is an object to be identified, by the diffractive optical element 112. In this embodiment, the dot pattern is irradiated with 4000 to 5000 laser beams.

[0037] The image sensor module 12 uses a CMOS (Complementary Metal Oxide Semiconductor) image sensor as a photoelectric conversion element. The image sensor module 12 captures the light emitted from the projector 11 and reflected by the object G, acquires it as point cloud data P, and outputs it to the shape determination unit 20 via the control unit 13.

[0038] Fig. 4 shows a schematic example of the trajectories of each laser dot (P0, Pn, Pf) observed on the image sensor module 12 in the distance measurement range. As shown in Fig. 4, the regular dot patterns are arranged so as not to overlap each other in the distance measurement range. Therefore, if the reflected light of each laser dot (P0, Pn, Pf) irradiated to the object G is received by the image sensor module 12 and the position can be detected by the three-dimensional sensor 10, the trajectories of each laser dot (P0, Pn, Pf) on the image sensor module 12 in the distance measurement range can be calculated as three-dimensional coordinate values ​​by referring to a lookup table (LUT) stored in advance.

[0039] The three-dimensional sensor 10 configured in this manner emits laser light from the projector 11 onto the object G, captures the light reflected from the surface of the object G with the image sensor module 12, and acquires point cloud data P that indicates the three-dimensional shape of the object to be identified in three-dimensional coordinate values.

[0040] The shape determination unit 20 has a processor that includes a CPU (Central Processing Unit) that provides various management functions through the execution of programs, a ROM (Read Only Memory) that serves as a storage area for storing a BIOS (Basic Input Output System) and the like, and a RAM (Random Access Memory) that is used as an execution area for the programs. The programs include firmware and an operating system (OS). The shape discrimination unit 20 executes processes such as discrimination and display of the external shape using the point cloud data P acquired by the three-dimensional sensor 10 through execution of a program.

[0041] As shown in FIG. 5, the shape recognition device 1 according to this embodiment can be configured to include a three-dimensional sensor 10, a conveyor 30 to which the three-dimensional sensor 10 is attached, and a terminal device 40 including a shape determination unit 20.

[0042] The conveyor 30 is a transport conveyor whose conveyor surface as a placement surface F moves at a predetermined speed, and distinguishes the external shape of the articles G and distributes them during the transport process. A three-dimensional sensor 10 is disposed midway along the conveyor 30 in the transport direction, and is capable of capturing an image of the articles G being transported from above. The conveyor 30 transports the articles G placed on the conveyor surface downstream in the transport direction.

[0043] The terminal device 40 is connected to the three-dimensional sensor 10 by wire or wirelessly, and is capable of communicating with the three-dimensional sensor 10. The terminal device 40 may have a communication interface or the like for such a connection. The terminal device 40 may be what is called a mobile PC (Personal Computer), or may be a device such as a stationary PC or a mobile terminal (smartphone).

[0044] (2) External shape discrimination processing FIG. 6 is a flow chart showing the flow of processing for determining the external shape of an object G in the shape recognition device 1. The shape discrimination unit 20 first turns on the three-dimensional sensor 10 in step S101, and irradiates the placement surface F with laser light from directly above while no object G is placed on the placement surface F (S101) to set an xy coordinate system along the placement surface F. Thousands of laser beams are irradiated simultaneously as a regular dot pattern that does not overlap with each other. The laser beam is always irradiated at regular intervals while the external shape of the object G is being discriminated, and the laser beam irradiated as aligned dots toward the placement surface F is reflected by the placement surface F and the surface of the object G on the placement surface F, and is captured by the image sensor module 12 to be acquired as point cloud data P.

[0045] Then, in step S102, coordinate conversion is performed to an xy coordinate system so that the z axis of 3D sensor 10 is perpendicular to placement surface F and the xy axes of 3D sensor 10 are horizontal to placement surface F (S102). This corrects the tilt when an image of item G placed on placement surface F is captured from directly above to obtain point cloud data P of item G. Here, placement surface F is a conveyor surface that moves at a constant speed and is at least planar, and directly above refers to a position above the reference surface of the conveyor.

[0046] Next, in step S103, it is determined whether the point cloud data P includes point cloud data P that is smaller than a pre-registered distance H0 from the three-dimensional sensor 10 to the placement surface F (S103). If point cloud data P that is smaller than the distance H0 is included (S103; Yes), the point cloud data P acquired as a discrimination trigger is stored (S104). If point cloud data P that is smaller than the distance H0 is not included (S103; No), the point cloud data P is not stored and the laser light continues to be irradiated (S101).

[0047] FIG. 7 is a conceptual diagram for explaining the tray data removal process. In the shape recognition device 1 of this embodiment, it is possible to pre-select between a tray-with mode in which discrimination is performed when an item G is contained in a tray T, and a tray-less mode in which discrimination is performed when an item G is not contained in a tray but is placed directly on the placement surface F. The shape discrimination unit 20 determines whether or not the tray-present mode has been selected in step S105 (S105), and if the tray-present mode has been selected (S105; Yes), removes the point cloud data P indicating the tray T from the point cloud data P stored in step S104, and extracts the point cloud data P of only the area where the item G is present (S106).

[0048] Specifically, as shown in FIG. 7(a), it is determined whether or not a tray-present mode in which an item G is contained in a tray T is selected (S105). If the tray-present mode is selected (S105; Yes), several tens of points are extracted from the stored point cloud data P in each of the x and y axes, and a variance σ1 in the z-axis direction perpendicular to the x and y axes is calculated. If the calculated variance σ1 in the z-axis direction is greater than a predetermined value, it is estimated that two side surfaces of the tray T in the x-axis direction and the y-axis direction have been detected. Then, the area of ​​the tray T is calculated from the size of the tray T registered in advance. Then, as shown in FIG. 7(b), the point cloud data P indicating the calculated area of ​​the tray T is removed from the stored point cloud data P, and point cloud data P of only the area where the item G exists is extracted using the distance to the placement surface F (S106).

[0049] FIG. 8 is a conceptual diagram illustrating the process of removing noise from the point cloud data P. In FIG. In step S107, the shape discrimination unit 20 removes noise from the point cloud data P from which the point cloud data P indicating the tray T has been removed, and extracts the point cloud data P of the article G (S107). Also, if it is determined in step S105 that there is no tray T (S105; No), the shape discrimination unit 20 removes noise from the point cloud data P stored in step S104, and extracts the point cloud data P of the article G (S107). In step S107, the shape determination unit 20 removes noise from the stored point cloud data P and extracts point cloud data P of only the article G (S107).

[0050] 8, the variance σ2 of neighboring points of the point cloud data that are outside a virtual sphere with a radius of a predetermined threshold value centered on any one point Pn in the acquired point cloud data P is calculated, and if the variance σ2 is greater than a predetermined value, it is determined to be noise and removed from the point cloud data P. This process is repeatedly performed on each point of the point cloud data to remove noise from the point cloud data P and extract the point cloud data P of the article G (S107). In addition, noise can be detected by calculating statistical outliers, where the average distance from any point Pn in the acquired point cloud data P to nearby points is far from the average of the entire point cloud data, and the noise can be removed from the acquired point cloud data P, or this process can be performed additionally.

[0051] In step S108, the shape determination unit 20 determines the outer shape of the article G from the point cloud data P of only the area where the article G exists, from which noise has been removed. 9A and 9B are schematic diagrams showing a process flow for determining the outer shape of an article G from point cloud data P (shown in FIG. 9A) of only the area where the article G exists. First, as shown in FIG. 9B, the shape determination unit 20 sets a convex polygon 100 that is the contour of the article G in an xy plane along the xy axes from the point cloud data P of only the area where the article G exists (S108). Then, as shown in FIG. 9C, the planar frame of a circumscribing rectangle 110 that circumscribes the convex polygon 100 is stretched in the z-axis direction so that the area of ​​the convex polygon 100 in the xy plane is minimized, and a cubic frame 120 that surrounds the article G at a height in the z-axis direction is imagined (S109).

[0052] FIG. 10 shows the relationship between a convex polygon 100 that forms the outline of an article G and a circumscribing rectangle 110, and FIG. 11 shows an example of calculating the ratio R1 of the area of ​​the convex polygon 100 to the circumscribing rectangle 110. Following step S109, as shown in FIG. 10, the area S1 of the convex polygon 100 and the area S2 of the circumscribed rectangle 110 are calculated to calculate the ratio R1 of the area of ​​the convex polygon 100 to the circumscribed rectangle 110 (S110). Then, in step S111, it is determined whether the area ratio R1 is greater than a predetermined threshold value Th1 (S111). As shown in FIG. 11(a), if the area ratio R1 is greater than the threshold value Th1 in step S111 (S111; Yes), the outer shape of the article G is determined to be a shaped body, specifically a rectangular parallelepiped (S112), and as shown in FIG. 11(b), if the area ratio R1 is less than the threshold value Th1 (S110; No), the outer shape of the article G is determined to be an indefinite body (S112). In this embodiment, the threshold value Th1 is set to 0.9. In other words, if the area of ​​the convex polygon 100 that defines the outline of the item G accounts for 90% or more of the area of ​​the circumscribed rectangle 110 that circumscribes the convex polygon 100, the item is presumed to be a rectangular prism, and is sorted by distinguishing it from amorphous objects other than rectangular prisms.

[0053] "Variations" Fig. 12 is a flowchart showing the flow of processing for determining the external shape of an object G in a shape recognition device 1 according to a modified example, and Fig. 13 shows a schematic flow of processing of a modified example for determining the external shape of an object G from point cloud data P (shown in Fig. 13(a)) of only the area in which the object G exists. In the flowchart of Fig. 12, the processing up to step S209 in which a cubic frame 120 surrounding the object G is imagined is similar to the processing from step S101 to step S109 described above, and therefore description thereof will be omitted.

[0054] The shape discrimination unit 20 discriminates the external shape of the object G based on the ratio R2 of the total height H2 obtained by multiplying the height Hr from the bottom surface 110A of the cubic frame 120 by the number N of point cloud data P within the cubic frame 120 to the total height H1 of the heights Hn from the bottom surface 110A of the cubic frame 120 of the point cloud data P within the cubic frame 120 surrounding the object G.

[0055] Specifically, as shown diagrammatically in FIG. 13(b), in step S210, the heights Hn of each point Pn of the point cloud data P within the cubic frame 120 are summed (S210), thereby obtaining data for determining the heightwise shape of the object G enclosed by the cubic frame 120, even if the object G has planes with different heights. Next, as shown in FIG. 13(c), in step S211, the height Hr from the bottom surface 110A of the cubic frame 120 is multiplied by the number N of point cloud data P within the cubic frame 120 to calculate a total height value H2 (S211), which is the height data for when the objects G surrounded by the cubic frame 120 are cubes of the same height.

[0056] Then, in step 212, the shape discrimination unit 20 calculates the ratio R2 of the sum H1 of the heights Hn to the sum H2 of the heights Hr (S212), and determines whether the ratio R2 is greater than a predetermined threshold Th2 (S213). If the ratio R2 is greater than the threshold Th2 (S213; Yes), the item G is determined to be a shaped body having an outer shape with a constant height, specifically a rectangular parallelepiped (S214), and if the ratio R2 is less than the threshold Th2 (S213; No), the item G is determined to be an indefinitely shaped body having an outer shape made up of planes with different heights (S215). Here, the threshold Th2 can be set based on the sorting criteria for the items G.

[0057] Through such processing, point cloud data P of the object G is extracted from the point cloud data acquired by the three-dimensional sensor 10, and it is determined whether the outer shape of the object G is regular or irregular. This makes it possible to automatically determine the outer shape of a three-dimensional object without contact. In particular, it is possible to automatically and continuously determine the outer shape of the object G moving on the conveyor without contact, and to sort the object G into regular and irregular objects.

[0058] "Second embodiment" FIG. 14 is a block diagram showing the functional configuration of a shape recognition device 1A according to this embodiment. The second embodiment will be described with reference to Figure 14, focusing on the differences from the first embodiment. However, the example described in the first embodiment can be applied to the three-dimensional sensor 10 as an example of an acquisition means for continuously acquiring point cloud data, which is a collection of three-dimensional coordinate values ​​of the surface of the three-dimensional object G, which is the object to be determined.

[0059] As shown in Figure 14, the shape recognition device 1A has a three-dimensional sensor 10 that continuously acquires point cloud data, which is a collection of three-dimensional coordinate values ​​of the surface of the object G to be discriminated, and a shape discrimination unit 20A that discriminates the external shape of the object G using a shape discrimination model S generated by machine learning in which point cloud data P, which is a collection of three-dimensional coordinate values ​​of the surface of the object G acquired by imaging the object G, is used as input data, and shape discrimination information of the object G corresponding to this input data is used as output data.

[0060] The shape discrimination unit 20A includes a database unit 21 and a machine learning unit 22. The database unit 21 accumulates a data set of shape discrimination information of the article G associated with the point cloud data, together with point cloud data, which is a collection of three-dimensional coordinate values ​​of the surface of the article G. The shape discrimination information of the article G associated with the point cloud data is information that can identify the outer shape classification of the article G, including a rectangular parallelepiped shape, a rectangular parallelepiped shape with a stepped surface, a circular shape, a flat shape, a rod-like shape, and a composite shape. In this embodiment, the shape discrimination information of the article G focuses on a fixed shape, and preferably includes classifications of a rectangular parallelepiped shape and a rectangular parallelepiped shape with a stepped surface. This is because the shape recognition device 1A discriminates and assigns the outer shape of the article G on the conveyor 30, whose conveyor surface moves at a predetermined speed.

[0061] The machine learning unit 22 uses the data set accumulated in the database unit 21 to generate a shape discrimination model S for discriminating the outer shape of the article G by machine learning using a plurality of learning data, in which point cloud data P, which is a set of three-dimensional coordinate values ​​of the surface of the article G, is used as input data, and shape discrimination information of the article G corresponding to the input data is used as output data. The type of machine learning (algorithm) for generating the shape discrimination model S is not particularly limited as long as it can obtain sufficient discrimination accuracy of the outer shape for practical use. For example, a commonly used neural network (including deep learning, convolutional neural network, etc.), decision tree learning, random forest, support vector regression, etc. may be used.

[0062] FIG. 15 is a diagram for explaining a method for determining the external shape of an object G in the shape recognition device 1A. First, the shape discrimination unit 20A removes noise from the point cloud data P, which is a collection of three-dimensional coordinate values ​​of the surface of the object G acquired by the three-dimensional sensor 10 installed on the conveyor 30, and extracts the point cloud data P of only the object G. Then, the shape discrimination unit 20A inputs the extracted point cloud data P as input data to the shape discrimination model S, and outputs shape discrimination information for discriminating the external shape of the article G as output data.

[0063] The shape discrimination information of the article G output from the shape discrimination model S is displayed, for example, on a display unit 41 (display) of the terminal device 40. By displaying the shape discrimination information of the article G on the display unit 41, the article G moving on the placement surface F of the conveyor 30 can be sorted into shapeless and shapeless articles.

[0064] In this way, by inputting the extracted point cloud data P into the shape discrimination model S as input data and discriminating the outer shape of the article G, which is the output data, the outer shape of the article G can be directly discriminated from the point cloud data P that acquires the outer shape of the article G.

[0065] "Third embodiment" FIG. 16 is a block diagram showing the functional configuration of a shape recognition device 1B according to this embodiment. The third embodiment will be described with reference to Figure 16, focusing on the differences from the first embodiment. However, the example described in the first embodiment can be applied to the three-dimensional sensor 10 as an example of an acquisition means for continuously acquiring point cloud data P, which is a collection of three-dimensional coordinate values ​​of the surface of the object G to be determined.

[0066] As shown in Figure 16, the shape recognition device 1B has a three-dimensional sensor 10 that continuously acquires point cloud data P, which is a collection of three-dimensional coordinate values ​​of the surface of the object G to be discriminated, a memory unit 50 that stores specific shape data indicating the external shape of at least a part of the object G, and a shape discrimination unit 20B that discriminates the external shape of the object G based on whether or not the extracted point cloud data P contains the specific external shape data.

[0067] The specific shape data stored in the storage unit 50 is three-dimensional point cloud data indicating the outer shape classification of the article G, including a rectangular parallelepiped shape, a shape having a stepped surface, a circular shape, a flat shape, a rod-like shape, and a composite shape. In this embodiment, focusing on a fixed shape, it is preferable that the specific shape data of the article G includes classifications of a rectangular parallelepiped shape and a rectangular parallelepiped shape having a stepped surface.

[0068] First, the shape discrimination unit 20B removes noise from the point cloud data P, which is a collection of three-dimensional coordinate values ​​of the surface of the object G acquired by the three-dimensional sensor 10 installed on the conveyor 30, and extracts the point cloud data P of only the object G.

[0069] Then, the shape discrimination unit 20B determines whether the point cloud data P of only the item G generated by the three-dimensional sensor 10 has a specific shape based on the specific shape data stored in the memory unit 50. Specifically, it is determined whether the shape is a rectangular parallelepiped. Whether the shape is the specific shape is determined based on whether the difference between the stored specific shape data and the three-dimensional coordinate information value generated by the three-dimensional sensor 10 is smaller than a predetermined threshold value.

[0070] When it is determined that the object has a specific shape, for example, the display unit 41 (display) of the terminal device 40 displays a message indicating that the object has a specific shape together with the point cloud data P of only the object G. By displaying the specific shape of the object G on the display unit 41, the object G moving on the placement surface F of the conveyor 30 can be sorted into object with a fixed shape and object with an irregular shape.

[0071] In this way, by determining whether the extracted point cloud data P contains specific outer shape data, the outer shape of the article G can be directly determined from the point cloud data P that acquires the outer shape of the article G. [Explanation of symbols]

[0072] 1, 1A, 1B...shape recognition device 10. 3D sensor 11...Projector, 111...VCSEL, 112...Diffractive optical element 12. Image sensor module 13. Control section 20, 20A, 20B...shape discrimination section 30...Conveyor 40 Terminal device F...Placement surface G...Goods P...Point cloud data

Claims

1. an acquisition means for continuously acquiring point cloud data that includes a placement surface of a moving object in an imaging range and is a set of three-dimensional coordinate values ​​of the placement surface and a surface of the object; a coordinate system setting means for converting the x and y axes of the acquisition means into an x ​​and y coordinate system along the placement surface; A discrimination means for extracting point cloud data of the article from the point cloud data acquired by the acquisition means and discriminating an outer shape of the article, A shape recognition device comprising:

2. The acquisition means projects invisible light from an array light source consisting of a plurality of light-emitting elements through a diffractive optical element as a regular dot pattern that does not overlap with each other, and receives reflected light reflected from the surface of the object and the placement surface to acquire the point cloud data.

2. The shape recognition device according to claim 1.

3. the discrimination means detects noise by calculating variance at neighboring points of any one point of the acquired point cloud data, and removes the noise from the acquired point cloud data to extract point cloud data of the article; 2. The shape recognition device according to claim 1.

4. The discrimination means extracts the point cloud data that is smaller than a pre-registered distance from the acquisition means to the placement surface as the point cloud data of the article.

4. The shape recognition device according to claim 3.

5. The discrimination means extracts point cloud data of the item by removing point cloud data indicating a tray that accommodates the item on the placement surface from the point cloud data acquired by the acquisition means.

5. The shape recognition device according to claim 4.

6. the discrimination means detects a tray containing the item by calculating a variance of the point cloud data in a z-axis direction perpendicular to the x-axis and y-axis directions, calculates an area of ​​the tray from a size of the tray that is stored in advance, and extracts the point cloud data of an area where the item exists based on a distance to the placement surface and the area of ​​the tray.

6. The shape recognition device according to claim 5.

7. the discrimination means imagines a convex polygon that is the contour of the article on an xy plane along the xy axes from the extracted point cloud data, estimates a circumscribing rectangular planar frame that minimizes the area of ​​the convex polygon on the xy plane, and stretches the planar frame in the z-axis direction to estimate a cubic frame that surrounds the article at a height in the z-axis direction, and discriminates the outer shape of the article based on the ratio of the area of ​​the convex polygon to the area of ​​the rectangle.

6. The shape recognition device according to claim 4 or 5.

8. the discrimination means discriminates the outer shape of the article based on a ratio of a total height obtained by multiplying the height from the bottom surface of the cubic frame by the number of the point cloud data within the cubic frame to a total height value of the height from the bottom surface of the cubic frame of the point cloud data within the cubic frame; 8. The shape recognition device according to claim 7.

9. the discrimination means discriminates the external shape of the article using a shape discrimination model generated by machine learning in which the point cloud data, which is a set of three-dimensional coordinate values ​​of the surface of the article acquired by imaging the article, is used as input data, and external shape discrimination information for discriminating the external shape of the article is used as output data; 6. The shape recognition device according to claim 4 or 5.

10. A storage means for storing specific shape data indicating an outer shape of at least a part of the article, The discrimination means discriminates an outer shape of the article based on whether or not the specific shape data is included in the extracted point cloud data.

6. The shape recognition device according to claim 4 or 5.

11. The discrimination means discriminates whether the article is of a regular shape or an irregular shape based on an external shape of the article.

2. The shape recognition device according to claim 1.

12. the acquiring means acquires the point cloud data by capturing an image of the article placed on the moving placement surface from directly above the article.

2. The shape recognition device according to claim 1.

13. On the computer, an acquisition step of continuously acquiring point cloud data that includes a placement surface of a moving object in an imaging range and is a set of three-dimensional coordinate values ​​of the placement surface and a surface of the object; a coordinate system setting step of converting the x and y axes of the acquiring step into an x ​​and y coordinate system along the placement surface; a determination step of extracting point cloud data of the article from the point cloud data acquired by the acquisition step and determining an outer shape of the article; A program characterized by:

Images