Material sorting identification method and system

Abstract

Embodiments of the invention provide a material sorting identification method and system. The method comprises the steps of obtaining initial point cloud data of a target material; performing first preprocessing based on the initial point cloud data to obtain target point cloud data; determining a first plane image based on the target point cloud data; performing second preprocessing on the first plane image to obtain a second plane image; performing convolution in at least one direction on the second plane image through at least one preset operator to obtain at least one image after convolution; generating a third plane image based on the at least one convoluted image; and performing image analysis based on the third plane image to determine the position of the target material. The system comprises an acquisition module, a preprocessing module, a convolution module and a position determination module.

Description

Technical Field

[0001] The present invention relates to the field of sorting and identification technology, and in particular to a material sorting and identification method and system. Background Art

[0002] Usually, material positioning uses an image processing library on a two-dimensional plane to perform two-dimensional coordinate positioning, identify lighting environment requirements, and have poor stability. In complex environments, such as low light or cluttered backgrounds, accurately determining the location of the target material may be limited. In the case of target changes or occlusion, it will lead to difficulties in identification.

[0003] Therefore, it is hoped that a technical solution can be provided to improve the stability and accuracy of material identification. Summary of the invention

[0004] One or more embodiments of the present specification provide a material sorting and identification method, including: acquiring initial point cloud data of a target material; performing a first preprocessing based on the initial point cloud data to obtain target point cloud data; determining a first plane image based on the target point cloud data; performing a second preprocessing on the first plane image to obtain a second plane image; performing convolution on the second plane image in at least one direction using at least one preset operator to obtain at least one convolved image; generating a third plane image based on the at least one convolved image; and performing image analysis based on the third plane image to determine the position of the target material.

[0005] One or more embodiments of the present specification provide a system for material sorting and identification, the system comprising: an acquisition module, a preprocessing module, a convolution module, and a position determination module. The acquisition module is configured to acquire the initial point cloud data of the target material; the preprocessing module is configured to: perform a first preprocessing based on the initial point cloud data to obtain the target point cloud data; determine a first plane image based on the target point cloud data; perform a second preprocessing on the first plane image to obtain a second plane image; the convolution module is configured to: perform a convolution on the second plane image in at least one direction through at least one preset operator to obtain at least one convolved image; generate a third plane image based on the at least one convolved image; the position determination module is configured to perform image analysis based on the third plane image to determine the position of the target material.

[0006] One or more embodiments of the present specification provide a computer-readable storage medium that stores computer instructions, and when the computer instructions are executed by a computer, a material sorting and identification method is implemented.

[0007] One or more embodiments of the present specification provide a sorting and identification device, wherein the device includes a processor, and the at least one processor is used to execute the computer instructions to implement a material sorting and identification method. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0009] Figure 1 is an exemplary module diagram of a material sorting and identification system according to some embodiments of this specification;

[0010] Figure 2 is an exemplary flow chart of a material sorting and identification method according to some embodiments of this specification;

[0011] Figure 3 is an exemplary schematic diagram of determining the first area and the second area according to some embodiments of this specification;

[0012] Figure 4 a-4d are exemplary schematic diagrams of images after convolution in four directions according to some embodiments of this specification;

[0013] Figure 5 Yes Figure 4 a-4d are exemplary schematic diagrams of the third plane image obtained by synthesis;

[0014] Figure 6 is an exemplary flow chart of determining a first plane image according to some embodiments of this specification;

[0015] Figure 7 is an exemplary flow chart for determining the location of a target material according to some embodiments of the present specification. DETAILED DESCRIPTION

[0016] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0017] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0018] As shown in this specification, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0019] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0020] Figure 1 It is an exemplary module diagram of a material sorting and identification system according to some embodiments of this specification.

[0021] In some embodiments, Figure 1 As shown, the material sorting and identification system 100 may include an acquisition module 110 , a preprocessing module 120 , a convolution module 130 and a position determination module 140 .

[0022] In some embodiments, the acquisition module 110 is configured to acquire initial point cloud data of the target material.

[0023] In some embodiments, the preprocessing module 120 is configured to: perform a first preprocessing based on the initial point cloud data to obtain target point cloud data; determine a first plane image based on the target point cloud data; and perform a second preprocessing on the first plane image to obtain a second plane image.

[0024] In some embodiments, the preprocessing module 120 is further configured to: fit the target point cloud data to obtain a target plane; determine a mapping image of the target point cloud data on the target plane based on the three-dimensional coordinate information of the target point cloud data; and determine a first plane image based on the mapping image.

[0025] In some embodiments, the second pre-processing includes at least one of a mean filtering process, a binarization process, and a closing operation process.

[0026] In some embodiments, the convolution module 130 is configured to: perform convolution on the second planar image in at least one direction using at least one preset operator to obtain at least one convolved image; and generate a third planar image based on the at least one convolved image.

[0027] In some embodiments, the location determination module 140 is configured to determine the location of the target material by performing image analysis based on the third plane image.

[0028] In some embodiments, the position determination module 140 is further configured to: determine at least one candidate contour in the third plane image; filter out the candidate contours whose contour features do not meet the preset contour conditions in at least one candidate contour to obtain the target material contour; perform vertex extraction based on the target material contour to determine the vertex position of the target material.

[0029] For details about the above embodiments, please refer to Figure 2-Figure 4 And related instructions.

[0030] In some embodiments, the acquisition module 110, preprocessing module 120, convolution module 130 and location determination module 140 of the material sorting and identification system 100 can be integrated into a processor, and the processor is configured to process information and / or data related to the material sorting and identification system 100. The processor can also process data obtained from other devices, such as user terminals, which may include personal computers, mobile devices, etc.

[0031] In some embodiments, the processor can be configured as a combination of one or more of a microcontroller (MCU), an embedded processor (EmbeddedProcessor), a graphics processing unit (GPU), etc. It should be noted that the above description of the material sorting and identification system 100 and its modules is only for the convenience of description and does not limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from this principle. In some embodiments, Figure 1 The acquisition module 110, preprocessing module 120, convolution module 130 and position determination module 140 disclosed in the specification can be different modules in a system, or a module can realize the functions of two or more modules mentioned above. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the protection scope of this specification.

[0032] Figure 2200 is an exemplary flow chart of a material sorting and identification method according to some embodiments of this specification. Figure 2 As shown, the process 200 includes the following steps 210 to 270. In some embodiments, the process 200 may be executed by a processor.

[0033] Step 210, obtaining initial point cloud data of the target material.

[0034] Target materials refer to materials to be identified, such as workpieces and parts to be identified.

[0035] The initial point cloud data is a data set composed of point cloud data of multiple points. The point cloud data may include the three-dimensional coordinates, grayscale values ​​or other attribute information of the points.

[0036] In some embodiments, the processor may acquire initial point cloud data acquired by an image acquisition device. The image acquisition device may be a 3D camera, etc. The initial point cloud data acquired by the 3D camera includes all point cloud data within the field of view.

[0037] Step 220: Perform a first preprocessing based on the initial point cloud data to obtain target point cloud data.

[0038] Target point cloud data refers to the point cloud data corresponding to the target material.

[0039] The first preprocessing is a method for processing the initial point cloud data. For example, the first preprocessing may include at least one of filtering, downsampling, segmentation, feature extraction, etc. Filtering can remove noise and smooth the point cloud data. The filtering methods include Gaussian filtering, median filtering, bilateral filtering, etc. Segmentation refers to dividing the initial point cloud data according to different objects or regions. Segmentation can be performed by clustering, regional growing, and other methods. In some embodiments, the processor may segment the initial point cloud data into point cloud data of multiple regions; and apply at least one filtering method to the point cloud data of multiple regions. Through filtering and segmentation techniques, noise and irrelevant point cloud data are removed, and the part related to the target material is retained.

[0040] Downsampling can reduce the amount of point cloud data and simplify the complexity of data processing. Downsampling can be performed through voxel grid, nearest neighbor or random sampling.

[0041] Feature extraction refers to extracting characteristic parameters related to the target material, such as shape, color, texture, etc. In some embodiments, the processor can extract the characteristic parameters based on the initial point cloud data through a preset algorithm; and use the initial point cloud data corresponding to the characteristic parameters that meet the preset conditions as the target point cloud data.

[0042] The preset condition may be that the similarity between the feature parameter and the initial feature parameter is less than a preset similarity threshold. The initial feature parameter may be manually set based on experience, and the similarity is positively correlated with the difference between the feature parameter and the initial feature parameter. If the target material includes multiple objects, different initial feature parameters may be set for different categories of target materials.

[0043] Step 230: Determine a first plane image based on the target point cloud data.

[0044] The first plane image refers to a plane mapping image of the target material.

[0045] In some embodiments, the processor may determine the first planar image by a variety of methods.

[0046] For example, the processor may map the three-dimensional coordinates of the target point cloud data onto the initial plane to obtain the two-dimensional coordinates and grayscale values ​​of the first plane image. The initial plane is a pre-set plane, and the initial plane may be manually set based on experience.

[0047] Exemplarily, the three-dimensional coordinates of the target point cloud data include X-axis coordinates, Y-axis coordinates, and Z-axis coordinates. The processor can map the X-axis coordinates and Y-axis coordinates to the initial plane as the horizontal coordinates and vertical coordinates of the corresponding points on the first plane image; and map the height data of the Z-axis coordinates to the grayscale value of the corresponding point on the first plane image. The larger the Z-axis coordinate, the larger the grayscale value, and the brighter it is presented on the first plane image.

[0048] The information of the target material on the same plane will be displayed in the first plane image in the form of grayscale, and the shape of the material can be preliminarily observed by the naked eye. The position information of the point cloud of the target material is represented by the coordinate position of the pixel on the first plane image.

[0049] In some embodiments, the processor may fit the target point cloud data to obtain a target plane; determine a mapping image of the target point cloud data on the target plane based on the three-dimensional coordinate information of the target point cloud data; and determine a first plane image based on the mapping image. Figure 2 And related instructions.

[0050] Step 240: Perform a second preprocessing on the first planar image to obtain a second planar image.

[0051] The second pre-processing is to make the edge of the target material clear.

[0052] In some embodiments, the second pre-processing includes at least one of a mean filtering process, a binarization process, and a closing operation process.

[0053] Mean filtering refers to a method of smoothing an image. For example, for a pixel in the first plane image, the processor may use the grayscale average value of the pixels within a preset range around the pixel as the grayscale value of the current pixel.

[0054] Binarization refers to a method of converting a grayscale image into a binary image. A binary image refers to an image containing only two colors. The processor can assign a first pixel value to pixels whose grayscale values ​​are less than a preset threshold, and assign a second pixel value to pixels whose grayscale values ​​are greater than or equal to the preset threshold, so as to achieve image binarization. The first pixel value and the second pixel value can be set manually, and the first pixel value and the second pixel value correspond to different colors and are easy to distinguish with the naked eye. The preset threshold can be preset based on experience and actual conditions, and the actual conditions may include environment, material batch and type, etc.

[0055] Closing operation processing refers to connecting the originally scattered material images into a complete image with the same gray value mean. For example, after binarization, the image may be an image with hollow points, and these hollow points can be eliminated through closing operation. The gray value mean can be set based on experience. In some embodiments, the gray value mean is negatively correlated with environmental factors, and the environmental factors can be ambient brightness, etc. If the sunlight is too strong and the ambient brightness is high, the point cloud captured by the 3D camera may be missing, which may cause the material in the first plane image to be missing.

[0056] In some embodiments, the closing operation processing may be implemented by a preset algorithm. The preset algorithm may be an open and closed convolution operator, such as a square operator of a preset specification, where the preset specification is an odd number, such as at least one of 3×3, 5×5, 9×9, 11×11, etc. The processor may connect discrete points into lines by an open and closed convolution operator.

[0057] The first plane image can be smoothed by the mean filter to remove the noise in the image; the target material can be presented more clearly in the image by the binarization process; the small areas in the image can be connected or filled by the closing operation to eliminate the noise or connect the discontinuous parts of the object, so as to facilitate the subsequent search for the contour of the target object. The second preprocessing can make the edge of the target material in the second plane image clearer, which is convenient for the subsequent identification of the edge of the target material.

[0058] Figure 3 This is an exemplary schematic diagram for determining the first area and the second area according to some embodiments of this specification.

[0059] In some embodiments, the binarization process includes: dividing the first plane image into multiple sub-regions; for each sub-region, calculating the grayscale feature value of the sub-region as a preset classification threshold; taking the pixel points in the sub-region whose pixel values ​​are greater than the preset classification threshold as the first pixel points, and taking the pixel points whose pixel values ​​are less than or equal to the preset classification threshold as the second pixel points.

[0060] In some embodiments, the processor may divide the first plane image into a plurality of sub-regions with a preset size. The preset size may be preset manually based on experience and demand. For example, Figure 3 As shown, the processor may divide the first planar image into sub-region 1, sub-region 2, ..., sub-region n.

[0061] Grayscale feature value refers to a parameter that reflects the overall characteristics of the grayscale value of a sub-region. For example, the grayscale feature value may be a grayscale mean or median. In some embodiments, for brighter image regions, the grayscale mean of the sub-region is selected as the threshold value to provide overall brightness information of the image; for darker image regions (such as image regions that are subject to greater noise interference), the grayscale median of the sub-region may be selected as the threshold value because the grayscale median is less sensitive to abnormal values ​​and can better reflect the true brightness level of objects in the image.

[0062] For example, Figure 3 As shown, the processor can calculate the grayscale feature values ​​q1, q2, ..., q corresponding to sub-region 1, sub-region 2, ..., sub-region n, respectively. n As the preset classification threshold for each sub-region, the pixel value in sub-region n greater than q n The pixel point is taken as the first pixel point, and the pixel value is less than or equal to q n The pixel point is taken as the second pixel point.

[0063] In some embodiments, the pixel value of the first pixel is different from the pixel value of the second pixel. For example, the processor may set the pixel value of the first pixel or the second pixel to 255, which is white, and set the pixel value of the second pixel or the first pixel to 0, which is black.

[0064] In some embodiments, an area formed by first pixels of a plurality of sub-areas is a first area, and an area formed by second pixels of a plurality of sub-areas is a second area.

[0065] For example, Figure 3 As shown, the processor may use the first pixel points corresponding to sub-area 1, sub-area 2 ... and sub-area n as the first area Q1, and use the second pixel points corresponding to sub-area 1, sub-area 2 ... and sub-area n as the second area Q2.

[0066] In some embodiments, the processor may combine the first region and the second region to obtain a second planar image.

[0067] By dividing the entire image into many small areas and using different thresholds for segmentation in each small area, we can better adapt to the characteristics of the local area of ​​the image and obtain more accurate segmentation results.

[0068] The second plane image is a plane image reflecting the approximate shape of the target material.

[0069] In some embodiments, the processor may perform mean filtering, binarization and / or closing operation on the first planar image to obtain the second planar image.

[0070] Step 250: Convolve the second planar image in at least one direction using at least one preset operator to obtain at least one convolved image.

[0071] The preset operator refers to a preset operation method, for example, the preset operator may be a convolution operator. The convolution operator may be manually set based on experience, for example, at least one API convolution operator (application programming interface convolution operator) is preset in OpenCV (Open Source Computer Vision Library).

[0072] In some embodiments, the processor may use Filter2D (two-dimensional filter) to perform convolution on the second plane image in at least one direction through a preset operator. In some embodiments, each preset operator corresponds to a convolved image.

[0073] Figure 4 a-4d are exemplary schematic diagrams of images after convolution in four directions according to some embodiments of the present specification.

[0074] For example, Figure 4 As shown in a-4d, the processor can perform convolution on the second plane image in four directions through four preset operators. The four convolution operators are [{-1,0,1}, {-1,0,1}, {-1,0,1}], [{1,0,-1}, {1,0,-1}, {1,0,-1}], [{-1,-1,-1}, {0,0,0}, {1,1,1}], [{1,1,1}, {0,0,0}, {-1,-1,-1}], and the four convolution operators are respectively substituted into OpenCV for convolution, and the convolved images obtained are respectively. Figure 4 a. Figure 4 b. Figure 4 c and Figure 4 d.

[0075] Step 260: Generate a third planar image based on at least one convolved image.

[0076] The third plane image is a plane image of the complete target material after convolution.

[0077] In some embodiments, the processor may synthesize the images through a preset operation based on at least one convolved image to obtain a third planar image.

[0078] The preset operation is a preset operation of synthesizing an image. For example, the preset operation may be an AddWeighted (image weighted overlay function) operation in Open CV.

[0079] Figure 5 Yes Figure 4 a-4d are schematic diagrams of the third plane image obtained by synthesizing. Figure 5 As shown, the processor can use the AddWeighted operation in Open CV to convolute the image. Figure 4 a-4d are synthesized to obtain a complete third plane image.

[0080] Step 270: Perform image analysis based on the third plane image to determine the location of the target material.

[0081] The position of the target material refers to the position of the target material in three-dimensional space. The position of the target material can be expressed by the three-dimensional coordinates of the target material.

[0082] In some embodiments, the processor may determine the location of the target material based on the third plane image by using a variety of image analysis methods.

[0083] For example, the processor may perform edge detection on the third plane image, obtain the two-dimensional coordinates and grayscale values ​​of the image edge, and map the coordinates and grayscale values ​​of the image edge to three-dimensional coordinates based on a mapping relationship, thereby determining the position of the target material.

[0084] The mapping relationship refers to the mapping relationship of mapping the three-dimensional coordinates to the initial plane. For the initial plane, please refer to the above text and related instructions. Exemplarily, the two-dimensional coordinates may include a horizontal coordinate and a vertical coordinate, and the three-dimensional coordinates include an X-axis coordinate, a Y-axis coordinate, and a Z-axis coordinate. The horizontal coordinate and the vertical coordinate can be mapped to the X-axis coordinate and the Y-axis coordinate; the gray value can be mapped to the Z-axis coordinate.

[0085] Edge detection can be implemented by at least one edge detection algorithm, such as Canny edge detection algorithm, convolutional neural network (CNN), etc.

[0086] For another example, the processor may determine at least one candidate contour and its contour features in the third plane image; filter the candidate contours whose contour features do not meet the preset contour conditions in the at least one candidate contour to obtain the contour of the target material; extract the target point of the target material to determine the target point position of the target material. Figure 4 Related instructions.

[0087] It should be noted that the above description of the process 200 is only for example and illustration, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the process 200 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0088] Figure 6 is an exemplary flow chart of determining a first plane image according to some embodiments of this specification. Process 600 is an exemplary process of a material sorting and identification method, such as Figure 6 As shown, the process 600 includes the following steps 610 to 630. In some embodiments, the process 600 may be executed by a processor.

[0089] Step 610: Fit the target point cloud data to obtain a target plane.

[0090] The target plane refers to the most prominent planar structure of the point cloud data.

[0091] In some embodiments, the processor can fit the target point cloud data based on the least squares method to obtain one or more candidate planes; calculate the number of target point cloud data on the candidate planes, and determine the candidate plane with the largest number of target point cloud data as the target plane.

[0092] The target plane can make the scene presentation more complete. For example, if the target material is located on a material table, the target plane can more clearly reflect all the materials placed on the entire plane, thereby accurately extracting the outlines of all materials.

[0093] In some embodiments, the processor can select a preset number or a preset proportion of point cloud data from the target point cloud data to obtain at least one point cloud data subset; perform fitting based on at least one point cloud data subset through a preset algorithm to obtain at least one candidate plane; calculate the distance from the target point cloud data to at least one candidate plane, and determine the number of target point cloud data with a distance less than a preset distance threshold as the number of internal points of the candidate plane; and determine the candidate plane with the largest number of internal points as the target plane.

[0094] The point cloud data subset is a collection of multiple point cloud data. The processor can randomly select a preset number of point cloud data from the target point cloud data as the point cloud data subset. In some embodiments, the preset number or preset ratio can be determined by a variety of methods. For example, the preset number or preset ratio can be manually set according to demand. For another example, the preset number or preset ratio can be determined by a random sampling (RANSAC) algorithm.

[0095] In some embodiments, the preset ratio is related to the uniformity of the target point cloud. The target point cloud is the point cloud corresponding to the target point cloud data. The uniformity of the target point cloud refers to the uniformity of the target point cloud distribution in space. The preset ratio is negatively correlated with the uniformity of the target point cloud. The greater the uniformity of the target point cloud, the smaller the difference in target point cloud data in different areas, and a smaller preset ratio can be set, such as 20%; if the uniformity is poor, it means that the difference in target point cloud data in different areas is greater, then the preset ratio needs to be increased, such as 50%, so that the randomly sampled point cloud data subset is more accurate.

[0096] In some implementations, the processor may divide the three-dimensional space where the target point cloud is located into multiple sub-regions of equal volume, calculate the number of target point clouds in each sub-region, and use the variance of the number of target point clouds in the multiple sub-regions as a measure of the uniformity of the target point cloud. The volume and shape of the sub-regions are equal, and the volume of the sub-regions can be set based on experience. The smaller the variance, the more uniform the distribution of the target point cloud.

[0097] The candidate plane is a plane to be confirmed as a target plane. The preset algorithm is a preset calculation method, for example, the preset algorithm may be a least squares method.

[0098] In some embodiments, for each point cloud data subset, the processor can calculate a plane with the smallest first distance sum by least squares method based on the point cloud data subset as the candidate plane corresponding to the point cloud data subset, where the first distance sum refers to the sum of distances from multiple points in the point cloud data subset to the plane.

[0099] In some embodiments, for a candidate plane, the processor may calculate the distance between each point of the target point cloud data and the candidate plane, and take the points whose distance is less than a preset distance threshold as inliers; and count the number of inliers corresponding to the candidate plane.

[0100] In some embodiments, the processor may perform random sampling multiple times to select other point cloud data subsets and repeat the above steps; in each iteration, a candidate plane and the corresponding number of internal points are fitted; after traversing all points in the target point cloud data, the candidate plane with the largest number of corresponding internal points is used as the target plane.

[0101] By determining multiple candidate planes through random sampling and least squares method, the candidate plane with the largest number of internal points is determined as the target plane, so that the point cloud near the target plane can be as large as possible, so that the first plane image obtained subsequently can more accurately reflect the relevant information of the target material.

[0102] Step 620, mapping the target point cloud data to the target plane to obtain a mapping image.

[0103] The target point cloud data may include three-dimensional coordinates of the target point cloud, for example, X-axis coordinates, Y-axis coordinates, and Z-axis coordinates.

[0104] In some embodiments, the processor can traverse each point of the target point cloud data, map the three-dimensional coordinates of the point in the target point cloud data to the target plane, and use the obtained image as the mapping image. For example, the processor can map the X-axis coordinate and Y-axis coordinate of the point in the target point cloud data to the target plane as the corresponding horizontal coordinate and vertical coordinate of the target point cloud data; and map the height data of the Z-axis coordinate to the corresponding grayscale value of the target point cloud data.

[0105] For example, the three-dimensional coordinates of a point in the target point cloud data are (point.x, point.y, point.z), and the processor traverses the points in the target point cloud data, and the coordinates of each point after being mapped to the target plane are (point.x+1024, point.y+768). Point.x, point.y, and point.z are the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of each point in the target point cloud data, respectively, [point.x+1024] is the horizontal coordinate of each point mapped to the target plane, and point.y+768 is the vertical coordinate of each point mapped to the target plane.

[0106] Step 630: Determine a first planar image based on the mapping image.

[0107] In some embodiments, the processor may determine the mapped image as the first planar image.

[0108] In some embodiments of this specification, the target point cloud data is mapped to the fitted target plane through plane fitting to obtain a first plane image, so that the shape of the target material can be observed by naked eyes on the plane image, and the position of the target material can be easily identified.

[0109] It should be noted that the above description of process 600 is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 600 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0110] Figure 7is an exemplary flow chart for determining the location of a target material according to some embodiments of the present specification.

[0111] Step 710: Determine at least one candidate contour and its contour features in the third planar image.

[0112] The candidate contour refers to the contour of the target material to be confirmed. The contour feature is the characteristic parameter that distinguishes the candidate contour, such as the area of ​​the figure formed by the contour, the contour perimeter, etc.

[0113] In some embodiments, the processor may fit at least one candidate contour and its contour features in the third plane image through a convex hull based on a preset operator. The preset operator may be manually preset, for example, the Findcounters operator in OpenCV may be directly used as the preset operator. Convex hull fitting is to connect all raised points in a figure to form a contour, for example, the convex hull of a quadrilateral is 4 points, and the convex hull of a pentagram is 5 points.

[0114] Step 720 , filtering the candidate contours whose contour features do not meet the preset contour conditions among at least one candidate contour to obtain the target material contour.

[0115] The shape of the material is usually a regular polygon, so it is necessary to filter out the distorted, odd-shaped and irregular contours in the candidate contours.

[0116] The preset contour condition is a condition that the preset contour feature satisfies.

[0117] In some embodiments, the preset contour condition includes that the contour area satisfies a first preset condition and / or the contour perimeter satisfies a second preset condition. The contour area refers to the area of ​​the figure formed by the contour; the contour perimeter refers to the perimeter of the figure formed by the contour.

[0118] The first preset condition may include that the contour area is less than a preset area threshold. The second preset condition may include that the contour perimeter is less than a preset perimeter threshold. The preset area threshold and / or the preset perimeter threshold may be manually set according to actual conditions. For example, the area threshold may be set to 1000 cm 2 , the perimeter threshold is set to 200cm.

[0119] By filtering by contour area and contour perimeter, distorted, odd-shaped and irregular contours can be filtered out, making the contour of the target material more accurate.

[0120] The target material contour refers to the contour corresponding to the target material. There may be one or more target materials in the same set of point cloud data, and there may also be one or more corresponding target material contours.

[0121] In some embodiments, the processor may filter out candidate contours whose contour-constructed figures have an area greater than a preset area threshold, and / or whose contour perimeters are greater than a preset perimeter threshold, and use the remaining candidate contours as target material contours.

[0122] Step 730, extracting target points from the target material contour to determine the target point positions of the target material.

[0123] The target point is a point on the target material contour used to determine the position of the target material. For example, the target point can be an inflection point of the target material contour. The position of the target point can be represented by the three-dimensional coordinates of the target point.

[0124] In some embodiments, the processor can extract the target point in the target material contour based on the corner detection algorithm; and map the two-dimensional coordinates and grayscale values ​​corresponding to the target point to three-dimensional coordinates. The corner detection algorithm can be a Harris corner point detection algorithm, etc. For the mapping method of two-dimensional coordinates and grayscale values ​​to three-dimensional coordinates, please refer to Figure 2 And related instructions.

[0125] In some embodiments, the target point is a vertex of the target material contour. In some embodiments, the processor can determine the curvature and / or angle of each point on the target material contour; and take the point on the target material contour whose curvature meets the third preset condition and / or whose angle meets the fourth preset condition as the vertex of the target material.

[0126] The third preset condition is a condition that the curvature satisfies, for example, the third preset condition may include that the curvature is greater than a preset curvature threshold. The fourth preset condition is a condition that the angle satisfies, for example, the fourth preset condition may include that the angle is greater than a preset angle threshold. The preset curvature threshold and the preset angle threshold may be set based on experience.

[0127] In some embodiments, the processor may segment the target material contour through a segmentation algorithm to obtain at least one sub-straight line segment and / or sub-curve segment; calculate the curvature and / or angle of the endpoint of each sub-straight line segment and / or sub-curve segment; and take the endpoint whose curvature satisfies a third preset condition and / or the angle satisfies a fourth preset condition as the vertex of the target material.

[0128] The segmentation algorithm is a method for dividing the contour into different straight line segments and / or curve segments. For example, the segmentation algorithm can be the Douglas-Peucker algorithm.

[0129] By identifying the vertices of the target material in segments, the positions of the vertices can be determined more reliably without being affected by complex edges; at the same time, the amount of data that needs to be calculated can be reduced and the data processing speed can be improved.

[0130] By determining the vertices of the target material through curvature and / or angle, the image determined by the vertices can contain more target points of the target material, thereby making the position of the identified target material more accurate.

[0131] In some embodiments of the present specification, by screening out qualified contour features, the obtained contour can be matched with the target material and the interference of non-target material contours can be eliminated; at the same time, the position of the target material can be determined by the position of the contour points of the target material, and the two-dimensional coordinates of the target points can be traced back to three-dimensional coordinates, thereby improving the effect and accuracy of the position determination of the target material.

[0132] It should be noted that the above description of process 700 is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 700 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0133] The embodiments of this specification provide a material sorting and identification device. In some embodiments, the material sorting and identification device includes a processor. The processor is configured to execute any of the material sorting and identification methods described above.

[0134] The embodiments of this specification provide a computer-readable storage medium. In some embodiments, the storage medium stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes any of the material sorting and identification methods described above.

[0135] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0136] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0137] In addition, the order of processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are only for illustrative purposes. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0138] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. In fact, the features of the embodiment are less than all the features of the single embodiment disclosed above.

[0139] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0140] Each patent, patent application, patent application publication and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions and / or use of terms in this specification shall prevail.

[0141] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A material sorting and identification method, comprising: Obtain the initial point cloud data of the target material; Performing a first preprocessing based on the initial point cloud data to obtain target point cloud data; Determine a first plane image based on the target point cloud data; Performing a second preprocessing on the first plane image to obtain a second plane image; Performing convolution on the second plane image in at least one direction by using at least one preset operator to obtain at least one convolved image; generating a third planar image based on the at least one convolved image; The position of the target material is determined by performing image analysis based on the third plane image.

2. The method according to claim 1, characterized in that The determining of the first plane image based on the target point cloud data comprises: Fitting the target point cloud data to obtain a target plane; Mapping the target point cloud data to the target plane to obtain a mapping image; Based on the mapped image, the first planar image is determined.

3. The method according to claim 1, characterized in that: The second preprocessing includes at least one of a mean filtering process, a binarization process, and a closing operation process.

4. The method according to claim 1, characterized in that: The determining the position of the target material by performing image analysis based on the third plane image comprises: Determine at least one candidate contour and contour features thereof in the third planar image; Filtering the candidate contours whose contour features do not meet the preset contour conditions among the at least one candidate contour to obtain the target material contour; Target points are extracted from the target material contour to determine the target point positions of the target material.

5. A material sorting and identification system, comprising: Acquisition module, preprocessing module, convolution module, location determination module: The acquisition module is configured to acquire initial point cloud data of the target material; The pre-processing module is configured as follows: Performing a first preprocessing based on the initial point cloud data to obtain target point cloud data; Determine a first plane image based on the target point cloud data; Performing a second preprocessing on the first plane image to obtain a second plane image; The convolution module is configured as: Performing convolution on the second plane image in at least one direction by using at least one preset operator to obtain at least one convolved image; generating a third planar image based on the at least one convolved image; The position determination module is configured to determine the position of the target material by performing image analysis based on the third plane image.

6. The method as claimed in claim 5, characterized in that The preprocessing module is further configured to: Fitting the target point cloud data to obtain a target plane; Mapping the target point cloud data to the target plane to obtain a mapping image; Based on the mapped image, the first planar image is determined.

7. The method according to claim 5, characterized in that The second preprocessing includes at least one of a mean filtering process, a binarization process, and a closing operation process.

8. The method according to claim 5, characterized in that The location determination module is further configured to: determining at least one candidate contour in the third planar image; Filtering the candidate contours whose contour features do not meet the preset contour conditions in the at least one candidate contour to obtain the target material contour; Vertex extraction is performed based on the target material contour to determine the vertex positions of the target material.

9. A material sorting and identification device, comprising a processor, wherein the processor is used to execute the material sorting and identification method according to any one of claims 1 to 4.

10. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the material sorting and identification method according to any one of claims 1 to 4.

Images