System and method for identifying materials based on 3D

Abstract

The invention discloses a system and method for recognizing materials based on 3D, and relates to the technical field of material recognition, and the system comprises a 3D visual collection module which collects material morphology and depth three-dimensional information through a plurality of groups of 3D sensors; and the 2D visual acquisition module synchronously works with the 3D module through a plurality of groups of 2D cameras and acquires material image two-dimensional information. The data processing and analysis module deploys a multi-feature deep learning fusion algorithm network, performs data fusion, feature extraction and AI identification analysis on two-dimensional and three-dimensional information, and generates a control instruction; and the material execution module executes material screening according to the instruction. According to the invention, a plurality of groups of 3D sensors and 2D cameras are arranged on a material conveying path, the full-angle morphology, depth and color information of the material is synchronously acquired, and a panoramic three-dimensional color model can be reconstructed to eliminate detection dead angles. And in combination with a microscopic feature deep learning algorithm, the recognition accuracy and recall rate of difficult-to-distinguish defects such as slight mildew and worm damage are greatly improved, and misjudgment caused by illumination and shadow is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of material identification, and in particular to a system and method for identifying materials based on 3D. BACKGROUND

[0002] In the fields of agricultural product processing, industrial material sorting, food processing, etc., as a key equipment for realizing material quality grading and impurity removal, the recognition accuracy and sorting efficiency of a color sorter directly determine the quality of downstream products, production efficiency and market competitiveness. Traditional color sorters generally rely on 2D color cameras as core perception components, and can only capture the planar color and two-dimensional contour information of the materials, and realize sorting operation by comparing the differences in color and shape between the materials and impurities.

[0003] However, in actual industrial application scenarios, such 2D recognition technology has the following problems: first, for materials with similar colors but significantly different physical properties, the 2D recognition technology is not easy to recognize. Second, for materials with slight defects or quality abnormalities on the surface, the 2D recognition technology is easily disturbed and misjudged. Third, the 2D recognition technology has visual dead angles and cannot cover the side and bottom surfaces.

[0004] For example, the application number 202510386525.1 discloses a control algorithm for adjusting the blowing of a spray valve based on shape and bad point position. The application scheme can realize dynamic adjustment of the blowing parameters of the spray valve through the cooperation of the image analysis module and the image recognition module, and ensure that the to-be-removed objects can be accurately removed. However, the scheme has the following problems: it only relies on two-dimensional image information and is not easy to realize accurate screening of materials.

[0005] Therefore, the traditional 2D color sorter has technical limitations in dealing with color similar material differentiation, slight defect recognition, irregular large material full-angle detection, etc. due to its ability to only obtain two-dimensional information of materials. Therefore, there is an urgent need for a material screening scheme capable of obtaining three-dimensional information of materials to meet the urgent needs of various industries for high-precision and high-efficiency material sorting. SUMMARY

[0006] In view of the above problems, the present application aims to provide a system and method for identifying materials based on 3D, which solves the problems of poor material differentiation and low material defect recognition rate during material screening.

[0007] The present application provides a system and method for identifying materials based on 3D.

[0008] The first aspect of the present application is a system for identifying materials based on 3D, comprising:

[0009] a 3D vision acquisition module for acquiring the three-dimensional information of the shape and depth of the materials based on a plurality of 3D sensors;

[0010] 2D vision acquisition module, based on multiple groups of 2D cameras, works synchronously with the 3D vision acquisition module, and collects image two-dimensional information of the material;

[0011] Data processing and analysis module, deployed with a multi-feature deep learning fusion algorithm network, performs data fusion, feature extraction and AI identification analysis on the collected three-dimensional information and two-dimensional information, and generates corresponding control instructions;

[0012] Material execution module; according to the control instructions, the material screening is executed.

[0013] In an embodiment of the present application, the 3D sensor adopts a structured light sensor or a laser profile sensor.

[0014] In an embodiment of the present application, the 3D sensor is combined with the 2D camera, and three groups are arranged above and on the front and rear sides of the conveyor material A point, and three-dimensional and two-dimensional information above the material is synchronously collected;

[0015] A group is arranged on the lower rear side of the material throwing B point, and three-dimensional and two-dimensional information below the material is synchronously collected.

[0016] In an embodiment of the present application, the multi-feature deep learning fusion algorithm network is an AI-driven convolutional neural network CNN and a point cloud processing network PointNet++.

[0017] In an embodiment of the present application, the multi-feature deep learning fusion algorithm network fuses three-dimensional and two-dimensional information, extracts RGB color, volume, surface area and spatial pose features of the material.

[0018] In an embodiment of the present application, when the material screening is executed, a spray valve is adopted, a material motion trajectory prediction model is established according to the RGB color, volume, surface area and spatial pose features of the material, and the opening and closing time, spray angle and air flow intensity of the spray valve are calculated in advance.

[0019] In an embodiment of the present application, the multi-feature deep learning fusion algorithm network is based on three-dimensional and two-dimensional information, AI identifies the material including reconstructing the microscopic features of material concave, crack and mildew spot, and assists in screening the material according to the microscopic features.

[0020] Second aspect: a method for identifying materials based on 3D, comprising:

[0021] S1, based on multiple groups of 3D sensors, collecting three-dimensional information of the topography and depth of the material;

[0022] S2, based on multiple groups of 2D cameras, working synchronously with the 3D vision acquisition module, and collecting image two-dimensional information of the material;

[0023] S3, based on the trained multi-feature deep learning fusion algorithm network, the collected three-dimensional information and two-dimensional information are subjected to data fusion, feature extraction and AI recognition analysis, and corresponding control instructions are generated;

[0024] S4, according to the control instructions, the material screening is performed.

[0025] Third aspect: an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method provided in the second aspect.

[0026] Fourth aspect: a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method provided in the second aspect.

[0027] Advantages of the present application:

[0028] 1. The present application arranges multiple groups of 3D sensors and 2D cameras on the material conveying path to synchronously collect the appearance, depth and color information of the material from all angles. After data fusion, the system can reconstruct the panoramic three-dimensional color model of the material, eliminate the detection dead angle, and combine the deep learning algorithm of the microscopic features to revolutionarily improve the recognition accuracy and recall rate of slight mildewing, insect damage, cracks and other defects that are difficult to discover by the naked eye, effectively avoiding misjudgment caused by light and shadow.

[0029] 2. The present application breaks through the limitation of traditional color or two-dimensional shape, uses CNN+PointNet++ fusion network to extract multi-dimensional features such as RGB color, volume, surface area, spatial pose and surface texture of the material, and only needs to switch the pre-trained AI model when changing the material, without manual parameter adjustment, so that the system can quickly adapt to new sorting tasks, significantly improve the universality and production efficiency of the equipment, and solve the pain points of poor adaptability and complex debugging of traditional equipment.

[0030] 3. The system of the present application establishes a motion trajectory prediction model for each material in real time according to the three-dimensional features such as the volume, centroid and spatial pose of the recognized material, combined with the conveying line speed, which realizes the precise attack on high-speed motion and irregular shape materials, greatly reduces the occurrence of mis-spraying and missed spraying, improves the sorting efficiency and production capacity, saves the consumption of compressed air, and reduces the running cost of the equipment. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a structural schematic diagram of the material 3D recognition system of the present application;

[0032] Figure 2 It is a flowchart of the material 3D recognition method of the present application;

[0033] Figure 3 Schematic diagram of the combination of 3D sensor and 2D camera of the present application;

[0034] Figure 4 Schematic diagram of the structure of the electronic device of the present application. DETAILED DESCRIPTION

[0035] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar symbols represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0036] The existing material recognition method has the problems of low recognition accuracy, difficulty in accurately distinguishing some appearance similar materials, easy to be disturbed in complex environment, leading to deviation of recognition result, and poor adaptability to material form change, and when the material placement posture, surface condition and the like change, the recognition accuracy will be greatly reduced.

[0037] Embodiment 1:

[0038] In view of the above problems, the present embodiment provides a system for recognizing materials based on 3D, Figure 1 The structure schematic diagram of the system of the present application, the system comprises: 3D vision acquisition module, 2D vision acquisition module, data processing and analysis module and material execution module.

[0039] Among them, the 3D vision acquisition module acquires the topography and depth three-dimensional information of the material based on multiple groups of 3D sensors. These data can clearly present the subtle features and spatial position relationship of the material, laying a foundation for subsequent accurate recognition.

[0040] The 3D sensor can adopt a structured light sensor or a laser profile sensor, and each of the two sensors has its unique advantages. The structured light sensor projects a specific structured light pattern onto the surface of the material, calculates the three-dimensional information of the material according to the deformation of the pattern, has high precision and fast acquisition speed, can quickly and accurately acquire the fine topography of the material, and is suitable for recognition scenes with high requirements for the details of the material surface

[0041] The laser profile sensor uses laser to scan the surface of the material, and constructs the three-dimensional profile of the material by measuring the time and angle of laser reflection. It has better adaptability to complex shapes and large size materials, and can more stably acquire accurate three-dimensional data when facing some materials with uneven surface or special shape.

[0042] The 2D vision acquisition module is based on multiple 2D cameras and works synchronously with the 3D vision acquisition module to acquire two-dimensional image information of materials.

[0043] The 2D camera uses a high-resolution color camera. The 2D camera is mainly used to capture the color information of materials. Through precise analysis of the color information, it is possible to initially screen and distinguish different types of materials.

[0044] Furthermore, such as Figure 3 As shown, a 3D sensor and a 2D camera are combined and set up. The 3D sensor is set up at the same location as the 2D camera to collect the three-dimensional and two-dimensional information of the material simultaneously.

[0045] Specifically, at material point A on the conveyor ( Figure 3 Three sets of combinations (such as) are set above and to the front, back and sides of position A in the middle. Figure 3 As shown in Figures 1, 2, and 3, the 3D sensors on the front and back sides work in conjunction with the 2D camera to identify information from the left and right sides of the material, including the three-dimensional shape and color information of the sides; the 3D sensor and 2D camera in the middle identify the three-dimensional shape and color information from directly above the material. During this stage, because the material is being conveyed on the conveyor belt, only the bottom surface within an approximately 80° range is difficult to identify.

[0046] Furthermore, such as Figure 3 As shown, at point B where the material is ejected ( Figure 3 A combination (such as) is set on the lower rear side of position B. Figure 3 As shown in Figure 4), the three-dimensional and two-dimensional information below the material is collected simultaneously. When the material is thrown to point B by the conveyor belt, the 3D sensor and 2D camera on the lower side of the conveyor belt cover position B, collect information on the 80° range of the bottom surface of the material that is not identified, and obtain the three-dimensional shape and color information of the bottom surface.

[0047] Subsequently, the three-dimensional and two-dimensional data collected at locations A and B are fused and processed by a multi-feature deep learning fusion algorithm network to automatically synthesize panoramic images of the material, thereby achieving the acquisition of information about the material from all angles.

[0048] Using the above structure, the 3D sensors and 2D cameras cover the entire angle of the material, collecting accurate data from each angle. During actual operation, the system controls the 3D sensors and 2D cameras to collect data at specific time points and frequencies according to a preset program. As the material moves from point A to point B on the conveyor belt, the sensors and cameras at different locations work in an orderly manner, ensuring the continuity and completeness of data collection.

[0049] Through the fusion processing of these data, the system can seamlessly splice the data from various angles together and automatically synthesize the panoramic image of the material. This panoramic image not only contains the appearance information of the material, but also the color information can be combined with the three-dimensional space information. In this way, when identifying the material, not only can the color characteristics be relied on, but also the three-dimensional information such as the shape and size of the material can be combined, greatly improving the accuracy and reliability of the identification. In the face of material identification tasks in complex environments, this multi-dimensional information fusion can effectively reduce the misjudgment and omission, ensuring the efficient progress of the material identification and screening work.

[0050] The data processing and analysis module is deployed with a multi-feature deep learning fusion algorithm network to perform data fusion, feature extraction, and AI recognition analysis on the collected three-dimensional information and two-dimensional information, and generate corresponding control instructions.

[0051] The multi-feature deep learning fusion algorithm network uses a dedicated convolutional neural network CNN and a point cloud processing network PointNet++, which performs deep mining and analysis on the collected two-dimensional image data and three-dimensional point cloud data of the material. The convolutional neural network CNN can automatically extract features such as texture and edge in two-dimensional images, and use its powerful feature extraction capability to capture subtle information in the material image. The point cloud processing network PointNet++ is used to process three-dimensional point cloud data, which can accurately model the three-dimensional spatial structure of the material and learn local and global features in the point cloud data.

[0052] These two networks cooperate with each other, first inputting the two-dimensional image data into the CNN for preliminary processing to obtain a series of representative two-dimensional feature vectors, and inputting the three-dimensional point cloud data into the PointNet++ to generate a three-dimensional feature description of the material. Then, the multi-feature deep learning fusion algorithm network fuses these two-dimensional and three-dimensional features by using feature splicing, feature weighted summation, etc., so that the two-dimensional and three-dimensional features can complement each other.

[0053] Then, the fused features are further input into the subsequent classification and recognition module. In this module, the algorithm classifies the material according to the fused features and judges the type, specification, etc. of the material. Through continuous training and optimization, the multi-feature deep learning fusion algorithm network can gradually improve the accuracy of material identification. The network has self-adaptive ability, and when it encounters new materials or complex environmental disturbances, it can quickly adapt to new situations by adjusting its parameters and weights, ensuring the stability and reliability of material identification.

[0054] The convolutional neural network (CNN) and the point cloud processing network (PointNet++) have multiple channels of information of images, topographies, and depth features as inputs instead of a single two-dimensional image. The algorithm automatically learns the optimal correlation features of color and three-dimensional morphology for decision-making, which can extract the RGB color, volume, surface area, and spatial posture features of the material, thereby providing more comprehensive information for material identification.

[0055] The multi-feature deep learning fusion algorithm network automatically learns the optimal correlation features of color and three-dimensional morphology for decision-making, and uses AI to autonomously find differences in recognition logic, which has strong adaptability. When changing materials, only the AI model needs to be switched, without complex parameter adjustment, greatly improving the flexibility and ease of use of the material identification system.

[0056] Further, a spray valve is used for material screening, and a material motion trajectory prediction model is established based on the RGB color, volume, surface area, and spatial posture features of the material to calculate the opening and closing time, spray angle, and air flow intensity of the spray valve in advance, achieving precise attack on irregular-shaped and high-speed moving materials. This can reduce mis-spraying and missed spraying, save compressed air, and allow the equipment to operate at a higher conveyor speed, improving productivity.

[0057] During the material screening process, real-time monitoring of the material by the multi-feature deep learning fusion algorithm network can dynamically adjust the parameters of the spray valve. When the characteristics of the material change slightly during movement, the system can quickly respond to ensure that the material is always screened in the best spray state.

[0058] Further, based on the volume, surface area, and spatial posture features of the material, the standing or lying state of the material can be determined, which can provide prediction parameters for subsequent spray valve control, improve the accuracy of spray valve blowing positioning, and improve the blowing hit rate to develop more accurate blowing strategies for the spray valve.

[0059] Further, the multi-feature deep learning fusion algorithm network is based on three-dimensional and two-dimensional information, AI identifies the material including the microscopic features of material concave, crack, and mildew spot, and assists in screening the material based on the microscopic features.

[0060] Utilizing high-precision 3D sensor technology, the surface of the material is scanned for topography, and based on the three-dimensional information of the material's topography and depth, the micro features of the material's concave, cracks and mildew spots are reconstructed. According to the micro features, slight mildew, insect damage or cracks on the surface of potatoes, peanuts and other materials that are difficult to detect with the naked eye can be accurately identified. This is a problem that 2D imaging is prone to misjudgment due to lighting effects, while 3D sensor technology effectively avoids such misjudgments by capturing depth information. During the scanning process, the system will analyze the three-dimensional data obtained in real time, compare the actual features of the material with the pre-set standard feature library, and once it finds that the micro features of the material deviate from the standard features, the system will quickly mark the material and classify it according to the type and degree of deviation.

[0061] At the same time, this 3D recognition-based method also has high flexibility and adaptability. It can adjust the scanning accuracy and range according to different material types and detection requirements. For example, for some small-sized materials with more complex surface features, the system's 3D sensor can increase the scanning accuracy to ensure accurate identification of minor defects. For materials with lower requirements for surface defects, the system can appropriately reduce the scanning accuracy to improve the efficiency of detection.

[0062] The material execution module is used to execute material screening according to control instructions. Material screening can use a nozzle or a cylinder to remove unqualified materials. For small and light unqualified materials, a nozzle quickly sprays air to remove them. For larger and heavier unqualified materials, a cylinder drives a finger to remove them. Qualified materials are then transported to the next process by a belt conveyor, completing the entire material sorting process.

[0063] During the material screening process, the material execution module continuously receives control instructions to ensure the accuracy and efficiency of the screening. Control instructions are accurately issued based on the analysis and classification results of the material's micro features. For materials with slight mildew, insect damage or cracks that are marked as unqualified, the system quickly transmits the corresponding control signals to the material execution module.

[0064] During the entire material sorting process, the material execution module works closely with the system. Each screening action is based on accurate data analysis and control instructions, achieving efficient and accurate sorting of materials and greatly improving the quality of materials and the efficiency of subsequent processing.

[0065] Embodiment 2:

[0066] Based on the system of Embodiment 1, this embodiment discloses a method for identifying materials based on 3D recognition, as shown in Figure 2 The method comprises the following steps:

[0067] S1, based on multiple sets of 3D sensors and 2D camera synchronous acquisition of material morphology and depth three-dimensional information and image two-dimensional information.

[0068] This step is completed by 3D vision acquisition module and 2D vision acquisition module, and the whole surface information of the material is obtained comprehensively.

[0069] Firstly, the material to be sorted is conveyed to the vision recognition area by a high-precision belt conveyor at a constant and adjustable speed. The system can accurately track the material position through an encoder or a photoelectric sensor.

[0070] Then, when the material reaches point A, the three sets of 3D sensors and their matching high-resolution 2D color cameras arranged above the station are started synchronously through a synchronous trigger.

[0071] Then, the two groups located on the left and right sides of the position synchronously collect the three-dimensional point cloud data (including topography and depth information) and two-dimensional color images (color information) of the left and right sides of the material; the middle group located directly above synchronously collects the three-dimensional point cloud data and two-dimensional color images of the top of the material. At this point, only the bottom surface cannot be photographed because it is in contact with the conveyor belt.

[0072] Then, the material continues to move with the conveyor belt until it is thrown off or runs to point B, starting a set of 3D sensors and 2D cameras arranged below to synchronously collect three-dimensional point cloud data and two-dimensional color images of the bottom of the material, thus completing the remaining 80° range of visual blind area and realizing 360° full coverage data collection of the material.

[0073] S2, based on the trained multi-feature deep learning fusion algorithm network, data fusion, feature extraction and AI recognition analysis are performed on the collected three-dimensional information and two-dimensional information to generate corresponding control instructions.

[0074] Firstly, data preprocessing and alignment are performed. Based on multi-view 3D point cloud data and multi-view 2D color images, time and space registration is performed. The 3D point cloud data is unified to the same world coordinate system through coordinate transformation, and the 2D image is aligned with the 3D point cloud at the pixel level. The RGB color information is accurately mapped to the corresponding point cloud through the camera intrinsic and extrinsic matrix.

[0075] Then, multi-modal feature extraction and fusion are performed. The generated color point cloud data set is input into the pre-trained multi-feature deep learning fusion algorithm network. This network is an AI-driven hybrid model that combines convolutional neural network (CNN) and point cloud processing network (PointNet++).

[0076] The CNN branch mainly processes 2D image information. Its powerful convolution and pooling operations can efficiently extract color features, texture features, and macroscopic contour morphology of the material surface.

[0077] PointNet++ branch is dedicated to processing 3D point cloud data, which can hierarchically learn the local and global features of the point cloud, accurately extract the three-dimensional morphological features of the material, such as volume, surface area, curvature, spatial pose features, such as rolling angle, pitch angle, and microscopic surface features, such as the depth of concave, the direction and length of crack, and the fluctuation of mold spot.

[0078] The multi-feature deep learning fusion algorithm network does not process the two kinds of data independently, but performs deep fusion at the feature layer. For example, the rich texture features extracted by CNN can be spliced with the geometric structure features extracted by PointNet++, or weighted fusion through attention mechanism, forming a joint feature vector that can represent the appearance and shape of the material. This fusion method enables AI to understand color and corresponding three-dimensional shape, and solves the problem of distinguishing materials with similar colors but different physical properties.

[0079] Then, the fused high-dimensional feature vector is input into the classifier, and the network based on the decision boundary trained by a large amount of data accurately identifies and classifies the material, identifies the defect type of the material, including identifying mold, insect, crack or other defects, and generates control instructions according to the identification result.

[0080] S3, according to the control instruction, performing material screening.

[0081] Performing material screening, material screening can use a nozzle or a cylinder to remove the unqualified material. For small and light unqualified materials, the nozzle quickly sprays air to remove them; for larger and heavier unqualified materials, the cylinder drives the finger to remove them, and the qualified materials continue to be conveyed to the next process by the belt conveyor, completing the entire material sorting process.

[0082] Further, for the material that needs to be removed, the system establishes a motion trajectory prediction model of the material according to the spatial pose features and volume of the current material, combined with the known running speed, acceleration and gravitational acceleration of the conveyor and other physical parameters, accurately predicts the spatial position and pose of the material at a certain future time point, and calculates the optimal removal parameters in advance based on the predicted accurate position and pose, to achieve accurate attack.

[0083] For the nozzle, including calculating and generating control parameters including opening and closing time, jet delay, jet angle and air flow strength, which ensures that the high-speed flying material is hit by the right strength of air flow at the right time point and blown off the track. For the cylinder finger, the action trigger time and stroke depth are calculated, and the calculated high-precision actuator parameters are bound with the removal instruction to form the final control instruction that can directly drive the cylinder finger to execute.

[0084] In the material screening process, to ensure the accuracy and efficiency of screening, the material execution module will continuously receive control instructions. Control instructions will be accurately issued according to the analysis and classification results of the micro features of the material in the early stage. For those slightly mildewed, worm-eaten or cracked materials marked as unqualified, the system will quickly transmit the corresponding control signal to the material execution module.

[0085] In the whole material sorting process, the material execution module works closely with the system. Each screening action is based on accurate data analysis and control instructions, so as to realize efficient and accurate sorting of materials, greatly improving the quality of materials and the efficiency of subsequent processing.

[0086] The application also provides an electronic device, Figure 4 The structure diagram of the electronic device provided by the embodiment of the application is shown in Figure 4 As shown in the figure, the electronic device can include a processor, a communications interface, a memory and a communications bus, wherein the processor, the communications interface and the memory complete mutual communication through the communications bus. The processor can call the logical instructions in the memory, for example, to execute the following method:

[0087] S1, based on a plurality of 3D sensors and 2D cameras, synchronously collecting the topography and depth three-dimensional information and image two-dimensional information of the material;

[0088] S2, based on the trained multi-feature deep learning fusion algorithm network, performing data fusion, feature extraction and AI recognition analysis on the collected three-dimensional information and two-dimensional information, and generating corresponding control instructions;

[0089] S3, executing material screening according to the control instructions.

[0090] In addition, the logical instructions in the memory described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various storage medium that can store program codes.

[0091] The embodiment of the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method provided by the above-mentioned embodiments, for example, comprising:

[0092] S1, synchronously collecting topography and depth three-dimensional information and image two-dimensional information of the material based on multiple groups of 3D sensors and 2D cameras;

[0093] S2, performing data fusion, feature extraction and AI recognition analysis on the collected three-dimensional information and two-dimensional information based on the trained multi-feature deep learning fusion algorithm network, and generating a corresponding control instruction;

[0094] S3, performing material screening according to the control instruction.

[0095] The system embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0097] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A system for recognizing materials based on 3D, characterized in that, include: The 3D vision acquisition module acquires the morphology and depth three-dimensional information of materials based on multiple sets of 3D sensors; The 2D vision acquisition module, based on multiple 2D cameras, works synchronously with the 3D vision acquisition module to acquire two-dimensional image information of materials. The data processing and analysis module deploys a multi-feature deep learning fusion algorithm network to perform data fusion, feature extraction, and AI recognition analysis on the collected 3D and 2D information, and generate corresponding control commands. The material execution module performs material screening based on control instructions.

2. The system according to claim 1, characterized in that, The 3D sensor is a structured light sensor or a laser contour sensor.

3. The system according to claim 1, characterized in that, The 3D sensor and 2D camera are combined and set up in three sets above and on the front and rear sides of the material point A on the conveyor, to simultaneously collect three-dimensional and two-dimensional information above the material; A set of arrays is set up below and to the rear of the material ejection point B to simultaneously collect three-dimensional and two-dimensional information below the material.

4. The system according to claim 1, characterized in that, The multi-feature deep learning fusion algorithm network is an AI-driven convolutional neural network (CNN) and a point cloud processing network (PointNet++).

5. The system according to claim 1, characterized in that, The multi-feature deep learning fusion algorithm network integrates three-dimensional and two-dimensional information to extract the RGB color, volume, surface area, and spatial orientation features of the material.

6. The system according to claim 5, characterized in that, When performing material screening, a spray valve is used. Based on the material's RGB color, volume, surface area, and spatial posture characteristics, a material movement trajectory prediction model is established to calculate the spray valve's opening and closing time, spray angle, and airflow intensity in advance.

7. The system according to claim 1, characterized in that, The multi-feature deep learning fusion algorithm network is based on three-dimensional and two-dimensional information. AI identifies materials by reconstructing the microscopic features of material dents, cracks and mold spots, and assists in screening materials based on microscopic features.

8. A method for identifying materials based on 3D recognition, characterized in that, Applied to the system according to any one of claims 1 to 7, comprising the steps of: S1. Based on multiple sets of 3D sensors and 2D cameras, the material's morphology and depth three-dimensional information and image two-dimensional information are collected simultaneously; S2. Based on the trained multi-feature deep learning fusion algorithm network, the collected three-dimensional and two-dimensional information are fused, feature extracted and AI-recognized and analyzed to generate corresponding control commands. S3. Perform material screening according to control instructions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in claim 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method as described in claim 8.

Images