Material online identification method and device based on image recognition

By using the silo and feeding turntable in the material recognition system, and using the deep learning network model for identification, the problems of inaccurate material identification and high hardware cost on the conveyor belt are solved, and high accuracy and low cost are achieved material recognition effect.

CN116495433BActive Publication Date: 2024-11-15SHANDONG BOSURE AUTOMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310273100.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-21
Publication Date
2024-11-15
Estimated Expiration
2043-03-21

AI Technical Summary

Technical Problem

The existing material recognition technology is difficult to obtain clear images on high-speed conveyor belts, resulting in inaccurate identification, and the scraping device is susceptible to material size differences, increasing safety risks and hardware costs.

Method used

The online material recognition method based on image recognition is adopted. The material is smoothed through the silo and feeding turntable and then transported to the image acquisition device stably. The variety and grading recognition are combined with the deep learning network model to reduce the performance requirements of the camera and light source.

Benefits of technology

It realizes stable acquisition of clear images on the conveyor belt, improves the accuracy and security of material recognition, and reduces hardware costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116495433B_ABST
    Figure CN116495433B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of material identification, and in particular to an online material identification method and device based on image recognition, wherein the identification method comprises the following steps: S1: performing image acquisition and establishing an identification model; S1 comprises the following sub-steps: step one: a material receiving mechanism receives a material, and performs image acquisition through an image acquisition device; step two: the image acquisition device sends the acquired image information to an industrial computer, and the industrial computer receives the image information and performs image processing; step three: the industrial computer performs identification model training; S2: the industrial computer recognizes material information, can more stably acquire clear images, and improve the accuracy of material identification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of material identification, and in particular to an online material identification method and device based on image recognition. Background Art

[0002] At present, the mainstream material identification method on the production line is mainly to install cameras and light sources above the conveyor belt. However, due to the fast movement speed of the conveyor belt of the production line, which is generally 1.5~2m / s, at the appropriate object distance (generally 20~30cm), in order to obtain a clear image, the camera exposure time needs to be set short enough (generally 200~500μs). At this time, the amount of light entering the camera is extremely small, and the camera aperture needs to be adjusted to the maximum, and equipped with a high-brightness light source, which requires high camera performance and light source intensity, and the cost is relatively high. After the aperture is increased, the depth of field of the camera will become smaller, so it is impossible to stably collect clear images for the uneven sand on the conveyor belt. It can be considered to set a scraping device above the conveyor belt to make the material flow highly consistent, but since the conveyor belt is also used to transport materials of different coarseness and fineness, such as large stones and small stones, it will cause rigid impact on the scraping device, which will cause accidents. All these have led to the problem that the camera cannot accurately collect and identify materials when performing material identification. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide an online material identification method and device based on image recognition, which can more stably collect clear images and improve the accuracy of material identification.

[0004] The technical solution adopted by the present invention to solve the technical problem is: an online material identification method based on image recognition, comprising the following steps:

[0005] S1: collect images and establish recognition models;

[0006] The S1 includes the following sub-steps:

[0007] Step 1: The material receiving mechanism receives the material and the image is captured by the image acquisition device;

[0008] Step 2: The image acquisition device sends the acquired image information to the industrial computer, and the industrial computer receives the image information and performs image processing;

[0009] Step 3: The industrial computer performs recognition model training. The industrial computer is provided with a variety data set and a gradation data set. The data in the variety data set and the gradation data set are split into a training set and a test set respectively. A deep learning network model is established based on a residual neural network. When the number of output categories is the number of material varieties, the model is a variety recognition model. When the number of output categories is the number of gradation classifications, the model is a gradation recognition model. The variety recognition model and the gradation recognition model are trained using the corresponding data sets to obtain a trained weight file.

[0010] S2: The industrial computer identifies the material information.

[0011] The step 1 includes the following sub-steps:

[0012] 1-1: The receiving mechanism receives the material and flattens the surface of the material;

[0013] 1-2: The image acquisition device acquires the image of the material after scraping and sends the acquired image to the industrial computer.

[0014] The step 2 includes the following sub-steps:

[0015] 2-1: The industrial computer receives the image information of the current material;

[0016] 2-2: The industrial computer receives the type and grading information corresponding to the current material input by the user;

[0017] 2-3: The industrial computer establishes a database, which includes a variety data set and a gradation data set, automatically marks the material variety and gradation information of each image, and stores them in the variety data set and the gradation data set;

[0018] 2-4: Replace materials of different types and gradations, return to step 1, and obtain image data of multiple materials with multiple gradations.

[0019] In S2, the material to be tested is put into the silo, and at set intervals, the image acquisition device acquires image information of the current material and sends it to the industrial computer, which inputs the image information into the trained variety recognition model and gradation recognition model to obtain the variety and gradation information of the current material.

[0020] The device applied to the above-mentioned material online identification method based on image recognition comprises a silo, a material outlet is arranged below the silo, and a material receiving mechanism is arranged below the material outlet of the silo;

[0021] The material receiving mechanism is connected to an image acquisition device, and the image acquisition device and the material receiving mechanism are connected to a support frame;

[0022] The image acquisition device and the material receiving mechanism are electrically connected to an industrial computer, and the industrial computer, the image acquisition device and the material receiving mechanism are all connected to a power source.

[0023] The silo is funnel-shaped, and an upper opening is provided on the silo.

[0024] The material receiving mechanism comprises an upper scraper plate and a lower scraper plate, and the material outlet of the silo is located between the upper scraper plate and the lower scraper plate;

[0025] A material receiving turntable is provided below the upper scraper plate and the lower scraper plate, and the material receiving turntable is connected to a driving device for driving the material receiving turntable to rotate, and the top of the upper scraper plate, the top of the lower scraper plate and the driving device are all fixedly connected to the support frame;

[0026] The image acquisition device is located above the material receiving turntable, the image acquisition device is located on a side of the driving device away from the lower scraper plate, and the image acquisition device is fixedly connected to the support frame.

[0027] The driving device includes a driving motor, the driving shaft of the driving motor is coaxially fixedly connected to the material receiving turntable, and the housing of the driving motor is fixedly connected to the support frame. The material falls from the material bin into the material receiving turntable, the driving motor is started, the material receiving turntable rotates, and the material on the material receiving turntable is smoothed when passing through the upper scraper plate; the smoothed material continues to rotate with the material receiving turntable to the bottom of the image acquisition device for photography, and then the photographed material is scraped off from the material receiving turntable by the lower scraper plate; the image collected by the image acquisition device is sent to the industrial computer.

[0028] The material receiving turntable includes a rotating shaft that is transmission-connected to the driving device, and the rotating shaft is fixedly connected to a plurality of fan-shaped receiving plates by a connecting rod. The fan-shaped receiving plates are distributed on the circumference of the rotating shaft with the rotating shaft as the center, and both ends of the fan-shaped receiving plates are opened. A gap is left between the upper scraper plate and the upper surface of the fan-shaped receiving plate, and the lower end of the lower scraper plate is in contact with the upper surface of the fan-shaped receiving plate.

[0029] Compared with the prior art, the present invention has the following beneficial effects: the present invention receives the material at the discharge port of the silo, transports the material to the camera field of view through the receiving turntable, ensures the top height of the material is consistent, the camera working distance is stable, the collected image is clear, and the industrial computer is more accurate in recognition. The motor speed is adjustable, the camera performance and light source brightness are low, and the hardware cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the three-dimensional structure of the present invention.

[0031] Figure 2 It is a structural schematic diagram of the material receiving mechanism of the present invention.

[0032] Figure 3 It is a schematic diagram of the network structure when the model of the present invention is established.

[0033] In the figure: 1. silo; 2. image acquisition device; 3. upper scraper plate; 4. lower scraper plate; 5. material receiving turntable; 6. driving motor; 7. supporting frame. DETAILED DESCRIPTION

[0034] The embodiments of the present invention are further described below in conjunction with the accompanying drawings:

[0035] Example 1

[0036] like Figure 1 to Figure 2 As shown, the material online identification method based on image recognition includes the following steps:

[0037] S1: collect images and establish recognition models;

[0038] The S1 includes the following sub-steps:

[0039] Step 1: The material receiving mechanism receives the material, the material receiving mechanism is connected to the image acquisition device 2, and the image acquisition device 2 performs image acquisition;

[0040] Step 2: The image acquisition device 2 and the material receiving mechanism are electrically connected to an industrial computer, and the industrial computer, the image acquisition device 2 and the material receiving mechanism are all connected to a power supply. The image acquisition device 2 sends the acquired image information to the industrial computer, and the industrial computer receives the image information for image processing;

[0041] Step 3: The industrial computer performs recognition model training;

[0042] S2: The industrial computer identifies the material information.

[0043] A silo 1 is disposed above the material receiving mechanism, a discharge port is disposed below the silo 1, and the material receiving mechanism is disposed below the discharge port of the silo 1.

[0044] The silo 1 is funnel-shaped, and the upper part of the silo 1 is open. When in use, the device can be set at the material production line, and the material passes through the silo 1 and falls on the material receiving mechanism, and the image acquisition device performs image acquisition and analysis.

[0045] The material receiving mechanism comprises an upper scraper plate 3 and a lower scraper plate 4, and the discharge port of the silo 1 is located between the upper scraper plate 3 and the lower scraper plate 4;

[0046] A material receiving turntable 5 is provided below the upper scraper plate 3 and the lower scraper plate 4, and the material receiving turntable 5 is connected to a driving device for driving the material receiving turntable 5 to rotate. The top of the upper scraper plate 3, the top of the lower scraper plate 4 and the driving device are fixedly connected to the support frame 7;

[0047] The image acquisition device 2 is located above the material receiving turntable 5 , and the image acquisition device 2 is located on a side of the driving device away from the lower scraper plate 4 . The image acquisition device 2 is fixedly connected to the support frame 7 .

[0048] The driving device comprises a driving motor 6 , a driving shaft of the driving motor 6 is coaxially and fixedly connected to the material receiving turntable 5 , and a housing of the driving motor 6 is fixedly connected to the supporting frame 7 .

[0049] The image acquisition device 2 includes a camera fixedly connected to the support frame 7, and the lens of the camera passes through the support frame 7 toward the receiving turntable 5. The lens cover of the camera is provided with a light shield. The image acquisition device 2 can also be a camera, etc., and when performing image acquisition, the acquired video stream can be decoded into an image.

[0050] The material receiving turntable 5 includes a rotating shaft that is transmission-connected to the driving device, and the rotating shaft is fixedly connected to a plurality of fan-shaped receiving plates by a connecting rod. The fan-shaped receiving plates are distributed on the circumference of the rotating shaft with the rotating shaft as the center, and both ends of the fan-shaped receiving plates are opened. A gap is left between the upper scraper plate 3 and the upper surface of the fan-shaped receiving plate, and the lower end of the lower scraper plate 4 is in contact with the upper surface of the fan-shaped receiving plate.

[0051] In this application, the speed of the driving motor can be controlled by an industrial computer to ensure the clarity of the camera when acquiring pictures. Figure 1 The support frame 7 can adopt an existing support frame that can be lifted and lowered and horizontally adjusted in length. Specifically, the support frame can include a vertical lifting rod that can be adjusted in height and a horizontal lifting rod that can be adjusted in length. The bottom of the vertical lifting rod is connected to a support base to achieve fixation and stability. Its structure can be implemented using existing technology, so its structure will not be described in detail.

[0052] When in use, the fan-shaped receiving plate of the material receiving turntable 5 located between the upper scraper plate 3 and the lower scraper plate 4 should be aligned with the unloading port of the silo 1 to ensure that the material falls into the fan-shaped receiving plate between the upper scraper plate 3 and the lower scraper plate 4.

[0053] Example 2

[0054] Based on Example 1, the step 1 includes the following sub-steps:

[0055] 1-1: The material falls from the silo 1 into the receiving turntable 5, the drive motor 6 is started, the receiving turntable 5 rotates, and the material on the receiving turntable 5 is smoothed when passing through the upper scraper plate 3;

[0056] 1-2: The smoothed material continues to rotate with the material receiving turntable 5 to the bottom of the image acquisition device for photographing, and then the photographed material is scraped off from the material receiving turntable 5 by the lower scraper plate 4;

[0057] 1-3: The image captured by the image acquisition device is sent to the industrial computer.

[0058] Specifically, the material discharge port of the silo 1 is located between the upper scraper plate 3 and the lower scraper plate 4. When in use, the material falls on the fan-shaped receiving plate between the upper scraper plate 3 and the lower scraper plate 4, and the drive motor 6 is started to drive the receiving turntable 5 to rotate. Figure 1 In this embodiment, the driving motor 6 drives the material receiving turntable 5 to rotate clockwise, and the upper scraper plate 3 smoothes the material falling on the fan-shaped receiving plate, and then moves to the bottom of the camera to take pictures. As the material receiving turntable 5 continues to move, the material after taking pictures is scraped off when it passes through the lower scraper plate 4.

[0059] The step 2 includes the following sub-steps:

[0060] 2-1: The industrial computer receives the image information of the current material;

[0061] 2-2: The industrial computer receives the type and grading information corresponding to the current material input by the user; the grading information includes the fineness modulus and confidence level of the material.

[0062] 2-3: The industrial computer establishes a database, which includes a variety data set and a gradation data set, automatically marks the material variety and gradation information of each image, and stores them in the variety data set and the gradation data set;

[0063] 2-4: Change the materials of different types and gradations, return to step 1, and obtain image data of multiple materials and multiple gradations. At least 500 images should be collected for a single material and a single gradation.

[0064] In the step three, the data in the variety data set and the grading data set are split into a training set and a test set respectively, and a deep learning network model is established based on a residual neural network to train and generate a variety recognition model and a grading recognition model to obtain a trained weight file.

[0065] Convolutional neural networks were proposed by four scholars from Microsoft Research. They are easy to optimize and can improve accuracy by increasing the depth. The residual blocks inside them use skip connections to alleviate the gradient vanishing problem caused by increasing the depth of deep neural networks.

[0066] Specifically, this application establishes a deep learning network model based on a residual neural network, and its network structure is as follows: Figure 3As shown in the figure, the numbers in the box represent the convolution kernel size and the number of channels. The input image size is 3×512×512 pixels. After 64 channels and 7×7 convolution, a 64×256×256 pixel feature map is obtained. Then, it passes through four network blocks based on a three-layer residual structure in sequence to obtain a 2048×16×16 pixel feature map. After passing through an average pooling layer and a 2048×c fully connected layer, the probability value of the classification category is output. The value of c is determined by the number of output categories.

[0067] In S2, the material to be tested is put into the silo 1. At set intervals, the image acquisition device 2 acquires image information of the current material and sends it to the industrial computer. The industrial computer inputs the image information into the trained variety recognition model and gradation recognition model respectively to obtain the variety and gradation information of the current material.

[0068] Example 3

[0069] Take concrete aggregate as an example. Concrete is a composite material made of gel material, aggregate and water in appropriate proportions, which is hardened after a certain period of time. Aggregate is one of the main materials of concrete. It occupies about 70% of the volume in concrete and is the main material of concrete engineering. Its usage is very large. Aggregates with a particle size greater than 4.75 mm are called coarse aggregates. Aggregates with a particle size of less than 4.75 mm are called fine aggregates. According to the national standard GT / B 14684 "Construction Sand", fine aggregates can be divided into natural sand and machine-made sand according to their sources. There are significant differences between the two in terms of particle shape, gradation, surface texture and the content of particles below 0.075 mm. The variety and gradation of concrete aggregates are directly related to the density, workability and strength of concrete.

[0070] The equipment is powered on, the drive motor keeps rotating at a constant speed of 2.5r / min, and the diameter of the receiving turntable is 300mm. The camera is connected to the industrial computer data receiving software to display the picture.

[0071] The material is put into the silo 1 and falls from the discharge port of the silo 1 to the receiving turntable 5. The receiving turntable 5 rotates and passes through the upper scraper plate 3 to scrape the top of the material flat.

[0072] In this embodiment, the distance between the camera and the top of the material is set to 80mm, the exposure time is set to 50ms, the light source brightness is adjusted, the aperture size is set to f / 4, the light intensity at the top of the sand material is 2000Lux, and the lens focal length is selected to be 12mm. The material is rotated to the bottom of the light shield and the image is collected.

[0073] A weighing bucket can be placed under the material receiving turntable 5. The material receiving turntable 5 continues to rotate, and the material flows through the lower scraper plate 4 and falls into the weighing bucket through the gaps between the fan-shaped receiving plates of the material receiving turntable 5. The gap distance between the fan-shaped receiving plates is 5 cm.

[0074] The user inputs the type and grading information of the current material into the industrial computer, such as the type is river sand and the grading is medium sand, for data labeling.

[0075] The industrial computer receives the captured images (if it is a video captured by a camera, the video is sent to the industrial computer, which first decodes the video stream into an image), automatically annotates the material type and gradation information of each image, and enters them into the type data set and gradation data set respectively.

[0076] 800 images were collected for a single material and a single gradation.

[0077] Repeat the above steps to obtain image data sets of two varieties and four gradations. The varieties are river sand and machine-made sand, and the gradations are fineness modulus 2.0, 2.5, 3.0, and 3.5.

[0078] According to Example 1 and Example 2, a deep learning network model is established based on a residual neural network.

[0079] The data in the variety data set and the gradation data set are split into a training set and a test set, and a deep learning network model is established based on a residual neural network. In the variety recognition model of this embodiment, the value of c is equal to the number of sand material varieties, and in the gradation recognition model, the value of c is equal to the number of gradation classifications. The variety recognition model and the gradation recognition model are trained using the corresponding data sets to obtain a trained weight file. Specifically, in the variety recognition model, the number of sand material varieties is set to 2, and in the gradation recognition model, the number of gradation classifications is set to 5. The variety recognition model and the gradation recognition model are trained using the corresponding data sets to obtain a trained weight file.

[0080] The material to be identified is put into the silo 1, and is brought to the camera for image acquisition through the material receiving turntable 5. The image acquisition is sent to the industrial computer, and the industrial computer inputs the image information into the variety identification model and the gradation identification model respectively to obtain the variety and gradation information of the current material.

[0081] In general, the discrete reasoning results of the material gradation fineness modulus can be calculated by confidence weighting to obtain the continuous target gradation fineness modulus.

[0082] When the fineness modulus of the material to be tested is a value between two adjacent marked gradations, the gradation fineness modulus value G0 is output by weighted calculation based on the two gradations with the highest confidence. The calculation formula is as follows:

[0083] ;

[0084] Where G0 is the output gradation fineness modulus value, G1 is the gradation fineness modulus value with the highest confidence, G2 is the gradation fineness modulus value with the second highest confidence, C1 is the highest confidence, and C2 is the second highest confidence.

[0085] In this application, the fineness modulus of the material is calculated according to the provisions of GB / T14684-2022 on material grading information.

[0086] Assuming the fineness modulus of the material to be tested is 2.7, ideally, the two gradation classification results with the highest confidence should be fineness modulus 2.5 and 3.0. Assuming the confidence of fineness modulus 2.5 is 45%, and the confidence of fineness modulus 3.0 is 30%, then through weighted calculation, it can be obtained that the target gradation classification is fineness modulus 2.7.

Claims

1. A material online identification method based on image recognition, characterized in that: The following steps are involved: S1: collect images and establish recognition models; The S1 includes the following sub-steps: Step 1: The material receiving mechanism receives the material and the image is captured by the image capture device (2); The step 1 includes the following sub-steps: 1-1: The receiving mechanism receives the material and flattens the surface of the material; 1-2: The image acquisition device acquires the image of the material after scraping, and sends the acquired image to the industrial computer; Step 2: The image acquisition device (2) sends the acquired image information to the industrial computer, and the industrial computer receives the image information and performs image processing; the step 2 includes the following sub-steps: 2-1: The industrial computer receives the image information of the current material; 2-2: The industrial computer receives the type and grading information corresponding to the current material input by the user; 2-3: The industrial computer establishes a database, which includes a variety data set and a gradation data set, automatically marks the material variety and gradation information of each image, and stores them in the variety data set and the gradation data set; 2-4: Replace materials of different types and gradations, return to step 1, and obtain image data of multiple materials with multiple gradations; Step 3: The industrial computer performs recognition model training. The industrial computer is provided with a variety data set and a gradation data set. The data in the variety data set and the gradation data set are split into a training set and a test set respectively. A deep learning network model is established based on a residual neural network. When the number of output categories is the number of material varieties, the model is a variety recognition model. When the number of output categories is the number of gradation classifications, the model is a gradation recognition model. The variety recognition model and the gradation recognition model are trained using the corresponding data sets to obtain a trained weight file. S2: The industrial computer identifies the material information; in S2, at a set time interval, the image acquisition device (2) acquires the image information of the current material and sends it to the industrial computer, and the industrial computer inputs the image information into the trained variety recognition model and gradation recognition model respectively to obtain the variety and gradation information of the current material; When the fineness modulus of the sand and gravel to be tested is a value between two adjacent marked gradations, the gradation fineness modulus value G0 is output by weighted calculation based on the two gradations with the highest confidence. The calculation formula is as follows: Where G0 is the output gradation fineness modulus value, G1 is the gradation fineness modulus value with the highest confidence, G2 is the gradation fineness modulus value with the second highest confidence, C1 is the highest confidence, and C2 is the second highest confidence.

2. The device applied to the material online identification method based on image recognition as claimed in claim 1 is characterized in that: It comprises a silo (1), a material outlet is arranged below the silo (1), and a material receiving mechanism is arranged below the material outlet of the silo (1); The material receiving mechanism is connected to an image acquisition device (2), and the image acquisition device (2) and the material receiving mechanism are connected to a support frame (7); The image acquisition device and the material receiving mechanism are electrically connected to an industrial computer, and the industrial computer, the image acquisition device and the material receiving mechanism are all connected to a power supply; The material receiving mechanism comprises an upper scraper plate (3) and a lower scraper plate (4), and the material outlet of the material bin (1) is located between the upper scraper plate (3) and the lower scraper plate (4); A material receiving turntable (5) is provided below the upper scraper plate (3) and the lower scraper plate (4), and the material receiving turntable (5) is connected to a driving device for driving the material receiving turntable (5) to rotate, and the top of the upper scraper plate (3), the top of the lower scraper plate (4) and the driving device are all fixedly connected to the support frame (7); The image acquisition device (2) is located above the material receiving turntable (5), the image acquisition device (2) is located on a side of the driving device away from the lower scraper plate (4), and the image acquisition device (2) is fixedly connected to the support frame (7); The driving device comprises a driving motor (6), a driving shaft of the driving motor (6) is coaxially fixedly connected to the material receiving turntable (5), and a housing of the driving motor (6) is fixedly connected to the supporting frame (7); The material receiving turntable (5) includes a rotating shaft that is transmission-connected to the driving device, and the rotating shaft is fixedly connected to a plurality of fan-shaped receiving plates through a connecting rod. The fan-shaped receiving plates are distributed on the circumference of the rotating shaft with the rotating shaft as the center. Both ends of the fan-shaped receiving plates are opened, and a gap is left between the upper scraper plate (3) and the upper surface of the fan-shaped receiving plate. The lower end of the lower scraper plate (4) is in contact with the upper surface of the fan-shaped receiving plate.

3. The device according to claim 2, characterized in that The silo (1) is funnel-shaped, and the top of the silo (1) is open.