A scrap steel recognition method based on image processing and DBN deep belief network

Abstract

The present invention discloses a scrap steel identification method based on image processing and DBN deep belief network, comprising: using a camera to collect images of a currently parked vehicle carrying scrap steel to obtain a vehicle image, and performing image segmentation on the collected vehicle image to obtain the vehicle compartment position; evaluating the compartment position to determine whether the compartment is located in a specified area; when the compartment is located in the specified area, collecting images of the scrap steel in the compartment to obtain a scrap steel image; preprocessing the collected scrap steel image to obtain a preprocessed scrap steel image; extracting features from the preprocessed scrap steel image to obtain corresponding scrap steel feature data; performing DBN deep belief network learning on the extracted scrap steel feature data to obtain the corresponding grade of the scrap steel. The present invention achieves the purpose of accurately identifying and judging the scrap steel grade, realizes efficient operation of the enterprise, and solves the problem of inaccurate scrap steel grade detection in traditional manual identification.

Description

Technical Field

[0001] The present invention relates to the technical field of scrap metal identification and recycling, and in particular to a scrap steel identification method based on image processing and DBN deep belief network. Background Art

[0002] When scrap steel is identified and recycled in a factory, it is usually transported to a designated location by a transport vehicle, and then the scrap steel is identified and judged to determine its scrap grade, and then it is recycled.

[0003] Traditional scrap steel determination and inspection work completely relies on personnel to visually grade scrap steel based on determination standards and with the help of simple measuring tools, and to manually fill in determination records. Scrap steel determination and inspection are only carried out under daytime visual conditions, with a poor determination environment, high safety risks, low determination efficiency, and a large number of human interference factors in the determination process. There are many risk loopholes, and risk prevention and control is difficult. The uneven determination levels of current determination personnel have an impact on the accuracy of scrap steel determination, and there is considerable controversy. In view of this situation, it is necessary to develop an efficient, fully automatic, unmanned, intelligent scrap steel determination system that can simultaneously eliminate disputes among all parties regarding scrap steel grade determination. Summary of the invention

[0004] The present invention provides a scrap steel identification method based on image processing and DBN deep belief network to solve the technical problems that the existing scrap steel identification and inspection method can only be used for identification under daytime visual conditions, the identification environment is poor, the safety hazard is high, the identification efficiency is low, the identification process is greatly affected by human interference factors, there are many risk loopholes, the risk prevention and control is difficult, and the accuracy is greatly affected by human factors, thus causing great controversy.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] On the one hand, the present invention provides a scrap steel identification method based on image processing and DBN deep belief network, which automatically determines the grade of scrap steel by collecting images of scrap steel in the carriage of a carrier, processing and extracting features of the images, and using a deep belief network. It includes the following steps:

[0007] A camera device is used to collect images of the currently parked vehicle carrying scrap steel to obtain a vehicle image, and the collected vehicle image is segmented to obtain the compartment position of the vehicle carrying scrap steel;

[0008] Evaluate the position of the carriage to determine whether the carriage is within the designated area;

[0009] When the carriage is located in the designated area, images of the scrap steel in the carriage are collected to obtain scrap steel images;

[0010] Preprocessing the collected scrap steel image to obtain a preprocessed scrap steel image;

[0011] Perform feature extraction on the preprocessed scrap steel image to obtain corresponding scrap steel feature data;

[0012] The extracted scrap steel feature data is subjected to DBN deep belief network learning to obtain the corresponding grade of the scrap steel.

[0013] Furthermore, the image segmentation of the collected vehicle image includes:

[0014] The grab cut image segmentation algorithm is used to calibrate the foreground and background of the collected vehicle image. The Gaussian mixture model is used to calibrate each pixel in the vehicle image to obtain the weight π, mean vector μ and covariance matrix Σ. The Gaussian mixture model is iteratively solved to remove the background in the vehicle image, segment the edge of the vehicle, obtain the vehicle's contour information, and separate the vehicle compartment from the image.

[0015] Furthermore, the position of the carriage is evaluated to determine whether the carriage is located in a designated area, including:

[0016] The minimum spanning tree algorithm is used to determine whether the segmented carriage position is within the designated area specified by the on-site marking. If the carriage boundary is consistent with the designated area, the scrap steel in the carriage is imaged. Otherwise, the vehicle is re-docked until the vehicle's carriage is within the designated area.

[0017] Furthermore, the image acquisition of the scrap steel in the carriage includes:

[0018] A spherical camera is used to collect images of scrap steel in the carriage. During shooting, the greedy algorithm is used to solve the shooting operation to obtain a clear, high-coverage and non-overlapping operation combination.

[0019] Furthermore, the shooting operation includes a shooting path, a shooting angle and a shooting focal length;

[0020] The method of solving the shooting operation by using the greedy algorithm includes:

[0021] The greedy algorithm is used to establish a mathematical model for the shooting path, shooting angle and shooting focal length, which is decomposed into three sub-problems. Then, the shooting path, shooting angle and shooting focal length are solved locally at the optimal level. Finally, the local optimal solutions are combined to obtain the optimal solution for the shooting operation, thereby obtaining the scrap steel image.

[0022] Furthermore, the preprocessing of the collected scrap steel images includes:

[0023] The collected scrap steel images are subjected to detection area extraction, image fusion, tilt correction, filtering and noise reduction, histogram equalization and image enhancement processing.

[0024] Furthermore, the extraction of the area to be detected from the collected scrap steel image includes: eliminating useless information in the image, performing a deduplication algorithm on the image to extract the area to be detected;

[0025] The filtering and noise reduction adopts adaptive Wiener filter filtering.

[0026] Furthermore, the feature extraction of the pre-processed scrap steel image to obtain corresponding scrap steel feature data includes:

[0027] The tree-structured wavelet transform algorithm is used to decompose the high-frequency band and low-frequency band of the preprocessed scrap steel image to obtain the image features. The wavelet decomposition is performed according to the difference in high-frequency and low-frequency energy of the image, and the wavelet transform coefficient is used as the feature vector to extract the color features, texture features and edge features of the image.

[0028] The geometric dimensions of the scrap steel are determined to obtain the geometric dimension data of the scrap steel.

[0029] Furthermore, the extracted scrap steel feature data is subjected to DBN deep belief network learning to obtain the corresponding grade of the scrap steel, including:

[0030] Based on DBN deep belief network technology, a restricted Boltzmann machine is constructed; according to the actual training situation, several layers of restricted Boltzmann machines are constructed, and a deep belief network is composed of several layers of restricted Boltzmann machines;

[0031] The extracted scrap steel feature data is used as the visible layer input of the restricted Boltzmann machine, and the level of scrap steel grade determination is used as the output layer of a deep belief network composed of several layers of restricted Boltzmann machines; wherein the scrap steel feature data includes: color features, texture features, edge features and geometric size data of the scrap steel.

[0032] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0033] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the above method.

[0034] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0035] The present invention overcomes the problem of inaccurate and inefficient manual detection of scrap steel in traditional enterprises. The present invention starts the process of judging the grade of scrap steel from the moment the carrier is parked. It verifies whether the carrier is close to the specified position through image acquisition, then acquires images of the scrap steel in the carrier compartment, and extracts features, including color, texture, position, and geometric size. Finally, these features are input into the deep belief network to output the scrap steel grade. The purpose of accurately identifying and judging the scrap steel grade is achieved, thereby realizing the efficient operation of the enterprise, and further solving the problem of inaccurate scrap steel grade detection in traditional manual identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 It is a schematic diagram of the execution flow of the scrap steel identification method based on image processing and DBN deep belief network provided by an embodiment of the present invention;

[0038] Figure 2 is a structural diagram of a deep belief network provided by an embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram of tree-structured wavelet decomposition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0041] First embodiment

[0042] This embodiment provides a scrap steel identification method based on image processing and DBN deep belief network, which can be implemented by electronic devices, and proposes a complete set of scrap steel identification methods for enterprises. It uses a camera to extract vehicle information, uses the grab cut algorithm to process the vehicle compartment carrying scrap steel to segment and identify the vehicle image, uses the minimum spanning tree algorithm to determine the parking position of the carrier, uses the greedy algorithm to solve the optimal shooting operation combination to collect scrap steel images, pre-processes the collected scrap steel images to enhance the scrap steel image information, uses the tree-like wavelet transform to extract the surface features of the scrap steel from the processed images, inputs the extracted features into the deep belief network model, and outputs the corresponding scrap steel grade.

[0043] Specifically, the execution process of this method is as follows Figure 1 As shown, the following steps are included:

[0044] S1, using a camera to collect images of a currently parked vehicle carrying scrap steel to obtain a vehicle image, and performing image segmentation on the collected vehicle image to obtain a compartment position of the vehicle carrying scrap steel;

[0045] Specifically, in this embodiment, the implementation process of S1 is as follows: the grab cut image segmentation algorithm is used to calibrate the foreground and background of the collected vehicle image, and the Gaussian mixture model is used to calibrate each pixel in the vehicle image, obtain the weight π, mean vector μ and covariance matrix Σ, and perform iterative solution of the Gaussian mixture model to remove the background in the vehicle image, segment the edge of the vehicle, obtain the contour information of the vehicle, and separate the vehicle compartment from the image. The mixed Gaussian density formula is as follows:

[0046] And 0≤π i ≤1 (1)

[0047]

[0048] Among them, π i Represents the number of samples of the i-th Gaussian model; N i In the total sample size N, g i is the probability model formula of the i-th Gaussian model.

[0049] S2, evaluating the position of the carriage to determine whether the carriage is located in the designated area;

[0050] Specifically, in this embodiment, the implementation process of the above S2 is as follows:

[0051] The minimum spanning tree algorithm is used to determine the position of the carriage, and the boundary of the segmented carriage is determined to determine whether it is located in the designated area specified by the on-site marking. If the boundary of the carriage is consistent with the designated area, that is, the complete calibration area is segmented, the scrap steel in the carriage can be imaged and the scrap steel identification process can be carried out; otherwise, the vehicle is re-parked until the vehicle's carriage is located in the designated area. The formula is as follows:

[0052]

[0053] The internal variance of a component C is defined as the maximum weight in the component's minimum spanning tree.

[0054]

[0055] Two areas C1, The difference between is defined as the minimum weight edge connecting the two regions.

[0056]

[0057] The region comparison evaluates whether there is evidence of a boundary between a pair or more regions by checking whether the difference (Dif(C1, C2)) between two regions (the designated parking area and the area outside the designated parking area) is large relative to the internal difference within at least one region Int(C1) and Int(C2). A threshold function is used to control how large the difference between regions must be to be greater than the minimum internal difference.

[0058] S3, when the carriage is located in the designated area, collecting images of the scrap steel in the carriage to obtain a scrap steel image;

[0059] Specifically, in this embodiment, the implementation process of the above S3 is as follows:

[0060] A spherical camera is used to collect images of scrap steel in the carriage, and when shooting, the shooting path, shooting angle, and shooting focal length are adjusted to find an optimal combination to collect non-overlapping and the clearest scrap steel images. Specifically, this embodiment uses the greedy algorithm to solve the shooting operation and solves a clear, high-coverage, and non-overlapping operation combination; the process includes: using the greedy algorithm to establish a mathematical model for the shooting path, shooting angle, and shooting focal length, decomposing it into three sub-problems, and then separately solving the shooting path, shooting angle, and shooting focal length locally optimally; finally, combining the local optimal solutions to obtain the optimal solution for the shooting operation, thereby obtaining the scrap steel image we need.

[0061] S4, preprocessing the collected scrap steel image to obtain a preprocessed scrap steel image;

[0062] Specifically, in this embodiment, the preprocessing process includes: extraction of the area to be detected, image fusion, tilt correction, filtering and noise reduction, histogram equalization and image enhancement processing. The specific implementation process is as follows:

[0063] Useless information in the image is eliminated, and the image is deduplicated using a deduplication algorithm to retain the precise area to be detected. The image segmentation and filtering noise reduction method uses an adaptive Wiener filter. In addition, in order to address the problem of low contrast in the original image, the image is histogram-equalized to enhance the image contrast and perform image enhancement.

[0064] The collected images will have overlapping and non-overlapping parts, so it is necessary to pick out the overlapping parts and remove the non-overlapping parts. The implementation process is: make weight matrices for the overlapping parts of the images, with all the weights of the parts with values ​​as 1 and the weights of the NaN values ​​as 0; then multiply the weight matrices, with the weight value of the overlapping parts as 1 and the weight value of the non-overlapping parts as 0. A new weight matrix is ​​obtained, and then multiplied with the original image to extract the information of the overlapping parts. Then, a non-overlapping image that can be used is obtained.

[0065] S5, extracting features from the preprocessed scrap steel image to obtain corresponding scrap steel feature data;

[0066] Specifically, in this embodiment, the implementation process of the above S5 is as follows:

[0067] Use Figure 3 The tree-like wavelet transform algorithm shown decomposes the high-frequency band and low-frequency band of the preprocessed scrap steel image to obtain more detailed image features, performs wavelet decomposition according to the difference in high-frequency and low-frequency energy of the image, and sets a threshold T for whether to decompose the sub-region. The wavelet transform coefficient is used as a feature vector to extract the color feature, texture feature and edge feature of the image;

[0068] Among them, the standard for decomposing the high and low frequencies of the sub-regions is as follows:

[0069]

[0070] Among them, E indicates whether to continue decomposition, E i Indicates the energy value of this block sub-region. The algorithm automatically iterates to compare the energy value of the sub-region with the threshold until it meets the requirements and the iteration ends.

[0071] The geometric size of scrap steel is determined. According to the number of pixels of the target scrap steel with the geometric size to be determined and the number of pixels of the scrap steel of the same grade in the picture pre-specified by the expert, the similarity between the two scrap steels of the same grade is calculated. The formula is:

[0072]

[0073] Among them, x i It represents the similarity ratio between the target scrap steel and the scrap steel of the same grade, M i Indicates the number of pixels of the target scrap steel, M j It represents the number of pixels of the same grade of scrap steel in the image that is pre-defined by experts, j represents the grade of scrap steel, and k represents the similarity coefficient.

[0074] S6, performing DBN deep belief network learning on the extracted scrap steel feature data to obtain the scrap steel grade.

[0075] Specifically, in this embodiment, the implementation process of the above S6 is as follows:

[0076] Based on DBN deep belief network technology, a restricted Boltzmann machine is constructed; according to the actual training situation, several layers of restricted Boltzmann machines are constructed, such as Figure 2 The deep belief network shown;

[0077] The color features, texture features, edge features and geometric size data of scrap steel extracted from S5 are used as the visible layer input of the restricted Boltzmann machine, and the level of scrap steel grade determination is used as the output layer of the deep belief network composed of several layers of restricted Boltzmann machines.

[0078] Specifically, this embodiment initializes a single-layer RBM restricted Boltzmann machine, trains it, and then adds a single-layer RBM restricted Boltzmann machine, so that the output of the previous layer is used as the input of this layer, and so on, and finally fine-tunes it in combination with the BP algorithm. The recognition results are 0-heavy 6mm, 1-heavy 10mm, 2-heavy special grade, 3-scrap grade one, 4-scrap packaged block, and 7-scrap grade two.

[0079] In summary, this embodiment provides a scrap steel identification method, which starts from when a vehicle carrying scrap steel enters a specified position, electronically photographs the scrap metal in the vehicle, completes image information collection, and then pre-processes the photographed photos, extracts information features of the scrap steel based on processing segmentation, and then combines the deep belief network to determine the scrap steel grade. The purpose of accurately identifying and determining the scrap steel grade is achieved, thereby achieving efficient operation of the enterprise, and further solving the problem of inaccurate scrap steel grade detection in traditional manual identification.

[0080] Second embodiment

[0081] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment.

[0082] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) and one or more memories, wherein the memory stores at least one instruction, and the instruction is loaded by the processor to execute the above method.

[0083] Third embodiment

[0084] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0085] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0086] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0088] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0089] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A scrap steel recognition method based on image processing and DBN deep belief network, characterized in that: The scrap steel identification method based on image processing and DBN deep belief network includes: A camera device is used to collect images of the currently parked vehicle carrying scrap steel to obtain a vehicle image, and the collected vehicle image is segmented to obtain the compartment position of the vehicle carrying scrap steel; Evaluate the position of the carriage to determine whether the carriage is within the designated area; When the carriage is located in the designated area, images of the scrap steel in the carriage are collected to obtain scrap steel images; Preprocessing the collected scrap steel image to obtain a preprocessed scrap steel image; Perform feature extraction on the preprocessed scrap steel image to obtain corresponding scrap steel feature data; Perform DBN deep belief network learning on the extracted scrap steel feature data to obtain the corresponding grade of the scrap steel; The step of performing image segmentation on the collected vehicle image comprises: The grab cut image segmentation algorithm is used to calibrate the foreground and background of the collected vehicle image. The Gaussian mixture model is used to calibrate each pixel in the vehicle image, obtain the weight π, mean vector μ and covariance matrix Σ, and iteratively solve the Gaussian mixture model to remove the background in the vehicle image, segment the edge of the vehicle, obtain the contour information of the vehicle, and separate the vehicle compartment from the image. The image acquisition of the scrap steel in the carriage includes: A spherical camera is used to collect images of scrap steel in the carriage. When shooting, the greedy algorithm is used to solve the shooting operation to obtain a clear, high-coverage and non-overlapping operation combination. The shooting operation includes shooting path, shooting angle and shooting focal length; The method of solving the shooting operation by using the greedy algorithm includes: The greedy algorithm is used to establish a mathematical model for the shooting path, shooting angle and shooting focal length, which is decomposed into three sub-problems. Then, the shooting path, shooting angle and shooting focal length are solved locally optimally separately. Finally, the local optimal solutions are combined to obtain the optimal solution for the shooting operation, thereby obtaining the scrap steel image. The feature extraction of the pre-processed scrap steel image to obtain corresponding scrap steel feature data includes: The tree-structured wavelet transform algorithm is used to decompose the high-frequency band and low-frequency band of the preprocessed scrap steel image to obtain the image features. The wavelet decomposition is performed according to the difference in high-frequency and low-frequency energy of the image, and the wavelet transform coefficient is used as the feature vector to extract the color features, texture features and edge features of the image. The geometric dimensions of the scrap steel are determined to obtain the geometric dimension data of the scrap steel.

2. The scrap steel identification method based on image processing and DBN deep belief network according to claim 1 is characterized in that: Evaluate the position of the carriage to determine whether the carriage is within the specified area, including: The minimum spanning tree algorithm is used to determine whether the segmented carriage position is within the designated area specified by the on-site marking. If the carriage boundary is consistent with the designated area, the scrap steel in the carriage is imaged. Otherwise, the vehicle is re-docked until the vehicle's carriage is within the designated area.

3. The scrap steel identification method based on image processing and DBN deep belief network according to claim 1 is characterized in that: The preprocessing of the collected scrap steel images includes: The collected scrap steel images are subjected to detection area extraction, image fusion, tilt correction, filtering and noise reduction, histogram equalization and image enhancement processing.

4. The scrap steel identification method based on image processing and DBN deep belief network as claimed in claim 3 is characterized in that: The extracting of the area to be detected from the collected scrap steel image includes: eliminating useless information in the image, performing a deduplication algorithm on the image to extract the area to be detected.

5. The scrap steel identification method based on image processing and DBN deep belief network as claimed in claim 3 is characterized in that: The filtering and noise reduction adopts adaptive Wiener filter filtering.

6. The scrap steel identification method based on image processing and DBN deep belief network according to claim 1 is characterized in that: The extracted scrap steel feature data is subjected to DBN deep belief network learning to obtain the corresponding grade of the scrap steel, including: Based on DBN deep belief network technology, a restricted Boltzmann machine is constructed; according to the actual training situation, several layers of restricted Boltzmann machines are constructed, and a deep belief network is composed of several layers of restricted Boltzmann machines; The extracted scrap steel feature data is used as the visible layer input of the restricted Boltzmann machine, and the level of scrap steel grade determination is used as the output layer of a deep belief network composed of several layers of restricted Boltzmann machines; wherein the scrap steel feature data includes: color features, texture features, edge features and geometric size data of the scrap steel.

Images