Mining conveying belt three-dimensional longitudinal tearing detection method and system based on machine vision

Abstract

The invention relates to the technical field of conveyor belt detection, in particular to a mining conveyor belt three-dimensional longitudinal tearing detection method and system based on machine vision, and the method comprises the steps: shooting a regional conveyor belt to obtain a conveyor belt image set, carrying out the texture enhancement of the conveyor belt image set through a filter bank to obtain an enhanced conveyor belt image set, and constructing a longitudinal tearing recognition network, and obtaining a detection feature vector according to the longitudinal tear identification network and the enhanced conveyor belt image, determining a longitudinal tear region based on the detection feature vector, comparing the topological feature library with the longitudinal tear region to obtain a longitudinal tear category group, identifying high-frequency longitudinal tear categories in the longitudinal tear category group, and summarizing the high-frequency longitudinal tear categories to obtain a plurality of high-frequency longitudinal tear categories. The intelligent degree of mining conveyor belt detection can be improved, and the accuracy of mining conveyor belt detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of conveyor belt detection, and particularly relates to a three-dimensional longitudinal tear detection method and system for mine conveyor belts based on machine vision. BACKGROUND

[0002] Mine conveyor belts play a key role in mine production, and longitudinal tearing may occur during operation due to mechanical impact, wear and tear, or material aging, which not only leads to material leakage and reduces production efficiency, but also may cause safety accidents and pose a serious threat to personnel and equipment. Therefore, effective longitudinal tear detection of mine conveyor belts is of great significance to ensure the safety of mine production.

[0003] Traditional longitudinal tear detection of mine conveyor belts relies on manual regular inspection or simple sensor monitoring. The manual detection in this method has low efficiency, is easily affected by the environment, and is difficult to monitor in real time, and therefore the intelligent level needs to be improved. In addition, sensor monitoring can only be used for specific positions and cannot comprehensively cover the entire conveyor belt, which may lead to missed detection and reduce the accuracy of longitudinal tear detection. SUMMARY

[0004] The present application provides a three-dimensional longitudinal tear detection method and system for mine conveyor belts based on machine vision, which aims to improve the intelligent level of mine conveyor belt detection and improve the accuracy of mine conveyor belt detection.

[0005] To achieve the above object, the application provides a mine conveying belt three-dimensional longitudinal tear detection method based on machine vision, which comprises the following steps: camera installation is performed on a pre-constructed mine conveying belt to obtain a shooting unit, wherein the mine conveying belt comprises a plurality of regional conveying belts, and the shooting unit comprises a plurality of high-definition industrial cameras; regional conveying belts are extracted in sequence from the plurality of regional conveying belts, and the shooting unit is used to shoot the regional conveying belts to obtain a conveying belt image set, the texture direction range and the texture cycle set are determined based on the conveying belt image set, and a filter bank is designed according to the texture direction range and the texture cycle set; the filter bank is used to perform texture enhancement on each conveying belt image in the conveying belt image set to obtain an enhanced conveying belt image set; a longitudinal tear identification network is constructed based on a pre-acquired historical conveying belt image set, wherein the longitudinal tear identification network comprises a shallow convolutional network, a deep convolutional network and a fully connected layer; enhanced conveying belt images are extracted in sequence from the enhanced conveying belt image set, longitudinal tear detection is performed according to the longitudinal tear identification network and the enhanced conveying belt images to obtain a detection feature vector, wherein the detection feature vector comprises a detection longitudinal tear position and a detection longitudinal tear degree; if the detection longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, the longitudinal tear region is determined in the regional conveying belt based on the detection longitudinal tear position in the detection feature vector; the longitudinal tear region is compared according to a pre-constructed topological feature library to obtain a longitudinal tear category, wherein the topological feature library comprises a plurality of historical topological data, and the historical topological data comprises a historical topological feature group and a historical longitudinal tear category; the longitudinal tear categories corresponding to the enhanced conveying belt images are summarized to obtain a longitudinal tear category group, the high-frequency longitudinal tear categories in the longitudinal tear category group are identified, and the high-frequency longitudinal tear categories corresponding to the regional conveying belts are summarized to obtain a plurality of high-frequency longitudinal tear categories, and the three-dimensional longitudinal tear detection of the mine conveying belt based on machine vision is completed.

[0006] Optionally, the camera installation on the pre-constructed mine conveying belt to obtain the shooting unit comprises the following operations performed on each regional conveying belt in the plurality of regional conveying belts of the mine conveying belt: a regional shooting unit is constructed on both sides of the regional conveying belt, wherein the regional shooting unit comprises a shooting support and a plurality of high-definition industrial cameras, and the angles and heights at which the plurality of high-definition industrial cameras are installed are different; and the regional shooting units corresponding to each regional conveying belt are summarized to obtain the shooting unit.

[0007] Optionally, the determining the texture direction range and the texture period set based on the conveyor belt image set comprises: sequentially extracting conveyor belt images in the conveyor belt image set, and preprocessing the conveyor belt images to obtain target conveyor belt images, wherein the preprocessing comprises: image graying and gray value normalization, and the target conveyor belt images comprise a plurality of target pixel points; sequentially extracting the target pixel points in the target conveyor belt images, respectively calculating a horizontal gradient of the target pixel points in a preset horizontal direction and a vertical gradient of the target pixel points in a preset vertical direction, and calculating a gradient direction of the target pixel points based on the horizontal gradient and the vertical gradient; aggregating the gradient directions of each target pixel point to obtain a gradient direction set, setting a target direction range group based on the gradient direction set, and constructing a direction histogram of the target direction range group according to the gradient direction set, wherein the horizontal axis of the direction histogram represents the target direction range, and the vertical axis represents the number of gradient directions distributed in the target direction range; identifying a main direction range based on the direction histogram, and determining a center direction of the main direction range; selecting a plurality of parallel line segments in the target conveyor belt image based on the center direction, and performing texture period analysis on each parallel line segment in the plurality of parallel line segments to obtain a plurality of unit texture periods, and performing mean value calculation on the plurality of unit texture periods to obtain a texture period; aggregating the main direction range and the texture period corresponding to each conveyor belt image to obtain a main direction range set and a texture period set; and merging each main direction range in the main direction range set to obtain the texture direction range.

[0008] Optionally, the designing the filter set according to the texture direction range and the texture period set comprises: extracting a texture direction group in the texture direction range according to a preset direction number, and performing the following operations on each texture direction in the texture direction group: sequentially extracting texture periods in the texture period set, calculating a filter frequency based on the texture period, wherein the filter frequency is the inverse of the texture period; generating a filter according to the filter frequency and the texture direction; and aggregating the filters to obtain the filter set.

[0009] Optionally, the performing texture enhancement on each conveyor belt image in the conveyor belt image set by using the filter set to obtain an enhanced conveyor belt image set comprises: sequentially extracting conveyor belt images in the conveyor belt image set; dividing the conveyor belt images to obtain a divided conveyor belt image group, and performing the following operations on each divided conveyor belt image in the divided conveyor belt image group: sequentially extracting filters in the filter set, performing convolution operation on the divided conveyor belt image by using the filter to obtain a local convolution image, and calculating a convolution energy of the local convolution image, wherein the convolution energy is represented as: wherein, represents the convolution energy, and represents the number of pixel points in the horizontal axis and the number of pixel points in the vertical axis of the local convolution image, respectively. represents a gray value of a pixel point with coordinates in a local convolution image, and respectively represent an abscissa and an ordinate of the pixel point; the convolution energies are summarized to obtain a convolution energy group, a maximum convolution energy in the convolution energy group is identified, and a maximum filter corresponding to the maximum convolution energy is determined; the local convolution image is filtered by using the maximum filter to obtain an enhanced region image, and the enhanced region images are merged to obtain an enhanced conveyor belt image; and the enhanced conveyor belt images are summarized to obtain an enhanced conveyor belt image set.

[0010] Optionally, the longitudinal tear identification network is constructed based on the pre-acquired historical conveyor belt image set, including: performing longitudinal tear labeling on each historical conveyor belt image in the historical conveyor belt image set to obtain a labeled conveyor belt image set, wherein the longitudinal tear labeling includes longitudinal tear position labeling and longitudinal tear degree labeling; constructing a shallow convolution network and a deep convolution network, wherein the shallow convolution network includes one convolution layer and one pooling layer, and the deep convolution network includes multiple convolution layers and multiple pooling layers; fusing the shallow convolution network, the deep convolution network and a full connection layer to obtain a multi-dimensional convolution neural network; dividing a training image set in the labeled conveyor belt image set, and training the multi-dimensional convolution neural network by using the training image set to obtain the longitudinal tear identification network.

[0011] Optionally, the training of the multi-dimensional convolutional neural network by using the training image set to obtain the longitudinal tear identification network comprises: extracting training images in the training image set, inputting the training images into a shallow convolutional network and a deep convolutional network in the multi-dimensional convolutional neural network respectively to obtain a shallow feature vector and a deep feature vector, wherein the shallow feature vector comprises a shallow longitudinal tear position and a shallow longitudinal tear degree, and the deep feature vector comprises a deep longitudinal tear position and a deep longitudinal tear degree; splicing the shallow feature vector and the deep feature vector to obtain a fusion feature vector, wherein the splicing refers to connecting the beginning and the end of the shallow feature vector and the deep feature vector; inputting the fusion feature vector into a full connection layer in the multi-dimensional convolutional neural network to obtain a target feature vector; determining labeled data in the training images and constructing a labeled feature vector based on the labeled data, wherein the labeled data comprises a labeled longitudinal tear position and a labeled longitudinal tear degree; calculating a loss value based on the labeled feature vector, the target feature vector and a preset loss function, and optimizing the multi-dimensional convolutional neural network according to the loss value to obtain an updated multi-dimensional convolutional neural network; removing the training images from the training image set to obtain a removed training image set; taking the updated multi-dimensional convolutional neural network and the removed training image set as the multi-dimensional convolutional neural network and the training image set respectively, and returning to the step of extracting the training images in the training image set until the removed training image set is empty; confirming a last optimization step, and recording the updated multi-dimensional convolutional neural network in the last optimization step as the longitudinal tear identification network.

[0012] Optionally, the comparing the longitudinal tear region with the pre-constructed topology feature library to obtain a longitudinal tear category comprises: performing connected domain analysis on the longitudinal tear region to obtain a connected domain group, and determining a connected domain number and a connected domain area range based on the connected domain group, wherein the connected domain group comprises a plurality of connected domains; identifying a number of holes in the longitudinal tear region, and calculating an Euler number based on the connected domain number and the number of holes, wherein the Euler number is a difference value between the connected domain number and the number of holes; merging the Euler number and the connected domain area range to obtain a longitudinal tear topology feature group; sequentially extracting historical topology data in the topology feature library, and calculating a feature similarity between a historical topology feature group in the historical topology data and the longitudinal tear topology feature group; aggregating the feature similarities to obtain a feature similarity group, identifying a maximum similarity in the feature similarity group, and determining a similar topology data corresponding to the maximum similarity, and recording a historical longitudinal tear category in the similar topology data as the longitudinal tear category.

[0013] Optionally, the calculation of the feature similarity between the historical topology feature group in the historical topology data and the longitudinal tear topology feature group comprises: determining a historical Euler number in the historical topology feature group and a historical area range, identifying an overlapping area range between the historical area range and the connected domain area range; and calculating the feature similarity according to the historical Euler number, the Euler number, the connected domain area range and the overlapping area range by using the following formula: wherein, denotes a feature similarity, denotes a historical Euler number, denotes an Euler number, and denote a maximum area and a minimum area in the overlapping area range, respectively, and denote a maximum area and a minimum area in the connected domain area range, respectively.

[0014] To achieve the above object, the application further provides a three-dimensional longitudinal tear detection system for a mine conveyor belt based on machine vision, comprising: a shooting unit installation module for installing a camera on a pre-constructed mine conveyor belt to obtain a shooting unit, wherein the mine conveyor belt comprises a plurality of regional conveyor belts, the shooting unit comprises a plurality of high-definition industrial cameras, a regional conveyor belt is extracted in the plurality of regional conveyor belts in turn, and the regional conveyor belt is shot by using the shooting unit to obtain a conveyor belt image set, a texture direction range and a texture period set are determined based on the conveyor belt image set, and a filter bank is designed according to the texture direction range and the texture period set; a recognition network training module for performing texture enhancement on each conveyor belt image in the conveyor belt image set by using the filter bank to obtain an enhanced conveyor belt image set, and constructing a longitudinal tear recognition network based on a pre-acquired historical conveyor belt image set, wherein the longitudinal tear recognition network comprises a shallow convolutional network, a deep convolutional network and a fully connected layer; a longitudinal tear region recognition module for extracting an enhanced conveyor belt image in the enhanced conveyor belt image set in turn, performing longitudinal tear detection according to the longitudinal tear recognition network and the enhanced conveyor belt image to obtain a detection feature vector, wherein the detection feature vector comprises a detection longitudinal tear position and a detection longitudinal tear degree, and if the detection longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, a longitudinal tear region is determined in the regional conveyor belt based on the detection longitudinal tear position in the detection feature vector; and a longitudinal tear category determination module for comparing the longitudinal tear region according to a pre-constructed topological feature library to obtain a longitudinal tear category, wherein the topological feature library comprises a plurality of historical topological data, and the historical topological data comprises a historical topological feature group and a historical longitudinal tear category, a longitudinal tear category group is obtained by summarizing longitudinal tear categories corresponding to the enhanced conveyor belt image, a high-frequency longitudinal tear category in the longitudinal tear category group is recognized, and a plurality of high-frequency longitudinal tear categories are obtained by summarizing high-frequency longitudinal tear categories corresponding to the regional conveyor belt.

[0015] To solve the above problems, the application further provides an electronic device, comprising: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the above-mentioned three-dimensional longitudinal tear detection method for a mine conveyor belt based on machine vision.

[0016] To solve the above problems, the application further provides a computer readable storage medium, wherein at least one instruction is stored in the computer readable storage medium, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned machine vision-based three-dimensional longitudinal tear detection method for a mine conveying belt.

[0017] To solve the problems in the background art, first, high-definition industrial cameras are installed on both sides of the mine conveying belt in multiple regions of the mine conveying belt. This multi-angle and multi-height image acquisition method effectively improves the comprehensiveness of image detection. Then, the texture direction range and texture period set are determined, and a filter bank is designed. This step can accurately extract the texture features of the conveying belt image, which helps to improve the distinction between longitudinal tear marks and normal textures. Further, the designed filter bank is used for texture enhancement of the conveying belt image to obtain an enhanced conveying belt image. Texture enhancement can make the difference between the longitudinal tear region and the normal region more significant in terms of texture features, forming a high-contrast feature, thereby improving the accuracy of detection. The present scheme further constructs a longitudinal tear recognition network comprising a shallow convolutional network, a deep convolutional network and a fully connected layer. This network can fuse the shallow features and deep features of the image, realize comprehensive learning and accurate recognition of the longitudinal tear features in the conveying belt image, and effectively improve the accuracy and efficiency of detection. Then, longitudinal tear detection is performed according to the longitudinal tear recognition network and the enhanced conveying belt image to obtain a detection feature vector. This step realizes intelligent analysis of the conveying belt image, which can quickly and accurately locate and evaluate longitudinal tear damage. If the detection longitudinal tear degree in the detection feature vector is greater than the longitudinal tear degree threshold, the longitudinal tear region is determined in the regional conveying belt, and the longitudinal tear region is compared according to the topological feature library to obtain the longitudinal tear category. This step can further refine the classification of longitudinal tear damage, thereby helping maintenance personnel to better understand the morphology and characteristics of longitudinal tear, and further improving the efficiency and intelligence of longitudinal tear detection. Finally, the longitudinal tear categories corresponding to the enhanced conveying belt image are summarized, the high-frequency longitudinal tear categories are recognized and summarized, and the detection is finally completed. This step can provide maintenance personnel with a comprehensive overview of the damage situation of the conveying belt, highlighting the common and serious longitudinal tear problem areas, which helps to reasonably arrange maintenance plans and resource allocation, and ensures the stable operation of the conveying system. Therefore, the present application can improve the intelligence of mine conveying belt detection and improve the accuracy of mine conveying belt detection. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a machine vision-based three-dimensional longitudinal tear detection method for a mine conveying belt is provided for an embodiment of the present application. Figure 2 A functional module diagram of a machine vision-based three-dimensional longitudinal tear detection system for a mine conveying belt is provided for an embodiment of the present application. Figure 3A structural schematic diagram of an electronic device for implementing the machine vision-based three-dimensional longitudinal tear detection method of a mine conveying belt is provided in an embodiment of the present application.

[0019] Legend of reference signs: 1, electronic device; 10, processor; 11, memory; 12, bus.

[0020] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0022] An embodiment of the present application provides a machine vision-based three-dimensional longitudinal tear detection method of a mine conveying belt. The execution subject of the machine vision-based three-dimensional longitudinal tear detection method of the mine conveying belt includes but is not limited to at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the present application. In other words, the machine vision-based three-dimensional longitudinal tear detection method of the mine conveying belt can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0023] Reference Figure 1 As shown in the figure, a flowchart of the machine vision-based three-dimensional longitudinal tear detection method of the mine conveying belt is provided in an embodiment of the present application. In the embodiment, the machine vision-based three-dimensional longitudinal tear detection method of the mine conveying belt includes: S1, installing a camera on a pre-constructed mine conveying belt to obtain a shooting unit, wherein the mine conveying belt includes a plurality of regional conveying belts, and the shooting unit includes a plurality of high-definition industrial cameras.

[0024] It can be understood that the mine conveying belt refers to a continuous conveying belt for mine material transportation, the shooting unit refers to a high-definition industrial camera installed on both sides of the mine conveying belt, and the regional conveying belt refers to a partial region of the mine conveying belt, wherein the partial region is divided by human.

[0025] Further, the high-definition industrial camera includes a binocular stereo vision depth camera, which refers to a device capable of obtaining depth information of a scene from different angles through two cameras, for example, a ZED 2K Stereo Camera. The working principle of the ZED 2K Stereo Camera is as follows: the same scene is photographed by the left and right cameras respectively, the depth information is obtained by calculating the parallax between the corresponding points of the left and right images, and the depth information is converted into a depth image. By introducing the binocular stereo vision depth camera, a depth image can be generated in real time, and the binocular stereo vision is relatively flexible and suitable for various environments.

[0026] In detail, the camera installation on the pre-constructed mine conveyor belt to obtain the shooting unit includes: performing the following operations on each regional conveyor belt in the plurality of regional conveyor belts of the mine conveyor belt: constructing a regional shooting unit on both sides of the regional conveyor belt, wherein the regional shooting unit includes: a shooting support and a plurality of high-definition industrial cameras, and the angles and heights of the plurality of high-definition industrial cameras are different; and collecting the regional shooting units corresponding to each regional conveyor belt to obtain the shooting unit.

[0027] It should be explained that the regional shooting unit refers to a plurality of high-definition industrial cameras installed on both sides of the regional conveyor belt. The shooting support refers to a rigid structure for fixing and supporting the high-definition industrial camera, which is made of metal material and can adjust the height and angle to adapt to different shooting requirements. The purpose of the different angles and heights of the plurality of high-definition industrial cameras is to obtain images of the regional conveyor belt at different angles and heights, so as to reflect the longitudinal tearing condition of the regional conveyor belt in the three-dimensional space.

[0028] S2, sequentially extracting regional conveyor belts in the plurality of regional conveyor belts, and shooting the regional conveyor belts by using the shooting unit to obtain a conveyor belt image set, determining a texture direction range and a texture period set based on the conveyor belt image set, and designing a filter bank according to the texture direction range and the texture period set.

[0029] It should be explained that since the shooting unit includes a binocular stereo vision depth camera, the step of shooting the regional conveyor belt by using the shooting unit to obtain the conveyor belt image is as follows: the left and right cameras of the binocular stereo vision depth camera are used to shoot the regional conveyor belt at the same time, images of two different angles are obtained respectively, and then a depth image is obtained by parallax calculation. Combined with the ordinary color image, a color image containing depth information is generated as the conveyor belt image.

[0030] It can be understood that the conveyor belt image set refers to a set of images of the region conveyor belt captured by the shooting unit. The texture direction range refers to a range of texture directions of the conveyor belt image, wherein the texture direction refers to the direction of the texture feature in the image, such as the fiber arrangement direction of the conveyor belt surface. The texture period set refers to a set of multiple texture periods, wherein the texture period refers to the minimum interval (in the number of pixels) of texture repetition. The filter set refers to a combination of multiple filters, wherein the filter refers to a mathematical operator for enhancing texture in a specific direction and frequency, such as a Gabor filter, and the filter corresponds to a texture direction in the texture direction range and a texture period in the texture period set. The filter highlights the differences in texture features between the normal region and the longitudinal tear region, facilitating subsequent detection.

[0031] In detail, the determination of the texture direction range and the texture period set based on the conveyor belt image set comprises: sequentially extracting the conveyor belt images in the conveyor belt image set, and pre-processing the conveyor belt images to obtain target conveyor belt images, wherein the preprocessing comprises image graying and gray value normalization, and the target conveyor belt image comprises multiple target pixel points; sequentially extracting the target pixel points in the target conveyor belt image, respectively calculating the horizontal gradient of the target pixel points in the preset horizontal direction and the vertical gradient of the target pixel points in the preset vertical direction, and calculating the gradient direction of the target pixel points based on the horizontal gradient and the vertical gradient; aggregating the gradient direction of each target pixel point to obtain a gradient direction set, setting a target direction range group based on the gradient direction set, and constructing a direction histogram of the target direction range group according to the gradient direction set, wherein the horizontal axis of the direction histogram represents the target direction range, and the vertical axis represents the number of gradient directions distributed in the target direction range; identifying the main direction range based on the direction histogram, and determining the center direction of the main direction range; selecting multiple parallel line segments in the target conveyor belt image based on the center direction, and performing texture period analysis on each parallel line segment in the multiple parallel line segments to obtain multiple unit texture periods, and performing mean value calculation on the multiple unit texture periods to obtain the texture period; aggregating the main direction range and the texture period corresponding to each conveyor belt image to obtain a main direction range set and a texture period set; and merging each main direction range in the main direction range set to obtain the texture direction range.

[0032] The transverse gradient refers to the rate of change of the pixel gray value in the horizontal direction in the target conveyor belt image, and the longitudinal gradient refers to the rate of change of the pixel gray value in the vertical direction in the target conveyor belt image. The transverse gradient and the longitudinal gradient can be calculated by using the Sobel operator. The gradient direction refers to the direction of the gradient vector (i.e., the direction of the texture change), and it is calculated by using the arctangent function to calculate the ratio of the transverse gradient to the longitudinal gradient, i.e., gradient direction = arctan (longitudinal gradient / transverse gradient), wherein arctan () is the arctangent function. The target direction range set refers to the combination of multiple target direction ranges obtained by classifying the gradient direction set, for example, if the gradient direction set is: , , the gradient direction set can be classified to obtain a target direction range: . The direction histogram refers to a column chart that statistically shows the frequency of different gradient directions, wherein the horizontal axis is the direction range, and the vertical axis is the frequency. The main direction range refers to the target direction range with the largest number in the direction histogram. The center direction refers to the midpoint direction of the main direction range, for example, the center direction of is . The parallel line segment refers to a straight line segment selected along the center direction in the target conveyor belt image for texture period analysis. The parallel line segment is selected in the following manner: taking the center direction as the reference, selecting according to a preset length and a preset interval.

[0033] Further, the texture period analysis is to perform autocorrelation function calculation on each gray value contained in the parallel line segment to obtain the texture period, for example, performing autocorrelation function calculation on the gray value sequence of a certain parallel line segment: [100, 120, 130, 110, 90, 80, 100, 120, 130, 110, 90, 80,...], it is found that each 10-pixel peak value appears in the gray value sequence, and therefore the unit texture period of the parallel line segment is 10-pixel quantity. The texture period refers to the average value of multiple unit texture periods. The merging each main direction range in the main direction range set refers to taking the minimum direction and the maximum direction appearing in the main direction range set as the texture direction range.

[0034] In detail, the filter bank is designed according to the texture direction range and the texture period set, including: extracting a texture direction group in the texture direction range according to a preset direction number, and performing the following operation on each texture direction in the texture direction group: sequentially extracting a texture period in the texture period set, and calculating a filter frequency based on the texture period, wherein the filter frequency is the inverse of the texture period; generating a filter according to the filter frequency and the texture direction; and collecting the filters to obtain the filter bank.

[0035] It is understandable that the direction number refers to an artificially set constant, and the texture direction group is a combination of multiple texture directions, for example: a texture direction range is ( ), the number of directions is 4, then 4 texture directions are taken at equal intervals within the texture direction range, which are: 、 、 and . The filter frequency refers to the inverse of the texture period. The filter refers to a linear filter for extracting texture features of a specific direction and frequency, such as a Gabor filter, wherein the detailed steps of generating a filter according to the filter frequency and texture direction (taking the Gabor filter as an example) are: generating a corresponding filter by adjusting the wavelength (inverse of frequency) and direction (texture direction) of the Gabor kernel function, and its mathematical expression is the product of a complex sine wave and a Gaussian function.

[0036] S3. Use a filter bank to perform texture enhancement on each conveyor belt image in the conveyor belt image set to obtain an enhanced conveyor belt image set.

[0037] It should be explained that because mining conveyor belts have a distinct periodic texture on their surface, and longitudinal tears disrupt this texture pattern, texture enhancement of the conveyor belt image can make the longitudinal tears appear as distinct, high-contrast features in the enhanced image, facilitating subsequent detection and identification. Texture enhancement refers to the use of a filter to highlight the difference in texture between normal areas and longitudinally torn areas in the conveyor belt image. The enhanced conveyor belt image refers to the conveyor belt image after texture enhancement.

[0038] In detail, the method of using a filter bank to perform texture enhancement on each conveyor belt image in a conveyor belt image set to obtain an enhanced conveyor belt image set includes: sequentially extracting conveyor belt images from the conveyor belt image set; dividing the conveyor belt images to obtain divided conveyor belt image groups, and performing the following operations on each divided conveyor belt image in the divided conveyor belt image groups; sequentially extracting filters from the filter bank, performing convolution operations on the divided conveyor belt images using the filters to obtain local convolution images, and calculating the convolution energy of the local convolution images, wherein the convolution energy is expressed as: ,in, represents the convolution energy, and Represents the number of pixels on the horizontal axis and the number of pixels on the vertical axis of the local convolution image, respectively. Indicates that the coordinates in the local convolution image are The gray value of the pixel, and respectively represent the horizontal coordinate and the vertical coordinate of a pixel point; the convolution energies are summarized to obtain a convolution energy group, the maximum convolution energy in the convolution energy group is identified, and the maximum filter corresponding to the maximum convolution energy is determined; the maximum filter is used to filter the local convolution image to obtain an enhanced region image, and the enhanced region images are merged to obtain an enhanced conveyor belt image; and the enhanced conveyor belt images are summarized to obtain an enhanced conveyor belt image set.

[0039] It should be explained that the divided conveyor belt image group refers to a combination of the divided conveyor belt images, wherein the purpose of the division is to divide the image into smaller regions for local feature analysis and processing, and the division is performed by grid division on the image according to a preset size and interval, wherein the image division is a common step in image recognition neighborhood and will not be described in detail here. The convolution operation refers to the operation of pixel-by-pixel multiplication and accumulation of the filter and the local region of the image, the local convolution image refers to the divided conveyor belt image after the convolution operation. The convolution energy refers to the sum of the squares of the gray values of each pixel point in the local convolution image, which reflects the matching degree of the local region and the filter, and the greater the convolution energy, the more consistent the texture features of the region and the direction and period set by the filter. The maximum convolution energy refers to the convolution energy with the largest value in the convolution energy group. The maximum filter refers to the filter corresponding to the maximum convolution energy. The use of the maximum filter to filter the local convolution image refers to the application of the maximum filter to the local image for secondary enhancement to highlight the texture features in the image. Merging the enhanced region images refers to splicing multiple enhanced region images to obtain a complete enhanced conveyor belt image, i.e., the enhanced conveyor belt image.

[0040] S4, constructing a longitudinal tear identification network based on the pre-acquired historical conveyor belt image set, wherein the longitudinal tear identification network comprises a shallow convolution network, a deep convolution network and a fully connected layer.

[0041] It can be understood that the historical conveyor belt image set refers to a collection of images of the mine conveyor belt detected in the past period. The longitudinal tear identification network refers to a neural network model for identifying the longitudinal tear position and the longitudinal tear strength of the input enhanced conveyor belt image.

[0042] It needs to be explained that the shallow convolutional network can learn the gray scale, edge and other shallow features of the image. These shallow features are simple and direct, and can preliminarily judge the shape and general structure of the object in the image. The deep convolutional network contains more convolutional layers and pooling layers than the shallow convolutional network, so the deep convolutional network can learn the texture, shadow and other deep features in the image. These deep features can provide more complex semantic information, such as the material texture of the conveyor belt, the light and shadow on the surface, etc. These deep features can more accurately identify the position and information of the object in the image (such as the longitudinal tearing area in the conveyor belt image). If only a single deep convolutional network is used, the shallow features will be too much to be fused and cannot be effectively identified due to the more convolutional layers and pooling layers in the deep convolutional network. Shallow features and deep features have their own emphasis when describing the object information in the image. For example, in longitudinal tearing detection, edge features can roughly outline the tearing profile, and texture features can further determine the difference in texture patterns between the tearing area and the normal area. By combining the two features, the model can better understand the image content and improve the accuracy of detection.

[0043] Further, the full connection layer refers to a module for fusing the data output by the shallow convolutional network and the deep convolutional network.

[0044] In detail, the longitudinal tearing recognition network is constructed based on the pre-acquired historical conveyor belt image set, including: performing longitudinal tearing labeling on each historical conveyor belt image in the historical conveyor belt image set to obtain a labeled conveyor belt image set, wherein the longitudinal tearing labeling includes longitudinal tearing position labeling and longitudinal tearing degree labeling; constructing a shallow convolutional network and a deep convolutional network, wherein the shallow convolutional network includes one convolutional layer and one pooling layer, and the deep convolutional network includes multiple convolutional layers and multiple pooling layers; fusing the shallow convolutional network, the deep convolutional network and the full connection layer to obtain a multi-dimensional convolutional neural network; dividing a training image set in the labeled conveyor belt image set, and training the multi-dimensional convolutional neural network using the training image set to obtain the longitudinal tearing recognition network.

[0045] It can be understood that the longitudinal tear annotation refers to manual annotation of the longitudinal tear region in the historical conveyor belt image by relevant personnel, wherein the longitudinal tear position annotation refers to annotation of the specific position of the longitudinal tear region, which can be represented as a manually drawn frame, and the longitudinal tear degree annotation refers to annotation of the severity of the longitudinal tear in the longitudinal tear region. The multi-dimensional convolutional neural network refers to an integrated body obtained by fusing a shallow convolutional network, a deep convolutional network and a fully connected layer, wherein the detailed steps of fusing the shallow convolutional network, the deep convolutional network and the fully connected layer are as follows: the shallow feature vector extracted by the shallow convolutional network and the deep feature vector extracted by the deep convolutional network are spliced to form a fusion feature vector, and then the fusion feature vector is nonlinearly transformed by the fully connected layer to generate a final target feature vector. The training image set refers to a set of annotated conveyor belt images used to train the multi-dimensional convolutional neural network. In addition to the training image set, a validation image set and a test image set need to be further annotated in the annotated conveyor belt image set, wherein the validation image set and the test image set refer to sets of annotated conveyor belt images for validating and testing the multi-dimensional convolutional neural network, respectively, and the division of the training image set, the validation image set and the test image set, i.e., the purposes of these image sets, are all general technologies in the field of artificial intelligence, and will not be described here. In the deep convolutional network, the number of convolutional layers and pooling layers is set by humans. Optionally, the structure of a certain deep convolutional network is as follows: an input layer, a convolutional layer 1, a ReLU activation layer, a pooling layer 1, a convolutional layer 2, a ReLU activation layer, a pooling layer 2, and an output layer.

[0046] In detail, the training of the multi-dimensional convolutional neural network by using the training image set to obtain the longitudinal tear identification network comprises: extracting training images in the training image set, inputting the training images into a shallow convolutional network and a deep convolutional network in the multi-dimensional convolutional neural network respectively to obtain a shallow feature vector and a deep feature vector, wherein the shallow feature vector comprises a shallow longitudinal tear position and a shallow longitudinal tear degree, and the deep feature vector comprises a deep longitudinal tear position and a deep longitudinal tear degree; splicing the shallow feature vector and the deep feature vector to obtain a fusion feature vector, wherein the splicing refers to connecting the beginning and the end of the shallow feature vector and the deep feature vector; inputting the fusion feature vector into a full connection layer in the multi-dimensional convolutional neural network to obtain a target feature vector; determining labeled data in the training images and constructing a labeled feature vector based on the labeled data, wherein the labeled data comprises a labeled longitudinal tear position and a labeled longitudinal tear degree; calculating a loss value based on the labeled feature vector, the target feature vector and a preset loss function, and optimizing the multi-dimensional convolutional neural network according to the loss value to obtain an updated multi-dimensional convolutional neural network; removing the training images from the training image set to obtain a removed training image set; taking the updated multi-dimensional convolutional neural network and the removed training image set as the multi-dimensional convolutional neural network and the training image set respectively, and returning to the step of extracting the training images in the training image set until the removed training image set is empty; confirming a last optimization step, and recording the updated multi-dimensional convolutional neural network in the last optimization step as the longitudinal tear identification network.

[0047] It can be understood that the training image refers to an image arranged at the first position in the training image set. The shallow feature vector and the deep feature vector respectively refer to vectors output by the shallow convolutional network and the deep convolutional network, and correspond to longitudinal tear information artificially labeled, wherein the shallow longitudinal tear position is represented as a set of pixel point coordinates, and these pixel point coordinates form a longitudinal tear region. wherein, represents the target feature vector, represents an activation function, represents a weight matrix, represents the fusion feature vector, represents a bias term. The annotated longitudinal tear position and the annotated longitudinal tear degree respectively represent a longitudinal tear position and a longitudinal tear degree manually annotated in the training image. The annotated feature vector represents a vector composed of the annotated data, and the annotated feature vector has the same form and dimension as the target feature vector. The loss function and the loss value respectively represent a function for measuring the difference between the output value of the multi-dimensional convolutional neural network and the true annotated value and a calculation result of the function. Optionally, the loss function is a mean square error function or a cross-entropy function. The optimization of the multi-dimensional convolutional neural network according to the loss value means that the weight matrix and the bias term of the multi-dimensional convolutional neural network are optimized by using an optimization algorithm, wherein the optimization algorithm includes a stochastic gradient descent algorithm, an Adam optimization algorithm, etc. The training image set after elimination represents the training image set after the training image is eliminated. The last optimization step represents the optimization step corresponding to the return step performed for the last time.

[0048] S5. sequentially extracting the enhanced conveyor belt images in the enhanced conveyor belt image set, performing longitudinal tear detection according to the longitudinal tear identification network and the enhanced conveyor belt images, and obtaining a detection feature vector, wherein the detection feature vector includes a detection longitudinal tear position and a detection longitudinal tear degree.

[0049] It can be understood that the detection feature vector represents a vector output by the longitudinal tear identification network, wherein the detection feature vector is represented as: , wherein, represents the detection feature vector, represents the detection longitudinal tear position, wherein, and respectively represent the coordinates of the first pixel point and the coordinates of the first pixel point of the edge of the longitudinal tear region, represents the number of pixel points of the edge of the longitudinal tear region, represents the detection longitudinal tear degree.

[0050] S6. If the detection longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, determining a longitudinal tear region in the area conveyor belt based on the detection longitudinal tear position in the detection feature vector.

[0051] It should be explained that the longitudinal tear degree threshold represents a constant artificially set, and when the detection longitudinal tear degree is greater than the longitudinal tear degree threshold, it indicates that a serious longitudinal tear phenomenon occurs in the area conveyor belt corresponding to the enhanced conveyor belt image, and manual repair is required. The longitudinal tear region represents an edge region composed of each coordinate in the detection longitudinal tear position.

[0052] S7. performing comparison on the longitudinal tear region according to a pre-constructed topological feature library, and obtaining a longitudinal tear category, wherein the topological feature library includes a plurality of historical topological data, and the historical topological data includes a historical topological feature group and a historical longitudinal tear category.

[0053] It can be understood that before comparison, the longitudinal tear topology feature group of the longitudinal tear region also needs to be extracted, which refers to the combination of topology features in the longitudinal tear region, wherein the topology features are used to represent the shape connectivity, hole number and area distribution and other structural characteristics of the longitudinal tear region. The topology feature library refers to a collection of a plurality of historical topology data artificially constructed, wherein the historical topology data includes: historical topology feature groups and historical longitudinal tear categories, the historical topology feature groups are obtained in the same way as the longitudinal tear topology feature group extraction, and the historical longitudinal tear categories are determined by relevant personnel.

[0054] In detail, the comparison of the longitudinal tear region according to the pre-constructed topology feature library to obtain the longitudinal tear category includes: performing connected domain analysis on the longitudinal tear region to obtain a connected domain group, and determining the number of connected domains and the area range of connected domains based on the connected domain group, wherein the connected domain group includes a plurality of connected domains; identifying the number of holes in the longitudinal tear region, and calculating the Euler number based on the number of connected domains and the number of holes, wherein the Euler number is the difference between the number of connected domains and the number of holes; merging the Euler number and the area range of connected domains to obtain the longitudinal tear topology feature group; extracting historical topology data in the topology feature library in turn, and calculating the feature similarity between the historical topology feature group in the historical topology data and the longitudinal tear topology feature group; aggregating the feature similarity to obtain a feature similarity group, identifying the maximum similarity in the feature similarity group, and determining the similar topology data corresponding to the maximum similarity, and recording the historical longitudinal tear category in the similar topology data as the longitudinal tear category.

[0055] It can be understood that the connected domain group refers to the combination of a plurality of connected domains, wherein the connected domain refers to a pixel region connected to each other in the image. The number of connected domains refers to the number of connected domains in the connected domain group, wherein each connected domain corresponds to a connected domain area, and the area range of connected domains refers to the range composed of the maximum connected domain area and the minimum connected domain area in the connected domain area group corresponding to the connected domain group. The number of holes refers to the number of holes in the longitudinal tear region, wherein the hole refers to a closed region not connected inside the connected domain. The Euler number refers to the difference between the number of connected domains and the number of holes, which represents the topological complexity of the longitudinal tear region. The larger the Euler number is, the simpler the structure is (the fewer the holes are or the more the connected domains are). The feature similarity refers to a numerical value quantifying the similarity between the historical topology feature group and the longitudinal tear topology feature group. The larger the feature similarity is, the more similar the historical topology feature group and the longitudinal tear topology feature group are. The maximum similarity refers to the feature similarity with the largest value in the feature similarity group. The similar topology data refers to the historical topology data corresponding to the maximum similarity.

[0056] In detail, the calculation of the feature similarity between the historical topological feature group and the longitudinal tear topological feature group in the historical topological data comprises: determining a historical Euler number and a historical area range in the historical topological feature group, and identifying an overlapping area range between the historical area range and the connected domain area range; and calculating the feature similarity according to the historical Euler number, the Euler number, the connected domain area range and the overlapping area range by using the following formula: wherein, represents the feature similarity, represents the historical Euler number, represents the Euler number, and respectively represent the maximum area and the minimum area in the overlapping area range, and respectively represent the maximum area and the minimum area in the connected domain area range.

[0057] It can be understood that the historical Euler number refers to the Euler number in the historical topological feature group, and the historical area range refers to the connected domain area range in the historical topological feature group. The overlapping area range refers to the intersection of the historical area range and the connected domain area range, for example: the historical area range is (0, 1), the connected domain area range is (0.5, 1), wherein, , , , the overlapping area range is (0.5, 1).

[0058] S8, the longitudinal tear categories corresponding to the enhanced conveyor belt images are summarized to obtain a longitudinal tear category group, a high-frequency longitudinal tear category in the longitudinal tear category group is identified, and the high-frequency longitudinal tear categories corresponding to the regional conveyor belts are summarized to obtain a plurality of high-frequency longitudinal tear categories, and the machine vision-based three-dimensional longitudinal tear detection of the mine conveyor belt is completed.

[0059] It can be understood that the high-frequency longitudinal tear category refers to the longitudinal tear category with the highest frequency in the longitudinal tear category group. It should be noted that if there is more than one longitudinal tear category with the highest frequency in the longitudinal tear category group, they are all recorded as high-frequency longitudinal tear categories. The plurality of high-frequency longitudinal tear categories are longitudinal tear categories occurring in different regions of the mine conveyor belt. Through the plurality of high-frequency longitudinal tear categories, detection and maintenance information can be provided for relevant personnel.

[0060] ​In order to solve the problems described in the background technology, the present invention first installs high-definition industrial cameras on both sides of the conveyor belt in multiple areas of the mining conveyor belt. This multi-angle and multi-height image acquisition method effectively improves the comprehensiveness of image detection. Then, the texture direction range and texture period set are determined, and a filter group is designed. This step can accurately extract the texture features of the conveyor belt image, which helps to improve the distinction between longitudinal tear marks and normal textures. Further, the conveyor belt image is texture enhanced using the designed filter group to obtain an enhanced conveyor belt image. Texture enhancement can make the difference in texture features between the longitudinal tear area and the normal area more significant, forming a high-contrast feature, thereby improving the accuracy of detection. This scheme also constructs a longitudinal tear recognition network including a shallow convolutional network, a deep convolutional network and a fully connected layer. The network can fuse the shallow and deep features of the image to achieve comprehensive learning and accurate recognition of the longitudinal tear features in the conveyor belt image, effectively improving the detection accuracy. Accuracy and efficiency, then longitudinal tear detection is performed based on the longitudinal tear recognition network and the enhanced conveyor belt image to obtain a detection feature vector. This step realizes the intelligent analysis of the conveyor belt image, and can quickly and accurately locate and evaluate the longitudinal tear damage. If the detected longitudinal tear degree in the detection feature vector is greater than the longitudinal tear degree threshold, the longitudinal tear area is determined in the regional conveyor belt, and the longitudinal tear area is compared according to the topological feature library to obtain the longitudinal tear category. This step can further refine the classification of longitudinal tear damage, thereby helping maintenance personnel to have a deeper understanding of the morphology and characteristics of longitudinal tearing, thereby improving the efficiency and intelligence of longitudinal tear detection. Finally, the longitudinal tear categories corresponding to the enhanced conveyor belt image are summarized, and high-frequency longitudinal tear categories are identified and summarized, and the detection is finally completed. This step can provide maintenance personnel with a comprehensive overview of the conveyor belt damage situation, highlighting common and serious longitudinal tear problem areas, helping to reasonably arrange maintenance plans and resource allocation, and ensuring the stable operation of the conveying system. Therefore, the present invention can improve the intelligence level of mining conveyor belt detection and improve the accuracy of mining conveyor belt detection.

[0061] like Figure 2 1 is a functional module diagram of a three-dimensional longitudinal tear detection system for a mining conveyor belt based on machine vision provided by one embodiment of the present invention.

[0062] The machine vision-based 3D longitudinal tear detection system 100 for mining conveyor belts described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the machine vision-based 3D longitudinal tear detection system 100 for mining conveyor belts can include a camera installation module 101, a recognition network training module 102, a longitudinal tear area identification module 103, and a longitudinal tear category determination module 104. A module, also referred to as a unit, is a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the electronic device's memory.

[0063] The photographing unit installation module 101 is used for installing a camera on a pre-constructed mine conveying belt to obtain a photographing unit, wherein the mine conveying belt comprises a plurality of regional conveying belts, the photographing unit comprises a plurality of high-definition industrial cameras, a regional conveying belt is sequentially extracted from the plurality of regional conveying belts, and the regional conveying belt is photographed by using the photographing unit to obtain a conveying belt image set; a texture direction range and a texture cycle set are determined based on the conveying belt image set, and a filter bank is designed according to the texture direction range and the texture cycle set; the identification network training module 102 is used for performing texture enhancement on each conveying belt image in the conveying belt image set by using the filter bank to obtain an enhanced conveying belt image set, and a longitudinal tear identification network is constructed based on a pre-acquired historical conveying belt image set, wherein the longitudinal tear identification network comprises a shallow convolutional network, a deep convolutional network and a fully connected layer; the longitudinal tear region identification module 103 is used for sequentially extracting an enhanced conveying belt image from the enhanced conveying belt image set, performing longitudinal tear detection according to the longitudinal tear identification network and the enhanced conveying belt image to obtain a detection feature vector, wherein the detection feature vector comprises a detection longitudinal tear position and a detection longitudinal tear degree, and if the detection longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, a longitudinal tear region is determined in the regional conveying belt based on the detection longitudinal tear position in the detection feature vector; and the longitudinal tear category determination module 104 is used for comparing the longitudinal tear region according to a pre-constructed topological feature library to obtain a longitudinal tear category, wherein the topological feature library comprises a plurality of historical topological data, and the historical topological data comprises a historical topological feature group and a historical longitudinal tear category; a longitudinal tear category group is obtained by summarizing longitudinal tear categories corresponding to the enhanced conveying belt image; a high-frequency longitudinal tear category is identified in the longitudinal tear category group; and a plurality of high-frequency longitudinal tear categories are obtained by summarizing high-frequency longitudinal tear categories corresponding to the regional conveying belt.

[0064] In detail, the modules in the machine vision-based mine conveying belt three-dimensional longitudinal tear detection system 100 in the embodiment of the present application use the same technical means as the machine vision-based mine conveying belt three-dimensional longitudinal tear detection method in the above Figure 1 , and can produce the same technical effects, which will not be described here.

[0065] As shown in Figure 3 , it is a structural schematic diagram of an electronic device for implementing the machine vision-based mine conveying belt three-dimensional longitudinal tear detection method according to an embodiment of the present application.

[0066] The electronic device 1 can comprise a processor 10, a memory 11 and a bus 12, and can further comprise a computer program stored in the memory 11 and executable on the processor 10, such as a machine vision-based mine conveying belt three-dimensional longitudinal tear detection method program.

[0067] The memory 11 includes at least one type of readable storage medium, such as flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Further, the memory 11 includes both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used not only to store application software and various data installed on the electronic device 1, such as the code of the machine vision-based mine conveyor belt three-dimensional longitudinal tear detection method program, but also to temporarily store data that has been output or will be output.

[0068] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips, etc. The processor 10 is the control core of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as the machine vision-based mine conveyor belt three-dimensional longitudinal tear detection method program, etc.), and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0069] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0070] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 3The illustrated structure does not constitute a limitation on the electronic device 1, and can include fewer or more components than illustrated, or combine certain components, or different component arrangements.

[0071] For example, although not shown, the electronic device 1 can also include a power source (such as a battery) to power the various components, and preferably the power source can be logically connected to the at least one processor 10 through a power management system, so that the power management system can implement functions such as charge management, discharge management, and power consumption management. The power source can also include one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0072] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, and the like), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0073] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, and the like. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0074] The machine vision-based mine conveying belt three-dimensional longitudinal tear detection method program stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when running in the processor 10, can realize: camera installation on a pre-constructed mine conveying belt to obtain a shooting unit, wherein the mine conveying belt comprises multiple regional conveying belts, and the shooting unit comprises multiple high-definition industrial cameras; regional conveying belts are sequentially extracted from the multiple regional conveying belts, and the regional conveying belts are shot by using the shooting unit to obtain a conveying belt image set, a texture direction range and a texture period set are determined based on the conveying belt image set, and a filter bank is designed according to the texture direction range and the texture period set; each conveying belt image in the conveying belt image set is subjected to texture enhancement by using the filter bank to obtain an enhanced conveying belt image set; a longitudinal tear identification network is constructed based on a pre-acquired historical conveying belt image set, wherein the longitudinal tear identification network comprises: a shallow convolutional network, a deep convolutional network, and a fully connected layer; enhanced conveying belt images are sequentially extracted from the enhanced conveying belt image set, and longitudinal tear detection is performed according to the longitudinal tear identification network and the enhanced conveying belt images to obtain a detection feature vector, wherein the detection feature vector comprises: a detection longitudinal tear position and a detection longitudinal tear degree; if the detection longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, a longitudinal tear region is determined in the regional conveying belt based on the detection longitudinal tear position in the detection feature vector; the longitudinal tear region is compared according to a pre-constructed topological feature library to obtain a longitudinal tear category, wherein the topological feature library comprises multiple historical topological data, and the historical topological data comprises: a historical topological feature group and a historical longitudinal tear category; longitudinal tear categories corresponding to the enhanced conveying belt images are summarized to obtain a longitudinal tear category group, high-frequency longitudinal tear categories in the longitudinal tear category group are identified, and high-frequency longitudinal tear categories corresponding to the regional conveying belts are summarized to obtain multiple high-frequency longitudinal tear categories, and the machine vision-based mine conveying belt three-dimensional longitudinal tear detection is completed.

[0075] Specifically, the specific implementation method of the processor 10 to the above instructions can refer to Figures 1 to 3 The description of related steps in the corresponding embodiments will not be repeated here.

[0076] Further, the modules / units integrated in the electronic device 1 are implemented in the form of software function units and sold or used as independent products, which can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or system capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory).

[0077] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device: camera installation is performed on a pre-constructed mine conveying belt to obtain a shooting unit, wherein the mine conveying belt comprises a plurality of regional conveying belts, and the shooting unit comprises a plurality of high-definition industrial cameras; regional conveying belts are sequentially extracted from the plurality of regional conveying belts, and the regional conveying belts are shot by using the shooting unit to obtain a conveying belt image set, a texture direction range and a texture cycle set are determined based on the conveying belt image set, and a filter bank is designed according to the texture direction range and the texture cycle set; texture enhancement is performed on each conveying belt image in the conveying belt image set by using the filter bank to obtain an enhanced conveying belt image set; a longitudinal tear identification network is constructed based on a pre-acquired historical conveying belt image set, wherein the longitudinal tear identification network comprises a shallow convolutional network, a deep convolutional network and a fully connected layer; enhanced conveying belt images are sequentially extracted from the enhanced conveying belt image set, longitudinal tear detection is performed according to the longitudinal tear identification network and the enhanced conveying belt images to obtain a detection feature vector, wherein the detection feature vector comprises a detection longitudinal tear position and a detection longitudinal tear degree; if the detection longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, a longitudinal tear region is determined in the regional conveying belt based on the detection longitudinal tear position in the detection feature vector; the longitudinal tear region is compared according to a pre-constructed topological feature library to obtain a longitudinal tear category, wherein the topological feature library comprises a plurality of historical topological data, and the historical topological data comprises a historical topological feature group and a historical longitudinal tear category; longitudinal tear categories corresponding to the enhanced conveying belt images are summarized to obtain a longitudinal tear category group, a high-frequency longitudinal tear category in the longitudinal tear category group is identified, and high-frequency longitudinal tear categories corresponding to the regional conveying belts are summarized to obtain a plurality of high-frequency longitudinal tear categories, and a machine vision-based three-dimensional longitudinal tear detection of the mine conveying belt is completed.

[0078] In several embodiments provided by the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other manners. For example, the system embodiments described above are merely schematic. In practice, the division of the system can be different from the system embodiments described above.

[0079] The modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0080] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or hardware plus software functional modules.

[0081] It is apparent for a person skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but that the present application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not to limit the present application, and although the present application has been described in detail with reference to the preferred embodiments, those ordinarily skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the present application.

Claims

1. A method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision, characterized in that: The method comprises: installing a camera on a pre-constructed mining conveyor belt to obtain a shooting unit, wherein the mining conveyor belt includes a plurality of regional conveyor belts, and the shooting unit includes a plurality of high-definition industrial cameras; sequentially extracting regional conveyor belts from the plurality of regional conveyor belts, and using the shooting unit to shoot the regional conveyor belts to obtain a conveyor belt image set, determining a texture direction range and a texture period set based on the conveyor belt image set, and designing a filter group according to the texture direction range and the texture period set; using the filter group to perform texture enhancement on each conveyor belt image in the conveyor belt image set to obtain an enhanced conveyor belt image set; constructing a longitudinal tear recognition network based on a pre-acquired historical conveyor belt image set, wherein the longitudinal tear recognition network includes: a shallow convolutional network, a deep convolutional network and a fully connected layer; sequentially extracting enhanced conveyor belt images from the enhanced conveyor belt image set, and determining the texture direction range and texture period set based on the texture direction range and texture period set; and designing a filter group according to the texture direction range and texture period set. The recognition network and the enhanced conveyor belt image are used to detect longitudinal tears to obtain a detection feature vector, wherein the detection feature vector includes: detecting the longitudinal tear position and detecting the longitudinal tear degree; if the detected longitudinal tear degree in the detection feature vector is greater than a preset longitudinal tear degree threshold, then based on the detected longitudinal tear position in the detection feature vector, the longitudinal tear area is determined in the regional conveyor belt; the longitudinal tear area is compared according to the pre-constructed topological feature library to obtain the longitudinal tear category, wherein the topological feature library includes multiple historical topological data, and the historical topological data includes: historical topological feature groups and historical longitudinal tear categories; the longitudinal tear categories corresponding to the enhanced conveyor belt image are summarized to obtain a longitudinal tear category group, the high-frequency longitudinal tear categories in the longitudinal tear category group are identified, and the high-frequency longitudinal tear categories corresponding to the regional conveyor belt are summarized to obtain multiple high-frequency longitudinal tear categories, thereby completing the three-dimensional longitudinal tear detection of the mining conveyor belt based on machine vision.

2. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 1, characterized in that: The camera is installed on the pre-constructed mining conveyor belt to obtain a shooting unit, including: performing the following operations on each of the multiple regional conveyor belts of the mining conveyor belt: constructing a regional shooting unit on both sides of the regional conveyor belt, wherein the regional shooting unit includes: a shooting bracket and multiple high-definition industrial cameras, and the multiple high-definition industrial cameras are installed at different angles and heights; summarizing the regional shooting units corresponding to each regional conveyor belt to obtain a shooting unit.

3. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 2, characterized in that: The method for determining the texture direction range and texture period set based on the conveyor belt image set includes: extracting conveyor belt images in sequence from the conveyor belt image set, preprocessing the conveyor belt images, and obtaining a target conveyor belt image, wherein the preprocessing includes: image grayscale and grayscale value normalization, and the target conveyor belt image includes a plurality of target pixel points; extracting target pixel points in sequence from the target conveyor belt image, respectively calculating the transverse gradient of the target pixel points in a preset horizontal direction and the longitudinal gradient of the target pixel points in a preset vertical direction, and calculating the gradient direction of the target pixel points based on the transverse gradient and the longitudinal gradient; summarizing the gradient direction of each target pixel point to obtain a gradient direction set, setting a target direction range group based on the gradient direction set, and constructing the target direction set according to the gradient direction set. A direction histogram of the target direction range group is obtained, wherein the horizontal axis of the direction histogram represents the target direction range and the vertical axis represents the number of gradient directions distributed in the target direction range; based on the direction histogram, the main direction range is identified, and the center direction of the main direction range is determined; based on the center direction, multiple parallel line segments are selected in the target conveyor belt image, and texture period analysis is performed on each of the multiple parallel line segments to obtain multiple unit texture periods, and the mean of the multiple unit texture periods is calculated to obtain the texture period; the main direction range and texture period corresponding to each conveyor belt image are respectively summarized to obtain a main direction range set and a texture period set; each main direction range in the main direction range set is merged to obtain a texture direction range.

4. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 3, characterized in that: The method of designing a filter group based on a texture direction range and a texture period set includes: extracting a texture direction group within the texture direction range according to a preset number of directions, performing the following operations on each texture direction in the texture direction group: sequentially extracting texture periods from the texture period set, calculating a filter frequency based on the texture period, wherein the filter frequency is the inverse of the texture period; generating a filter based on the filter frequency and the texture direction; and summarizing the filters to obtain a filter group.

5. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 4, characterized in that: The method uses a filter bank to perform texture enhancement on each conveyor belt image in a conveyor belt image set to obtain an enhanced conveyor belt image set, comprising: sequentially extracting conveyor belt images from the conveyor belt image set; dividing the conveyor belt images to obtain divided conveyor belt image groups, and performing the following operations on each divided conveyor belt image in the divided conveyor belt image group; sequentially extracting filters from the filter bank, performing convolution operations on the divided conveyor belt images using the filters to obtain local convolution images, and calculating convolution energy of the local convolution images, wherein the convolution energy is expressed as: ,in, represents the convolution energy, and Represents the number of pixels on the horizontal axis and the number of pixels on the vertical axis of the local convolution image, respectively. Indicates that the coordinates in the local convolution image are The gray value of the pixel, and Represent the horizontal coordinate and vertical coordinate of the pixel point respectively; summarize the convolution energy to obtain a convolution energy group, identify the maximum convolution energy in the convolution energy group, and determine the maximum filter corresponding to the maximum convolution energy; use the maximum filter to filter the local convolution image to obtain an enhanced area image, merge the enhanced area images to obtain an enhanced conveyor belt image; summarize the enhanced conveyor belt images to obtain an enhanced conveyor belt image set.

6. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 5, characterized in that: The method constructs a longitudinal tear recognition network based on a pre-acquired historical conveyor belt image set, including: longitudinal tear annotation of each historical conveyor belt image in the historical conveyor belt image set to obtain a labeled conveyor belt image set, wherein the longitudinal tear annotation includes longitudinal tear position annotation and longitudinal tear degree annotation; constructs a shallow convolutional network and a deep convolutional network, wherein the shallow convolutional network includes a convolution layer and a pooling layer, and the deep convolutional network includes multiple convolution layers and multiple pooling layers; fuses the shallow convolutional network, the deep convolutional network and the fully connected layer to obtain a multidimensional convolutional neural network; divides a training image set in the labeled conveyor belt image set, and uses the training image set to train the multidimensional convolutional neural network to obtain a longitudinal tear recognition network.

7. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 6, characterized in that: The method uses a training image set to train a multidimensional convolutional neural network to obtain a vertical tear recognition network, including: extracting training images from the training image set, inputting the training images into a shallow convolutional network and a deep convolutional network in the multidimensional convolutional neural network respectively, obtaining a shallow feature vector and a deep feature vector, wherein the shallow feature vector includes: a shallow vertical tear position and a shallow vertical tear degree, and the deep feature vector includes: a deep vertical tear position and a deep vertical tear degree; splicing the shallow feature vector and the deep feature vector to obtain a fused feature vector, wherein the splicing refers to connecting the beginning and the end of the shallow feature vector and the deep feature vector; inputting the fused feature vector into a fully connected layer in the multidimensional convolutional neural network to obtain a target feature vector; determining the target feature vector in the training image. Annotate data, and construct a labeled feature vector based on the labeled data, wherein the labeled data includes: a labeled vertical tear position and a labeled vertical tear degree; calculate a loss value based on the labeled feature vector, a target feature vector and a preset loss function, and optimize the multidimensional convolutional neural network according to the loss value to obtain an updated multidimensional convolutional neural network; eliminate the training image from the training image set to obtain a eliminated training image set; use the updated multidimensional convolutional neural network and the eliminated training image set as the multidimensional convolutional neural network and the training image set respectively, and return to the step of extracting training images from the training image set until the eliminated training image set is an empty set; confirm the last optimization step, and record the updated multidimensional convolutional neural network in the last optimization step as a vertical tear recognition network.

8. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 7, characterized in that: The method compares the longitudinal tear area according to the pre-constructed topological feature library to obtain the longitudinal tear category, including: performing a connected domain analysis on the longitudinal tear area to obtain a connected domain group, and determining the number of connected domains and the area range of the connected domains based on the connected domain group, wherein the connected domain group includes multiple connected domains; identifying the number of holes in the longitudinal tear area, and calculating the Euler number based on the number of connected domains and the number of holes, wherein the Euler number is the difference between the number of connected domains and the number of holes; combining the Euler number and the area range of the connected domain to obtain a longitudinal tear topological feature group; extracting historical topological data in the topological feature library in sequence, and calculating the feature similarity between the historical topological feature group and the longitudinal tear topological feature group in the historical topological data; summarizing the feature similarity to obtain a feature similarity group, identifying the maximum similarity in the feature similarity group, and determining the similar topological data corresponding to the maximum similarity, and recording the historical longitudinal tear category in the similar topological data as the longitudinal tear category.

9. The method for detecting three-dimensional longitudinal tearing of a mining conveyor belt based on machine vision according to claim 8, characterized in that: The calculation of feature similarity between the historical topological feature group and the longitudinal tearing topological feature group in the historical topological data includes: determining the historical Euler number and the historical area range in the historical topological feature group, identifying the overlapping area range between the historical area range and the connected domain area range; and calculating the feature similarity using the following formula based on the historical Euler number, the Euler number, the connected domain area range, and the overlapping area range: ,in, Represents feature similarity, represents the historical Euler number, represents the Euler number, and Respectively represent the maximum area and minimum area in the overlapping area range, and They represent the maximum and minimum areas in the connected domain area range respectively.

10. A three-dimensional longitudinal tear detection system for mining conveyor belts based on machine vision, characterized in that: The system includes: a shooting unit installation module, which is used to install a camera on a pre-built mining conveyor belt to obtain a shooting unit, wherein the mining conveyor belt includes multiple regional conveyor belts, and the shooting unit includes multiple high-definition industrial cameras. Regional conveyor belts are sequentially extracted from the multiple regional conveyor belts, and the regional conveyor belts are photographed by using the shooting unit to obtain a conveyor belt image set, based on the conveyor belt image set, a texture direction range and a texture period set are determined, and a filter group is designed according to the texture direction range and the texture period set; a recognition network training module, which is used to use the filter group to perform texture enhancement on each conveyor belt image in the conveyor belt image set to obtain an enhanced conveyor belt image set, and a longitudinal tear recognition network is constructed based on the pre-acquired historical conveyor belt image set, wherein the longitudinal tear recognition network includes: a shallow convolutional network, a deep convolutional network and a fully connected layer; a longitudinal tear region recognition module, which is used to enhance the conveyor belt image in the enhanced conveyor belt image set. The enhanced conveyor belt images are extracted in sequence from the belt images, and longitudinal tear detection is performed based on the longitudinal tear recognition network and the enhanced conveyor belt images to obtain a detection feature vector, wherein the detection feature vector includes: detecting the longitudinal tear position and detecting the longitudinal tear degree. If the detected longitudinal tear degree in the detection feature vector is greater than the preset longitudinal tear degree threshold, the longitudinal tear area is determined in the regional conveyor belt based on the detected longitudinal tear position in the detection feature vector; a longitudinal tear category determination module is used to compare the longitudinal tear area according to a pre-constructed topological feature library to obtain the longitudinal tear category, wherein the topological feature library includes multiple historical topological data, and the historical topological data includes: historical topological feature groups and historical longitudinal tear categories, summarizing the longitudinal tear categories corresponding to the enhanced conveyor belt images to obtain a longitudinal tear category group, identifying the high-frequency longitudinal tear categories in the longitudinal tear category group, and summarizing the high-frequency longitudinal tear categories corresponding to the regional conveyor belt to obtain multiple high-frequency longitudinal tear categories.

Images