Fault detection method and electronic equipment

Abstract

The invention provides a fault detection method and electronic equipment, and relates to the technical field of fault detection.The method comprises the steps that a belt image of conveying equipment is obtained, and image feature information in the belt image is recognized; performing image segmentation on the belt image based on the image feature information to obtain a plurality of segmented images; screening out an abnormal image from the plurality of segmented images based on the gray level of each pixel point in each segmented image; and determining an abnormal feature region in the abnormal image, and determining a fault degree corresponding to the abnormal feature region and a fault region in the conveying equipment based on the number of pixel points in the abnormal feature region. According to the invention, the fault degree of the abnormal feature region and the fault region of the conveying equipment are finally determined through the number of the pixel points in the abnormal feature region, the influence of human factors is avoided, the accuracy of positioning the fault region of the conveying equipment is improved, and a worker can accurately maintain the conveying equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of fault detection, and in particular, to a fault detection method and an electronic device. Background Art

[0002] In modern industrial production, as a widely used material conveying device, the belt conveyor undertakes the important task of efficient transportation. With the development of industrial automation, the stability and safety of the belt conveyor during operation have gradually attracted attention. In order to detect the operation state of the belt conveyor in real time, timely discover potential faults and give early warnings, and ensure the safety and continuity during the operation of the belt conveyor, therefore, it is necessary to perform fault detection on the belt conveyor and perform corresponding repairs based on the results of the fault detection.

[0003] Traditional fault detection methods for belt conveyors mostly rely on manual inspections. During the manual inspection process, since the fault detection results of manual inspections are easily affected by human factors, it is impossible to accurately locate the fault area of the belt conveyor, nor can the fault degree of each fault area be accurately determined. Furthermore, it is impossible to perform precise maintenance on the belt conveyor. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a fault detection method and an electronic device, which can solve the technical problems in the prior art that it is impossible to accurately determine the fault degree of each area of the belt conveyor and the low accuracy in identifying the fault area of the belt conveyor.

[0005] In a first aspect, an embodiment of the present invention provides a fault detection method, including:

[0006] Obtain a belt image of a conveying device, and identify image feature information in the belt image;

[0007] Perform image segmentation on the belt image based on the image feature information to obtain a plurality of segmented images;

[0008] Screen out abnormal images from the plurality of segmented images based on the gray levels of the pixel points in each of the segmented images;

[0009] Determine an abnormal feature area in the abnormal image, and determine the fault degree corresponding to the abnormal feature area and the fault area in the conveying device based on the number of pixel points in the abnormal feature area.

[0010] Further, an embodiment of the present invention provides a first possible implementation manner of the first aspect, wherein the step of screening out abnormal images from the plurality of segmented images based on the gray levels of the pixel points in each of the segmented images includes:

[0011] Determine the average gray value of each of the segmented images based on the gray level of each pixel point in each of the segmented images and the number of pixel points in the segmented image;

[0012] Determine the average gray value of the belt image based on the gray level of each pixel point in the belt image and the number of pixel points in each of the belt images;

[0013] Screen out the abnormal image from multiple segmented images based on the average gray value of the segmented image and the average gray value of the belt image.

[0014] Furthermore, the embodiment of the present invention provides a second possible implementation manner of the first aspect, wherein the step of screening out the abnormal image from multiple segmented images based on the average gray value of the segmented image and the average gray value of the belt image includes:

[0015] Calculate the absolute value of the difference between the average gray value of each segmented image and the average gray value of the belt image;

[0016] Determine the segmented image with the absolute value greater than the preset gray difference as the abnormal image.

[0017] Furthermore, the embodiment of the present invention provides a third possible implementation manner of the first aspect, wherein the step of determining the average gray value of each segmented image based on the gray level of each pixel point in each segmented image and the number of pixel points in the segmented image includes:

[0018] Obtain the number of pixel points in each segmented image;

[0019] Respectively calculate the product of the gray level of each level in the segmented image and the corresponding number of pixel points to obtain the pixel gray product value corresponding to each gray level;

[0020] Perform a summation calculation on the pixel gray product values corresponding to the gray levels of the segmented image to obtain the total gray level value of the segmented image;

[0021] Respectively calculate the ratio of the total gray level value of each segmented image to its number of pixel points to obtain the average gray value of each segmented image.

[0022] Furthermore, the embodiment of the present invention provides a fourth possible implementation manner of the first aspect, wherein the step of determining the average gray value of the belt image based on the gray level of each pixel point in the belt image and the number of pixel points in each of the belt images includes:

[0023] Perform a summation calculation on the number of pixel points in each segmented image to obtain the number of pixel points in the belt image;

[0024] Calculate the product of the gray level of each level in the belt image and the corresponding number of pixel points to obtain the pixel gray product value corresponding to each gray level of the belt image;

[0025] Perform a summation calculation on the pixel gray product values corresponding to each gray level of the belt image to obtain the total gray level value of the belt image;

[0026] Calculate the ratio of the total gray level value of the belt image to the number of its pixel points to obtain the average gray value of the belt image.

[0027] Furthermore, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein the steps of determining the abnormal feature area in the abnormal image and determining the fault degree corresponding to the abnormal feature area and the fault area in the conveying device based on the number of pixel points in the abnormal feature area include:

[0028] Obtain the abnormal feature information of the abnormal image, and determine the abnormal feature area in the abnormal image based on the abnormal feature information;

[0029] Calculate the ratio of the number of pixel points in the abnormal feature area to the number of pixel points in the belt image to obtain the fault degree corresponding to the abnormal feature area;

[0030] Use the abnormal feature area with the fault degree greater than the preset fault degree threshold as the target abnormal feature area, and determine the corresponding area of the target abnormal feature area in the conveying device as the fault area.

[0031] Furthermore, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, wherein the steps of the fault detection method further include:

[0032] Obtain the sound data, vibration data, and environmental data of the conveying device;

[0033] Extract features from the sound data, the vibration data, and the environmental data respectively to obtain the sound feature information, vibration feature information, and environmental feature information of the conveying device;

[0034] Determine the multi-modal features of the conveying device based on the image feature information, the sound feature information, the vibration feature information, and the environmental feature information.

[0035] Furthermore, an embodiment of the present invention provides a seventh possible implementation manner of the first aspect, wherein the steps of performing image segmentation on the belt image based on the image feature information to obtain a plurality of segmented images include:

[0036] Encode the environmental features in the environmental feature information to obtain an environmental embedding vector;

[0037] Determine image segmentation parameters based on the environmental embedding vector, and perform image segmentation on the belt image based on the image segmentation parameters and the image feature information to obtain a plurality of the segmented images.

[0038] Further, an eighth possible implementation manner of the first aspect is provided in an embodiment of the present invention, wherein the steps of the fault detection method further include:

[0039] Determine a prediction threshold based on the environmental embedding vector;

[0040] Perform fault type prediction on the conveying device based on the multimodal features and the prediction threshold to obtain the fault type of the conveying device.

[0041] In a second aspect, an embodiment of the present invention provides an electronic device, including: a processor and a storage device;

[0042] A computer program is stored on the storage device, and when the computer program is run by the processor, it executes the fault detection method described in any one of the above.

[0043] An embodiment of the present invention provides a fault detection method, which includes: acquiring a belt image of a conveying device, and identifying image feature information in the belt image; performing image segmentation on the belt image based on the image feature information to obtain a plurality of segmented images; screening out abnormal images from the plurality of segmented images based on the gray levels of the pixel points in each segmented image; determining abnormal feature regions in the abnormal images, and determining the fault degree corresponding to the abnormal feature regions and the fault regions in the conveying device based on the number of pixel points in the abnormal feature regions. The present invention can quickly and accurately determine the fault degree of the abnormal feature regions based on the number of pixel points in the abnormal feature regions of the abnormal images by judging the abnormal states of the segmented images according to the gray levels of the pixel points in the segmented images and screening out the abnormal images from the segmented images, and automatically and accurately locate the fault regions in the conveying device, avoiding the influence of human factors, improving the accuracy when locating the fault regions of the conveying device, and enabling the staff to perform precise maintenance on the conveying device according to the fault regions and the fault degrees.

[0044] Other features and advantages of the embodiments of the present invention will be described in the subsequent description, or some features and advantages can be inferred from the description or determined without doubt, or can be known by implementing the above technologies of the embodiments of the present invention.

[0045] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. Description of the Drawings

[0046] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 It shows a schematic structural diagram of a fault detection system provided by an embodiment of the present invention;

[0048] Figure 2 It shows a schematic flowchart of a fault detection method provided by an embodiment of the present invention;

[0049] Figure 3 It shows a flowchart of a real-time video processing module provided by an embodiment of the present invention. Specific embodiments

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will describe the technical solutions of the present invention in conjunction with the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0051] At present, belt conveyors bear the heavy responsibility of efficient transportation. During the use of belt conveyors, it is necessary to perform fault detection on belt conveyors to ensure their safety and stability during operation. The fault detection of belt conveyors in the prior art is usually achieved through manual inspection. During the manual inspection process, it is necessary to rely on manual experience to judge the fault detection results, and often requires staff to master professional knowledge and skills, resulting in low efficiency of fault detection. Often, corresponding measures are only taken after a fault occurs, which easily damages the belt conveyor and causes greater economic losses. At the same time, there are great subjectivity and uncertainty in the process of manual inspection, and the detection results are easily affected by human factors. Therefore, when using the existing methods to perform fault detection on belt conveyors, it is impossible to accurately determine the degree of faults in each area of the belt conveyor and accurately locate the fault area of the belt conveyor, making it impossible for staff to perform precise maintenance on the belt conveyor.

[0052] To improve the above problems, the embodiments of the present invention provide a fault detection method and an electronic device. The following will introduce the embodiments of the present invention in detail.

[0053] This embodiment provides a fault detection method, which can be applied to the fault detection system of conveying equipment. See, for example, Figure 1Schematic structural diagram of a fault detection system. Specifically, the fault detection system includes: an input module, a multimodal fusion module, an adaptive deep learning module, a video data transmission module, a main control module, a video image processing module, a real-time video processing module, a real-time video data storage module, a historical video data storage module, a fault prediction module, a fault prediction result optimization module, a fault alarm and operating parameter setting module, and a fault diagnosis module;

[0054] See Figure 2 Schematic flow diagram of a fault detection method. This method mainly includes the following steps:

[0055] Step S201: Obtain the belt image of the conveying equipment and identify the image feature information in the belt image;

[0056] When performing fault detection on a conveying equipment (including a belt conveyor), the belt image of the conveying equipment is usually obtained through the input module in the fault detection system. Among them, the input module includes a video data input unit and a parameter data input unit. Specifically, the video data input unit is responsible for controlling a high-definition camera to collect the real-time video data of the conveying equipment to ensure the high definition and real-time nature of the obtained real-time video data. After the video data input unit collects the real-time video data of the conveying equipment, it sends the real-time video data to the video image processing module through a data transmission interface;

[0057] The parameter data input unit collects the real-time video data acquisition parameters related to the operation of the conveying equipment. Among them, the real-time video data acquisition parameters include: the number of input devices, transmission speed, resolution, frame rate, scanning speed, number of scanning lines, and image size. Specifically, the number of input devices refers to the number of high-definition cameras, which affects the acquisition range of the conveying equipment in the real-time video data and the specific details of the conveying equipment; the transmission speed affects the operation stability of the conveying equipment under different conditions; the resolution affects the clarity of the real-time video data; the frame rate determines the smoothness of the real-time video data and is usually set at 30 frames per second or higher to ensure the real-time nature of the conveying equipment in the working scenario in the real-time video data; the scanning speed and the number of scanning lines affect the speed and accuracy of image capture of the conveying equipment, especially on high-speed running conveying equipment; the image size affects the computational amount of subsequent processing and is usually set to standard sizes such as 640×480 or 1280×720. These real-time video data acquisition parameters are crucial for the processing and analysis of real-time video data. After obtaining the real-time video data acquisition parameters, the real-time video data acquisition parameters are transmitted to the video image processing module through a dedicated interface to ensure the consistency and accuracy of the real-time video data during the video processing process;

[0058] The main control module in the fault detection system controls the video image processing module to perform image preprocessing on the real-time video data. By analyzing each frame of the real-time video data frame by frame, each frame image of the conveying device is obtained. Then, through image processing algorithms such as edge detection and optical flow method, the feature information of each frame image is extracted. Among them, the image feature information of each frame image extracted carries the key feature content of the image itself. For example, it reflects the distribution of the edges of the conveying device in the image, the movement trends of different regions, etc. Specifically, the image feature information includes the color histogram of the image (reflecting the distribution characteristics of the image color), texture features (reflecting the texture characteristics of the image surface), and shape information (reflecting the contours of each region and objects in the image), etc.; the video feature information is obtained by extracting features from the real-time video data. Among them, the video feature information includes metadata such as timestamps (to prevent the real-time video data from being tampered with) and frame numbers (to ensure the integrity of the real-time video data); the extracted image feature information and video feature information are transmitted to the deep learning algorithm unit in the fault detection system through a high-bandwidth data channel to provide data support for subsequent deep learning processing;

[0059] In a specific implementation, the video image processing module extracts the feature information of each frame image through a Convolutional Neural Networks (CNN). The basic formula of the convolutional neural network is:

[0060] y = f(W * x + b);

[0061] Among them, y is the output image feature information, f is the activation function in the convolutional neural network, W is the convolutional kernel in the convolutional neural network, x is the input image, and b is the bias term in the convolutional neural network; through the convolutional operation of the convolutional neural network, the high-level features of the image can be effectively extracted, thereby improving the recognition accuracy of the subsequent deep learning algorithm unit for the belt image. For example, in the fault detection of the conveying device, by training the convolutional neural network model, the different image feature information of the conveying device in the normal and fault states can be automatically learned, so as to achieve fast and accurate fault detection.

[0062] Step S203, perform image segmentation on the belt image based on the image feature information to obtain multiple segmented images;

[0063] Input the real-time video data, image feature information, and video feature information of the conveying device into the real-time video processing module in the fault detection system to achieve image segmentation of the belt image. See Figure 3The flowchart of a real-time video processing module is shown. The real-time video processing module includes an image preprocessing unit, an image segmentation unit, an image recognition unit, and an image classification and merging unit. Specifically, the image preprocessing unit usually uses methods such as high-pass filtering or median filtering to convert real-time video data into grayscale images, removing clutter and noise in the real-time video data. After the real-time video data is converted into grayscale images by the image preprocessing unit, the output real-time video data will be clearer, facilitating subsequent processing.

[0064] The real-time video data that has been converted into grayscale images by the image preprocessing unit is input to the image segmentation unit for image segmentation of the belt image. Among them, in the image segmentation unit, image feature information and video feature information are usually used as input information for deep learning algorithms. The belt image is segmented through deep learning algorithms (such as convolutional neural networks or U-Net deep learning models). During the image segmentation process, the belt image can be segmented according to the regions corresponding to different components of the belt in the conveying equipment, or according to the fault-prone areas of the conveying equipment, or relevant standards can be set according to the actual application scenario of the conveying equipment to segment the belt image, or the belt image can be segmented based on an adaptive segmentation method of deep learning algorithms, enabling efficient and accurate acquisition of the belt image.

[0065] Step S205: Screen out abnormal images from multiple segmented images based on the gray levels of each pixel point in each segmented image.

[0066] One or more of the above-screened abnormal images may exist. The image recognition unit in the real-time video processing module quantifies the gray levels of each segmented image and establishes a feature extraction model. Specifically, the gray levels of each pixel point in each segmented image are extracted through the feature extraction model. In the feature extraction model, the extraction of the gray levels of each pixel point in each segmented image follows a set time interval to ensure the analysis of each segmented image at appropriate time points. The scanning rows and columns are preset and scanned along the row and column directions of the segmented image respectively. This process can carefully capture the gray levels of each pixel point in each segmented image, thereby improving the accuracy of fault detection.

[0067] In a specific implementation, the abnormality degree of each segmented image can also be quantified by the difference between the gray level of each pixel point and the average gray value of the belt image:

[0068] M d = M ipq - M b ;

[0069] Among them, M ddenotes the difference between the gray level of the pixel at the p-th row and q-th column in the i-th segmented image and the average gray value of the belt image, M ipq is the gray level of the pixel at the p-th row and q-th column in the i-th segmented image, M b is the average gray value of the belt image;

[0070] By calculating the difference between the gray level of each pixel and the average gray value of the belt image, the quantification of the abnormality degree of each segmented image is realized. During the scanning process, when it is found that the difference between the average gray value of a certain row or column of pixels in the segmented image and the average gray value of the belt image exceeds the set threshold, that row or column is marked as an abnormal area to remind the staff to pay key attention. For example, the set threshold is 20. When the absolute value of the difference between the average gray value of a certain row of pixels in a certain segmented image and the average gray value of the belt image exceeds 20, that row is considered an abnormal area to remind the staff to pay key attention.

[0071] Step S207, determine the abnormal feature area in the abnormal image, and determine the fault degree corresponding to the abnormal feature area and the fault area in the conveying equipment based on the number of pixels in the abnormal feature area;

[0072] By obtaining the abnormal feature area in the abnormal image, extracting features from the abnormal feature area to obtain the abnormal feature information of the abnormal feature area, and merging the abnormal feature information. For example, when a certain type of abnormal feature information continuously appears at the same position in consecutive video frames of real-time video data, this type of abnormal feature information is determined to be the abnormal feature information of the same abnormal feature area, which can effectively reduce the false detection of the abnormal feature area and improve the accuracy of fault detection.

[0073] The above-mentioned fault detection method provided by the embodiments of the present invention can accurately judge the abnormal state of each segmented image according to the gray levels of each pixel in the segmented image, and screen out the abnormal images from the segmented images. It can quickly and accurately determine the fault degree of the abnormal feature area based on the number of pixels in the abnormal feature area of the abnormal image, and automatically and accurately locate the fault area in the conveying equipment, avoiding the influence of human factors, improving the accuracy when positioning the fault area of the conveying equipment, and enabling the staff to perform precise maintenance on the conveying equipment according to the fault area and the fault degree.

[0074] In one embodiment, the specific implementation manner of screening out abnormal images from multiple segmented images based on the gray levels of each pixel in each segmented image includes:

[0075] Determine the average gray value of each segmented image based on the gray levels of each pixel in each segmented image and the number of pixels in the segmented image;

[0076] Determine the average gray value of the belt image based on the gray levels of each pixel point in the belt image and the number of pixel points in each belt image;

[0077] Obtain the average gray values of each segmented image and the belt image, quantify the situations of each segmented image and the belt image, which is convenient for subsequent comparison, and can better screen out abnormal images from multiple segmented images.

[0078] Screen out abnormal images from multiple segmented images based on the average gray value of the segmented image and the average gray value of the belt image;

[0079] The average gray value of the segmented image reflects the degree of abnormality of each segmented image. When the difference between the average gray value of the segmented image and the average gray value of the belt image is large, it can be determined that the segmented image is an abnormal image, and abnormal images can be accurately screened out from multiple segmented images;

[0080] In a specific embodiment, obtain the real-time video data during the operation of the conveying device. Among them, the real-time video data acquisition parameters show that the conveying speed is 1.5 meters per second, the image resolution is 1280x720, and the frame rate is 30 frames per second. After image preprocessing and image segmentation of the real-time video data, multiple segmented images are obtained. After calculating the average gray value, it is determined that the average gray value of a certain segmented image is significantly lower than the average gray values of other segmented images and the belt image. Mark this segmented image as an abnormal image, and in subsequent detections, this abnormal image continuously appears in subsequent frames of the real-time video data. Determine this segmented image as an abnormal image. Subsequently, the specific location of this abnormal image in the conveying device can be reported to the staff through an alarm, which is convenient for the staff to perform maintenance, ensuring the safety and reliability of the conveying device. By using the fault detection method provided by the present invention to locate abnormal images, the real-time video data is efficiently analyzed and used for fault detection, providing a guarantee for the stable operation of the conveying device.

[0081] In one embodiment, the specific implementation manner of screening out abnormal images from multiple segmented images based on the average gray value of the segmented image and the average gray value of the belt image provided in this embodiment includes:

[0082] Calculate the absolute value of the difference between the average gray value of each segmented image and the average gray value of the belt image;

[0083] Calculate the absolute value of the difference between the average gray value of each segmented image and the average gray value of the belt image:

[0084] M = |M ai - M b |;

[0085] where M is the absolute value of the difference between the average gray value of the segmented image and the average gray value of the belt image, Mai is the average gray value of the i-th segmented image, and M b is the average gray value of the belt image.

[0086] Determine the segmented images with absolute values greater than the preset gray difference as abnormal images;

[0087] Set the preset gray difference to M c , if the absolute value M of the difference between the average gray value of the segmented image and the average gray value of the belt image is greater than the preset gray difference M c , then determine that the segmented image is an abnormal image.

[0088] In one embodiment, this embodiment provides a specific implementation manner for determining the average gray value of each segmented image based on the gray levels of each pixel point in each segmented image and the number of pixel points in the segmented image, including:

[0089] Obtain the number of pixel points in each segmented image;

[0090] Sum up the number of pixel points corresponding to each gray level of each segmented image to obtain the number of pixel points in each segmented image:

[0091]

[0092] where S i is the number of pixel points in the i-th segmented image, N - 1 is the total number of levels of gray levels, and p ij is the number of pixel points corresponding to the j-th gray level in the i-th segmented image, and j is the level of the gray level.

[0093] Calculate the product of each gray level in the segmented image and the corresponding number of pixel points respectively to obtain the pixel gray product value corresponding to each gray level;

[0094] Calculate the product of each gray level in the segmented image and the corresponding number of pixel points respectively to obtain the pixel gray product value corresponding to each gray level:

[0095] Z ij = p ij ·j;

[0096] where Z ij is the pixel gray product value corresponding to the j-th gray level in the i-th segmented image.

[0097] Sum up the pixel gray product values corresponding to each gray level of the segmented image to obtain the total gray level value of the segmented image;

[0098] Sum up the pixel gray product values corresponding to each gray level of the segmented image to obtain the total gray level value of the segmented image:

[0099]

[0100] Among them, E i is the total gray level value of the i-th segmented image.

[0101] Calculate the ratio of the total gray level value of each segmented image to the number of its pixel points respectively, to obtain the average gray value of each segmented image;

[0102] Calculate the ratio of the total gray level value of each segmented image to the number of its pixel points respectively, to obtain the average gray value of each segmented image:

[0103]

[0104] Among them, M ai is the average gray value of the i-th segmented image.

[0105] In one embodiment, this embodiment provides a specific implementation manner for determining the average gray value of the belt image based on the gray levels of each pixel point in the belt image and the number of pixel points of each belt image, including:

[0106] Perform a summation calculation on the number of pixel points in each segmented image to obtain the number of pixel points in the belt image;

[0107] Perform a summation calculation on the number of pixel points in each segmented image to obtain the number of pixel points in the belt image:

[0108]

[0109] Among them, S is the number of pixel points in the belt image, and L is the number of segmented images.

[0110] Calculate the product of the gray level of each level in the belt image and the corresponding number of pixel points to obtain the pixel gray product value corresponding to each gray level of the belt image;

[0111] Calculate the product of the gray level of each level in the belt image and the corresponding number of pixel points to obtain the pixel gray product value corresponding to each gray level of the belt image:

[0112] Z b = p bj ·j;

[0113] Among them, Z b is the pixel gray product value corresponding to each gray level of the belt image, p bj is the number of pixel points corresponding to the j-th gray level in the belt image, and j is the level of the gray level.

[0114] Perform a summation calculation on the pixel gray product values corresponding to each gray level of the belt image to obtain the total gray level value of the belt image;

[0115] Sum the pixel gray product values corresponding to each gray level of the belt image to obtain the total gray level value of the belt image:

[0116]

[0117] Where E is the total gray level value of the belt image, and N - 1 is the total number of gray level grades.

[0118] Calculate the ratio of the total gray level value of the belt image to the number of its pixel points to obtain the average gray value of the belt image;

[0119] Calculate the ratio of the total gray level value of the belt image to the number of its pixel points to obtain the average gray value of the belt image:

[0120]

[0121] Where M b The average gray value of the belt image.

[0122] In one embodiment, this embodiment provides a specific implementation manner for determining the abnormal feature region in the abnormal image, determining the fault degree corresponding to the abnormal feature region based on the number of pixel points in the abnormal feature region, and the fault region in the conveying device, including:

[0123] Obtain the abnormal feature information of the abnormal image, and determine the abnormal feature region in the abnormal image based on the abnormal feature information;

[0124] The above abnormal feature information is obtained through the image classification and merging unit. Among them, the abnormal feature information of the abnormal image includes the shape of the abnormal feature region in the abnormal image, the coordinates of the central pixel point in the abnormal feature region, the number of pixel points in the abnormal feature region, the proportion of the abnormal feature region in the belt image, and the brightness value of the abnormal feature region; specifically, usually, the contour of the edge of the abnormal feature region is extracted by an edge detection algorithm (such as the Canny edge detection algorithm or the Sobel operator edge detection algorithm), and the shape of the abnormal feature region can be efficiently recognized; when obtaining the coordinates of the central pixel point in the abnormal feature region, a two-dimensional coordinate system is established based on the belt image, and the average value of the abscissas and the average value of the ordinates of all pixel points in the abnormal feature region are calculated to obtain the coordinates of the central pixel point in the abnormal feature region:

[0125]

[0126] Where C x Is the abscissa of the central pixel point in the abnormal feature region, x u Is the abscissa of the u-th pixel point in the abnormal feature region, C yis the ordinate of the central pixel point within the abnormal feature region, y u is the ordinate of the u-th pixel point within the abnormal feature region, V is the number of pixel points in the abnormal feature region. By obtaining the coordinates of the central pixel point within the abnormal feature region, the position of the abnormal feature region in the belt image can be accurately located;

[0127] Determine the proportion of the abnormal feature region in the belt image based on the number of pixel points in the abnormal feature region and the number of pixel points in the belt image; The brightness value of the abnormal feature region can be obtained by summing the brightness values of all pixel points within the abnormal feature region to get the total brightness value of the abnormal feature region, and calculating the ratio of the total brightness value of the abnormal feature region to the number of pixel points in the abnormal feature region to obtain the brightness value of the abnormal feature region, which can reflect the brightness and darkness degree of the abnormal feature region.

[0128] Calculate the ratio of the number of pixel points in the abnormal feature region to the number of pixel points in the belt image to obtain the fault degree corresponding to the abnormal feature region;

[0129] The number of pixel points in the abnormal feature region directly reflects the severity of the damage in the abnormal feature region, while the ratio of the number of pixel points in the abnormal feature region to the number of pixel points in the belt image can provide the influence of the abnormal feature region in the belt image (i.e., the fault degree of the corresponding region in the belt of the conveying equipment). By calculating the ratio of the number of pixel points in the abnormal feature region to the number of pixel points in the belt image, the fault degree corresponding to the abnormal feature region is obtained:

[0130]

[0131] where, R is the fault degree corresponding to the abnormal feature region, S′ is the number of pixel points in the abnormal feature region, and S is the number of pixel points in the belt image;

[0132] In a specific embodiment, it is detected that the number of pixel points in the abnormal feature region is 500, while the number of pixel points in the belt image is 64000, then the fault degree of this abnormal feature region is:

[0133]

[0134] Through this fault degree, the importance of the abnormal feature region in the belt image (i.e., the fault degree of the corresponding region in the belt of the conveying equipment) can be judged.

[0135] Regard the abnormal feature region with a fault degree greater than the preset fault degree threshold as the target abnormal feature region, and determine the corresponding region of the target abnormal feature region in the conveying equipment as the fault region;

[0136] The target abnormal feature area is screened out from the abnormal feature area by presetting the fault degree threshold, and the corresponding area of the target abnormal feature area in the conveying equipment is determined as the fault area, so as to quickly and accurately locate the fault area of the conveying equipment, improve the accuracy during the detection of the fault area, and feed back the information of the fault area to the fault diagnosis module to trigger a fault alarm;

[0137] In a specific embodiment, it is found that the number of pixel points in a certain abnormal feature area is 800, and the number of pixel points in the belt image is 64000. The fault degree of this abnormal feature area is as follows:

[0138]

[0139] The preset fault degree threshold is set to 1%. The fault degree of this abnormal feature area is greater than the preset fault degree threshold, and it is determined during subsequent analysis that the brightness value of this abnormal feature area is lower than the brightness value when the conveying equipment is operating normally. It can be determined that the fault degree of this abnormal feature area is too high. This abnormal feature area is used as the target abnormal feature area, and the corresponding area of this target abnormal feature area in the belt of the conveying equipment is determined as the fault area, triggering a fault area alarm to prompt the staff to repair this fault area, which provides an important guarantee for the stable operation of the conveying equipment.

[0140] In an embodiment, the specific implementation manner of the fault detection method provided in this embodiment further includes:

[0141] Obtain the sound data, vibration data and environmental data of the conveying equipment;

[0142] Add multi-modal data input channels in the input module to respectively collect the sound data, vibration data and environmental data of the conveying equipment (including: temperature data, light intensity data, type of conveying equipment, etc.). After synchronizing the time of the sound data, vibration data and temperature data in the environmental data of the conveying equipment with the real-time video data of the conveying equipment, they are transmitted to the multi-modal fusion module in the fault detection system.

[0143] Respectively extract features from the sound data, vibration data and environmental data to obtain the sound feature information, vibration feature information and environmental feature information of the conveying equipment;

[0144] Through the deep learning fusion network in the multi-modal fusion module, extract the complementary features between different data of the conveying equipment;

[0145] In a specific embodiment, a multi-channel convolutional neural network is used to respectively process the sound data X audio 、vibration data X vib and temperature data X tempFeature extraction is performed to obtain the sound feature information, vibration feature information, and temperature feature information of the conveying equipment. Among them, the feature representation of the sound feature information is F audio ; the feature representation of the vibration feature information is F vib ; the feature representation of the temperature feature information is F temp ; and the feature representation of the image feature information obtained based on the real-time video data is F image .

[0146] Determine the multi-modal features of the conveying equipment based on the image feature information, sound feature information, vibration feature information, and environmental feature information;

[0147] The multi-modal fusion module performs weighted summation on the above-mentioned image feature information, sound feature information, vibration feature information, and temperature feature information through attention weights, and finally obtains the multi-modal feature F multi :

[0148] F multi =∑ w∈{image，audio，vib，temp} α w F w ;

[0149] Among them, F w is the feature representation of a certain feature information, and α w is the weight of a certain feature information, which is dynamically generated by the attention mechanism and represents the importance of this feature information in the current operating scenario of the conveying equipment. The calculation formula is:

[0150]

[0151] Among them, W w is the weight matrix corresponding to a certain feature information. After training the weight matrix, the weight can adapt to the change of the importance degree of different feature information;

[0152] On the basis of the original real-time video data and image feature information, multi-modal data fusion is introduced. Integrate various sensor data such as sound data, vibration signals, and temperature data with the original real-time video data, depict the operating state of the conveying equipment from multiple dimensions, improve the accuracy of fault detection, ensure that the fault detection method has the ability of early warning, and can significantly improve the performance in fault type detection;

[0153] By integrating various feature information of the conveying equipment through the above multi-modal data fusion module, it can not only overcome the limitations of single real-time video data in specific scenarios of the conveying equipment, but also form more comprehensive multi-modal features, improve the reliability of fault detection, and at the same time enhance the robustness and adaptability of each module.

[0154] In one embodiment, the specific implementation manner of performing image segmentation on a belt image based on image feature information to obtain multiple segmented images includes:

[0155] Encoding the environmental features in the environmental feature information to obtain an environmental embedding vector;

[0156] To further improve the adaptability and generalization ability of the fault detection method, the fault detection system further includes an adaptive deep learning module, enabling the adaptive deep learning module to automatically adjust the image segmentation parameters and prediction thresholds according to the type of conveying equipment and the environment where it is located, ensuring that the fault detection method still maintains the ability to efficiently identify fault regions and predict fault types under different conveying equipment and different usage environments of the conveying equipment;

[0157] Design a dynamic adjustment strategy, where the automatic adjustment of the image segmentation parameters and prediction thresholds is based on environmental features, and the set of environmental features is set as:

[0158] E′=[E1,E2,…,E n

[0159] where E’ is the set of environmental features, and E1, E2, …, E n are various environmental features included in the environmental feature set, such as light intensity, temperature, type of conveying equipment, etc.;

[0160] Encoding the environmental feature set based on the feature extraction network to obtain an environmental embedding vector H env .

[0161] Determine the image segmentation parameters based on the environmental embedding vector, and divide the belt image based on the image segmentation parameters and image feature information to obtain multiple segmented images;

[0162] Guided by the environmental embedding vector, calculate the adaptive image segmentation parameters through the parameter generation network:

[0163] P seg =W seg ·H env +b seg ;

[0164] T pred =W pred ·H env +b pred ;

[0165] where P seg is the image segmentation parameter, which affects the results of image preprocessing and segmented images, W seg is the weight matrix used to generate the image segmentation parameter, and b seg ​is a bias term for generating image segmentation parameters;

[0166] Automatically calculate the image segmentation parameter P according to the current environmental characteristics seg , and input the image segmentation parameter into the image segmentation unit to determine the way of image segmentation for the belt image. For example, when the lighting condition is poor, the image segmentation parameter will be adjusted to enhance the contrast of the belt image, so as to more effectively identify the fault area of the conveying equipment.

[0167] In the embodiment of the present invention, after obtaining the real-time video data of the conveying equipment through the input module, the main control module controls the video data transmission module in the fault detection system to transmit the real-time video data from the input module to the main control module. The video data transmission module adopts an efficient video coding algorithm, such as H.264 or H.265, to achieve the compression and fast transmission of the real-time video data, and ensure the real-time performance and fluency of the real-time video data;

[0168] The main control module outputs an instruction to control the real-time video data storage module in the fault detection system to save the real-time video data to the storage medium for subsequent query and analysis. The format of the real-time video data storage can adopt the standard MP4 or AVI format to ensure data compatibility with the video image processing module; The main control module outputs an instruction to control the historical video data storage module in the fault detection system, which is responsible for storing the historical video data of the conveying equipment collected before, providing a data basis for fault detection and fault type prediction.

[0169] In one embodiment, the specific implementation manner of the fault detection method provided in this embodiment further includes:

[0170] Determine the prediction threshold based on the environmental embedding vector;

[0171] Guided by the environmental embedding vector, calculate an adaptive prediction threshold through the parameter generation network:

[0172] T pred =W pred ·H env +b pred ;

[0173] Among them, T pred is the prediction threshold, which affects the sensitivity of fault type judgment, W pred is the weight matrix for generating the prediction threshold, and b pred is the bias term for generating the prediction threshold;

[0174] The prediction threshold T pred can be automatically adjusted according to different environmental characteristics, so that the fault detection method adapts to the actual operating state of the conveying equipment and improves the prediction accuracy of the fault type of the conveying equipment.

[0175] The above adaptive deep learning module generates adaptive image segmentation parameters and prediction thresholds through various types of environmental features in the environmental feature set, ensuring that the fault detection of the conveying equipment can automatically adapt to different environments. For example, when the conveying equipment operates in a high-temperature and high-humidity environment, the environmental feature embedding vector will indicate high light and humidity. In the adaptive deep learning module, the image segmentation parameters are adjusted through an adaptive algorithm to enhance the recognition of the detail areas in the belt image, and at the same time, the prediction threshold is reduced to adapt to the high-temperature and high-humidity environment, increasing the sensitivity of early warning during fault detection. Through this adaptive adjustment, the best effect of fault area recognition and fault type prediction can always be maintained, improving the robustness and flexibility of the fault detection system, and enabling it to still have high accuracy and low false alarm rate in different environments.

[0176] Predict the fault type of the conveying equipment based on the multi-modal features and prediction threshold to obtain the fault type of the conveying equipment.

[0177] Take the multi-modal features and prediction threshold as the input parameters of the deep learning algorithm unit in the fault prediction module of the fault detection system. This deep learning algorithm unit extracts key features from the real-time video data and historical video data of the conveying equipment through deep learning technologies such as convolutional neural networks, and obtains the feature vector:

[0178] h = σ(W g *x g +b g );

[0179] Among them, h is the feature vector extracted by the deep learning algorithm unit, x g is the input video data, W g is the weight of the convolutional layer in the deep learning algorithm unit, σ is the activation function (such as ReLU activation function, etc.), b g is the bias of the convolutional layer in the deep learning algorithm unit;

[0180] After the multiple convolutional layers and pooling layers in the deep learning algorithm unit process the real-time video data and historical video data of the conveying equipment, the deep learning algorithm unit can learn the deep features (such as edges, textures, and shapes, etc.) of the belt image of the conveying equipment, thereby providing a basis for subsequent image recognition;

[0181] After extracting the feature vectors, the deep learning algorithm unit will perform image recognition based on the extracted feature vectors. It will use the pre-trained classification models in the fault model database of the fault detection system (built based on classification algorithms such as Softmax regression or support vector machines, and trained with real-time video datasets and historical video datasets marked with fault types to predict the fault types of the conveying equipment. The classification model is equipped with a self-learning function that can update and improve itself) to predict the possible fault types of the conveying equipment at the next moment and the probability information of each fault type (such as conveyor belt slippage, material blockage, or mechanical wear) according to the image feature information, video feature information, and the images recognized by the deep learning algorithm unit through feature vectors. Then, the newly recognized video data and the corresponding fault types and the probability information of each fault type will be stored in the fault model database and the historical video data storage module, which is convenient for subsequent review, statistics, and comparison of information such as the fault types and occurrence probabilities of the conveying equipment corresponding to the video data in the historical video data storage module and the fault model database, so as to analyze the laws and trends of various faults of the conveying equipment, and thus optimize the classification model and the fault prediction module, providing a reference basis for formulating maintenance strategies in advance;

[0182] And if the classification model does not predict the fault information of relevant fault types during the process of identifying the fault types of the conveying equipment, the deep learning algorithm unit will activate the self-learning function. By analyzing the feature data and recognition results at this time, it will update the fault model database, enabling the classification model to continuously learn and adapt to new fault types, and improving the recognition rate and accuracy. Through the combined action of the fault prediction module and the classification model, the accuracy and real-time performance of fault type prediction can be effectively improved, ensuring timely response to potential fault risks;

[0183] At the same time, the prediction threshold and multi-modal features also play a key role in fault type prediction. Using the prediction threshold and multi-modal features as the input of the deep learning algorithm unit, the classification model predicts the possible fault types and probabilities of the conveying equipment at the next moment. For example, when the load of the conveying equipment increases and the vibration intensifies, the prediction threshold will automatically decrease, enabling the classification model to detect the possible fault types of the conveying equipment at an early stage. When the conveying equipment has abnormal vibration, the vibration feature in the multi-modal features will occupy a higher attention weight, while the video image data may not be able to accurately capture the abnormality due to occlusion or lighting problems. In this case, the fusion model can automatically bias towards the vibration signal feature, thereby improving the accuracy of fault type prediction.

[0184] In an embodiment of the present invention, a fault prediction result optimization module is further provided in the fault detection system. The probability value of the most likely fault type of the conveying equipment obtained by prediction is used as the prediction probability value, and the probability value of this fault type stored in the historical video data storage module is used as the actual probability value. A probability error threshold is set, and the absolute value of the difference between the prediction probability value and the actual probability value is calculated as the probability error value:

[0185] E g = |A - H|;

[0186] Wherein, E g is the probability error value, A is the prediction probability value, and H is the actual probability value;

[0187] If the probability error value is less than or equal to the set probability error threshold, the prediction result is considered accurate. For example, assume that the current prediction probability value is 75, and the actual probability value in the historical video data storage module is 70. The calculated probability error value is 5, and the probability error threshold is set to 5, indicating that the current fault prediction result is correct; if the probability error value is greater than the set probability error threshold, the prediction result is considered incorrect, and the classification model in the fault model database will be corrected according to the actual fault type of the current conveying equipment. The corrected data will be transmitted back to the video image processing module to further improve the accuracy of fault type prediction.

[0188] In an embodiment of the present invention, a fault alarm and operating parameter setting module is further provided in the fault detection system. Among them, the fault alarm and operating parameter setting module includes three sub-modules: a fault alarm threshold setting module, a fault alarm processing module, and a parameter operation optimization module; specifically, the fault alarm threshold setting module allows staff to flexibly adjust the alarm threshold according to the actual use of the conveying equipment to adapt to different operating conditions; the fault alarm processing module is responsible for processing the triggered alarm events to ensure that the staff can respond quickly and take appropriate maintenance measures; the parameter operation optimization module is used to adjust the real-time video data acquisition parameters to maintain the efficiency of fault detection;

[0189] The fault alarm threshold setting module in the fault detection system sets the fault alarm threshold. The fault diagnosis module obtains the fault severity of the conveying equipment according to the fault area of the current conveying equipment and the predicted fault type of the conveying equipment, and compares the severity of the fault with the fault alarm threshold:

[0190]

[0191] Wherein, L represents the fault level, C is the fault severity, and T is the set fault alarm threshold;

[0192] When L is greater than 1, it indicates that the current severity of the fault exceeds the set fault alarm threshold, and an alarm will be issued. For example, if the calculated result of the current fault level is 1.2, a fault alarm will be triggered and an operation instruction will be sent; when L is less than or equal to 1, it indicates that the current severity of the fault does not exceed the set fault alarm threshold, and no alarm will be issued, and the parameters of the real-time video data acquisition will be optimized. The video image processing module will optimize the real-time video data based on the modified real-time video data acquisition parameters to improve the reading speed and operation efficiency of the real-time video data. For example, by adjusting the resolution and frame rate in the real-time video data acquisition parameters, the data volume of the real-time video data can be reduced, thereby accelerating the processing speed; through these measures, the accuracy of fault detection and the stability and response ability of the fault detection system are improved.

[0193] Based on the foregoing embodiments, this embodiment provides an example of applying the foregoing fault detection method to detect faults in a belt conveyor, which can be specifically implemented according to the following steps:

[0194] Step S302, obtain the belt image, vibration data, sound data, and ambient data of the belt conveyor through the input module in the fault detection system, and control the video image processing module to obtain image feature information and video feature information from the real-time video data based on the main control module in the fault detection system; among them, the video data input unit in the input module obtains the real-time video data of the belt conveyor; the parameter data input unit of the input module obtains the real-time video data acquisition parameters; the multi-modal data input channel of the input module collects the vibration data, sound data, and ambient data of the belt conveyor.

[0195] Step S304, control the video data transmission module through the main control module to transmit the real-time video data to the main control module. The main control module controls the real-time video data storage module to store the real-time video data of the belt conveyor, and the main control module controls the historical video data storage module to store the historical video data of the belt conveyor.

[0196] Step S306, the multi-modal fusion module in the fault detection system obtains multi-modal features based on the real-time video data, vibration data, sound data, and ambient data of the belt conveyor; the adaptive deep learning module in the fault detection system obtains image segmentation parameters and prediction thresholds based on the ambient data of the belt conveyor.

[0197] Step S308, use the image segmentation parameters as the input parameters of the real-time video processing module, and determine the fault degree of the abnormal feature area and the fault area of the belt conveyor based on the image feature information.

[0198] Step S310: Use the multi-modal features and prediction threshold as the input parameters of the fault prediction module in the fault detection system. Extract the feature vectors from the real-time video data and historical video data through the fault prediction module, and perform image recognition based on the feature vectors. The classification model predicts the fault types of the belt conveyor according to the image feature information, video feature information, and the image recognition results of the fault prediction module, and obtains the probabilities of the possible fault types that the belt conveyor may occur.

[0199] Step S312: Judge whether the prediction result of the fault type is accurate according to the fault prediction result optimization module in the fault detection system. If it is inaccurate, optimize the classification model.

[0200] Step S314: Set the fault alarm threshold according to the fault alarm and operation parameter setting module in the fault detection system. Determine the severity of the fault of the belt conveyor through the fault diagnosis module, compare the severity of the fault with the fault alarm threshold, and issue an alarm or optimize the real-time video data acquisition parameters according to the comparison result.

[0201] The above method provided by the embodiment of the present invention stores and analyzes the historical video data of the conveying equipment, providing an important decision-making support basis for the enterprise. The historical video data can be used for fault type prediction. Based on the results of the fault type prediction, it is convenient to formulate maintenance strategies, thereby optimizing the enterprise's resource allocation and maintenance plan, promoting the enterprise's transformation to intelligent management, improving the operation efficiency, enhancing the enterprise's market competitiveness, and promoting the digitization and intelligence of the fault detection of the conveying equipment.

[0202] The deep learning algorithm used in the fault detection method has a self-learning function and can be dynamically trained and optimized according to the historical video data, enabling the fault detection method to continuously improve the accuracy and reliability of its fault area recognition and fault type prediction in different working conditions and operating environments of the conveying equipment. Thus, it ensures that the fault detection method can still operate efficiently in different production environments and ensure timely and effective response.

[0203] Shorten the fault response time and reduce the maintenance cost: Determine the severity of the fault of the conveying equipment, monitor the fault situation of the conveying equipment in real time, and based on the comparison result of the severity of the fault and the fault alarm threshold, issue an alarm or optimize the real-time video data acquisition parameters. When the severity of the fault of the conveying equipment is too high, quickly trigger the alarm mechanism, reduce the fault handling time of the conveying equipment. The efficient response can reduce the damage risk of the conveying equipment, reduce the maintenance cost and manual intervention, improve the utilization efficiency of resources, and help the enterprise achieve more economical operation.

[0204] By accurately judging the abnormal state of each segmented image through the gray levels of each pixel in the image, abnormal images can be accurately screened out from the segmented images. The fault degree of the abnormal feature area can be quickly and accurately determined based on the number of pixels in the abnormal feature area of the abnormal image, and the fault area in the conveying device can be automatically and accurately located, avoiding the influence of human factors, reducing the false alarm rate, and improving the accuracy when positioning the fault area of the conveying device. This enables the staff to perform precise maintenance on the conveying device according to the fault area and fault degree, ensuring the continuity and safety during the use of the conveying device and reducing the risk of unexpected shutdowns.

[0205] This embodiment also provides an electronic device, including: a processor and a storage device;

[0206] A computer program is stored on the storage device, and when the computer program is run by the processor, it executes the above-mentioned fault detection method.

[0207] This embodiment of the present invention provides a computer-readable medium, wherein the computer-readable medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions cause the processor to implement the method described in the above embodiment.

[0208] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0209] Finally, it should be noted that the above embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A fault detection method, characterized in that, Including: Obtain the belt image of the conveying device, and identify the image feature information in the belt image; Perform image segmentation on the belt image based on the image feature information to obtain a plurality of segmented images; Screen out abnormal images from the plurality of segmented images based on the gray levels of each pixel point in each segmented image; Determine the abnormal feature region in the abnormal image, and determine the fault degree corresponding to the abnormal feature region and the fault region in the conveying device based on the number of pixel points in the abnormal feature region.

2. The fault detection method according to claim 1, wherein The step of screening out abnormal images from the plurality of segmented images based on the gray levels of each pixel point in each segmented image includes: Determine the average gray value of each segmented image based on the gray level of each pixel point in each segmented image and the number of pixel points in the segmented image; Determine the average gray value of the belt image based on the gray level of each pixel point in the belt image and the number of pixel points in each belt image; Screen out the abnormal images from the plurality of segmented images based on the average gray value of the segmented image and the average gray value of the belt image.

3. A fault detection method according to claim 2, characterized in that, The step of screening out the abnormal images from the plurality of segmented images based on the average gray value of the segmented image and the average gray value of the belt image includes: Calculate the absolute value of the difference between the average gray value of each segmented image and the average gray value of the belt image; Determine the segmented image with the absolute value greater than the preset gray difference as the abnormal image.

4. A fault detection method according to claim 3, characterized in that, The step of determining the average gray value of each segmented image based on the gray level of each pixel point in each segmented image and the number of pixel points in the segmented image includes: Obtain the number of pixel points in each segmented image; Calculate the product of the gray level of each level in the segmented image and the corresponding number of pixel points respectively to obtain the pixel gray product value corresponding to each gray level; Perform a summation calculation on the pixel gray product values corresponding to the gray levels of the segmented image to obtain the total gray level value of the segmented image; Calculate the ratio of the total gray level value of each segmented image to its number of pixel points respectively to obtain the average gray value of each segmented image.

5. A fault detection method according to claim 3, characterized in that The step of determining the average gray value of the belt image based on the gray level of each pixel point in the belt image and the number of pixel points in each belt image includes: Perform a summation calculation on the number of pixel points in each segmented image to obtain the number of pixel points in the belt image; Calculate the product of the gray level of each level in the belt image and the corresponding number of pixel points to obtain the pixel gray product value corresponding to each gray level of the belt image; Perform a summation calculation on the pixel gray product values corresponding to the gray levels of the belt image to obtain the total gray level value of the belt image; Calculate the ratio of the total gray level value of the belt image to its number of pixel points to obtain the average gray value of the belt image.

6. A fault detection method according to claim 1, characterized in that, The step of determining the abnormal feature region in the abnormal image, and determining the fault degree corresponding to the abnormal feature region and the fault region in the conveying device based on the number of pixel points in the abnormal feature region includes: Obtain the abnormal feature information of the abnormal image, and determine the abnormal feature area in the abnormal image based on the abnormal feature information; Calculate the ratio of the number of pixel points in the abnormal feature area to the number of pixel points in the belt image to obtain the fault degree corresponding to the abnormal feature area; Use the abnormal feature area with the fault degree greater than the preset fault degree threshold as the target abnormal feature area, and determine the corresponding area of the target abnormal feature area in the conveying device as the fault area.

7. A fault detection method according to claim 1, characterized in that, It further includes: Obtain the sound data, vibration data and environmental data of the conveying device; Extract features from the sound data, the vibration data and the environmental data respectively to obtain the sound feature information, vibration feature information and environmental feature information of the conveying device; Determine the multi-modal features of the conveying device based on the image feature information, the sound feature information, the vibration feature information and the environmental feature information.

8. A fault detection method according to claim 7, characterized in that, The step of performing image segmentation on the belt image based on the image feature information to obtain a plurality of segmented images includes: Encode the environmental features in the environmental feature information to obtain an environmental embedding vector; Determine image segmentation parameters based on the environmental embedding vector, and perform image segmentation on the belt image based on the image segmentation parameters and the image feature information to obtain a plurality of the segmented images.

9. A fault detection method according to claim 8, characterized in that, It further includes: Determine a prediction threshold based on the environmental embedding vector; Predict the fault type of the conveying device based on the multi-modal features and the prediction threshold to obtain the fault type of the conveying device.

10. An electronic device, characterized in that, It includes: A processor and a storage device; A computer program is stored on the storage device, and the computer program, when run by the processor, executes the method according to any one of claims 1 to 9.

Images