Method and device for monitoring running state of coal conveying belt and storage medium

Abstract

The invention discloses a running state monitoring method and device for a coal conveying belt and a storage medium, and the method comprises the steps: obtaining a video image used for describing the running state of the belt, and randomly extracting a plurality of image frames containing a belt region from the video image; identifying a belt area in each image frame, and determining a contrast area for detecting the running state of the belt based on the belt area, the contrast area corresponding to an object conveyed by the belt; determining a center point of the contrast area as a contrast point, and determining a motion vector of the contrast point between two adjacent image frames, thereby constructing a motion vector sequence based on a plurality of motion vectors corresponding to the plurality of image frames; and inputting the motion vector sequence into a preset state monitoring model, and judging whether the belt has an operation fault or not based on an output result of the state monitoring model. Therefore, the technical effects that the running state of the belt can be monitored in real time, and workers can conveniently adjust the faults of the belt in time are achieved.

Description

Technical Field

[0001] The present application relates to the technical field of belt operation status monitoring, and in particular to a method, device and storage medium for monitoring the operation status of a coal conveyor belt. Background Art

[0002] Since the safe operation of the coal conveyor belt directly affects the production efficiency of the power plant, it is necessary to monitor the operating status of the coal conveyor belt. In actual production, the most common faults of the coal conveyor belt are belt deviation and belt slippage. The most common way to monitor the operating status of the coal conveyor belt is manual inspection, but manual inspection has disadvantages such as high labor intensity of the inspectors and poor real-time performance. At present, there is also a monitoring method for the operating status of the coal conveyor belt. That is, a mechanical device is used to detect the operating status of the coal conveyor belt. However, the method of using a mechanical device to monitor the operating status of the coal conveyor belt has disadvantages such as poor reliability and low degree of automation.

[0003] The publication number is CN118992461A, and the name is a belt conveyor inspection method, system, medium, or program product. The method includes: determining the corresponding monitoring area according to the account information input by the user, and displaying the three-dimensional regional model of the monitoring area; in response to the user's selection operation of the target belt conveyor in the three-dimensional regional model, displaying the operating status of the target belt conveyor; after the user clicks the inspection start control, sending an inspection request to the on-site monitoring terminal of the target belt conveyor; receiving the on-site monitoring data returned by the on-site monitoring terminal; determining real-time warning information according to the on-site monitoring data, and displaying the real-time warning information.

[0004] The publication number is CN118387519A, and the name is a warehouse boring intelligent handling system and method. Among them; the real-time monitoring module is responsible for collecting real-time data during the handling process, including the running status of the conveyor belt, the type of material and the workshop environmental parameters; the intelligent scheduling module is responsible for issuing handling instructions, planning the material handling route and scheduling strategy, using the optimization algorithm to plan the optimal material handling route, and automatically issuing handling instructions to the automatic handling device based on the planning results; the automatic handling device is composed of a conveyor belt network, and performs the material handling task according to the instructions of the intelligent scheduling module, and is used for the handling of materials in the workshop.

[0005] There is no effective solution to the technical problems in the above-mentioned prior art that the method of monitoring the operating status of the coal conveyor belt by manual inspection has high economic cost and poor real-time performance; the method of monitoring the operating status of the coal conveyor belt by mechanical device has poor reliability and low automation level. Summary of the invention

[0006] The embodiments of the present disclosure provide a method, device and storage medium for monitoring the operating status of a coal conveyor belt, so as to at least solve the technical problems existing in the prior art that a method of monitoring the operating status of a coal conveyor belt by manual inspection has high economic cost and poor real-time performance; and a method of monitoring the operating status of a coal conveyor belt by mechanical devices has poor reliability and low automation.

[0007] According to one aspect of an embodiment of the present disclosure, there is provided a method for monitoring the operating status of a coal conveyor belt, comprising: acquiring a video image for describing the operating status of the belt, and randomly extracting a plurality of image frames including a belt area from the video image; identifying the belt area in each image frame, and determining a control area for detecting the operating status of the belt based on the belt area, wherein the control area corresponds to an object conveyed by the belt; determining a center point of the control area as a control point, and determining a motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on a plurality of motion vectors corresponding to a plurality of image frames; and inputting the motion vector sequence into a preset status monitoring model, and determining whether the belt has an operating fault based on an output result of the status monitoring model.

[0008] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0009] According to another aspect of the embodiment of the present disclosure, there is also provided an operation status monitoring device for a coal conveyor belt, including: an image frame extraction module, used to obtain a video image used to describe the operation status of the belt, and randomly extract multiple image frames containing the belt area from the video image; a control area determination module, used to identify the belt area in each image frame, and determine the control area for detecting the operation status of the belt based on the belt area, wherein the control area corresponds to the object conveyed by the belt; a motion vector determination module, used to determine the center point of the control area as the control point, and determine the motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on multiple motion vectors corresponding to multiple image frames; and an operation fault monitoring module, used to input the motion vector sequence into a preset state monitoring model, and determine whether the belt has an operation fault based on the output result of the state monitoring model.

[0010] According to another aspect of the embodiment of the present disclosure, there is also provided an operation status monitoring device for a coal conveyor belt, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: obtaining a video image for describing the operation status of the belt, and randomly extracting a plurality of image frames containing a belt area from the video image; identifying the belt area in each image frame, and determining a control area for detecting the operation status of the belt based on the belt area, wherein the control area corresponds to an object conveyed by the belt; determining a center point of the control area as a control point, and determining a motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on a plurality of motion vectors corresponding to a plurality of image frames; and inputting the motion vector sequence into a preset state monitoring model, and determining whether the belt has an operation fault based on an output result of the state monitoring model.

[0011] The present application provides a method for monitoring the operating status of a coal conveyor belt. First, the processor obtains a video image used to describe the operating status of the belt, and randomly extracts multiple image frames containing the belt area from the video image. Then, the processor identifies the belt area in each image frame, and determines a control area for detecting the operating status of the belt based on the belt area. Further, the processor determines the center point of the control area as the control point, and determines the motion vector of the control point between two adjacent image frames, thereby generating a motion vector sequence based on multiple motion vectors corresponding to multiple image frames. Finally, the processor inputs the motion vector sequence into a pre-set status monitoring model, and determines whether the belt has an operating fault based on the output result of the status monitoring model.

[0012] Referring to the above content, it can be seen that, unlike the prior art which uses manual inspection of the belt running status or uses mechanical devices to monitor the belt running status, in the present application, only the video image of the belt running status captured by a pre-installed camera and a pre-set status monitoring model are needed to monitor the belt running status.

[0013] Since the pre-installed camera can collect video images in real time, the staff can use the status monitoring model to monitor the running status of the belt in real time based on the real-time collected video images. This not only greatly reduces the economic cost of monitoring the running status of the belt, but also enables real-time monitoring of the running status of the belt, making it easier for the staff to make timely adjustments to the belt failure.

[0014] Furthermore, since the camera in the present application directly transmits the video image to the status monitoring model after capturing the video image, the degree of automation of monitoring the running status of the belt can be greatly improved without the need for additional human intervention.

[0015] This solves the technical problems in the prior art that the method of monitoring the operating status of the coal conveyor belt by manual inspection has high economic cost and poor real-time performance; and the method of monitoring the operating status of the coal conveyor belt by mechanical devices has poor reliability and low automation.

[0016] In addition, the status monitoring model in the present application is divided into a slip monitoring model and an offset monitoring model, wherein the slip monitoring model is used to determine whether the belt has a slip fault, and the offset monitoring model is used to determine whether the belt has an offset fault. Therefore, compared with the prior art, the present application greatly improves the scope of application of fault monitoring for the belt operation status. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings: Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present application; Figure 2 is a schematic diagram of the operating status monitoring system of the coal conveyor belt according to Example 1 of the present application; Figure 3 is a flow chart of the method for monitoring the operating status of a coal conveyor belt according to Example 1 of the present application; Figure 4 is a schematic diagram of belt slip states corresponding to different image frames according to Embodiment 1 of the present application; Figure 5 is a schematic diagram of a belt deviation state corresponding to different image frames according to Embodiment 1 of the present application; Figure 6 is a schematic diagram of the control area according to Example 1 of the present application; Figure 7 is a schematic diagram of another belt deviation state corresponding to different image frames according to Embodiment 1 of the present application; Figure 8 is a schematic diagram of a belt equipped with multiple vibration sensors according to Example 1 of the present application; Fig. 9 is a schematic diagram of the operating status monitoring device of the coal conveyor belt according to Example 2 of the present application; Fig.10 It is a schematic diagram of the coal conveyor belt operation status monitoring device according to Example 3 of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0020] Example 1 According to this embodiment, a method embodiment of monitoring the operating status of a coal conveyor belt is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0021] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a method for monitoring the operating status of a coal conveyor belt. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1Different configurations are shown.

[0022] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computing device. As involved in the embodiments of the present disclosure, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0023] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for monitoring the operating status of the coal conveyor belt in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the method for monitoring the operating status of the coal conveyor belt of the above-mentioned application program is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0025] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0026] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0027] Figure 2 Schematic diagram of the coal conveyor belt operation status monitoring system according to this embodiment. Figure 2 As shown, the system includes: a camera and a terminal device. The camera is installed at a position corresponding to the belt and is used to collect video images when the belt transports coal. The terminal device is connected to the camera for communication, and a processor of the terminal device is used to obtain the video images collected by the camera, and based on the obtained video images, determine whether the belt has an operating fault.

[0028] It should be noted that the terminal equipment in the system can be applicable to the hardware structure described above.

[0029] Under the above operating environment, according to the first aspect of this embodiment, a method for monitoring the operating status of a coal conveyor belt is provided. The method comprises: Figure 2 The processor implementation of the terminal device shown in . Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes: S302: acquiring a video image for describing the running state of the belt, and randomly extracting a plurality of image frames including the belt area from the video image; S304: identifying a belt area in each image frame, and determining a control area for detecting a running state of the belt based on the belt area, wherein the control area corresponds to an object conveyed by the belt; S306: determining the center point of the control area as the control point, and determining the motion vector of the control point between two adjacent image frames, thereby generating a motion vector sequence based on a plurality of motion vectors corresponding to a plurality of image frames; and S408: Input the motion vector sequence into a preset state monitoring model, and determine whether the belt has an operation fault based on an output result of the state monitoring model.

[0030] Specifically, refer to Figure 2 As shown, the staff controls the camera through the terminal device to start collecting video images of the belt conveying coal. When the camera collects the video images of the belt conveying coal, the video images are transmitted to the processor of the terminal device in real time. Thus, the processor of the terminal device obtains the video images used to describe the running state of the belt, and randomly extracts multiple image frames containing the belt area from the video images (S302). For example, the camera is T 1 Start collecting video images at the moment T n Stop collecting video images at all times so that the processor of the terminal device can obtain the same T 1 ~ Tn The video image corresponding to the time period. And the processor obtains T 1 ~ T n In the case of video images corresponding to the time period, further acquisition and T 1 The image frame corresponding to the moment T 2 The image frame corresponding to the moment T 3 The image frame corresponding to the moment, ..., and T n Corresponding image frames. Thus, the processor can finally extract n image frames, and each image frame contains the belt area.

[0031] It is worth noting that this embodiment takes the example of the processor extracting multiple image frames from the video image at the same time interval. In fact, the processor can extract multiple image frames from the video image at different time intervals, which will not be described in detail here.

[0032] Afterwards, when the processor extracts multiple image frames, the belt area in each image frame is identified, and a control area for detecting the running state of the belt is determined based on the belt area (S304). The control area corresponds to the object conveyed by the belt. For example, since the n image frames all contain the belt area, when the processor extracts n image frames, the n image frames can be input into a pre-trained convolutional neural network model, so as to identify the belt area in the n image frames using the convolutional neural network model. The staff pre-labeled the belt area in the historical image frames, and trained the convolutional neural network model using the labeled historical image frames.

[0033] In the case where the processor identifies the belt area in each image frame, the conveying object area in each belt area is further determined, and each conveying object area in each image frame is divided into multiple image blocks. Thus, the processor can determine a plurality of image block sets corresponding to each image frame based on a plurality of image blocks corresponding to rows or a plurality of image blocks corresponding to columns in each image frame. The conveying object area is used to indicate an area for conveying objects in the belt area. Then, the processor inputs a plurality of image block sets corresponding to rows or a plurality of image block sets corresponding to columns of two adjacent image frames in each image frame into a neural network model based on a self-attention mechanism, and determines the similarity between image blocks at corresponding positions of two adjacent image frames in each image frame. Further, the processor determines a plurality of similarity matrices corresponding to a plurality of image frames based on the similarity between image blocks at corresponding positions of two adjacent image frames. Then, the processor determines a target similarity greater than a preset similarity threshold in each similarity matrix, and determines an image block corresponding to the target similarity in each image frame. Finally, the processor determines a control area in each image frame based on a plurality of image blocks in each image frame. The above content will be described in detail later, so it will not be repeated here.

[0034] Furthermore, when determining the control area in each image frame, the processor determines the center point of the control area as the control point, and determines the motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on multiple motion vectors corresponding to multiple image frames (S306). Finally, the motion vector sequence is input into a pre-set state monitoring model, and based on the output result of the state monitoring model, it is determined whether the belt has an operation fault (S308). Specifically, since the operation faults of the belt in the actual conveying process include belt slip faults and belt deviation faults, the monitoring of the operation faults of the belt includes the monitoring of belt slip faults and belt deviation faults.

[0035] Among them, the monitoring of whether the belt has a slip fault includes: first, the processor determines the first sub-vector corresponding to the reference point of two adjacent image frames in multiple image frames, and constructs a first sub-vector sequence based on multiple first sub-vectors corresponding to multiple image frames. Among them, the first sub-vector is used to indicate the motion vector of the reference point conveyed along the belt. Then the processor inputs the first sub-vector sequence into a pre-set slip monitoring model, and determines whether the belt has a slip fault based on the output result of the slip monitoring model. The above content will be described in detail later, so it will not be repeated here.

[0036] And wherein, monitoring whether there is an offset fault of the belt includes: first, the processor determines the second sub-vector corresponding to the reference point of two adjacent image frames in multiple image frames, and constructs a second sub-vector sequence based on the multiple second sub-vectors corresponding to the multiple image frames. The second sub-vector is used to indicate the motion vector of the reference point perpendicular to the conveying direction of the belt. Then, the processor inputs the second sub-vector sequence into a pre-set offset monitoring model, and determines whether there is an offset fault of the belt based on the output result of the offset monitoring model. The above content will be described in detail later, so it will not be repeated here.

[0037] Therefore, it can be seen from the above content that since the state monitoring model (including the slip monitoring model and the offset monitoring model) is set in the processor, the belt can be monitored for slip faults and offset faults, thereby greatly improving the scope of application of the model.

[0038] As described in the background technology, since the safe operation of the coal conveyor belt directly affects the production efficiency of the power plant, it is very necessary to monitor the operating status of the coal conveyor belt. In actual production, the most common faults of the coal conveyor belt are belt deviation fault and belt slip fault. The most common way to monitor the operating status of the coal conveyor belt is manual inspection, but manual inspection has the disadvantages of high labor intensity of the inspectors and poor real-time performance. At present, there is also a monitoring method for the operating status of the coal conveyor belt. That is, a mechanical device is used to detect the operating status of the coal conveyor belt. However, the method of using a mechanical device to monitor the operating status of the coal conveyor belt has the disadvantages of poor reliability and low degree of automation.

[0039] In view of this, the present application provides a method for monitoring the running status of a coal conveyor belt. Different from the prior art that uses manual inspection of the running status of the belt or uses a mechanical device to monitor the running status of the belt, the present application only needs to use the video image of the running status of the belt collected by a pre-installed camera and a pre-set status monitoring model to monitor the running status of the belt.

[0040] Since the pre-installed camera can collect video images in real time, the staff can use the status monitoring model to monitor the running status of the belt in real time based on the real-time collected video images. This not only greatly reduces the economic cost of monitoring the running status of the belt, but also enables real-time monitoring of the running status of the belt, making it easier for the staff to make timely adjustments to the belt failure.

[0041] Furthermore, since the camera in the present application directly transmits the video image to the status monitoring model after capturing the video image, the degree of automation of monitoring the running status of the belt can be greatly improved without the need for additional human intervention.

[0042] This solves the technical problems in the prior art that the method of monitoring the operating status of the coal conveyor belt by manual inspection has high economic cost and poor real-time performance; and the method of monitoring the operating status of the coal conveyor belt by mechanical devices has poor reliability and low automation.

[0043] Optionally, the motion vector includes a first sub-vector of a control point along the conveying direction of the belt, the state monitoring model includes a slip monitoring model, and the motion vector sequence is input into a preset state monitoring model, and based on the output result of the state monitoring model, an operation of determining whether the belt has an operating fault comprises: determining a first sub-vector corresponding to a control point of two adjacent image frames in a plurality of image frames, and constructing a first sub-vector sequence based on a plurality of first sub-vectors corresponding to the plurality of image frames; and inputting the first sub-vector sequence into a preset slip monitoring model, and determining whether the belt has a slip fault based on the output result of the slip monitoring model.

[0044] Specifically, Figure 4 is a schematic diagram of belt slip states corresponding to different image frames according to an embodiment of the present application. Figure 4 As shown, for example, the processor extracts n image frames from the video image captured by the camera, and the processor extracts T 1 Image frame corresponding to the moment A 1 ,and T 2 Image frame corresponding to the moment A 2 ,and T 3 Image frame corresponding to the moment A 3 ,...,and T n Image frame corresponding to the moment A n .

[0045] And from Figure 4 It can be seen that the image frame A 1 ~A n There is a control area in the belt area in the image, and the control area is A 1 ~A n The running status in the image can represent the belt area in each image frame. A 1 ~A n The processor can then perform the control region in each image frame.A 1 ~A n Observe the movement of the belt to determine whether the belt is slipping.

[0046] refer to Figure 4 As shown, first, the processor determines a plurality of image frames A 1 ~A n The first subvector corresponding to the reference point of two adjacent image frames. A 1 The center point of the control area in the image frame A 2 There is a first subvector between the center points of the control area in f 1 , image frame A 2 The center point of the control area in the image frame A 3 There is a first subvector between the center points of the control area in f 2 , and so on, the image frame A n-1 The center point of the control area in the image frame A n There is a first subvector between the center points of the control area in f n-1 .

[0047] Thus, when the processor determines multiple first sub-vectors, it can further construct a first sub-vector sequence. f 1 ~f n-1 , then the first sub-vector sequence is .

[0048] Further, when the processor constructs the first sub-vector sequence, the first sub-vector sequence is input into a preset slip monitoring model, and based on the output result of the slip monitoring model, it is determined whether the belt has a slip fault. The slip monitoring model includes a plurality of binary classification models. The number of binary classification models in the slip monitoring model corresponds to the dimension of the first sub-vector sequence.

[0049] For example, the first sub-vector sequence is , then the skid monitoring model includes n-1 binary classification models. When input into the slip monitoring model, n-1 output results can be output, and each output result is used to indicate whether the belt has a slip fault or not.

[0050] For example, if the output of the slip monitoring model is consistent with the first sub-vector f 1 The corresponding first output result indicates that there is a slip fault, which means T 1 Image frame corresponding to the moment A 1 and with T 2 Image frame corresponding to the moment A 2 There is a slip fault.

[0051] Therefore, from the above content, it can be seen that the slip monitoring model in the present application can not only monitor whether there is a slip fault in the belt, but also determine the time when the belt starts to slip, thereby enabling the staff to better adjust the belt to ensure the normal operation of the belt.

[0052] Optionally, the motion vector includes a second sub-vector at a reference point perpendicular to the conveying direction of the belt, the state monitoring model includes an offset monitoring model, and the motion vector sequence is input into a preset state monitoring model, and based on the output result of the state monitoring model, an operation of determining whether the belt has an operating fault comprises: determining a second sub-vector corresponding to a reference point of two adjacent image frames in a plurality of image frames, and constructing a second sub-vector sequence based on a plurality of second sub-vectors corresponding to the plurality of image frames; and inputting the second sub-vector sequence into a preset offset monitoring model, and determining whether the belt has an offset fault based on the output result of the offset monitoring model.

[0053] Specifically, Figure 5 is a schematic diagram of a belt deviation state corresponding to different image frames according to an embodiment of the present application. Figure 5 As shown, for example, the processor extracts m image frames from the video image captured by the camera, and the processor extracts T 1 Image frame corresponding to the moment B 1 ,and T 2 Image frame corresponding to the moment B 2 ,and T 3 Image frame corresponding to the moment B 3 ,...,and T m Image frame corresponding to the momentB m .

[0054] And from Figure 5 It can be seen that the image frame B 1 ~B m There is a control area in the belt area in the image, and the control area is B 1 ~B m The running status in the image can represent the belt area in each image frame. B 1 ~B m The processor can then perform the control region in each image frame. B 1 ~B m Observe the movement of the belt to determine whether the belt has an offset fault.

[0055] refer to Figure 5 As shown, first, the processor determines a plurality of image frames B 1~ B m In , the second subvector corresponding to the reference point of two adjacent image frames. B 1 The center point of the control area in the image frame B 2 There is a second subvector between the center points of the control area in q 1 , image frame B 2 The center point of the control area in the image frame B 3 There is a second subvector between the center points of the control area in q 2 , and so on, the image frame B m-1 The center point of the control area in the image frame B m There is a second subvector between the center points of the control area in q m-1 .

[0056] Thus, when the processor determines multiple second sub-vectors, it can further construct a second sub-vector sequence. q 1 ~q m-1 , then the second sub-vector sequence is .

[0057] Further, when the processor constructs the second sub-vector sequence, the second sub-vector sequence is input into a preset offset monitoring model, and based on the output result of the offset monitoring model, it is determined whether the belt has an offset fault. The offset monitoring model includes a plurality of binary classification models. The number of binary classification models in the offset monitoring model corresponds to the dimension of the second sub-vector sequence.

[0058] For example, the second sub-vector sequence is , then the offset monitoring model includes m-1 binary classification models. When input into the deviation monitoring model, m-1 output results can be output, and each output result is used to indicate that there is a deviation fault of the belt or there is no deviation fault of the belt.

[0059] For example, if the offset monitoring model output is consistent with the second sub-vector q 2 The corresponding second output result indicates that there is an offset fault, which means that T 2 Image frame corresponding to the moment A 2 and with T 3 Image frame corresponding to the moment A 3 There is an offset fault between them.

[0060] Therefore, from the above content, it can be seen that the offset monitoring model in the present application can not only monitor whether there is an offset fault in the belt, but also determine the time when the belt starts to offset, thereby enabling the staff to better adjust the belt to ensure the normal operation of the belt.

[0061] Optionally, the operation of determining whether the belt has an offset fault based on the output result of the offset monitoring model includes: when determining the output result of the offset monitoring model, comparing the size relationship between each second sub-vector and a preset first offset threshold; and when the output result of the offset monitoring model shows that an offset fault exists and the second sub-vector is greater than the first offset threshold, determining that the belt has an offset fault.

[0062] Specifically, refer to Figure 5As shown, during the actual movement of the belt, even if the belt is offset, a certain degree of offset can be ignored as long as it does not affect the normal operation state. Therefore, when determining the output result of the offset monitoring model, the processor compares each second sub-vector with the preset first offset threshold. If the output result of the offset monitoring model shows that there is an offset fault, and the second sub-vector is greater than the first offset threshold, it means that there is an offset fault in the belt; if the output result of the offset monitoring shows that there is an offset fault, but the second sub-vector is less than or equal to the first offset threshold, it means that there is no offset fault in the belt.

[0063] For example, if the offset monitoring model output is consistent with the second sub-vector q 2 The corresponding second output result indicates that there is an offset fault, but the second subvector q 2 Less than the preset first offset threshold q k , it means that the belt offset distance is within the acceptable range and does not affect the normal operation of the belt. This means that there is no belt offset fault.

[0064] For another example, if the offset monitoring model outputs the same value as the second sub-vector q 2 The corresponding second output result indicates that there is an offset fault, and the second sub-vector q 2 Greater than the preset first offset threshold q k , it means that the belt offset distance has exceeded the acceptable range and has affected the normal operation of the belt. This indicates that the belt has an offset fault.

[0065] Optionally, the operation of determining a control area for detecting the running status of the belt based on the belt area includes: determining a conveying object area in each belt area corresponding to each image frame, and dividing the conveying object area corresponding to each image frame into multiple image blocks, wherein the conveying object area includes an object conveyed by the belt; determining multiple image block sets corresponding to each image frame based on multiple image blocks corresponding to rows or columns of each image frame, wherein each image block has the same size; inputting the multiple image block sets corresponding to each image frame into a neural network model based on a self-attention mechanism, and determining the similarity between image blocks at corresponding positions of two adjacent image frames in the multiple image frames; determining multiple similarity matrices corresponding to the multiple image frames based on the similarity between image blocks at corresponding positions in two adjacent image frames; respectively determining a target similarity greater than a preset similarity threshold in each similarity matrix, and determining an image block corresponding to the target similarity in each image frame; and determining a control area in each image frame based on the multiple image blocks in each image frame.

[0066] Specifically, Figure 6 is a schematic diagram of the control area according to the embodiment of the present application. Figure 6 As shown, first, the processor determines each belt area corresponding to each image frame S 1 Transport target area S 2 However, since not all belt areas have conveying objects, and the belt areas without conveying objects have similar shapes, the belt areas without conveying objects cannot be used as control areas. S 2 .

[0067] The processor then compares the individual image frames A 1 ~A n Corresponding transport area S 2 Further, based on the A 1 ~A n The multiple image blocks corresponding to the rows or columns of each image frame are determined A 1 ~A n A corresponding set of multiple image blocks, where each image block has the same size. Figure 6 As shown, in Figure 6 The transport target area in the image frame shownS 2 , there are multiple sets of image blocks corresponding to rows or multiple sets of image blocks corresponding to columns.

[0068] Thus, when there are multiple image frames A 1 ~A n In this case, it can be determined that the A 1 ~A n Corresponding multiple image block sets. For example, A 1 The corresponding image block set , with the image frame A 2 The corresponding image block set , with the image frame A 3 The corresponding image block set , ..., with the image frame A n The corresponding image block set .

[0069] The processor then inputs a plurality of sets of image blocks corresponding to each image frame into a neural network model based on a self-attention mechanism, and determines the similarity between image blocks at corresponding positions of two adjacent image frames in the plurality of image frames. A 1 Corresponding multiple image block sets Input to the neural network model based on the self-attention mechanism, the processor will be combined with the image frame A 2 Corresponding multiple image block sets Input to the neural network model based on the self-attention mechanism,..., the processor will be connected with the image frame A n Corresponding multiple image block sets Input to the neural network model based on the self-attention mechanism.

[0070] Therefore, the neural network model based on the self-attention mechanism can determine multiple image frames A 1 ~A n The similarity between the image blocks at the corresponding positions of two adjacent image frames. For example, a neural network model based on the self-attention mechanism receives an image frame. A 1 and image frames A 2In the case of a plurality of image blocks in columns, the image frame can be determined A 1 Center and position (i.e., image frame A 1 The image block of the first row and first column in the image frame A 2 With location (ie, image frame A 2 The similarity corresponding to the image block in the first row and first column of .

[0071] The neural network model based on the self-attention mechanism can determine the image frame A 1 Center and position (ie, image frame A 1 The image block in the second row and first column in the image frame A 2 With location (ie, image frame A 2 The similarity corresponding to the image block in the second row and first column .

[0072] And so on.

[0073] The neural network model based on the self-attention mechanism can determine the image frame A 1 Center and position (ie, image frame A 1 The image block in the tth row and hth column in the image frame A 2 With location (ie, image frame A 2 The similarity corresponding to the image block in the tth row and hth column .

[0074] It is worth noting that the above is an image frame A 1 and image frames A 2 The similarity between the multiple image blocks and the similarity between other two adjacent image frames can also be determined based on the above method, which will not be described in detail here.

[0075] Furthermore, the processor determines a plurality of similarity matrices corresponding to the plurality of image frames based on the similarity between the image blocks at corresponding positions in the two adjacent image frames.A 1 ~A 2 There is similarity between the image blocks at corresponding positions in the image frame, and each image frame has multiple image blocks, so the processor can determine multiple similarities corresponding to the multiple image blocks in two adjacent image frames. Thus, the processor can determine the similarity matrix based on the multiple similarities corresponding to the multiple image blocks in the two adjacent image frames. . And because there are multiple image frames A 1 ~A n , so that the processor can determine the multiple image frames A 1 ~ A n Corresponding multiple similarity matrices Q 1 ~Q n-1 .

[0076] After that, the processor determines the A 1 ~A n Corresponding multiple similarity matrices Q 1 ~Q n-1 In the case of Q 1 ~Q n-1 is greater than the preset similarity threshold Q k The target similarity and in each image frame A 1 ~A n For example, the processor determines the similarity matrix Q 1 is greater than the preset similarity threshold Q k The similarity is , , , , , , , , . Thus, the processor will be greater than the similarity threshold The similarity is determined as the target similarity. In addition, when the processor determines the target similarity, the image blocks corresponding to the target similarity are further determined. Thus, the processor finally determines the area composed of the image blocks corresponding to the target similarity as the control area.

[0077] Optionally, it also includes: identifying the belt area boundary in the belt area along the belt conveying direction, and determining any one of the belt area boundaries as a control boundary; determining whether the control boundaries of two adjacent image frames in each image frame are offset; and when the control boundaries of two adjacent image frames are offset and the offset is greater than a pre-set second offset threshold, it indicates that there is an offset fault in the belt.

[0078] Specifically, Figure 7 is a schematic diagram of another belt deviation state corresponding to different image frames according to an embodiment of the present application. Figure 7 As shown, in an embodiment of the present application, in addition to monitoring whether the belt has an offset fault through the offset monitoring model as mentioned above, the processor can also directly determine whether the belt has an offset fault by judging the size relationship between the offset amount of the control boundary of two adjacent image frames and a preset second offset threshold.

[0079] First, the processor identifies the belt area boundary along the belt conveying direction in the belt area (i.e., Figure 6 Belt zone boundaries along the x-direction in ). For example, Figure 7 In the image frame B 1 There are two belt area boundaries, corresponding to the upper and lower belt areas, and the image frame B 2 There are two belt area boundaries, corresponding to the upper and lower belt areas, and the image frame B 3 There are two belt area boundaries, and so on, corresponding to the image frame B m There are two corresponding belt area boundaries, upper and lower.

[0080] The processor then determines any one of the belt area boundaries as a control boundary. Figure 7 As shown, the processor can convert the image frame B 1 ~B m The upper belt area boundary is determined as the control boundary.

[0081] Then, the processor determines whether the comparison boundary between two adjacent image frames in each image frame is offset. And if the comparison boundary between two adjacent image frames is offset and the offset is greater than the second offset threshold, the processor determines that the belt has an offset fault. For example, referring to Figure 7As shown, the processor determines the B 1 ~B m Two adjacent image frames B 1 ~B 2 Whether the control boundary in the image is offset; the processor determines whether the control boundary in the image is offset; B 1 ~B m Two adjacent image frames B 2 ~ B 3 Whether the control boundary in the image is offset; and so on; the processor determines whether the control boundary in the image is offset; and so on; B 1 ~B m Two adjacent image frames B m-1 ~B m Whether the control boundary in is offset.

[0082] The processor then determines two adjacent image frames among the plurality of image frames. B 2 ~B 3 The control boundary of p 1 Greater than the preset second offset threshold p k , it means that the belt has an offset fault.

[0083] From the above, it can be seen that in the present application, not only can the deviation monitoring model be used to determine whether the belt has a deviation fault, but also the deviation fault can be determined by determining the magnitude relationship between the deviation amount corresponding to the reference boundary and the second deviation threshold. Thus, the technical effect of determining whether the belt has a deviation fault in multiple ways can be achieved.

[0084] Optionally, it also includes: in the event that the belt has an operating fault, determining the starting position of the belt operating fault, wherein the operation of determining the starting position of the belt operating fault in the event that the belt has an operating fault includes: collecting vibration data corresponding to each position of the belt at a preset distance interval, and constructing a vibration data sequence based on multiple vibration data; and inputting the vibration data sequence and the motion vector sequence into a preset fault position detection model, and determining the starting position of the belt operating fault, wherein the fault position detection model includes a multi-layer perceptron model and a fully connected layer.

[0085] Specifically, in the above-mentioned content, the processor can not only determine whether the belt has a slip fault by using the slip monitoring model, but also determine the starting time of the slip fault of the belt. Similarly, the processor can not only determine whether the belt has a deviation fault by using the deviation monitoring model, but also determine the starting time of the deviation fault of the belt.

[0086] However, the above two methods cannot determine the starting position of the belt slippage fault or the belt deviation fault. Therefore, if the processor wants to further accurately determine the starting position of the belt slippage fault or the belt deviation fault, it can use a preset fault position detection model to determine it.

[0087] Figure 8 is a schematic diagram of a belt equipped with multiple vibration sensors according to an embodiment of the present application. Figure 8 As shown, first, the processor collects vibration data corresponding to each position of the belt based on a plurality of pre-set vibration sensors. C 1 , collection and belt position L 1 Corresponding vibration data g 1 ; Processor based vibration sensor C 2 , collection and belt position L 2 Corresponding vibration data g 2 ; And so on; The processor is based on the vibration sensor C v , collection and belt position L v Corresponding vibration data g v .

[0088] The processor can then collect multiple vibration data g 1 ~g v , construct vibration data sequence .

[0089] Further, the processor converts the vibration data sequence The motion vector sequence is input into a preset fault position detection model to determine the starting position of the belt fault. The motion vector sequence can be a first sub-vector sequence It can also be the second sub-vector sequence . And wherein, the fault location detection model includes a multi-layer perceptron model and a fully connected layer.

[0090] And if the processor wants to determine the specific location of the belt slippage fault, the vibration data sequence With the first sub-vector sequence The multi-layer perceptron model in the fault location detection model can transform the vibration data sequence into With the first sub-vector sequence Perform data fusion. Then the fully connected layer performs classification based on the fused data sequence and finally determines the specific location where the belt slips.

[0091] In the case where the processor wants to determine the specific location of the belt deviation fault, the vibration data sequence can be With the second sub-vector sequence The multi-layer perceptron model in the fault location detection model can transform the vibration data sequence into With the second sub-vector sequence Perform data fusion. Then the fully connected layer performs classification based on the fused data sequence and finally determines the specific location of the belt offset fault.

[0092] Thereby, the technical effect of accurately determining the specific position where the belt has a slip fault or a deviation fault can be achieved.

[0093] Therefore, according to the first aspect of this embodiment, the technical effect of being able to monitor the running status of the belt in real time and facilitating the staff to make timely adjustments to the belt failure is achieved.

[0094] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0095] Therefore, according to this embodiment, the technical effect of being able to monitor the running status of the belt in real time and facilitating the staff to make timely adjustments to the belt faults is achieved.

[0096] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0098] Example 2 Fig. 9 The running state monitoring device 900 of the coal conveyor belt according to this embodiment is shown, and the device 900 corresponds to the method according to the embodiment 1. Fig. 9 As shown, the device 900 includes: an image frame extraction module 910, which is used to obtain a video image for describing the running state of the belt, and randomly extract a plurality of image frames containing the belt area from the video image; a control area determination module 920, which is used to identify the belt area in each image frame, and determine the control area for detecting the running state of the belt based on the belt area, wherein the control area corresponds to the object conveyed by the belt; a motion vector determination module 930, which is used to determine the center point of the control area as the control point, and determine the motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on the plurality of motion vectors corresponding to the plurality of image frames; and an operation fault monitoring module 940, which is used to input the motion vector sequence into a preset state monitoring model, and determine whether the belt has an operation fault based on the output result of the state monitoring model.

[0099] Optionally, the motion vector includes a first sub-vector of the control point along the conveying direction of the belt, the state monitoring model includes a slip monitoring model, and the running fault monitoring module 940 includes: a first vector sequence construction module, used to determine the first sub-vector corresponding to the control point of two adjacent image frames in multiple image frames, and construct a first sub-vector sequence based on multiple first sub-vectors corresponding to the multiple image frames; and a first judgment module, used to input the first sub-vector sequence into a preset slip monitoring model, and determine whether the belt has a slip fault based on the output result of the slip monitoring model.

[0100] Optionally, the motion vector includes a second sub-vector at a reference point perpendicular to the conveying direction of the belt, the state monitoring model includes an offset monitoring model, and the fault monitoring module 940 is run, including: a second vector sequence construction module, used to determine the second sub-vector corresponding to the reference point of two adjacent image frames in multiple image frames, and construct a second sub-vector sequence based on the multiple second sub-vectors corresponding to the multiple image frames; and a second determination module, used to input the second sub-vector sequence into a preset offset monitoring model, and determine whether the belt has an offset fault based on the output result of the offset monitoring model.

[0101] Optionally, the second determination module includes: a comparison module, which is used to compare the size relationship between each second sub-vector and a preset first offset threshold when determining the output result of the offset monitoring model; and an offset fault determination module, which is used to determine that the belt has an offset fault when the output result of the offset monitoring model indicates the existence of an offset fault and the second sub-vector is greater than the first offset threshold.

[0102] Optionally, the control area determination module 920 includes: an image block division module, which is used to determine the conveying object area in each belt area corresponding to each image frame, and divide the conveying object area corresponding to each image frame into multiple image blocks, wherein the conveying object area includes the object conveyed by the belt; an image block set determination module, which is used to determine multiple image block sets corresponding to each image frame based on multiple image blocks corresponding to rows or columns of each image frame, wherein the sizes of each image block are the same; a similarity determination module, which is used to input the multiple image block sets corresponding to each image frame into a neural network model based on a self-attention mechanism, and determine the similarity between image blocks at corresponding positions of two adjacent image frames in the multiple image frames; a similarity matrix determination module, which is used to determine multiple similarity matrices corresponding to the multiple image frames based on the similarity between image blocks at corresponding positions in two adjacent image frames; a target similarity determination module, which is used to respectively determine the target similarity greater than a preset similarity threshold in each similarity matrix, and determine the image block corresponding to the target similarity in each image frame; and a control area determination submodule, which is used to determine the control area in each image frame based on the multiple image blocks in each image frame.

[0103] Optionally, the device 900 also includes: a control boundary identification module, which is used to identify the belt area boundary in the belt area along the belt conveying direction, and determine any one of the belt area boundaries as the control boundary; an offset judgment module, which is used to determine whether the control boundaries of two adjacent image frames in each image frame are offset; and an offset fault determination module, which is used to indicate that the belt has the offset fault when the control boundaries between two adjacent image frames are offset and the offset amount is greater than a preset second offset threshold.

[0104] Optionally, the device 900 also includes: a fault position determination module, which is used to determine the starting position of the belt fault when the belt has an operating fault, wherein the fault position determination module includes: a data sequence construction module, which is used to collect vibration data corresponding to each position of the belt, and construct a vibration data sequence based on multiple vibration data; and a starting position determination module, which is used to input the vibration data sequence and the motion vector sequence into a pre-set fault position detection model, and determine the starting position of the belt fault, wherein the fault position detection model includes a multi-layer perceptron model and a fully connected layer.

[0105] Therefore, according to this embodiment, the technical effect of being able to monitor the running status of the belt in real time and facilitating the staff to make timely adjustments to the belt faults is achieved.

[0106] Example 3 Fig.10 The running state monitoring device 1000 of the coal conveyor belt according to this embodiment is shown, and the device 1000 corresponds to the method according to embodiment 1. As shown in reference 10, the device 1000 includes: a processor 1010; and a memory 1020, connected to the processor 1010, for providing the processor 1010 with instructions for processing the following processing steps: acquiring a video image for describing the running state of the belt, and randomly extracting a plurality of image frames containing the belt area from the video image; identifying the belt area in each image frame, and determining a control area for detecting the running state of the belt based on the belt area, wherein the control area corresponds to an object conveyed by the belt; determining the center point of the control area as the control point, and determining the motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on a plurality of motion vectors corresponding to the plurality of image frames; and inputting the motion vector sequence into a preset state monitoring model, and determining whether the belt has an operation fault based on the output result of the state monitoring model.

[0107] Optionally, the motion vector includes a first sub-vector of a control point along the conveying direction of the belt, the state monitoring model includes a slip monitoring model, and the motion vector sequence is input into a preset state monitoring model, and based on the output result of the state monitoring model, an operation of determining whether the belt has an operating fault comprises: determining a first sub-vector corresponding to a control point of two adjacent image frames in a plurality of image frames, and constructing a first sub-vector sequence based on a plurality of first sub-vectors corresponding to the plurality of image frames; and inputting the first sub-vector sequence into a preset slip monitoring model, and determining whether the belt has a slip fault based on the output result of the slip monitoring model.

[0108] Optionally, the motion vector includes a second sub-vector at a control point perpendicular to the conveying direction of the belt, the state monitoring model includes an offset monitoring model, and the motion vector sequence is input into a preset state monitoring model, and based on the output result of the state monitoring model, an operation of determining whether the belt has an operating fault comprises: determining a second sub-vector corresponding to a control point of two adjacent image frames in a plurality of image frames, and constructing a second sub-vector sequence based on a plurality of second sub-vectors corresponding to the plurality of image frames; and inputting the second sub-vector sequence into a preset offset monitoring model, and based on the output result of the offset monitoring model, determining whether the belt has an offset fault.

[0109] Optionally, the operation of determining whether the belt has an offset fault based on the output result of the offset monitoring model includes: when determining the output result of the offset monitoring model, comparing the size relationship between each second sub-vector and a preset first offset threshold; and when the output result of the offset monitoring model indicates the existence of an offset fault and the second sub-vector is greater than the first offset threshold, determining that the belt has an offset fault.

[0110] Optionally, the operation of determining a control area for detecting the running status of the belt based on the belt area includes: determining a conveying object area in each belt area corresponding to each image frame, and dividing the conveying object area corresponding to each image frame into multiple image blocks, wherein the conveying object area includes an object conveyed by the belt; determining multiple image block sets corresponding to each image frame based on multiple image blocks corresponding to rows or columns of each image frame, wherein the sizes of each image block are the same; inputting the multiple image block sets corresponding to each image frame into a neural network model based on a self-attention mechanism, and determining the similarity between image blocks at corresponding positions of two adjacent image frames in the multiple image frames; determining multiple similarity matrices corresponding to the multiple image frames based on the similarity between image blocks at corresponding positions in two adjacent image frames; respectively determining a target similarity greater than a preset similarity threshold in each similarity matrix, and determining an image block corresponding to the target similarity in each image frame; and determining a control area in each image frame based on the multiple image blocks in each image frame.

[0111] Optionally, the device also includes: identifying the belt area boundary in the belt area along the belt conveying direction, and determining any one of the belt area boundaries as a control boundary; determining whether the control boundaries of two adjacent image frames in each image frame are offset; and when the control boundaries of two adjacent image frames are offset and the offset is greater than a pre-set second offset threshold, it indicates that there is an offset fault in the belt.

[0112] Optionally, the device also includes: in the event that the belt has an operating fault, determining the starting position of the belt operating fault, wherein in the event that the belt has an operating fault, the operation of determining the starting position of the belt operating fault includes: collecting vibration data corresponding to each position of the belt, and constructing a vibration data sequence based on multiple vibration data; and inputting the vibration data sequence and the motion vector sequence into a pre-set fault position detection model, and determining the starting position of the belt fault, wherein the fault position detection model includes a multi-layer perceptron model and a fully connected layer.

[0113] Therefore, according to this embodiment, the technical effect of being able to monitor the running status of the belt in real time and facilitating the staff to make timely adjustments to the belt faults is achieved.

[0114] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0115] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0120] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for monitoring the operating status of a coal conveyor belt, characterized in that: include: Acquire a video image for describing the running state of the belt, and randomly extract a plurality of image frames containing the belt area from the video image; Identifying a belt area in each image frame, and determining a control area for detecting a running state of the belt based on the belt area, wherein the control area corresponds to an object conveyed by the belt; Determine the center point of the control area as the control point, and determine the motion vector of the control point between two adjacent image frames, so as to construct a motion vector sequence based on a plurality of motion vectors corresponding to the plurality of image frames; as well as The motion vector sequence is input into a preset condition monitoring model, and based on the output result of the condition monitoring model, it is determined whether the belt has an operation fault.

2. The method according to claim 1, characterized in that: The motion vector includes a first sub-vector of the control point along the conveying direction of the belt, the state monitoring model includes a slip monitoring model, and the motion vector sequence is input into a preset state monitoring model, and based on the output result of the state monitoring model, the operation of determining whether the belt has an operation fault includes: Determine a first sub-vector corresponding to a comparison point of two adjacent image frames in the plurality of image frames, and construct a first sub-vector sequence based on the plurality of first sub-vectors corresponding to the plurality of image frames; and The first sub-vector sequence is input into a preset slip monitoring model, and based on an output result of the slip monitoring model, it is determined whether the belt has a slip fault.

3. The method according to claim 1, characterized in that The motion vector includes a second sub-vector of the reference point perpendicular to the conveying direction of the belt, the state monitoring model includes an offset monitoring model, and the motion vector sequence is input into a preset state monitoring model, and based on the output result of the state monitoring model, the operation of determining whether the belt has an operation fault includes: Determine a second sub-vector corresponding to a comparison point of two adjacent image frames in the plurality of image frames, and construct a second sub-vector sequence based on the plurality of second sub-vectors corresponding to the plurality of image frames; and The second sub-vector sequence is input into a preset deviation monitoring model, and based on the output result of the deviation monitoring model, it is determined whether the belt has a deviation fault.

4. The method according to claim 3, characterized in that The operation of determining whether the belt has a deviation fault based on the output result of the deviation monitoring model includes: When determining the output result of the deviation monitoring model, comparing the magnitude relationship between each second sub-vector and a preset first deviation threshold; and When the output result of the deviation monitoring model indicates that a deviation fault exists, and the second sub-vector is greater than the first deviation threshold, it is determined that the belt has the deviation fault.

5. The method according to claim 4, characterized in that The operation of determining a control area for detecting the running state of the belt based on the belt area includes: Determine a conveying object area in each belt area corresponding to each image frame, and divide the conveying object area corresponding to each image frame into a plurality of image blocks, wherein the conveying object area includes an object conveyed by the belt; Based on a plurality of image blocks corresponding to rows or columns of the respective image frames, determining a plurality of sets of image blocks corresponding to the respective image frames, wherein the sizes of the respective image blocks are the same; Inputting a plurality of sets of image blocks corresponding to the respective image frames into a neural network model based on a self-attention mechanism, and determining the similarity between image blocks at corresponding positions of two adjacent image frames in the plurality of image frames; Determining a plurality of similarity matrices corresponding to the plurality of image frames based on similarities between image blocks at corresponding positions in two adjacent image frames; Determine target similarities greater than a preset similarity threshold in each similarity matrix respectively, and determine image blocks corresponding to the target similarities in each image frame; and Based on the plurality of image blocks in the respective image frames, a comparison area in the respective image frames is determined.

6. The method according to claim 4, characterized in that Also includes: Identifying the belt area boundaries in the belt area along the belt conveying direction, and determining any one of the belt area boundaries as a control boundary; Determining whether a comparison boundary between two adjacent image frames in each of the image frames is offset; as well as When the comparison boundaries between two adjacent image frames are offset and the offset is greater than a preset second offset threshold, it indicates that the belt has the offset fault.

7. The method according to claim 1, characterized in that Also includes: In the case where the belt has a running fault, determining a starting position of the belt having the running fault, wherein in the case where the belt has a running fault, determining the starting position of the belt having the running fault includes: Collecting vibration data corresponding to each position of the belt, and constructing a vibration data sequence based on the plurality of vibration data; as well as The vibration data sequence and the motion vector sequence are input into a preset fault position detection model, and the starting position of the belt fault is determined, wherein the fault position detection model includes a multi-layer perceptron model and a fully connected layer.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 7.

9. A coal conveyor belt operation status monitoring device, characterized in that: include: An image frame extraction module, used for acquiring a video image for describing the running state of the belt, and randomly extracting a plurality of image frames containing the belt area from the video image; A control area determination module, used for identifying a belt area in each image frame, and determining a control area for detecting a belt running state based on the belt area, wherein the control area corresponds to an object conveyed by the belt; a motion vector determination module, configured to determine a center point of the control area as a control point, and determine a motion vector of the control point between two adjacent image frames, thereby constructing a motion vector sequence based on a plurality of motion vectors corresponding to the plurality of image frames; as well as The operation fault monitoring module is used to input the motion vector sequence into a preset state monitoring model, and determine whether the belt has an operation fault based on an output result of the state monitoring model.

10. A coal conveyor belt operation status monitoring device, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Acquire a video image for describing the running state of the belt, and randomly extract a plurality of image frames containing the belt area from the video image; Identifying a belt area in each image frame, and determining a control area for detecting a running state of the belt based on the belt area, wherein the control area corresponds to an object conveyed by the belt; Determine the center point of the control area as the control point, and determine the motion vector of the control point between two adjacent image frames, so as to construct a motion vector sequence based on a plurality of motion vectors corresponding to the plurality of image frames; as well as The motion vector sequence is input into a preset condition monitoring model, and based on the output result of the condition monitoring model, it is determined whether the belt has an operation fault.

Images