Method and device for detecting state of conveyor belt of transport tool
By cleaning and adding historical video data of transport belts of transport tools, the convolutional layer model is trained to detect the transport belt status, solving the problem of misjudgment of the transport belt operating status detection, real-time and accurate status detection of the transport belt is realized, and improving the Underground operation safety and production efficiency.
Patent Information
- Application Number
- CN202110600892.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-31
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-05-31
AI Technical Summary
In coal mines, iron ore and other mines, the operation status detection of the transport belt of the transport tool is prone to misjudgment, resulting in emergency braking or safety threats of the transport tool, and cannot meet the requirements of safe and smart production.
A transport belt state detection method of a transport tool is adopted. By obtaining historical videos, data cleaning and label addition are cleaned, convolutional layer model is trained to extract feature data, and the operation status of the transport belt is determined using the softmax function to realize the state detection of real-time video.
Real-time and accurate status detection of the transport belt is realized, maintenance costs are reduced, safety protection of underground operators is enhanced, and safety is met in accordance with the requirements of safe and smart production.
Smart Images

Figure CN113269106B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing, and in particular to a method and device for detecting the state of a conveyor belt of a transport tool. Background Art
[0002] In coal mines, iron mines and other mines, conveyors and other transportation tools are used to transport the collected minerals and garbage in the mines out of the mines. Transportation tools are also used to transport tools and other items and equipment required by operators to the mines. In order to ensure safe production, it is necessary to detect the operating status of the conveyor belt of the transportation tool. However, due to the influence of factors such as the harsh production environment under the mine, the detection method of the operating status of the conveyor belt in the relevant technology is prone to misjudgment of the operating status of the conveyor belt. If a conveyor belt without faults is mistakenly determined to be faulty, the conveyor will be emergency braked, and the underground equipment maintenance personnel will be notified to repair the conveyor in time, which will affect the execution efficiency of the entire transportation system for the transportation task. If a conveyor belt with faults is mistakenly determined to be fault-free, it will pose a certain threat to the safety of underground workers. Both misjudgments cannot meet the requirements of safe and intelligent production. Therefore, how to detect the operating status of the conveyor belt of the transportation tool in real time and accurately is a technical problem that needs to be solved urgently. Summary of the invention
[0003] In view of this, the present disclosure proposes a method and device for detecting the status of a conveyor belt of a transportation tool.
[0004] According to one aspect of the present disclosure, a method for detecting a state of a conveyor belt of a transport tool is provided, the method comprising:
[0005] Get multiple historical videos of the conveyor belt;
[0006] Perform data cleaning on each historical video to obtain multiple target videos indicating the running status of the conveyor belt;
[0007] Adding a label for indicating the running state of the conveyor belt in the target video to each target video to obtain a training sample;
[0008] A preset conveyor belt state discrimination model is trained based on the training samples to obtain a conveyor belt state discrimination model, and the conveyor belt discrimination model is used to detect the running state of the conveyor belt in the real-time video.
[0009] In a possible implementation, the method further includes:
[0010] Acquire a real-time video of the conveyor belt, wherein the real-time video includes a real-time video stream;
[0011] The conveyor belt state discrimination model is used to detect the running state of the conveyor belt in the real-time video.
[0012] In a possible implementation, the preset conveyor belt state discrimination model includes a convolution layer and an activation layer, the convolution layer includes a three-dimensional convolution kernel, and the function used by the activation layer includes a softmax function.
[0013] The convolution layer uses a three-dimensional convolution kernel to extract features from each target video to obtain feature data of each target video, wherein the feature data includes spatial feature data and temporal feature data;
[0014] The activation layer determines the running status of the conveyor belt in each target video according to the feature data.
[0015] In a possible implementation, before adding a label for indicating the running status of the conveyor belt in each target video to each target video, the method further includes:
[0016] Segmenting the target videos whose duration is greater than the maximum value of the preset duration range among the multiple target videos so that the duration of each target video used to add the label is within the preset duration range; and / or
[0017] The target video whose duration is less than the minimum value of the preset duration range is deleted from the multiple target videos.
[0018] In a possible implementation manner, the running state of the conveyor belt in each target video includes a stationary state or a running state.
[0019] In a possible implementation, before training the preset conveyor belt state discrimination model based on the training samples, the method further includes:
[0020] Perform data enhancement processing on the training samples.
[0021] In a possible implementation, the step of training a preset conveyor belt state discrimination model based on the training samples to obtain the conveyor belt state discrimination model includes:
[0022] The conveyor belt state discrimination model is optimized by using a stochastic gradient descent method. When the learning rate of the preset conveyor belt state discrimination model is greater than or equal to the preset learning rate, the model training is terminated to obtain the conveyor belt state discrimination model.
[0023] According to another aspect of the present disclosure, a device for detecting a state of a conveyor belt of a transport tool is provided, the device comprising:
[0024] A video acquisition module, used to acquire multiple historical videos shot with the conveyor belt;
[0025] A data cleaning module performs data cleaning on each historical video to obtain multiple target videos indicating the running status of the conveyor belt;
[0026] A label adding module, adding a label for indicating the running state of the conveyor belt in the target video to each target video, to obtain a training sample;
[0027] The model training module trains a preset conveyor belt state discrimination model based on the training samples to obtain a conveyor belt state discrimination model, wherein the conveyor belt discrimination model is used to detect the running state of the conveyor belt in the real-time video.
[0028] According to another aspect of the present disclosure, a device for detecting a state of a conveyor belt of a transport tool is provided, the device comprising:
[0029] processor;
[0030] a memory for storing processor-executable instructions;
[0031] Wherein, the processor is configured to execute the above method.
[0032] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.
[0033] According to the state detection method and device of the conveyor belt of the transportation tool provided by the embodiment of the present disclosure, the operating state of the conveyor belt can be determined in real time and accurately based on the real-time video according to the trained conveyor belt state discrimination model, thereby reducing the maintenance cost of the conveyor belt, strengthening the safety protection of underground workers, and meeting the requirements of safe and intelligent production.
[0034] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0036] Figure 1 A flow chart showing a method for detecting the state of a conveyor belt of a transport tool according to an embodiment of the present disclosure.
[0037] Figure 2 Another flow chart of the method for detecting the state of a conveyor belt of a transport tool according to an embodiment of the present disclosure is shown.
[0038] Figure 3A block diagram showing a state detection device of a conveyor belt of a transport tool according to an embodiment of the present disclosure.
[0039] Figure 4 A block diagram of a device 800 for detecting a state of a conveyor belt of a transport means according to an embodiment of the present disclosure is shown.
[0040] Figure 5 A block diagram of a device 1900 for detecting a state of a conveyor belt of a transport tool according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0041] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0042] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0043] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.
[0044] In the related art, various sensors, such as speed sensors, current sensors, etc., are installed on the conveyor belt, and various operating parameters of the conveyor belt are measured in real time based on these sensors to analyze the operating status of the conveyor belt. However, in complex production environments, such as environments with excessive dust, high air humidity, or under the influence of factors such as sensor aging or fault shutdown, directly determining the operating status of the conveyor belt based on various operating parameters measured by various sensors installed on the conveyor belt is prone to problems such as misjudgment of the conveyor belt status, affecting the reliability of the entire transportation system in performing transportation tasks. Therefore, how to detect the operating status of the conveyor belt of a transportation tool in real time and accurately is a technical problem that needs to be solved urgently.
[0045] Based on this, the present disclosure provides a method and device for detecting the status of a conveyor belt of a transportation tool. Figure 1 A flow chart of a method for detecting the state of a conveyor belt of a transport tool according to an embodiment of the present disclosure is shown. The method can be applied to detect the running state of a conveyor belt during transportation of a transport tool in a mine or the like. When the transport tool is a belt conveyor, the conveyor belt can be a belt.
[0046] like Figure 1As shown, the method includes steps S11 to S14.
[0047] In step S11, a plurality of historical videos of the conveyor belt are acquired.
[0048] In step S12, data cleaning is performed on each historical video to obtain a plurality of target videos indicating the operating status of the conveyor belt.
[0049] By performing data cleaning on each historical video, the historical videos that are blurred, contain a lot of noise, and are highly repetitive can be removed, thereby obtaining multiple target videos that meet the data quality requirements and are used to indicate the operating status of the conveyor belt.
[0050] Among them, the method of data cleaning of historical videos may include one or more of the following methods, which are not limited in the present disclosure.
[0051] Method 1: retain one of the multiple identical videos in each historical video and delete the other identical videos.
[0052] Method 2: Delete the videos whose clarity is lower than the clarity threshold in each historical video.
[0053] The clarity threshold can be set according to whether the video can indicate the running state of the conveyor belt, so as to ensure that the obtained target video can detect the running state of the conveyor belt through the preset conveyor belt state discrimination model. In this way, when it is determined that there is a historical video with a clarity lower than the clarity threshold among multiple historical videos, it can be determined that the historical video with a clarity lower than the clarity threshold cannot indicate the running state of the conveyor belt, and the historical video with a clarity lower than the clarity threshold is deleted.
[0054] In step S13, a label indicating the running state of the conveyor belt in the target video is added to each target video to obtain a training sample, wherein the running state of the conveyor belt in each target video includes a stationary state or a running state.
[0055] Among them, a label for indicating the running status of the conveyor belt in each target video can be added to each target video by, for example, naming each target video, for example, a target video in which the running status of the conveyor belt in the target video is a stationary state is named "stop x", and a target video in which the running status of the conveyor belt in the target video is a running state is named "move x", and the present disclosure does not impose any restrictions on this.
[0056] By adding labels to each target video, the running status of the conveyor belt in each target video can be determined directly according to the label of each target video.
[0057] In a possible implementation, before step S13, the method may further include: segmenting a target video whose duration is greater than a maximum value of a preset duration range among the multiple target videos, so that the duration of each target video for adding a label is within the preset duration range; and / or
[0058] Delete the target videos whose duration is less than the minimum value of the preset duration range.
[0059] Among them, the preset duration range can be set according to the detection accuracy, the data processing capability of the device executing the method, etc. Among them, the higher the detection accuracy, the shorter the preset duration range can be set. The stronger the data processing capability of the device executing the method, the longer the preset duration range can be set. For example, the preset duration range can be 5-10 seconds, and the present disclosure does not limit this. As an example, when it is determined that the duration of a certain target video A is 13 seconds, the target video A can be divided into video segments A1 and A2 with durations of 5 seconds and 8 seconds respectively, and video segments A1 and A2 are used as new target videos. The target video A can also be divided into video segments A3 and A4 with durations of 6 seconds and 7 seconds respectively, and video segments A3 and A4 are used as new target videos. Or the target video A can also be divided into video segments A5 and A6 with durations of 10 seconds and 3 seconds respectively, and video segment A5 is used as a new target video, and video segment A6 is deleted. Those skilled in the art can set the implementation method of the segmentation process according to actual needs, and the present disclosure does not limit this, as long as the duration of each target video used for adding tags is within the preset duration range.
[0060] By processing target videos among multiple target videos whose duration is not within the preset duration range and ensuring that labels are added to target videos that meet the preset duration range, the efficiency of subsequent processing of each target video in the training samples can be improved, thereby improving the efficiency and speed of model training using training samples.
[0061] In a possible implementation, while processing the target videos whose duration is not within the preset duration range among the multiple target videos, the method may further include: performing standardization processing on the target videos whose duration is within the preset duration range to form video segments of a preset video size "X×Y×Z", and using each video segment obtained after the standardization processing as a new target video, so that each target video for adding a label is a video of X×Y×Z. Wherein, X represents the number of video frames, Y×Z represents the size of the video frame, Y is the width, and Z is the height, and the values of Y and Z may be the same or different, such as the preset video size may be X×Y×Y.
[0062] Among them, the preset video size "X×Y×Z" is pre-set, and X needs to refer to the preset duration range and the following preset conveyor belt state discrimination model to set the size limit of the input target video with labels, so that each target video with a duration within the preset duration range can form an integer number of "X×Y×Z" video clips after standardized processing. X and Y can be set based on the video frame size of different historical videos themselves, the following preset conveyor belt state discrimination model to set the size limit of the input target video with labels, etc. Those skilled in the art can set the preset video size "X×Y×Z" according to actual needs, and the present disclosure does not limit this.
[0063] Among them, the standardization process may include segmentation of the target video and / or video frame size adjustment. For example, assume that the preset video size is "X×Y×Y". If the total number of video frames Q of the target video 1 whose duration has been obtained and is within the preset duration range is greater than X, and the video frame size is consistent with Y×Y, then the target video 1 needs to be segmented, and the target video 1 is segmented into Q / X video segments, and Q / X video segments are used as new target videos. If the total number of video frames Q of the target video 2 whose duration has been obtained and is within the preset duration range is equal to X, and the video frame size is inconsistent with Y×Y, then the video frame size of the target video 2 needs to be adjusted, and the target video 2 may be scaled, cropped, and / or data supplemented, so that the video frame size of the target video 2 is adjusted from y×z to Y×Z. Exemplarily, the preset video size "X×Y×Z" may be 128×64×64.
[0064] In this way, the speed of subsequent model training can be further improved after standardization.
[0065] In step S14, a preset conveyor belt state discrimination model is trained based on the training samples to obtain a conveyor belt state discrimination model, and the conveyor belt discrimination model is used to detect the running state of the conveyor belt in the real-time video.
[0066] According to the state detection method of the conveyor belt of the transportation tool provided by the embodiment of the present disclosure, the operating state of the conveyor belt can be determined in real time and accurately based on the real-time video according to the trained conveyor belt state discrimination model, thereby reducing the maintenance cost of the conveyor belt, strengthening the safety protection of underground workers, and meeting the requirements of safe and intelligent production.
[0067] In a possible implementation, the preset conveyor belt state discrimination model may include a convolution layer and an activation layer. The convolution layer includes a three-dimensional convolution kernel, and the convolution layer may use the three-dimensional convolution kernel to extract features of each target video to obtain feature data of each target video. The feature data of each target video may include spatial feature data of each target video and temporal feature data of each target video. The function used by the activation layer includes a softmax function, and the activation layer may determine the running state of the conveyor belt in each target video according to the feature data of each target video. The three-dimensional convolution kernel may be a 3*3*3 convolution kernel.
[0068] Among them, by cutting each target video in the training sample into X×Y×Y segments, it is beneficial to better extract features of each target video according to the convolution layer in the preset conveyor belt state discrimination model.
[0069] Wherein, after the convolution layer, the preset conveyor belt state discrimination model may further include a pooling layer for changing the size of each video frame in each target video. After the pooling layer and before the activation layer, the preset conveyor belt state discrimination model may further include a fully connected layer for connecting the feature data of each target video.
[0070] In a possible implementation, step S14 may include: optimizing the conveyor belt state discrimination model using a stochastic gradient descent method, and when a learning rate of a preset conveyor belt discrimination model is greater than or equal to a preset learning rate, ending model training to obtain a conveyor belt state discrimination model.
[0071] For example, the initial learning rate of the preset conveyor belt state discrimination model can be set to 0.0001, the dropout can be set to 0.5, the relu function can be used for activation, and the preset learning rate can be set to 0.01, and the present disclosure does not impose any restrictions on this.
[0072] By optimizing the conveyor belt state discrimination model, the accuracy of detecting the running state of the conveyor belt in the real-time video according to the obtained conveyor belt state discrimination model can be improved.
[0073] Before training the preset conveyor belt state model based on the training samples, the method may further include: dividing the training samples into a training set, a validation set, and a test set according to a preset ratio, wherein the preset ratio may be 7:2:1, which is not limited in the present disclosure.
[0074] The preset conveyor belt state model can be trained based on the training set. When the learning rate of the model is greater than or equal to the preset learning rate, the model training based on the training set is terminated to obtain the first conveyor belt state model. The first conveyor belt state model is preliminarily evaluated using each target video in the validation set and the hyperparameters of the first conveyor belt state model are adjusted to obtain the conveyor belt state model. Finally, the generalization ability of the conveyor belt state model is evaluated using the test set.
[0075] By dividing the training samples into a training set, a validation set and a test set according to a preset ratio, and training the preset conveyor belt state discrimination model, the accuracy of the conveyor belt state discrimination model in detecting the running state of the conveyor belt can be improved.
[0076] In a possible implementation, before step S14, the method may further include: performing data enhancement processing on the training samples. The data enhancement processing on the training samples may be performed by horizontal flipping, vertical flipping, angle rotation, etc., which is not limited in the present disclosure.
[0077] By performing data enhancement processing on the training samples, the training samples can be expanded, the jitter of the training samples in space and time can be overcome, and the robustness and generalization of the conveyor belt state discrimination model obtained by training the preset conveyor belt discrimination model based on the training samples can be improved.
[0078] According to the state detection method of the conveyor belt of the transportation tool provided by the embodiment of the present disclosure, the operating state of the conveyor belt in the real-time video can be accurately detected based on the trained conveyor belt state discrimination model, thereby reducing the number of underground workers and meeting the safety production requirements.
[0079] Figure 2 Another flow chart of the method for detecting the state of a conveyor belt of a conveyor according to an embodiment of the present disclosure is shown. Figure 2 As shown, after executing step S14, the method may further include steps S15 to S16.
[0080] In step S15, a real-time video of the conveyor belt is obtained, and the real-time video includes a real-time video stream. Transmitting the real-time video in the form of a video stream can increase the transmission speed of the real-time video, increase the speed of determining the running state of the conveyor belt in the real-time video, realize real-time and accurate determination of the running state of the conveyor belt, and can respond and handle in time when there is a problem with the conveyor belt.
[0081] The real-time video may be obtained based on an image acquisition device pre-installed on a mine tunnel and / or a transportation vehicle. The installation position and number of the image acquisition device on the mine tunnel and / or the transportation vehicle may be selected based on information such as the mine tunnel structure, the number of conveyor belts in the transportation vehicle, and the relative position between the mine tunnel and the conveyor belt, and the image acquisition device may be adjusted after installation, such as adjusting the shooting angle, pixel, and exposure of the captured image of the image acquisition device, so that the real-time video of the conveyor belt captured by the image acquisition device may meet the preset requirements. The image acquisition device may include a camera, a fill light, etc. The preset requirements may include one or more of the following: the conveyor belt is located at the center of the video screen of the captured real-time video, the video screen of the captured real-time video includes only one target conveyor belt, the video frame size of the captured real-time video is consistent with Y and Z in the above-mentioned preset video size "X×Y×Z", etc.
[0082] In a possible implementation, step S15 may further include: after acquiring the real-time video, the video size of the real-time video may be determined. When it is determined that the video size of the real-time video is inconsistent with the above-mentioned preset video size "X×Y×Z", the real-time video is standardized so that the video size of the real-time video after the standardization is the above-mentioned preset video size "X×Y×Z", and then step S16 is executed. When it is determined that the video size of the real-time video is consistent with the above-mentioned preset video size "X×Y×Z", step S16 is directly executed.
[0083] In one possible implementation, when it is determined that the clarity of multiple videos of a conveyor belt acquired by an image acquisition device within a certain time period is lower than a clarity threshold, the internal parameters of the image acquisition device can also be adjusted to improve the clarity of the real-time video captured of the conveyor belt.
[0084] In step S16, the conveyor belt state discrimination model is used to detect the running state of the conveyor belt in the real-time video. That is, the real-time video is input into the conveyor belt state discrimination model to obtain the running state of the conveyor belt in the real-time video.
[0085] According to the state detection method of the conveyor belt of the transport tool provided by the embodiment of the present disclosure, by inputting the real-time video of the conveyor belt into the conveyor belt state judgment model, the transport state of the conveyor belt in the real-time video can be directly obtained. Compared with the method of using sensors to detect the transport state of the conveyor belt in the prior art, it is simpler, more efficient and accurate, thereby improving the safety of equipment maintenance personnel.
[0086] Figure 3 A block diagram of a device for detecting the state of a conveyor belt of a conveyor according to an embodiment of the present disclosure is shown. Figure 3As shown, the device includes a video acquisition module 11, a data cleaning module 12, a label adding module 13 and a model training module 14.
[0087] The video acquisition module 11 is used to acquire a plurality of historical videos shot with the conveyor belt.
[0088] The data cleaning module 12 performs data cleaning on each historical video to obtain a plurality of target videos indicating the running status of the conveyor belt.
[0089] The label adding module 13 adds a label for indicating the running status of the conveyor belt in the target video to each target video to obtain a training sample.
[0090] The model training module 14 trains a preset conveyor belt state discrimination model based on the training samples to obtain a conveyor belt state discrimination model, wherein the conveyor belt discrimination model is used to detect the running state of the conveyor belt in the real-time video.
[0091] According to the state detection device of the conveyor belt of the transportation tool provided by the embodiment of the present disclosure, the operating state of the conveyor belt can be determined in real time and accurately based on the real-time video according to the trained conveyor belt state discrimination model, thereby reducing the maintenance cost of the conveyor belt, strengthening the safety protection of underground workers, and meeting the requirements of safe and intelligent production.
[0092] In a possible implementation, the device may further include: a real-time video acquisition module and a detection module. The real-time video acquisition module is used to acquire a real-time video of the conveyor belt, wherein the real-time video includes a real-time video stream. The detection module is used to detect the running state of the conveyor belt in the real-time video using the conveyor belt state discrimination model.
[0093] In a possible implementation, the preset conveyor belt state discrimination model includes a convolution layer and an activation layer, the convolution layer includes a three-dimensional convolution kernel, and the function used by the activation layer includes a softmax function.
[0094] The convolution layer uses a three-dimensional convolution kernel to extract features from each target video to obtain feature data of each target video, wherein the feature data includes spatial feature data and temporal feature data;
[0095] The activation layer determines the running status of the conveyor belt in each target video according to the feature data.
[0096] In a possible implementation, the label adding module is further used to segment the target videos whose duration is greater than the maximum value of the preset duration range among the multiple target videos before adding the label for indicating the running status of the conveyor belt in the target video to each target video, so that the duration of each target video for adding the label is within the preset duration range; and / or
[0097] The target video whose duration is less than the minimum value of the preset duration range is deleted from the multiple target videos.
[0098] In a possible implementation manner, the running state of the conveyor belt in each target video includes a stationary state or a running state.
[0099] In a possible implementation, the model training module is further used to perform data enhancement processing on the training samples before training the preset conveyor belt state discrimination model based on the training samples.
[0100] In a possible implementation, the model training module includes a training sub-module, which is used to optimize the conveyor belt state discrimination model using a stochastic gradient descent method. When the learning rate of the preset conveyor belt state discrimination model is greater than or equal to the preset learning rate, the model training is terminated to obtain the conveyor belt state discrimination model.
[0101] The embodiment of the present disclosure further provides a device for detecting the state of a conveyor belt of a transport tool, the device comprising:
[0102] processor;
[0103] a memory for storing processor-executable instructions;
[0104] Wherein, the processor is configured to execute the state detection method of the conveyor belt of the above-mentioned transport tool.
[0105] The embodiment of the present disclosure further provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the state detection method of the conveyor belt of the above-mentioned conveyance tool.
[0106] Figure 4 A block diagram of a device 800 for detecting the state of a conveyor belt of a conveyance means according to an embodiment of the present disclosure is shown. For example, the device 800 may be a mobile phone, a computer, a messaging device, a tablet device, etc.
[0107] Reference Figure 4, the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .
[0108] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
[0109] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0110] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.
[0111] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0112] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
[0113] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.
[0114] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0115] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0116] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0117] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by the processor 820 of the device 800 to perform the above method.
[0118] Figure 5 FIG. 1 is a block diagram showing a device 1900 for detecting the state of a conveyor belt of a conveyance means according to an embodiment of the present disclosure. For example, the device 1900 may be provided as a server. Figure 5 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.
[0119] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output (I / O) interface 1958. The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.
[0120] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.
[0121] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0122] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0123] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0124] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0125] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0126] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0127] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0128] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0129] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for detecting the state of a conveyor belt of a transport vehicle, It is characterized in that include: Get multiple historical videos of the conveyor belt; Perform data cleaning on each historical video to obtain multiple target videos indicating the running status of the conveyor belt; Adding a label for indicating the running state of the conveyor belt in the target video to each target video to obtain a training sample; Based on the training samples, a preset conveyor belt state discrimination model is trained to obtain a conveyor belt state discrimination model, wherein the conveyor belt discrimination model is used to detect the running state of the conveyor belt in the real-time video; The preset conveyor belt state discrimination model includes a convolution layer and an activation layer. The convolution layer includes a three-dimensional convolution kernel. The function used by the activation layer includes a softmax function. The convolution layer uses a three-dimensional convolution kernel to extract features from each target video to obtain feature data of each target video, wherein the feature data includes spatial feature data and temporal feature data; The activation layer determines the running status of the conveyor belt in each target video according to the feature data.
2. The method according to claim 1, It is characterized in that The method further comprises: Acquire a real-time video of the conveyor belt, wherein the real-time video includes a real-time video stream; The conveyor belt state discrimination model is used to detect the running state of the conveyor belt in the real-time video.
3. The method according to claim 1, It is characterized in that Before adding a label for indicating the running state of the conveyor belt in each target video to each target video, the method further includes: Segmenting the target videos whose duration is greater than the maximum value of the preset duration range among the multiple target videos so that the duration of each target video used to add the label is within the preset duration range; and / or The target video whose duration is less than the minimum value of the preset duration range is deleted from the multiple target videos.
4. The method according to any one of claims 1 to 3, It is characterized in that The running state of the conveyor belt in each target video includes a stationary state or a running state.
5. The method according to claim 1, It is characterized in that Before training the preset conveyor belt state discrimination model based on the training samples, the method further includes: Perform data enhancement processing on the training samples.
6. The method according to claim 1, It is characterized in that The step of training a preset conveyor belt state discrimination model based on the training samples to obtain a conveyor belt state discrimination model includes: The conveyor belt state discrimination model is optimized by using a stochastic gradient descent method. When the learning rate of the preset conveyor belt state discrimination model is greater than or equal to the preset learning rate, the model training is terminated to obtain the conveyor belt state discrimination model.
7. A device for detecting the state of a conveyor belt of a transport vehicle, It is characterized in that The device comprises: A video acquisition module, used to acquire multiple historical videos shot with the conveyor belt; A data cleaning module performs data cleaning on each historical video to obtain multiple target videos indicating the running status of the conveyor belt; A label adding module, adding a label for indicating the running state of the conveyor belt in the target video to each target video, to obtain a training sample; A model training module is used to train a preset conveyor belt state discrimination model based on the training samples to obtain a conveyor belt state discrimination model, wherein the conveyor belt discrimination model is used to detect the running state of the conveyor belt in the real-time video; The preset conveyor belt state discrimination model includes a convolution layer and an activation layer. The convolution layer includes a three-dimensional convolution kernel. The function used by the activation layer includes a softmax function. The convolution layer uses a three-dimensional convolution kernel to extract features from each target video to obtain feature data of each target video, wherein the feature data includes spatial feature data and temporal feature data; The activation layer determines the running status of the conveyor belt in each target video according to the feature data.
8. A device for detecting the state of a conveyor belt of a transport vehicle, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 6.
9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Detection system for material on conveying belt based on deep learning
CN108764023A
Belt running state abnormity monitoring method, system and device, and storage medium
CN110490995A
Method, device and system for monitoring running state of belt and storage medium
CN110619625A
