A computer image-based ore transportation monitoring method and system

Abstract

The present invention provides a computer image-based ore conveying monitoring method and system, which relates to the field of image processing technology. The method includes: setting a first camera above the conveyor belt and setting a second camera on the side of the conveyor belt; obtaining a first video frame sequence and a second video frame sequence, and obtaining ore stacking feature information and belt status feature information through the feature extraction layer of the abnormality recognition model; obtaining stacking status information based on the ore stacking feature information; obtaining conveying status information based on the ore stacking feature information and the belt status feature information; and determining conveying smoothness information based on the stacking status information and the conveying status information. According to the present invention, the first video and the second video can be processed in combination with the conveying abnormality recognition model to automatically determine whether there is an abnormality in the ore stacking and whether there is an abnormality in the load of the conveyor belt, thereby accurately determining whether the ore conveying process is smooth.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a computer image-based ore transportation monitoring method and system. Background Art

[0002] In related technologies, ore is transported to a designated location via a conveyor belt, and image processing technology can be used to monitor whether the conveyor belt is deviating or whether there are foreign objects on the conveyor belt, thereby determining whether there are any abnormalities in the conveying process of the conveyor belt. However, in addition to the above abnormal factors, the stacking of ore on the conveyor belt may also cause abnormalities in the conveying process. For example, a small stacking volume leads to low conveying efficiency, or an excessive stacking volume leads to a heavy load on the conveyor belt, which can easily cause deformation of the conveyor belt. However, related technologies make it difficult to monitor the stacking of ore on the conveyor belt and the load of the conveyor belt itself. Summary of the Invention

[0003] The present invention provides a computer image-based ore conveying monitoring method and system, which can solve the technical problem that related technologies are difficult to monitor the stacking condition of ore on a conveyor belt and the load condition of the conveyor belt itself.

[0004] According to a first aspect of the present invention, there is provided a computer image-based ore conveyance monitoring method, comprising:

[0005] A first camera is set above the conveyor belt, and a second camera is set on the side of the conveyor belt. The moving direction of the conveyor belt is the X-axis direction, the width direction of the conveyor belt is the Y-axis direction, and the direction perpendicular to the plane where the X-axis and Y-axis are located is the Z-axis direction. The X-axis coordinates of the first camera and the second camera are consistent;

[0006] In a current monitoring cycle, a first video is captured by a first camera, a second video is captured by a second camera, and a first video frame sequence of the first video and a second video frame sequence of the second video are acquired;

[0007] The i-th first video frame in the first video frame sequence and the i-th second video frame in the second video frame sequence are combined into a video frame group, and the video frame group is input into the feature extraction layer of the conveying abnormality recognition model to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame respectively;

[0008] Obtaining stacking status information based on ore stacking feature information corresponding to the plurality of first video frames;

[0009] Obtaining conveying state information based on ore stacking feature information corresponding to the plurality of first video frames and belt state feature information corresponding to the plurality of second video frames;

[0010] The conveying stability information of the current monitoring period is determined according to the stacking state information and the conveying state information.

[0011] According to the present invention, obtaining stacking status information based on ore stacking feature information corresponding to a plurality of first video frames includes:

[0012] Processing the ore stacking feature information corresponding to each first video frame through an edge enhancement layer to obtain ore edge enhancement feature information;

[0013] Obtain enhanced difference feature information between the ore edge enhancement feature information corresponding to the (i+1)th first video frame and the ore edge enhancement feature information corresponding to the (i)th first video frame;

[0014] Aggregating the plurality of enhanced difference feature information to obtain first aggregated feature information;

[0015] The first aggregated feature information is input into the first state recognition level to obtain stacking state information.

[0016] According to the present invention, the enhanced difference feature information includes feature information of multiple feature channels;

[0017] The step of aggregating the plurality of enhanced difference feature information to obtain first aggregated feature information includes:

[0018] Aggregating enhanced difference feature information between multiple adjacent first video frames having the same feature channel to obtain aggregated difference feature information corresponding to the feature channel;

[0019] First aggregated feature information is obtained according to the aggregated difference feature information corresponding to the multiple feature channels.

[0020] According to the present invention, the enhanced difference feature information between a plurality of adjacent first video frames having the same feature channel is aggregated to obtain aggregated difference feature information corresponding to the feature channel, including:

[0021] According to the formula

[0022]

[0023] The coordinates of the aggregated difference feature information corresponding to the kth feature channel are obtained as The brightness value of the pixel ,in, The coordinates of the enhanced difference feature information between the i-th first video frame and the i+1-th first video frame of the k-th feature channel are The brightness value of the pixel, n is the number of the first video frame, is the preset brightness value.

[0024] According to the present invention, conveying state information is obtained based on ore stacking feature information corresponding to a plurality of first video frames and belt state feature information corresponding to a plurality of second video frames, including:

[0025] Inputting the ore stacking feature information corresponding to the first video frame into the second state recognition level to obtain stacking risk information;

[0026] Inputting belt state feature information corresponding to a second video frame in the same group as the first video frame into a third state recognition layer to obtain belt state risk information;

[0027] Input the stacking risk information and belt status risk information into the risk identification level to obtain the conveying risk information;

[0028] The delivery risk information corresponding to the plurality of video frame groups is input into the delivery status recognition layer to obtain delivery status information.

[0029] According to the present invention, the training step of the transport anomaly recognition model includes:

[0030] Acquire a first training video frame sequence captured by a first camera and a second training video frame sequence captured by a second camera during a training cycle;

[0031] Acquire, through a feature extraction layer, training ore stacking feature information of each first training video frame in the first training video frame sequence, and training belt state feature information of each second training video frame in the second training video frame sequence;

[0032] Processing the training ore stacking feature information through the edge enhancement level to obtain training edge enhancement feature information;

[0033] Obtaining, through a first decoding level, first training edge information in a first training video frame;

[0034] Obtaining an edge loss function according to the first training edge information and the labeling information of the first training video;

[0035] Obtaining multiple pieces of training enhanced difference feature information based on the multiple pieces of training edge enhanced feature information, and aggregating the multiple pieces of training enhanced difference feature information to obtain training aggregated feature information;

[0036] Inputting the training aggregate feature information into the first state recognition layer to obtain training stacking state information;

[0037] Input the training stacking status information into the second fully connected layer to obtain the training stacking risk probability distribution information;

[0038] Obtaining a stacking risk loss function based on the training stacking risk probability distribution information and the labeling information of the first training video;

[0039] Input the training ore stacking feature information corresponding to the first training video frame into the second state recognition level to obtain training stacking risk information, and input the training belt state feature information corresponding to the second training video frame with the same sequence number as the first training video frame into the third state recognition level to obtain training belt state risk information;

[0040] Input the training stacking risk information and the training belt status risk information into the risk identification layer to obtain the training conveying risk information;

[0041] Input the training transport risk information into the third fully connected layer to obtain the transport risk probability distribution information;

[0042] Obtaining a transportation risk loss function based on the transportation risk probability distribution information and the annotation information of the second training video;

[0043] Inputting a plurality of training transport risk information into the transport status identification layer to obtain training transport status information;

[0044] Input the training delivery state information into the fourth fully connected layer to obtain the delivery state risk probability distribution information;

[0045] Obtaining a transportation state risk loss function based on the transportation state risk probability distribution information and the annotation information of the second training video;

[0046] The training stacking state information and the training conveying state information are combined to determine the training stability state information;

[0047] Input the training stationarity state information into the first fully connected layer to obtain the training stationarity result;

[0048] Obtaining a stationarity loss function based on the labeling information of the second training video and the training stationarity result;

[0049] The transport anomaly recognition model is trained according to the edge loss function, the stacking risk loss function, the transport risk loss function, the transport state risk loss function and the stationarity loss function to obtain a trained transport anomaly recognition model.

[0050] According to the present invention, a transport anomaly recognition model is trained based on the edge loss function, the stacking risk loss function, the transport risk loss function, the transport state risk loss function, and the stationarity loss function to obtain a trained transport anomaly recognition model, including:

[0051] According to the formula

[0052]

[0053] Obtain the loss function of the transport anomaly recognition model ,in, is the marginal loss function, is the transmission risk loss function corresponding to the first training video frame and the second training video frame with sequence number i, n is the number of first video frames, is the stacking risk loss function, is the transport state risk loss function, is the stationary loss function, and max(*) is the maximum value function;

[0054] The transport anomaly recognition model is trained using the loss function of the transport anomaly recognition model to obtain a trained transport anomaly recognition model.

[0055] According to a second aspect of the present invention, there is provided a computer image-based ore conveyance monitoring system comprising:

[0056] A setting module is used to set a first camera above the conveyor belt and set a second camera on the side of the conveyor belt, wherein the movement direction of the conveyor belt is the X-axis direction, the width direction of the conveyor belt is the Y-axis direction, and the direction perpendicular to the plane where the X-axis and Y-axis are located is the Z-axis direction, and the X-axis coordinates of the first camera and the second camera are consistent;

[0057] a shooting module, configured to shoot a first video through a first camera and a second video through a second camera in a current monitoring period, and obtain a first video frame sequence of the first video and a second video frame sequence of the second video;

[0058] an extraction module, configured to combine the i-th first video frame in the first video frame sequence and the i-th second video frame in the second video frame sequence into a video frame group, and input the video frame group into a feature extraction layer of the conveying anomaly recognition model to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame;

[0059] A stacking status information module is used to obtain stacking status information based on ore stacking feature information corresponding to the plurality of first video frames;

[0060] A conveying state information module is used to obtain conveying state information based on ore stacking feature information corresponding to the plurality of first video frames and belt state feature information corresponding to the plurality of second video frames;

[0061] The conveying stability information module is used to determine the conveying stability information of the current monitoring period according to the stacking state information and the conveying state information.

[0062] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0063] According to the present invention, a first camera captures a first video of the conveyor belt from above to determine the stacking of ore on the conveyor belt. A second camera captures a second video of the conveyor belt from the side to determine the deformation of the conveyor belt itself, and thus the load on the conveyor belt. By processing the first and second videos in conjunction with a conveyor anomaly recognition model, it is possible to automatically determine whether there are any anomalies in the ore stacking and the conveyor belt load, thereby accurately determining whether the ore conveying process is smooth. When determining the aggregated difference feature information, the pixel points whose brightness values are not 0 after the ore edge enhancement feature information is subtracted can be highlighted by taking the absolute value of the brightness value and then averaging it, thereby highlighting the pixel points representing uneven stacking. Moreover, since the brightness value at the edge position in the ore edge enhancement feature information is enhanced, the brightness value at the edge position is higher than the brightness value at other positions. Therefore, the pixel value at the edge position in the aggregated difference feature information can be strengthened by binarization, so that the pixel points at the edge position that are not 0 are used to represent the unevenness of the ore stacking. Furthermore, the pixel points representing the uneven ore stacking at the edge positions in multiple ore edge enhancement feature information can be aggregated on the same feature map to express the unevenness of the edge of the ore pile based on the feature map. When determining the conveying status information, the top view and side view of the conveyor belt at the same moment can be recorded based on the first video frame and the second video frame of the same group, respectively. The ore stacking feature information of the first video frame and the belt status feature information of the second video frame of the same group can be combined to jointly determine whether the conveying process is at risk at that moment. Then, the conveying risk information at multiple moments can be combined to determine whether the conveying process is at risk during the current monitoring period, thereby improving the comprehensiveness and accuracy of process monitoring. When training the conveying anomaly recognition model, auxiliary training can be performed through the first decoding layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer, so that different layers in the conveying anomaly recognition model can be trained separately through multiple paths. When training the entire conveying anomaly recognition model through the stationarity loss function, the coefficient of the stationarity loss function is set through the stacking risk loss function and the conveying status risk loss function, thereby strengthening the process of training some layers of the conveying anomaly recognition model through one of the stacking risk loss function and the conveying status risk loss function, quickly correcting the layers with misjudgment, improving training efficiency, and quickly improving the accuracy of each layer of the conveying anomaly recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram exemplarily illustrates a flow chart of a method for monitoring ore transportation based on computer images according to an embodiment of the present invention;

[0065] Figure 2 and Figure 3 A schematic diagram exemplarily illustrating the arrangement positions of a first camera and a second camera according to an embodiment of the present invention;

[0066] Figure 4 A schematic diagram exemplarily illustrates a transport abnormality recognition model according to an embodiment of the present invention;

[0067] Figure 5 A block diagram of an ore conveyance monitoring system based on computer images according to an embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0069] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0070] Figure 1 A schematic flow chart of a computer image-based ore transportation monitoring method according to an embodiment of the present invention is exemplarily shown. The method includes:

[0071] Step S1: a first camera is set above the conveyor belt, and a second camera is set on the side of the conveyor belt, wherein the movement direction of the conveyor belt is the X-axis direction, the width direction of the conveyor belt is the Y-axis direction, and the direction perpendicular to the plane where the X-axis and Y-axis are located is the Z-axis direction. The X-axis coordinates of the first camera and the second camera are consistent;

[0072] Step S2: In the current monitoring period, a first video is captured by a first camera, and a second video is captured by a second camera, and a first video frame sequence of the first video and a second video frame sequence of the second video are obtained;

[0073] Step S3, combining the i-th first video frame in the first video frame sequence and the i-th second video frame in the second video frame sequence into a video frame group, and inputting the video frame group into a feature extraction layer of a conveying anomaly recognition model to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame respectively;

[0074] Step S4, obtaining stacking status information based on the ore stacking feature information corresponding to the plurality of first video frames;

[0075] Step S5, obtaining conveying state information based on the ore stacking feature information corresponding to the plurality of first video frames and the belt state feature information corresponding to the plurality of second video frames;

[0076] Step S6: Determine the conveying stability information of the current monitoring period according to the stacking state information and the conveying state information.

[0077] According to an embodiment of the present invention, a computer image-based ore conveyor monitoring method can use a first camera to capture a first video of the conveyor belt from above to determine the ore stacking condition on the conveyor belt. A second camera can also capture a second video of the conveyor belt from the side to determine the deformation of the conveyor belt itself, and thus the conveyor belt load. By processing the first and second videos in conjunction with a conveyor anomaly recognition model, it is possible to automatically determine whether there are any anomalies in the ore stacking and the conveyor belt load, thereby accurately determining whether the ore conveying process is smooth.

[0078] Figure 2 and Figure 3 A schematic diagram exemplarily illustrates the arrangement positions of a first camera and a second camera according to an embodiment of the present invention.

[0079] According to one embodiment of the present invention, in step S1 , a first camera may be arranged above the conveyor belt, and a second camera may be arranged at the side of the conveyor belt. Figure 2 This is a schematic diagram of the installation positions of the first camera and the second camera from the side view of the conveyor belt. Figure 3 This diagram illustrates the placement of the first and second cameras from a top-down perspective. The first camera's lens faces the top surface of the conveyor belt, with its optical axis pointing toward the centerline of the conveyor belt's top surface. A second camera is positioned to the side of the conveyor belt. The first and second cameras have the same X-axis coordinates, and the second camera's Z-axis coordinate is the same as the Z-axis coordinate of the conveyor belt's top surface. The pulley drives the conveyor belt, conveying ore. The ore can be placed on the conveyor belt, and the first camera captures the ore stacking from a top-down perspective, allowing for subsequent processing to determine if the ore stacking is normal. The weight of the ore can load the conveyor belt, causing deformation. Specifically, the Z-axis coordinate of the top surface of the conveyor belt, where ore is piled, will not be equal to the Z-axis coordinate of the second camera. The second camera captures the deformation from the side, allowing for subsequent processing to determine if the belt load is normal.

[0080] According to one embodiment of the present invention, in step S2, the duration of the monitoring period can be preset, for example, 10 seconds, 30 seconds, one minute, etc., or can be set based on the conveying speed and length of the conveyor belt. For example, if the conveyor belt is 10 meters long and the conveying speed is 0.5 meters per second, the duration of the monitoring period is 10 meters / (0.5 meters per second) = 20 seconds, i.e., the duration required to transport a piece of ore from one end of the conveyor belt to the other end. The present invention does not impose any limitation on the specific value of the duration of the monitoring period.

[0081] According to one embodiment of the present invention, a first video is captured by a first camera and parsed to obtain a first video frame sequence. That is, the first video frame sequence may include all video frames in the first video. In another example, the video frames in the first video may be uniformly sampled, for example, once per second, to obtain a first video frame sequence. That is, the first video frame sequence includes some video frames in the first video. Similarly, a second video frame sequence of a second video may be obtained.

[0082] According to one embodiment of the present invention, in step S3, the conveying anomaly recognition model is a deep learning neural network model, such as a convolutional neural network model. The conveying anomaly recognition model may include multiple levels. For example, the feature extraction level may be used to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame. The i-th first video frame and the i-th second video frame have the same sequence number and are shot at the same time. That is, the i-th first video frame and the i-th second video frame respectively record the top view and side view of the conveyor belt at the same time. The ore stacking feature information of the i-th first video frame and the belt status feature information of the i-th second video frame are used to reflect the ore stacking status and the deformation status of the belt at the same time.

[0083] Figure 4 A schematic diagram exemplarily shows a transport abnormality recognition model according to an embodiment of the present invention.

[0084] According to one embodiment of the present invention, the first video frame sequence includes a plurality of first video frames, and the second video frame sequence includes a plurality of second video frames. The first video frames and the second video frames with the same sequence number can be combined into a video frame group. The first video frame in the video frame group is input into the feature extraction layer to obtain ore stacking feature information, and the second video frame in the video frame group is input into the feature extraction layer to obtain belt status feature information. The structure of the feature extraction layer for receiving the first video frame and the second video frame is the same, but the model parameters may be different. For example, the feature extraction layer for receiving the first video frame and the second video frame can include layers such as convolution layers, activation layers, and pooling layers.

[0085] According to one embodiment of the present invention, in step S4, the ore stacking feature information corresponding to multiple first video frames can be processed by other levels of the conveying abnormality recognition model to obtain stacking status information, which is used to describe whether there is any abnormality in the stacking of ore on the conveyor belt during the current monitoring period, for example, whether there is a situation where too much stacking causes the conveyor belt to be heavily loaded and prone to spillage, or whether there is a situation where too little stacking causes low conveying efficiency.

[0086] According to one embodiment of the present invention, in step S4, stacking status information is obtained based on the ore stacking feature information corresponding to multiple first video frames, including: processing the ore stacking feature information corresponding to each first video frame through the edge enhancement level to obtain ore edge enhancement feature information; obtaining enhanced difference feature information between the ore edge enhancement feature information corresponding to the i+1th first video frame and the ore edge enhancement feature information corresponding to the i-th first video frame; aggregating multiple enhanced difference feature information to obtain first aggregated feature information; and inputting the first aggregated feature information into the first state recognition level to obtain stacking status information.

[0087] According to one embodiment of the present invention, the ore pile feature information may include feature maps for multiple feature channels. Each feature map can serve as a perspective for observing the first video frame. Each feature map has a different observation focus. For example, in one feature map, certain areas may have higher brightness values, while in another feature map, the brightness values at the same locations may be lower. Therefore, the presentation of each feature map is different, and the resolution of each feature map is lower than that of the first video frame. However, the field of view of each feature map is the same as that of the first video frame, and the content within the image is the ore on the conveyor belt. The edge of the ore pile can be enhanced using an edge enhancement layer, increasing the difference between pixels on the edge and other pixels, thereby making the edge more prominent. For example, the edge enhancement layer can be a functional module that executes an edge detection algorithm such as the Canny algorithm and increases the brightness of the detected edges by a preset ratio. It can process the ore pile feature information from multiple feature channels in each first video frame to obtain edge-enhanced ore edge feature information. The ore edge enhancement feature information has the same number of feature channels as the ore pile feature information, and the resolution remains unchanged.

[0088] According to one embodiment of the present invention, the ore edge enhancement feature information corresponding to the (i+1)th first video frame can be subtracted from the ore edge enhancement feature information corresponding to the (i)th first video frame. During the subtraction, the ore edge enhancement feature information of the same feature channel is subtracted to obtain enhanced difference feature information for each feature channel. If the ore is deposited uniformly, then the brightness value of each pixel in the enhanced difference feature information obtained after subtracting the ore edge enhancement feature information at different times in the same feature channel is all zero. However, if the ore is deposited unevenly, then the ore edge enhancement feature information at different times in the same feature channel may contain pixels with non-zero brightness values. Furthermore, the uneven edge positions will increase the brightness values of the pixels at corresponding positions in the enhanced difference feature information at different times in the edge enhancement feature information. In other words, the enhanced edge still exists in the enhanced difference feature information. This edge can be used as an observation focus. In subsequent processing, the uniformity of the ore deposit is determined based on the edge observation.

[0089] According to one embodiment of the present invention, multiple enhanced difference feature information of the same feature channel can be aggregated to obtain first aggregated feature information, which is used to aggregate pixels with non-zero brightness values (pixels indicating uneven stacking) in the enhanced difference feature information of the same feature channel onto a single feature map, thereby observing whether the ore stacking is uniform within the current monitoring period from the perspective of the feature channel. The enhanced difference feature information includes feature information of multiple feature channels; wherein, aggregating the multiple enhanced difference feature information to obtain the first aggregated feature information includes: aggregating the enhanced difference feature information between multiple adjacent first video frames with the same feature channel to obtain aggregated difference feature information corresponding to the feature channel; and obtaining the first aggregated feature information based on the aggregated difference feature information corresponding to the multiple feature channels.

[0090] According to one embodiment of the present invention, the enhanced difference feature information between multiple adjacent first video frames with the same feature channel is aggregated to obtain the aggregated difference feature information corresponding to the feature channel, including: obtaining the coordinates of the aggregated difference feature information corresponding to the kth feature channel according to formula (1) The brightness value of the pixel ,

[0091] (1)

[0092] in, The coordinates of the enhanced difference feature information between the i-th first video frame and the i+1-th first video frame of the k-th feature channel are The brightness value of the pixel, n is the number of the first video frame, is the preset brightness value.

[0093] According to one embodiment of the present invention, in formula (1), The coordinates of multiple enhanced difference feature information are The average value of the absolute values of the brightness values of the pixels of the aggregated difference feature information is obtained, and in order to highlight the differences between the ore edge enhancement feature information, that is, to highlight the pixels indicating uneven stacking, for the case where there may be pixels with brightness values less than 0 after the ore edge enhancement feature information is subtracted, the absolute value can be used for processing, so that the pixels with brightness values not equal to 0 after the subtraction are more prominent. After averaging the above absolute values, a binarization process can be performed, that is, when the average value is greater than or equal to a preset brightness value (for example, 100), the brightness value of the pixel in the aggregated difference feature information is set to 255, otherwise it is set to 0, so that the pixels at the enhanced edge position are more prominent, that is, the pixels indicating uneven stacking at the edge position are highlighted, and the influence of the pixels at the non-edge position in the enhanced difference feature information is weakened, so that the unevenness of the ore stacking can be represented based on the non-zero pixels at the edge position in each enhanced difference feature information. The pixel points indicating uneven ore stacking at the edge position in the ore edge enhancement feature information at multiple moments in the current monitoring period can be aggregated on the same feature map to obtain the aggregated difference feature information of the feature channel.

[0094] In this way, the pixel points whose brightness values are not 0 after the ore edge enhancement feature information is subtracted can be highlighted by taking the absolute value of the brightness value and then averaging it, thereby highlighting the pixel points representing uneven stacking. Moreover, since the brightness value at the edge position in the ore edge enhancement feature information is enhanced, the brightness value at the edge position is higher than the brightness value at other positions. Therefore, the pixel value at the edge position in the aggregated difference feature information can be strengthened by binarization, so that the pixel points at the edge position that are not 0 are used to represent the unevenness of the ore stacking. Furthermore, the pixel points representing the uneven ore stacking at the edge positions in multiple ore edge enhancement feature information can be aggregated on the same feature map to express the unevenness of the edge of the ore pile based on the feature map.

[0095] According to one embodiment of the present invention, similarly, aggregated difference feature information of each feature channel can be obtained, and the aggregated difference feature information of multiple feature channels can constitute the first aggregated feature information, that is, the first aggregated feature information includes the aggregated difference feature information of multiple feature channels.

[0096] According to one embodiment of the present invention, the first state recognition layer may include a deconvolution layer and other layers, which can decode the aggregated difference feature information of multiple feature channels to obtain stacking state information. The stacking state information can be information in vector form and can be used to describe the probability of uneven ore stacking during the current monitoring cycle. For example, if the amount of ore stacked in the first video frame captured at multiple times during the current monitoring cycle varies, causing the edge of the ore pile to frequently change, then the probability of uneven ore stacking is high, for example, higher than 0.5. The stacking state information is input into the fully connected layer for processing to obtain the probability of uneven ore stacking. Uneven ore stacking can affect the smoothness of the ore conveying process, causing the ore conveying efficiency to fluctuate, the load on the conveyor belt to fluctuate, and may also increase the risk of ore spillage.

[0097] According to one embodiment of the present invention, in step S5, the ore stacking feature information corresponding to multiple first video frames and the belt status feature information corresponding to multiple second video frames can be combined to obtain conveying status information, which is used to describe the risks of the ore conveying process during the monitoring period, for example, the risk of the conveyor belt being deformed due to a large load caused by a large amount of ore stacking, the risk of low conveying efficiency caused by a small amount of ore stacking, etc.

[0098] According to one embodiment of the present invention, in step S5, conveying status information is obtained based on ore stacking feature information corresponding to multiple first video frames and belt status feature information corresponding to multiple second video frames, including: inputting the ore stacking feature information corresponding to the first video frame into the second state identification level to obtain stacking risk information; inputting the belt status feature information corresponding to the second video frame in the same group as the first video frame into the third state identification level to obtain belt status risk information; inputting the stacking risk information and the belt status risk information into the risk identification level to obtain conveying risk information; inputting the conveying risk information corresponding to multiple video frame groups into the conveying state identification level to obtain conveying status information.

[0099] According to one embodiment of the present invention, as described above, the first video frame and the second video frame with the same serial number respectively record the overhead view and the side view of the conveyor belt at the same moment. Therefore, the ore stacking feature information of the first video frame with the same serial number in the video frame group and the belt status feature information corresponding to the second video frame can be combined to jointly determine whether there is a risk at that moment.

[0100] According to one embodiment of the present invention, the second and third state recognition layers are similar and can both include layers such as deconvolution layers. They can decode feature maps from multiple feature channels to obtain information in vector form. The model parameters of the first, second, and third state recognition layers can be different. The second state recognition layer processes the ore stacking feature information from multiple feature channels of a first video frame at a given moment to obtain vector-based stacking risk information. This information can be used to describe whether the ore stacking risk exists at that moment, for example, whether there is a risk of excessive stacking, such as spillage, or a risk of excessive load on the conveyor belt. The third state recognition layer processes the belt state feature information from multiple feature channels of a second video frame with the same sequence number as the first video frame (i.e., a second video frame in the same group) to obtain vector-based belt state risk information. This information can be used to describe whether the belt has undergone significant deformation or is bearing a significant load at that moment. Combining these two layers allows for a comprehensive assessment of whether the conveying process at that moment is risky.

[0101] According to one embodiment of the present invention, the risk identification level may include a linear operation level and a nonlinear operation level (for example, an activation level), which may perform operations on the stacking risk information corresponding to the first video frame of the same group and the belt status risk information corresponding to the second video frame to obtain conveying risk information, which may be used to describe whether there is a risk in the conveying process at the moment of shooting the first video frame and the second video frame of the same group. The conveying risk information is information in vector form, and its dimension is the same as that of the belt status risk information and the stacking risk information.

[0102] According to one embodiment of the present invention, the conveyor status recognition layer may include a linear operation layer and a nonlinear operation layer. It can operate on the conveyor risk information corresponding to multiple video frame groups to obtain conveyor status information, which can be used to describe the risk of ore being transported by the conveyor belt during the current monitoring period. The conveyor status information is in vector form and has the same dimensionality as the stacking status information.

[0103] In this way, the overhead view and side view of the conveyor belt at the same moment can be recorded respectively based on the first video frame and the second video frame of the same group, and the ore stacking characteristic information of the first video frame and the belt status characteristic information of the second video frame of the same group can be combined to jointly determine whether there is a risk in the conveying process at that moment, and then combine the conveying risk information at multiple moments to determine whether there is a risk in the conveying process within the current monitoring period, thereby improving the comprehensiveness and accuracy of process monitoring.

[0104] According to one embodiment of the present invention, in step S6, the stacking state information and the conveying state information can be combined to determine the conveying smoothness information for the current monitoring period. For example, the stacking state information and the conveying state information can be concatenated to obtain a conveying smoothness description vector, which is then processed through the first fully connected layer to obtain conveying smoothness information. The conveying smoothness information is information about the probability of a smooth conveying process and can be used to indicate the smoothness of the conveying process. The higher the value of the conveying smoothness information, the smoother the conveying process during the current monitoring period, and the lower the probability of uneven ore stacking, uneven belt load, and uneven belt deformation.

[0105] According to one embodiment of the present invention, the above-mentioned conveying anomaly recognition model can be trained before use, and the training steps of the conveying anomaly recognition model include: obtaining a first training video frame sequence shot by a first camera and a second training video frame sequence shot by a second camera in a training cycle; obtaining training ore stacking feature information of each first training video frame in the first training video frame sequence and training belt state feature information of each second training video frame in the second training video frame sequence through a feature extraction layer; processing the training ore stacking feature information through an edge enhancement layer to obtain training edge enhancement feature information; obtaining first training edge information in the first training video frame through a first decoding layer. ; According to the first training edge information and the annotation information of the first training video, an edge loss function is obtained; according to multiple training edge enhancement feature information, multiple training enhancement difference feature information is obtained, and the multiple training enhancement difference feature information is aggregated to obtain training aggregate feature information; the training aggregate feature information is input into the first state recognition layer to obtain training stacking state information; the training stacking state information is input into the second fully connected layer to obtain training stacking risk probability distribution information; according to the training stacking risk probability distribution information and the annotation information of the first training video, a stacking risk loss function is obtained; the training ore stacking feature information corresponding to the first training video frame is input into the second state recognition layer to obtain training The stacking risk information is input into the third state recognition layer, and the training belt state feature information corresponding to the second training video frame with the same sequence number as the first training video frame is input into the third state recognition layer to obtain the training belt state risk information; the training stacking risk information and the training belt state risk information are input into the risk recognition layer to obtain the training conveying risk information; the training conveying risk information is input into the third fully connected layer to obtain the conveying risk probability distribution information; according to the conveying risk probability distribution information and the annotation information of the second training video, the conveying risk loss function is obtained; the multiple training conveying risk information are input into the conveying state recognition layer to obtain the training conveying state information; the training conveying state information is input into the fourth fully connected layer to obtain the conveying state information. state risk probability distribution information; obtain the transportation state risk loss function according to the transportation state risk probability distribution information and the annotation information of the second training video; splice the training stacking state information and the training transportation state information to determine the training stationarity state information; input the training stationarity state information into the first fully connected layer to obtain the training stationarity result; obtain the stationarity loss function according to the annotation information of the second training video and the training stationarity result; train the transportation anomaly recognition model according to the edge loss function, the stacking risk loss function, the transportation risk loss function, the transportation state risk loss function and the stationarity loss function to obtain the trained transportation anomaly recognition model.

[0106] According to one embodiment of the present invention, the training cycle is a time period during the system testing process. During the training cycle, a video of the conveyor belt transporting ore can be captured by the first camera and the second camera, and a first training video frame sequence and a second training video frame sequence can be obtained. The method for obtaining the first training video frame sequence and the second training video frame sequence is similar to the method for obtaining the first video frame sequence and the second video frame sequence, and will not be repeated here.

[0107] According to one embodiment of the present invention, the training ore stacking feature information of each first training video frame in the first training video frame sequence and the training belt state feature information of each second training video frame in the second training video frame sequence can be obtained through the feature extraction level. The acquisition method is similar to the acquisition method of the above-mentioned ore stacking feature information and belt state feature information, and will not be repeated here.

[0108] According to one embodiment of the present invention, the training ore pile feature information can be processed through the edge enhancement layer to obtain the training edge enhancement feature information. The acquisition method is similar to the acquisition method of the edge enhancement feature information and will not be repeated here. Auxiliary training can be performed for the edge enhancement effect of the edge enhancement layer. For example, the training edge enhancement feature information of multiple feature channels of each first training video frame can be input into the first decoding layer to obtain the first training edge information in the first training video frame. The first decoding layer may include a deconvolution layer and other layers. The training edge enhancement feature information can be processed to obtain the position of the edge of the ore pile in the first training video frame and marked in the first training video frame to obtain the first training edge information. Similarly, the first training edge information in each first training video frame can be obtained.

[0109] According to one embodiment of the present invention, the annotation information of the first training video may include the edge locations of the ore pile in each first training video frame (for example, manually annotated by a mining engineer). Therefore, an edge loss function can be obtained by comparing the edge locations in the annotation information with the first training edge information output by the first decoding layer. In this example, the edge loss function can be a cross-entropy loss function or a dice loss function, etc. The present invention does not limit the specific type of edge loss function.

[0110] According to one embodiment of the present invention, the training aggregate feature information is obtained in a similar manner to the above-mentioned aggregate feature information, and the training stacking state information is obtained in a similar manner to the above-mentioned stacking state information, which will not be repeated here. The training stacking state information is similar to the stacking state information, and is information in vector form, which is used to describe the probability of uneven ore stacking during the training cycle. The training stacking state information is processed by the second fully connected layer to obtain the probability of uneven ore stacking during the training cycle, that is, the training stacking risk probability distribution information. Furthermore, the annotation information of the first training video may include annotation information on whether the ore stacking is uneven, for example, the probability of uneven ore stacking is 1, and the probability of uniform ore stacking is 0. Therefore, a stacking risk loss function can be obtained based on the training stacking risk probability distribution information and the above-mentioned annotation information. The stacking risk loss function can be a cross entropy loss function.

[0111] According to one embodiment of the present invention, the methods for obtaining training stacking risk information, training belt status risk information, and training conveying risk information are similar to the methods for obtaining the above-mentioned stacking risk information, belt status risk information, and conveying risk information, and are not repeated here. Among them, the training conveying risk information is similar to the conveying risk information, both of which are information in vector form, used to describe whether there is a risk when shooting the first training video frame and the second training video frame with the same sequence number during the conveying process. The training conveying risk information is processed by the third fully connected layer to obtain the conveying risk probability distribution information, that is, the probability of whether there is a risk when shooting the first training video frame and the second training video frame with the same sequence number.

[0112] According to one embodiment of the present invention, the annotation information of the second training video can be used to mark whether each second training video frame has a transportation risk (for example, manually marked by a mining engineer). For example, when there is too much ore piled up and the conveyor belt is overloaded, it can be marked as having a transportation risk, that is, the probability of having a transportation risk is 1, otherwise it is 0. Therefore, the transportation risk loss function can be determined based on the transportation risk probability distribution information and the annotation information of the second training video. The transportation risk loss function can be a cross-entropy loss function.

[0113] According to one embodiment of the present invention, the training conveying state information is similar to the conveying state information, both of which are information in vector form and can be used to describe the risk of the conveyor belt conveying ore during the training cycle. The training conveying state information is processed by the fourth fully connected layer to obtain the probability that there is a risk in the conveyor belt conveying ore during the training cycle, that is, the conveying state risk probability distribution information. In the annotation information of the second training video, it can be marked whether there is a risk in the process of conveying ore by the conveyor belt, for example, the risk of spillage due to excessive ore stacking, the risk of deformation of the conveyor belt due to a large load on the conveyor belt, and the risk of low transportation efficiency due to insufficient ore stacking. If these risks exist, the probability of the risk is marked as 1, otherwise it is 0. Furthermore, the conveying state risk loss function can be determined by the conveying state risk probability distribution information and the annotation information of the second training video. The conveying state risk loss function can be a cross-entropy loss function.

[0114] According to one embodiment of the present invention, the training stacking state information and the training conveying state information can be spliced to determine the training stability state information and input into the first fully connected layer to obtain the training stability result. Similar to the conveying stability information, the training stability result is the probability information of the smooth conveying process. The higher the value of the training stability state information, the smoother the conveying process, and the lower the probability of uneven ore stacking, uneven belt load, and uneven belt deformation. The annotation information of the second training video can mark the ore conveying process of the training cycle. If the ore conveying process is smooth and there are no abnormalities, it is marked as 1, that is, the probability of the ore conveying process being smooth is 1, otherwise it is 0. Furthermore, based on the annotation information of the second training video and the training stability result, a stability loss function can be obtained. The stability loss function can be a cross entropy loss function.

[0115] According to one embodiment of the present invention, the transport anomaly recognition model is trained based on the edge loss function, the stacking risk loss function, the transport risk loss function, the transport state risk loss function and the stationarity loss function to obtain the trained transport anomaly recognition model, including: obtaining the loss function of the transport anomaly recognition model according to formula (2): ,

[0116] (2)

[0117] in, is the marginal loss function, is the transmission risk loss function corresponding to the first training video frame and the second training video frame with sequence number i, n is the number of first video frames, is the stacking risk loss function, is the transport state risk loss function, is the stationary loss function, and max(*) is the maximum value function. The transport anomaly recognition model is trained through the loss function of the transport anomaly recognition model to obtain the trained transport anomaly recognition model.

[0118] According to one embodiment of the present invention, It means that the edge loss function, the transportation risk loss function, the stacking risk loss function and the transportation state risk loss function corresponding to the first training video frame and the second training video frame of each serial number are summed up, so that the loss function of the transportation anomaly recognition model can include the above-mentioned multiple loss functions, so that different levels in the transportation anomaly recognition model can be trained separately through multiple paths.

[0119] According to one embodiment of the present invention, It is expressed as the coefficient setting for the stationary loss function. In the case of , the coefficient is 1, otherwise, the coefficient is . Indicates that both the stacking risk loss function and the transportation state risk loss function are less than , since the stacking risk loss function and the transportation state risk loss function are both cross entropy loss functions, the values of both are less than , which means that the gap between the training stacking risk probability distribution information and the labeled information, as well as the gap between the transportation status risk probability distribution information and the labeled information are both less than 0.5, which means that the transportation anomaly recognition model is correct in judging the uniformity of ore stacking and whether there is a risk in the process of conveying ore on the conveyor belt. However, there is an error in the probability value. Therefore, the coefficient can be set to 1. If any one of the stacking risk loss function and the transportation status risk loss function is greater than or equal to , it means that there is a value greater than 0.5 in the gap between the training stacking risk probability distribution information and the labeled information, as well as the gap between the conveying state risk probability distribution information and the labeled information. That is, the conveying anomaly recognition model has at least one error in judging the uniformity of ore stacking and whether there is a risk in the process of conveying ore by the conveyor belt. In this case, As the coefficient of the stationary loss function, when the transport anomaly recognition model is trained by the stationary loss function, the process of training part of the transport anomaly recognition model through one of the stacking risk loss function and the transport state risk loss function is strengthened, so that the stacking risk loss function and the transport state risk loss function are quickly reduced to The following improves the training efficiency. When the stacking risk loss function and the transport state risk loss function are both greater than In the case of , the maximum value of the two is taken as the coefficient, and through multiple training, the stacking risk loss function and the transportation state risk loss function are alternately reduced, and finally all are reduced to Below, for example, In the case of As a coefficient, the process of training some levels of the transport anomaly recognition model through the stacking risk loss function is strengthened. After the stacking risk loss function drops to a level less than the transport state risk loss function, As a coefficient, strengthen the process of training some levels of the transport anomaly recognition model through the transport state risk loss function, and make the two decrease alternately until both are less than ,if , the coefficient can be set to 1.

[0120] According to one embodiment of the present invention, the above items are added together to obtain the loss function of the transport anomaly recognition model, and back propagation is performed. After multiple trainings and verification of the accuracy of the transport anomaly recognition model, a trained transport anomaly recognition model can be obtained, which can be used in the process of determining transport smoothness information.

[0121] In this way, auxiliary training can be performed through the first decoding layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer, so that different layers in the transport anomaly recognition model can be trained separately through multiple paths, and when the entire transport anomaly recognition model is trained through the stationarity loss function, the coefficient of the stationarity loss function is set through the stacking risk loss function and the transport state risk loss function, thereby strengthening the process of training part of the layers of the transport anomaly recognition model through one of the stacking risk loss function and the transport state risk loss function, quickly correcting the layers with misjudgment, improving training efficiency, and quickly improving the accuracy of each layer of the transport anomaly recognition model.

[0122] According to an embodiment of the present invention, a computer image-based ore conveyor monitoring method can use a first camera to capture a first video of the conveyor belt from above to determine the ore stacking condition on the conveyor belt. A second camera can also capture a second video of the conveyor belt from the side to determine the deformation of the conveyor belt itself, and thus the conveyor belt load. By processing the first and second videos in conjunction with a conveyor anomaly recognition model, it is possible to automatically determine whether there are any anomalies in the ore stacking and the conveyor belt load, thereby accurately determining whether the ore conveying process is smooth. When determining the aggregated difference feature information, the pixel points whose brightness values are not 0 after the ore edge enhancement feature information is subtracted can be highlighted by taking the absolute value of the brightness value and then averaging it, thereby highlighting the pixel points representing uneven stacking. Moreover, since the brightness value at the edge position in the ore edge enhancement feature information is enhanced, the brightness value at the edge position is higher than the brightness value at other positions. Therefore, the pixel value at the edge position in the aggregated difference feature information can be strengthened by binarization, so that the pixel points at the edge position that are not 0 are used to represent the unevenness of the ore stacking. Furthermore, the pixel points representing the uneven ore stacking at the edge positions in multiple ore edge enhancement feature information can be aggregated on the same feature map to express the unevenness of the edge of the ore pile based on the feature map. When determining the conveying status information, the top view and side view of the conveyor belt at the same moment can be recorded based on the first video frame and the second video frame of the same group, respectively. The ore stacking feature information of the first video frame and the belt status feature information of the second video frame of the same group can be combined to jointly determine whether the conveying process is at risk at that moment. Then, the conveying risk information at multiple moments can be combined to determine whether the conveying process is at risk during the current monitoring period, thereby improving the comprehensiveness and accuracy of process monitoring. When training the conveying anomaly recognition model, auxiliary training can be performed through the first decoding layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer, so that different layers in the conveying anomaly recognition model can be trained separately through multiple paths. When training the entire conveying anomaly recognition model through the stationarity loss function, the coefficient of the stationarity loss function is set through the stacking risk loss function and the conveying status risk loss function, thereby strengthening the process of training some layers of the conveying anomaly recognition model through one of the stacking risk loss function and the conveying status risk loss function, quickly correcting the layers with misjudgment, improving training efficiency, and quickly improving the accuracy of each layer of the conveying anomaly recognition model.

[0123] Figure 5 A block diagram of a computer image-based ore conveyance monitoring system according to an embodiment of the present invention is exemplarily shown, wherein the system comprises:

[0124] A setting module is used to set a first camera above the conveyor belt and set a second camera on the side of the conveyor belt, wherein the movement direction of the conveyor belt is the X-axis direction, the width direction of the conveyor belt is the Y-axis direction, and the direction perpendicular to the plane where the X-axis and Y-axis are located is the Z-axis direction, and the X-axis coordinates of the first camera and the second camera are consistent;

[0125] a shooting module, configured to shoot a first video through a first camera and a second video through a second camera in a current monitoring period, and obtain a first video frame sequence of the first video and a second video frame sequence of the second video;

[0126] an extraction module, configured to combine the i-th first video frame in the first video frame sequence and the i-th second video frame in the second video frame sequence into a video frame group, and input the video frame group into a feature extraction layer of the conveying anomaly recognition model to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame;

[0127] A stacking status information module is used to obtain stacking status information based on ore stacking feature information corresponding to the plurality of first video frames;

[0128] A conveying state information module is used to obtain conveying state information based on ore stacking feature information corresponding to the plurality of first video frames and belt state feature information corresponding to the plurality of second video frames;

[0129] The conveying stability information module is used to determine the conveying stability information of the current monitoring period according to the stacking state information and the conveying state information.

[0130] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0131] Those skilled in the art will appreciate that the embodiments of the present invention described above and shown in the accompanying drawings are intended to be illustrative only and are not intended to limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Any variations or modifications may be made to the embodiments of the present invention without departing from the principles described.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring ore transportation based on computer images, characterized in that: include: A first camera is set above the conveyor belt, and a second camera is set on the side of the conveyor belt. The moving direction of the conveyor belt is the X-axis direction, the width direction of the conveyor belt is the Y-axis direction, and the direction perpendicular to the plane where the X-axis and Y-axis are located is the Z-axis direction. The X-axis coordinates of the first camera and the second camera are consistent; In a current monitoring cycle, a first video is captured by a first camera, a second video is captured by a second camera, and a first video frame sequence of the first video and a second video frame sequence of the second video are acquired; The i-th first video frame in the first video frame sequence and the i-th second video frame in the second video frame sequence are combined into a video frame group, and the video frame group is input into the feature extraction layer of the conveying abnormality recognition model to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame respectively; Obtaining stacking status information based on ore stacking feature information corresponding to the plurality of first video frames; Obtaining conveying state information based on ore stacking feature information corresponding to the plurality of first video frames and belt state feature information corresponding to the plurality of second video frames; Determining the transport stability information of the current monitoring period according to the stacking state information and the transport state information; The training steps of the transport anomaly recognition model include: Acquire a first training video frame sequence captured by a first camera and a second training video frame sequence captured by a second camera during a training cycle; Acquire, through a feature extraction layer, training ore stacking feature information of each first training video frame in the first training video frame sequence, and training belt state feature information of each second training video frame in the second training video frame sequence; Processing the training ore stacking feature information through the edge enhancement level to obtain training edge enhancement feature information; Obtaining, through a first decoding level, first training edge information in a first training video frame; Obtaining an edge loss function according to the first training edge information and the labeling information of the first training video; Obtaining multiple pieces of training enhanced difference feature information based on the multiple pieces of training edge enhanced feature information, and aggregating the multiple pieces of training enhanced difference feature information to obtain training aggregated feature information; Inputting the training aggregate feature information into the first state recognition layer to obtain training stacking state information; Input the training stacking status information into the second fully connected layer to obtain the training stacking risk probability distribution information; Obtaining a stacking risk loss function based on the training stacking risk probability distribution information and the labeling information of the first training video; Input the training ore stacking feature information corresponding to the first training video frame into the second state recognition level to obtain training stacking risk information, and input the training belt state feature information corresponding to the second training video frame with the same sequence number as the first training video frame into the third state recognition level to obtain training belt state risk information; Input the training stacking risk information and the training belt status risk information into the risk identification layer to obtain the training conveying risk information; Input the training transport risk information into the third fully connected layer to obtain the transport risk probability distribution information; Obtaining a transportation risk loss function based on the transportation risk probability distribution information and the annotation information of the second training video; Inputting a plurality of training transport risk information into the transport status identification layer to obtain training transport status information; Input the training delivery state information into the fourth fully connected layer to obtain the delivery state risk probability distribution information; Obtaining a transportation state risk loss function based on the transportation state risk probability distribution information and the annotation information of the second training video; The training stacking state information and the training conveying state information are combined to determine the training stability state information; Input the training stationarity state information into the first fully connected layer to obtain the training stationarity result; Obtaining a stationarity loss function based on the labeling information of the second training video and the training stationarity result; The transport anomaly recognition model is trained according to the edge loss function, the stacking risk loss function, the transport risk loss function, the transport state risk loss function and the stationarity loss function to obtain a trained transport anomaly recognition model.

2. The computer image-based ore transportation monitoring method according to claim 1, characterized in that: Obtaining stacking status information based on ore stacking feature information corresponding to the plurality of first video frames includes: Processing the ore stacking feature information corresponding to each first video frame through an edge enhancement layer to obtain ore edge enhancement feature information; Obtain enhanced difference feature information between the ore edge enhancement feature information corresponding to the (i+1)th first video frame and the ore edge enhancement feature information corresponding to the (i)th first video frame; Aggregating the plurality of enhanced difference feature information to obtain first aggregated feature information; The first aggregated feature information is input into the first state recognition level to obtain stacking state information.

3. The computer image-based ore transportation monitoring method according to claim 2, characterized in that: The enhanced difference feature information includes feature information of multiple feature channels; The step of aggregating the plurality of enhanced difference feature information to obtain first aggregated feature information includes: Aggregating enhanced difference feature information between multiple adjacent first video frames having the same feature channel to obtain aggregated difference feature information corresponding to the feature channel; First aggregated feature information is obtained according to the aggregated difference feature information corresponding to the multiple feature channels.

4. The computer image-based ore transportation monitoring method according to claim 3, characterized in that: Aggregating enhanced difference feature information between a plurality of adjacent first video frames having the same feature channel to obtain aggregated difference feature information corresponding to the feature channel includes: According to the formula ， The coordinates of the aggregated difference feature information corresponding to the kth feature channel are obtained as The brightness value of the pixel ,in, The coordinates of the enhanced difference feature information between the i-th first video frame and the i+1-th first video frame of the k-th feature channel are The brightness value of the pixel, n is the number of the first video frame, is the preset brightness value.

5. The computer image-based ore transportation monitoring method according to claim 1, characterized in that: According to the ore stacking feature information corresponding to the plurality of first video frames and the belt state feature information corresponding to the plurality of second video frames, conveying state information is obtained, including: Inputting the ore stacking feature information corresponding to the first video frame into the second state recognition level to obtain stacking risk information; Inputting belt state feature information corresponding to a second video frame in the same group as the first video frame into a third state recognition layer to obtain belt state risk information; Input the stacking risk information and belt status risk information into the risk identification level to obtain the conveying risk information; The delivery risk information corresponding to the plurality of video frame groups is input into the delivery status recognition layer to obtain delivery status information.

6. The computer image-based ore transportation monitoring method according to claim 1, characterized in that: The transport anomaly recognition model is trained according to the edge loss function, the stacking risk loss function, the transport risk loss function, the transport state risk loss function, and the stationarity loss function to obtain a trained transport anomaly recognition model, including: According to the formula ， Obtain the loss function of the transport anomaly recognition model ,in, is the marginal loss function, is the transmission risk loss function corresponding to the first training video frame and the second training video frame with sequence number i, n is the number of first video frames, is the stacking risk loss function, is the transport state risk loss function, is the stationary loss function, and max(*) is the maximum value function; The transport anomaly recognition model is trained using the loss function of the transport anomaly recognition model to obtain a trained transport anomaly recognition model.

7. A computer image-based ore conveyance monitoring system for executing the method according to any one of claims 1 to 6, characterized in that: include: A setting module is used to set a first camera above the conveyor belt and set a second camera on the side of the conveyor belt, wherein the movement direction of the conveyor belt is the X-axis direction, the width direction of the conveyor belt is the Y-axis direction, and the direction perpendicular to the plane where the X-axis and Y-axis are located is the Z-axis direction, and the X-axis coordinates of the first camera and the second camera are consistent; a shooting module, configured to shoot a first video through a first camera and a second video through a second camera in a current monitoring period, and obtain a first video frame sequence of the first video and a second video frame sequence of the second video; an extraction module, configured to combine the i-th first video frame in the first video frame sequence and the i-th second video frame in the second video frame sequence into a video frame group, and input the video frame group into a feature extraction layer of the conveying anomaly recognition model to obtain ore stacking feature information of the i-th first video frame and belt status feature information of the i-th second video frame; A stacking status information module is used to obtain stacking status information based on ore stacking feature information corresponding to the plurality of first video frames; A conveying state information module is used to obtain conveying state information based on ore stacking feature information corresponding to the plurality of first video frames and belt state feature information corresponding to the plurality of second video frames; The conveying stability information module is used to determine the conveying stability information of the current monitoring period according to the stacking state information and the conveying state information.

Images