A dust suppression method and system for falling material pile based on multi-view images

Abstract

The present invention discloses a dust suppression method and system for a material pile based on multi-perspective images, the method comprising the following steps: S1: pre-processing multi-perspective video data to obtain multi-perspective image data, detecting feature points in the image data under different perspectives and matching them into pairs; S2: constructing a target detection model, identifying and tracking a forklift and a material pile; S3: combining the multi-perspective image data, analyzing the movement trajectory and action mode of the forklift, and inferring the forklift's intention; S4: combining the material pile characteristics and the forklift's intention, predicting the dust situation and setting dust removal parameters; S5: controlling the dust removal equipment to remove dust according to the dust removal parameters. The present invention solves the problem that the traditional dust suppression method can only spray according to a fixed preset scheme, but cannot adjust the dust suppression strategy in real time according to the forklift behavior and the size of the material pile; it also solves the problem that the dust suppression effect is limited due to the inability to accurately locate the forklift's production and transportation position and the dust range of the dust suppression strategy based on a single-perspective image.

Description

Technical Field

[0001] The present invention relates to the field of material transportation, and in particular to a dust suppression method and system for a falling material pile based on multi-view images. Background Art

[0002] Intelligent dust suppression technology is an environmental protection technology based on machine learning and image processing methods, which aims to reduce or eliminate dust and particulate dust generated by industrial production and construction. Traditional dust suppression methods mainly use spray dust suppression equipment to spray dry mist to suppress dust. However, for the dust generated when the forklift operates around the material pile, due to the unfixed shoveling position, the traditional spray dust suppression equipment cannot adjust the dust suppression strategy in real time according to the shoveling position of the forklift and the size of the pile, and can only spray according to a fixed preset plan, resulting in that even if spraying is performed, a good dust suppression effect cannot be achieved, thereby affecting the air quality of the surrounding environment and even affecting the health of the workers.

[0003] In recent years, research on video-based dust suppression strategies has gradually attracted attention. Through the analysis and processing of video images, real-time monitoring and precise control of dust emission can be achieved. However, the dust suppression control strategy based on a single video image is limited by the field of view and cannot accurately locate the shoveling position and dust emission range of the forklift, resulting in unsatisfactory dust suppression effect. Summary of the invention

[0004] In view of this, the present invention aims to provide a dust suppression method and system for a falling material pile based on multi-perspective images, aiming to solve the problem that the traditional dust suppression method cannot adjust the dust suppression strategy in real time according to the forklift behavior and the size of the pile, resulting in poor dust suppression effect.

[0005] The technical solution disclosed in the present invention is as follows:

[0006] A dust suppression method for a material pile based on multi-view images comprises the following steps:

[0007] S1: Preprocess the multi-view video data to obtain multi-view image data, detect feature points in the image data at different viewpoints and match them into pairs;

[0008] S2: Build a target detection model to identify and track the forklift and the material pile;

[0009] Identify the forklift and the material pile in the multi-view image data, align the multi-view image data into the same coordinate system based on the matched feature point pairs, and match and track the same forklift and the material pile;

[0010] S3: Combine multi-view image data to analyze the movement trajectory and action pattern of the forklift and infer the forklift's intention;

[0011] Construct a material pile feature extraction network and a forklift motion pattern recognition model to calculate the forklift motion trajectory and material pile importance and infer the forklift intention;

[0012] S4: Combine the characteristics of the material pile and the intention of the forklift to predict the dust situation and set the dust removal parameters;

[0013] S5: Control the dust removal equipment to perform dust removal according to the dust removal parameters.

[0014] Furthermore, the step S1 includes the following steps:

[0015] S11: Setting a fixed interval, dividing the multi-view video data into separate frames, and obtaining multi-view image data;

[0016] S12: Use SIFT algorithm to detect feature points in multi-view image data;

[0017] S13: For each detected feature point, calculate the corresponding SIFT descriptor;

[0018] S14: In the multi-view image data, the RANSAC algorithm is used to find corresponding feature point pairs and establish corresponding relationships between different videos.

[0019] Furthermore, the step S2 comprises the following steps:

[0020] S21: Build a target detection model to identify the forklift and the material pile in the multi-view image data. The calculation method is:

[0021] detections = YOLOv8(image)

[0022] Among them, detections is the target coordinate box, confidence and category probability, YOLOv8() is the YOLOv8 network operation, and image is the multi-view image data;

[0023] S22: Based on the matched feature point pairs, geometrically transform the multi-view image data and align them to the same coordinate system;

[0024] S23: In the multi-view image data, the same forklift and the material pile are matched using the position information and a unique identification code is assigned, which is calculated as follows:

[0025]

[0026]

[0027]

[0028] Threshold=MeanScore-γ·StdDev

[0029]

[0030] Among them, m i is the matching score of feature point pairs, d i is the distance between feature point pairs, i is the feature point index, MeanScore is the average matching score, N is the total number of feature points, StdDev is the average matching variance, Threshold is the dynamic threshold, γ is the adjustment coefficient, Distance is the distance between the center points of the target boxes in the two images, is the horizontal coordinate of the center point of the target frame in the first image, j is the target frame index, is the horizontal coordinate of the center point of the target box in the second image, is the ordinate of the center point of the target box in the first image, is the ordinate of the center point of the target box in the second image;

[0031] If Distance < Threshold, the forklift or the material pile is considered to match in the two perspectives, and a unique identification code is assigned to this target;

[0032] If Distance ≥ Threshold, the forklift or the material pile is considered unmatched in the two perspectives, and the next round of matching is performed;

[0033] S24: In the continuous frames of each view, the Hungarian algorithm is used to track the identification code of each forklift and the pile of falling materials.

[0034] Furthermore, step S3 includes the following steps:

[0035] S31: Construct a blanking pile feature extraction network, and the calculation method is:

[0036] f u =CNN(I u )

[0037] g1=Canny(detections material )

[0038] g2=VGG(detection material )

[0039] h=MLP(Concat(f u , g1, g2))

[0040] Among them, f u is the feature of the material pile at the u-th perspective, u is the perspective index, CNN() is the convolutional neural network operation, I uis the image of the material pile at the u-th viewing angle, g1 is the edge feature of the material pile, Canny() is the Canny edge detection operation, detections material is the multi-view target frame image of the blanking pile output by YOLOv8, g2 is the texture feature of the blanking pile, VGG() is the VGGNet network operation, h is the blanking pile feature, MLP() is the multi-layer perceptron operation, and Concat() is the concatenation operation;

[0041] S32: Construct a forklift action pattern recognition model, and the calculation method is:

[0042] f′ u =CNN(I′ u )

[0043] z u =Softmax(GlobalAvgPool(f′ u ))

[0044]

[0045]

[0046] Among them, f′ u is the forklift feature of the u-th view, I′ u is the forklift image at the u-th perspective, z u is the attention score of the u-th perspective, Softmax() is the Softmax function, GlobalAvgPool() is the global average pooling operation, α u is the attention weight of the u-th perspective, e is a natural constant, z v is the attention score of the vth view, M is the total number of views, and f fusion It is the action characteristic of forklift;

[0047] S33: Calculate the forklift motion trajectory and the importance of the material pile. The operation steps are as follows:

[0048] The motion trajectory of the forklift is represented as a coordinate sequence of time points, and the calculation method is:

[0049] T={(p1, q1), (p2, q2),..., (p t ,q t )}

[0050] Where T is the coordinate sequence of the time point, (p1, q1) is the horizontal and vertical coordinates of the forklift at the first moment, (p2, q2) is the horizontal and vertical coordinates of the forklift at the second moment, (p t ,q t ) are the horizontal and vertical coordinates of the forklift at the tth moment, and t is the time index;

[0051] Calculate the distance from the forklift to each material pile at each time point, and select the nearest material pile as the associated material pile;

[0052] For each material pile, count the number of times it is associated with the forklift, and sort the material piles according to the number of associations. The material pile with the most associations is the most important material pile for the forklift. The calculation method is:

[0053]

[0054] Among them, C k is the number of times the kth material pile is associated with the forklift, k is the material pile index, δ(t, k) is the indicator function, and Δt is the observation time period;

[0055] S34: Based on the characteristics of the material pile, the movement trajectory and action pattern of the forklift, the forklift’s intention is inferred. The calculation method is:

[0056] r = MLP(Concat(h, T, f fusion , log(C k )))

[0057] pro=Softmax(r)

[0058] Among them, r is the forklift intention vector, log() is the logarithm operation, and pro is the probability distribution of each forklift intention; the forklift intention includes loading, unloading, moving to a certain pile of materials, etc.

[0059] Furthermore, the step S4 comprises the following steps:

[0060] S41: Construct dust prediction model, the calculation method is:

[0061] Y = VGG (Concat (r, h))

[0062] Among them, Y is the dust height and range;

[0063] S42: Based on the predicted dust situation and the characteristics of the material pile, the dust removal parameters are predicted and calculated as follows:

[0064] Z = MLP(Concat(Y, h))

[0065] Among them, Z is the dust removal parameter.

[0066] Furthermore, the step S5 comprises the following steps:

[0067] S51: Sending dust removal parameters to the dust removal equipment control system;

[0068] S52: The dust removal device sets the spray range, spray intensity and spray time according to the received dust removal parameters;

[0069] S53: The dust removal equipment performs spray dust removal according to the set dust removal parameters.

[0070] A dust suppression system for a material pile based on multi-view images, comprising:

[0071] Multi-view video data preprocessing module: preprocess the multi-view video data to obtain multi-view image data, detect feature points in image data at different viewpoints and match them into pairs;

[0072] Forklift and material pile identification module: Build a target detection model to identify and track forklifts and material piles;

[0073] Forklift intention analysis module: Combines multi-view image data to analyze the movement trajectory and action pattern of the forklift and infer the forklift intention;

[0074] Dust prediction and dust removal parameter setting module: Combine the characteristics of the material pile and the intention of the forklift to predict the dust situation and set the dust removal parameters;

[0075] Dust removal control module: controls the dust removal equipment to remove dust according to the dust removal parameters;

[0076] The above-mentioned method for suppressing dust in a material pile based on multi-view images is realized by applying the material pile dust suppression system.

[0077] Compared with the prior art, the advantages of the present invention are:

[0078] (1) In terms of the overall solution, the present invention uses multi-view images to monitor and analyze the forklift and the material pile in real time, and dynamically adjusts the spray range, intensity and time of the dust removal equipment according to the forklift behavior and the material pile situation, thereby improving the dust suppression effect.

[0079] (2) In terms of algorithm improvement, the present invention proposes to perform target alignment based on matched feature points, set a dynamic threshold, and adapt to the feature changes of multi-view images, thereby avoiding the uncertainty of manually setting the target matching distance threshold. At the same time, the present invention uses a convolutional neural network with shared weights to extract features of the material pile in combination with the edge information and texture information of the material pile, thereby enhancing the effect of feature extraction and providing an important basis for the subsequent prediction of the forklift's intention and the prediction of dust conditions.

[0080] (3) The present invention proposes a perspective weighted attention mechanism, which automatically reduces the attention weight of the obstructed perspective, effectively reduces the data interference of the forklift when a certain perspective is obstructed, adaptively adjusts the importance of the corresponding perspective, and improves the accurate recognition rate of the forklift's intention, thereby making the dust removal parameter setting more reasonable and the dust removal effect better. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A schematic flow chart of the method for dust suppression in a material pile based on multi-view images provided by the present invention.

[0082] Figure 2 This is a flow chart of the algorithm for building a target detection model provided by the present invention.

[0083] Figure 3 A flow chart of the algorithm for inferring the forklift's intention provided by the present invention. DETAILED DESCRIPTION

[0084] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0085] Embodiment 1:

[0086] See also Figure 1 , a dust suppression method for a material pile based on multi-view images, comprising the following steps:

[0087] S1: Preprocess the multi-view video data to obtain multi-view image data, detect feature points in the image data at different viewpoints and match them into pairs;

[0088] S11: Setting a fixed interval, dividing the multi-view video data into separate frames, and obtaining multi-view image data;

[0089] S12: Use the SIFT algorithm to detect feature points in multi-view image data. The SIFT algorithm is a scale-invariant feature transform algorithm.

[0090] S13: For each detected feature point, calculate the corresponding SIFT descriptor;

[0091] S14: In the multi-view image data, the RANSAC algorithm is used to find the corresponding feature point pairs and establish the correspondence between different videos. The RANSAC algorithm is a random sample consensus algorithm.

[0092] For example, it is set to extract 5 frames per second, that is, 5 frames of images in a video with a duration of 1 second are divided into separate frames.

[0093] S2: See Figure 2 , build a target detection model to identify and track forklifts and falling material piles;

[0094] S21: Build a target detection model to identify the forklift and the material pile in the multi-view image data. The calculation method is:

[0095] detections = YOLOv8(image)

[0096] Among them, detections are the target coordinate box, confidence and category probability, YOLOv8() is the YOLOv8 network operation, YOLOv8 is You Only Look Once v8 network, and image is multi-view image data;

[0097] S22: Based on the matched feature point pairs, geometrically transform the multi-view image data and align them to the same coordinate system;

[0098] S23: In the multi-view image data, the same forklift and the material pile are matched using the position information and a unique identification code is assigned, which is calculated as follows:

[0099]

[0100]

[0101]

[0102] Threshold=MeanScore-γ·StdDev

[0103]

[0104] Among them, m i is the matching score of feature point pairs, d i is the distance between feature point pairs, i is the feature point index, MeanScore is the average matching score, N is the total number of feature points, StdDev is the average matching variance, Threshold is the dynamic threshold, γ is the adjustment coefficient, Distance is the distance between the center points of the target boxes in the two images, is the horizontal coordinate of the center point of the target frame in the first image, j is the target frame index, is the horizontal coordinate of the center point of the target box in the second image, is the ordinate of the center point of the target box in the first image, is the ordinate of the center point of the target box in the second image;

[0105] If Distance < Threshold, the forklift or the material pile is considered to match in the two perspectives, and a unique identification code is assigned to this target;

[0106] If Distance ≥ Threshold, the forklift or the material pile is considered unmatched in the two perspectives, and the next round of matching is performed;

[0107] The present invention utilizes the distance and score of SIFT feature point pairs to achieve accurate target matching of the forklift and the material pile in the multi-view image; compared with the traditional method of directly performing multi-view target alignment, SIFT feature points can extract key information in the image, thereby better preserving the characteristics and structure of the image, effectively improving the transformation and deformation problems during multi-view image alignment, and improving the accuracy and stability of target alignment.

[0108] In addition, the present invention performs target alignment by calculating the distance difference of the targets in the multi-view images and comparing it with a threshold, and the threshold is calculated based on the SIFT feature point matching score, which can adapt to the feature changes of the multi-view images, avoids the uncertainty of manually setting the target matching threshold, and is more objective and accurate.

[0109] For example, suppose there are image data from two perspectives, and the target detection result of each perspective includes the coordinate frame information of the forklift and the material pile, where:

[0110] m i They are: [0.8, 0.6, 0.9, 0.7, 0.5]

[0111] γ is: 0.5

[0112] Then we have:

[0113]

[0114]

[0115] Threshold=0.7-0.1×0.5=0.65

[0116] For the first feature point pair, if Distance = 1, then Distance > Threshold (1 > 0.65), it is considered that the forklift or the material pile does not match in the two perspectives;

[0117] For the second feature point pair, if Distance=0.6, then Distance<Threshold (0.6<0.65), and it is considered that the forklift or the material pile matches in the two perspectives.

[0118] S24: In the continuous frames of each view, the Hungarian algorithm is used to track the identification code of each forklift and the pile of falling materials.

[0119] S3: See Figure 3 ,Combined with multi-view image data, the movement trajectory and action pattern of the forklift are analyzed to infer the forklift’s intention;

[0120] S31: Construct a blanking pile feature extraction network, and the calculation method is:

[0121] f u =CNN(I u )

[0122] g1=Canny(detections material )

[0123] g2=VGG(detection material )

[0124] h=MLP(Concat(f u , g1, g2))

[0125] Among them, f u is the feature of the material pile at the u-th perspective, u is the perspective index, CNN() is the convolutional neural network operation, I u is the image of the material pile at the u-th viewing angle, g1 is the edge feature of the material pile, Canny() is the Canny edge detection operation, detections material is the multi-view target frame image of the blanking pile output by YOLOv8, g2 is the texture feature of the blanking pile, VGG() is the VGGNet network operation, VGGNet is the Visual Geometry Group Network, h is the blanking pile feature, MLP() is the multi-layer perceptron operation, and Concat() is the concatenation operation;

[0126] Step S31 uses a weight-sharing convolutional neural network to facilitate the fusion of multi-view image data. Secondly, the edge information of the material pile is extracted through Canny edge detection, so as to better obtain the height and range characteristics of the material pile. At the same time, the VGGNet network is used to extract the texture features of the material pile, and finally the multi-layer perceptron is used to perform feature fusion to obtain the material pile features. Compared with the traditional convolutional neural network, the material pile feature extraction network takes into account the edge information and texture information, improves the feature richness and representation ability, and provides an important basis for the subsequent prediction of the forklift intention and the dust situation.

[0127] S32: Construct a forklift action pattern recognition model, and the calculation method is:

[0128] f′ u =CNN(I′ u )

[0129] z u =Softmax(GlobalAvgPool(f′ u ))

[0130]

[0131]

[0132] Among them, f′ u is the forklift feature of the u-th view, I′ u is the forklift image at the u-th perspective, z u is the attention score of the u-th perspective, Softmax() is the Softmax function, GlobalAvgPool() is the global average pooling operation, α u is the attention weight of the u-th perspective, e is a natural constant, z v is the attention score of the vth view, M is the total number of views, and f fusion It is the action characteristic of forklift;

[0133] The present invention proposes a perspective weighted attention mechanism, which uses a weight-sharing convolutional neural network to extract the features of the forklift at each perspective, and calculates the attention score of each perspective through global average pooling and Softmax function. Finally, the forklift action features are obtained by weighted fusion of the forklift features at each perspective. Compared with the traditional method, the perspective weighted attention mechanism automatically reduces the attention weight of the occluded perspective, effectively reduces the data interference of the forklift when a certain perspective is occluded, thereby reducing the impact of occlusion on behavior recognition, improving the stability and robustness of the model, and making forklift intention recognition more accurate.

[0134] S33: Calculate the forklift motion trajectory and the importance of the material pile. The operation steps are as follows:

[0135] The motion trajectory of the forklift is represented as a coordinate sequence of time points, and the calculation method is:

[0136] T={(p1, q1), (p2, q2),..., (p t ,q t )}

[0137] Where T is the coordinate sequence of the time point, (p1, q1) is the horizontal and vertical coordinates of the forklift at the first moment, (p2, q2) is the horizontal and vertical coordinates of the forklift at the second moment, (p t ,q t ) are the horizontal and vertical coordinates of the forklift at the tth moment, and t is the time index;

[0138] Calculate the distance from the forklift to each material pile at each time point, and select the nearest material pile as the associated material pile;

[0139] For each material pile, count the number of times it is associated with the forklift, and sort the material piles according to the number of associations. The material pile with the most associations is the most important material pile for the forklift. The calculation method is:

[0140]

[0141] Among them, C k is the number of times the kth material pile is associated with the forklift, k is the material pile index, δ(t, k) is the indicator function, and Δt is the observation time period;

[0142] S34: Based on the characteristics of the material pile, the movement trajectory and action pattern of the forklift, the forklift’s intention is inferred. The calculation method is:

[0143] r = MLP(Concat(h, T, f fusion , log(C k )))

[0144] pro=Softmax(r)

[0145] Among them, r is the forklift intention vector, log() is the logarithm operation, and pro is the probability distribution of each forklift intention; the forklift intention includes loading, unloading, moving to a certain pile of materials, etc.

[0146] For example, suppose that at a certain point in time, the No. 1 material pile is still not full, and the forklift trajectory data shows that the forklift transferred materials from the No. 2 material pile to the No. 1 material pile last time; based on this situation, the model will infer that the forklift will continue to transfer materials from the No. 2 material pile to the No. 1 material pile. Therefore, the dust suppression parameters are set in advance according to the characteristics of the No. 1 and No. 2 material piles, and the spray operation is prepared.

[0147] On the other hand, suppose that pile No. 1 has been emptied, and pile No. 3 is the closest pile that has not been emptied. In this case, the model infers that the next operation of the shovel truck will be to shovel pile No. 3. Therefore, the dust suppression parameters are set in advance based on the characteristics of pile No. 3, and the spray operation is prepared.

[0148] S4: Combine the characteristics of the material pile and the intention of the forklift to predict the dust situation and set the dust removal parameters;

[0149] S41: Construct dust prediction model, the calculation method is:

[0150] Y = VGG (Concat (r, h))

[0151] Among them, Y is the dust height and range;

[0152] S42: Based on the predicted dust situation and the characteristics of the material pile, the dust removal parameters are predicted and calculated as follows:

[0153] Z = MLP(Concat(Y, h))

[0154] Among them, Z is the dust removal parameter.

[0155] S5: Control the dust removal equipment to remove dust according to the dust removal parameters;

[0156] S51: Sending dust removal parameters to the dust removal equipment control system;

[0157] S52: The dust removal device sets the spray range, spray intensity and spray time according to the received dust removal parameters;

[0158] S53: The dust removal equipment performs spray dust removal according to the set dust removal parameters.

[0159] The present invention is particularly suitable for scenes with complex working environments, frequent forklift activities, and multiple material piles. By using multi-view images to monitor forklifts and material piles in real time, the present invention can dynamically adjust the spray range, intensity, and time of the dust removal equipment according to the forklift behavior and material pile conditions, thereby significantly improving the dust suppression effect.

[0160] Embodiment 2:

[0161] The present invention also discloses a dust suppression system for a material pile based on multi-view images, which is used to implement the dust suppression method for a material pile based on multi-view images, and comprises:

[0162] Multi-view video data preprocessing module: preprocess the multi-view video data to obtain multi-view image data, detect feature points in image data at different viewpoints and match them into pairs;

[0163] Forklift and material pile identification module: Build a target detection model to identify and track forklifts and material piles;

[0164] Forklift intention analysis module: Combines multi-view image data to analyze the movement trajectory and action pattern of the forklift and infer the forklift intention;

[0165] Dust prediction and dust removal parameter setting module: Combine the characteristics of the material pile and the intention of the forklift to predict the dust situation and set the dust removal parameters;

[0166] Dust removal control module: controls the dust removal equipment to perform dust removal according to the dust removal parameters.

[0167] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0169] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A dust suppression method for a material pile based on multi-view images, characterized in that: The following steps are involved: S1: Preprocess the multi-view video data to obtain multi-view image data, detect feature points in the image data at different viewpoints and match them into pairs; S2: Build a target detection model to identify and track the forklift and the material pile; Identify the forklift and the material pile in the multi-view image data, align the multi-view image data into the same coordinate system based on the matched feature point pairs, and match and track the same forklift and the material pile; S3: Combine multi-view image data to analyze the movement trajectory and action pattern of the forklift and infer the forklift's intention, including the following steps: S31: Construct a feature extraction network for the material pile, and the calculation method is: ； ； ； ； in, is the blanking pile feature of the u-th viewing angle, u is the viewing angle index, For convolutional neural network operations, is the image of the material pile at the u-th viewing angle, is the edge feature of the blanking pile, For the Canny edge detection operation, for The output multi-view drop pile target frame image, is the texture feature of the blanking pile, For VGGNet network operation, For the characteristics of the falling pile, For multi-layer perceptron operation, For splicing operation; S32: Construct a forklift action pattern recognition model, and the calculation method is: ； ； ； ； in, is the forklift feature at the u-th perspective, is the forklift image from the u-th perspective, is the attention score of the u-th view, for function, is the global average pooling operation, is the attention weight of the u-th perspective, e is a natural constant, is the attention score of the vth view, M is the total number of views, It is the action characteristic of forklift; S33: Calculate the forklift motion trajectory and the importance of the material pile. The operation steps are as follows: The motion trajectory of the forklift is represented as a coordinate sequence of time points, and the calculation method is: ； Where T is the coordinate sequence of the time points, are the horizontal and vertical coordinates of the forklift at the first moment, are the horizontal and vertical coordinates of the forklift at the second moment, are the horizontal and vertical coordinates of the forklift at the tth moment, and t is the time index; Calculate the distance from the forklift to each material pile at each time point, and select the nearest material pile as the associated material pile; For each material pile, count the number of times it is associated with the forklift, and sort the material piles according to the number of associations. The material pile with the most associations is the most important material pile for the forklift. The calculation method is: ； in, is the number of times the kth material pile is associated with the forklift, k is the material pile index, is the indicator function, is the observation period; S34: Based on the characteristics of the material pile, the movement trajectory and action pattern of the forklift, the forklift’s intention is inferred. The calculation method is: ； ； Among them, r is the forklift intention vector, To take the logarithm operation, is the probability distribution of each intention of the forklift; S4: Combine the characteristics of the material pile and the intention of the forklift to predict the dust situation and set the dust removal parameters; S5: Control the dust removal equipment to perform dust removal according to the dust removal parameters.

2. The dust suppression method for falling material piles based on multi-view images according to claim 1 is characterized in that: The step S1 comprises the following steps: S11: Setting a fixed interval, dividing the multi-view video data into separate frames, and obtaining multi-view image data; S12: Use SIFT algorithm to detect feature points in multi-view image data; S13: For each detected feature point, calculate the corresponding SIFT descriptor; S14: In the multi-view image data, the RANSAC algorithm is used to find corresponding feature point pairs and establish corresponding relationships between different videos.

3. The dust suppression method for falling material pile based on multi-view images according to claim 2 is characterized in that: The step S2 comprises the following steps: S21: Build a target detection model to identify the forklift and the material pile in the multi-view image data. The calculation method is: ； in, is the target coordinate box, confidence and category probability, For YOLOv8 network operation, is multi-view image data; S22: Based on the matched feature point pairs, geometrically transform the multi-view image data and align them to the same coordinate system; S23: In the multi-view image data, the same forklift and the material pile are matched using the position information and a unique identification code is assigned, which is calculated as follows: ； ； ； ； ； in, is the matching score of feature point pairs, is the distance between feature point pairs, i is the feature point index, is the average matching score, is the total number of feature points, is the average matching variance, is the dynamic threshold, is the adjustment coefficient, is the distance between the center points of the target boxes in the two images, is the horizontal coordinate of the center point of the target frame in the first image, j is the target frame index, is the horizontal coordinate of the center point of the target box in the second image, is the ordinate of the center point of the target box in the first image, is the ordinate of the center point of the target box in the second image; like , then the forklift or the material pile is considered to match in the two perspectives, and a unique identification code is assigned to this target; like , then the forklift or the material pile is considered to be unmatched in the two perspectives, and the next round of matching is performed; S24: In the consecutive frames of each view, the Hungarian algorithm is used to track the identification code of each forklift and the pile of falling materials.

4. The dust suppression method for falling material piles based on multi-view images according to claim 3 is characterized in that: The step S4 comprises the following steps: S41: Construct dust prediction model, the calculation method is: ； Among them, Y is the dust height and range; S42: Based on the predicted dust situation and the characteristics of the material pile, the dust removal parameters are predicted and calculated as follows: ； in, Dust removal parameters.

5. The method for dust suppression of falling material pile based on multi-view images according to claim 4 is characterized in that: The step S5 comprises the following steps: S51: Sending dust removal parameters to the dust removal equipment control system; S52: The dust removal device sets the spray range, spray intensity and spray time according to the received dust removal parameters; S53: The dust removal equipment performs spray dust removal according to the set dust removal parameters.

6. A dust suppression system for falling material piles based on multi-view images, characterized in that: include: Multi-view video data preprocessing module: preprocess the multi-view video data to obtain multi-view image data, detect feature points in image data at different viewpoints and match them into pairs; Forklift and material pile identification module: Build a target detection model to identify and track forklifts and material piles; Forklift intention analysis module: Combines multi-view image data to analyze the movement trajectory and action pattern of the forklift and infer the forklift intention; Dust prediction and dust removal parameter setting module: Combine the characteristics of the material pile and the intention of the forklift to predict the dust situation and set the dust removal parameters; Dust removal control module: controls the dust removal equipment to remove dust according to the dust removal parameters; To realize the dust suppression method for a falling material pile based on multi-view images as described in any one of claims 1 to 5.

Images