Material level height early warning system and method based on point cloud and visual linkage

Abstract

The invention provides a material level height early warning system and method based on point cloud and visual linkage, and relates to the field of early warning. According to the system, through cooperative work of the point cloud scanning module, the visual identification module, the data processing module, the material level height detection strategy making module, the material level detection module, the control module and other modules, all-dimensional intelligent monitoring of the bulk cargo bin is achieved. The point cloud scanning module obtains three-dimensional point cloud data of a stock bin, the visual recognition module recognizes material types, the data processing module extracts stock bin location and stock bin opening information, the material level height detection strategy module determines a differential threshold value and an early warning mechanism, the detection module monitors the material level in real time, and the control module generates a discharging strategy according to an early warning signal. The accuracy of early warning is improved.

Description

Technical Field

[0001] The present invention relates to the field of early warning, and in particular to a material level height early warning system and method based on point cloud and vision linkage. Background Art

[0002] In bulk cargo operations, silos are important facilities for storing and loading and unloading bulk materials. In order to ensure the safe and efficient operation of silos, it is necessary to monitor and warn the material level height of the materials in the silo in real time. At present, material level detection mainly uses single sensor technology, such as ultrasound, radar, etc. These methods transmit sound waves or electromagnetic waves to the surface of the pile, receive reflected echoes, and calculate the material level height based on the round-trip time of the signal. However, in practical applications, the above traditional methods have certain limitations. The pile surface of bulk materials is usually irregular in shape, and the dust is flying seriously, which can easily interfere with and block the ultrasonic or radar signals, resulting in reduced detection accuracy. At the same time, the physical properties of different materials vary greatly, and a single sensor is difficult to adapt to the changing detection environment, and it is easy to have false alarms or missed alarms. Summary of the invention

[0003] The present application provides a material level height warning system and method based on point cloud and vision linkage to improve the accuracy of the warning.

[0004] In the first aspect, the present application provides a material level height warning system based on point cloud and visual linkage, the system comprising: a point cloud scanning module, a visual recognition module, a data processing module, a material level height detection strategy module, a detection module and a control module; Point cloud scanning module, used to obtain target three-dimensional point cloud data of the silo and surrounding environment through a laser scanner; The visual recognition module is used to collect silo images using a camera, identify material types through image processing and machine learning algorithms, and obtain material type data; The data processing module is used to perform point cloud segmentation based on the target three-dimensional point cloud data to obtain the location information of the silo and the rectangular frame of the silo opening; The material level height detection strategy module is used to determine the material level height threshold based on the material type data, and formulate a detection strategy based on the material level height threshold, the silo location information and the silo opening rectangular frame to obtain the material level height detection strategy; The detection module is used to detect the material level according to the material level detection strategy, generate material level data, and generate an early warning signal when the material level data exceeds a threshold; The control module is used to generate a discharge control strategy according to the early warning signal after receiving the early warning signal.

[0005] In the above technical solution, the point cloud scanning module uses a laser scanner to obtain the target three-dimensional point cloud data of the silo and the surrounding environment, providing high-precision spatial geometric information. By analyzing the three-dimensional point cloud data, the shape characteristics of the pile surface can be accurately depicted, overcoming the detection error problem caused by the irregular surface of the pile in the traditional method, and providing a solid data foundation for the accuracy of the early warning.

[0006] The visual recognition module collects silo images through the camera, and uses image processing and machine learning algorithms to identify material types and obtain material type data. Material type information provides important semantic constraints for material level detection, enabling the system to dynamically adjust detection strategies and thresholds based on the characteristics of different material types. By introducing material type information into early warning decisions, false alarms and missed alarms caused by material type changes are effectively reduced, and the adaptability and accuracy of early warnings are improved.

[0007] The data processing module segments the point cloud scanning data and extracts the key silo location information and the silo opening rectangular frame. Through fine-grained segmentation and semantic extraction of three-dimensional spatial information, the data processing module provides accurate location references for the formulation of subsequent detection strategies, ensuring the spatial positioning accuracy of the detection and warning process, and further improving the reliability of the warning results.

[0008] The material level detection strategy module comprehensively utilizes multi-source information such as material type data, silo location information, and silo opening rectangular frame to adaptively determine the material level threshold and develop an optimized detection strategy. By deeply integrating semantic information and spatial information, the module can generate a highly targeted and reliable material level detection strategy, providing a reliable decision-making basis for real-time monitoring and early warning, greatly improving the accuracy and credibility of early warning.

[0009] In a second aspect of the present application, a material level height warning method based on point cloud and visual linkage is provided, the method comprising: Use laser scanner to obtain target 3D point cloud data of the silo and surrounding environment; Use the camera to collect silo images, identify material types through image processing and machine learning algorithms, and obtain material type data; Perform point cloud segmentation based on the target 3D point cloud data to obtain the silo location information and the silo opening rectangular frame; Determine the material level height threshold based on the material type data, and formulate a detection strategy based on the material level height threshold, the silo location information, and the silo opening rectangular frame to obtain the material level height detection strategy; Detect the material level according to the material level detection strategy, generate material level data, and generate an early warning signal when the material level data exceeds the threshold; When the early warning signal is received, a discharge control strategy is generated according to the early warning signal.

[0010] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. This application uses a laser scanner to obtain the target three-dimensional point cloud data of the silo and the surrounding environment through the point cloud scanning module, providing high-precision spatial geometric information. By analyzing the three-dimensional point cloud data, the shape characteristics of the pile surface can be accurately depicted, overcoming the detection error problem caused by the irregular surface of the pile in the traditional method, and providing a solid data foundation for the accuracy of the early warning.

[0011] 2. The visual recognition module of this application collects silo images through a camera, and uses image processing and machine learning algorithms to identify material types and obtain material type data. Material type information provides important semantic constraints for material level detection, enabling the system to dynamically adjust detection strategies and thresholds based on the characteristics of different material types. By introducing material type information into early warning decisions, false alarms and missed alarms caused by changes in material types are effectively reduced, and the adaptability and accuracy of early warnings are improved.

[0012] 2. This application uses the data processing module to segment and process the point cloud scanning data to extract key silo location information and the silo opening rectangular frame. Through fine-grained segmentation and semantic extraction of three-dimensional spatial information, the data processing module provides accurate location references for the formulation of subsequent detection strategies, ensuring the spatial positioning accuracy of the detection and warning process, and further improving the reliability of the warning results. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 An architectural diagram of a material level height warning system based on point cloud and visual linkage provided in an embodiment of the present application; Figure 2 An architectural diagram of a point cloud scanning module provided in an embodiment of the present application; Figure 3 A schematic diagram of a rectangular frame of a bin opening provided in this application; Figure 4 This application provides a material level height warning method based on point cloud and visual linkage. DETAILED DESCRIPTION

[0014] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0015] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0016] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0017] In order to facilitate the understanding of the method and system provided by the embodiments of the present application, before introducing the embodiments of the present application, the background of the embodiments of the present application is first introduced. In the operation and management of modern bulk cargo, the safe and efficient operation of the silo as the core facility for storing and loading and unloading bulk materials is of vital importance. In order to realize the intelligent management of the silo, real-time monitoring and early warning of the material level height in the silo has become a key technical requirement. However, the current methods used for material level height detection mainly rely on single sensor technology, such as ultrasound, radar, etc. The basic principle of these traditional methods is to emit sound waves or electromagnetic waves to the surface of the pile, receive the reflected echo, and calculate the material level height based on the round-trip time of the signal.

[0018] Although the above methods have achieved the measurement of material level height to a certain extent, they have some limitations that cannot be ignored in the actual port operation environment. First, the accumulation form of bulk materials in the silo is usually complex and changeable, and the surface shape of the pile is irregular, which has a significant impact on the reflection and reception of the sensor signal. Secondly, during the loading and unloading operation, the material dust is seriously flying, and the dust cloud formed is easy to interfere with and block the ultrasonic or radar signal, resulting in a decrease in detection accuracy. In addition, different types of bulk materials have significant differences in physical properties, such as particle size, bulk density, moisture content, etc., which makes it difficult for a single sensor to adapt to the changing detection environment and prone to false alarms or missed alarms.

[0019] After the background introduction of the above content, those skilled in the art can understand the problems existing in the prior art. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0020] Based on the above background technology, please refer to Figure 1 , Figure 1 The architecture diagram of a material level height warning system based on point cloud and visual linkage provided in the embodiment of the present application, the system can be implemented by a computer program, and can also be run as an independent tool application. Specifically, in the embodiment of the present application, the method can be applied on a server, but can also be applied to electronic devices such as servers. A material level height warning system based on point cloud and visual linkage includes: a point cloud scanning module 1, a visual recognition module 2, a data processing module 3, a material level height detection strategy module 4, a detection module 5 and a control module 6; Point cloud scanning module 1, used to obtain target three-dimensional point cloud data of the silo and the surrounding environment through a laser scanner; Specifically, obtaining three-dimensional point cloud data through laser scanners has the following advantages: First, laser scanning technology can quickly and efficiently obtain three-dimensional information of a large range of space. Compared with traditional contact measurement methods, it has the characteristics of non-contact and high degree of automation, which can significantly improve data acquisition efficiency; secondly, laser scanners have high measurement accuracy, millimeter-level ranging resolution and angle resolution, which can depict the fine structure of the silo space in detail and provide a high-quality data foundation for subsequent point cloud processing; thirdly, laser scanners obtain spatial information by actively emitting laser beams, which can effectively overcome the influence of complex environmental factors such as dust interference and occlusion, and show good adaptability and robustness in the harsh working environment of bulk cargo.

[0021] After acquiring the target three-dimensional point cloud data, the point cloud scanning module transmits the original point cloud data to the data processing module for preprocessing and feature extraction. The data processing module first performs preprocessing operations such as denoising and filtering on the original point cloud data to remove abnormal points and outliers, thereby improving the quality and reliability of the point cloud data. Then, the preprocessed point cloud data is divided into regions using the point cloud segmentation algorithm, and the point cloud data is divided into different sub-regions such as the silo area, the pile area, and the ground area according to the spatial geometric features and semantic information. On this basis, the data processing module further extracts key features such as the silo location information and the rectangular frame of the silo opening, providing accurate spatial references for the subsequent formulation of material level height detection strategies.

[0022] Through point cloud segmentation and feature extraction, the data processing module achieves fine-grained understanding and semantic description of three-dimensional spatial information, converts raw point cloud data into structured spatial information, and lays a solid data foundation for intelligent material level detection. Compared with traditional two-dimensional image analysis methods, spatial information processing based on three-dimensional point clouds can more comprehensively and accurately depict the geometric characteristics of the silo space, has stronger adaptability and robustness for complex and irregular material pile surfaces, and effectively overcomes the influence of material pile shape changes on detection accuracy.

[0023] Based on the above embodiment, as an optional embodiment, please refer to Figure 2 The point cloud scanning module comprises: a laser scanning unit 11, a point cloud preprocessing unit 12 and a coordinate conversion unit 13; The laser scanning unit 11 is used to obtain preliminary three-dimensional point cloud data of the silo and the surrounding environment by emitting laser beams to the silo and the surrounding environment through a laser scanner and receiving the returned reflected signals; Specifically, the laser scanning unit 11 is the data acquisition core of the point cloud scanning module, and its main task is to perform all-round, high-precision three-dimensional measurement of the silo and the surrounding environment through a laser scanner. Specifically, the laser scanning unit 11 uses a laser scanner to emit a beam of high-energy, narrow-pulse laser, and through rotation or swinging, the laser beam quickly scans the surface of the silo and the surrounding environment to form a dense dot matrix. When the laser beam is irradiated on the surface of an object, reflection will occur, and the reflected signal carries rich spatial position and distance information. The receiver of the laser scanner captures these reflected signals and calculates the three-dimensional coordinates of each sampling point through a ranging algorithm, thereby obtaining preliminary three-dimensional point cloud data of the silo and the surrounding environment.

[0024] The main reason why the laser scanning unit 11 adopts laser scanning technology is that laser scanning has the advantages of non-contact, high precision, high density, and high efficiency. Compared with the traditional contact measurement method, laser scanning does not need to directly contact the object to be measured, avoiding interference and damage to the silo; at the same time, laser scanning can obtain a large amount of three-dimensional data in a short time, meeting the timeliness requirements of the operation. In addition, the measurement accuracy of the laser scanner can reach the millimeter level, which can truly reflect the subtle changes in the material pile and provide a high-quality data basis for subsequent material level height detection.

[0025] The point cloud preprocessing unit 12 is used to preprocess the preliminary three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data; Specifically, the point cloud preprocessing unit 12, as an important component of the point cloud scanning module, is responsible for preprocessing the preliminary three-dimensional point cloud data, aiming to improve the data quality and lay the foundation for subsequent coordinate transformation, feature extraction and material level height detection.

[0026] The point cloud preprocessing unit 12 is indispensable for the following reasons: First, the preliminary point cloud data acquired by the laser scanning unit 11 often has defects and interferences to varying degrees, such as noise points, outliers, uneven distribution, etc. These problems will affect the accuracy and reliability of the point cloud data, and thus affect the accuracy of material level detection. Secondly, the amount of raw point cloud data is usually very large, and direct processing and analysis will consume a lot of computing resources and time, reducing the real-time performance and efficiency of the system. Thirdly, point cloud data acquired in different regions and at different times may have differences in scale, density, coordinate system, etc., and need to be unified and standardized for data fusion and comparison.

[0027] To address the above problems, the point cloud preprocessing unit 12 uses a series of algorithms and strategies to optimize and refine the preliminary three-dimensional point cloud data in multiple dimensions, including the following steps: 1. Noise removal: The point cloud preprocessing unit 12 first uses algorithms such as statistical filtering and radius filtering to remove high-frequency noise and random errors in the point cloud data, thereby improving the signal-to-noise ratio and smoothness of the point cloud.

[0028] 2. Outlier removal: The point cloud preprocessing unit 12 identifies and removes outliers and abnormal points in the point cloud through indicators such as distance threshold and density threshold, and eliminates erroneous data introduced by factors such as occlusion and reflection.

[0029] 3. Downsampling: The point cloud preprocessing unit 12 uses voxel gridding, uniform sampling and other methods to perform downsampling on the point cloud data, which greatly reduces the amount of data and improves the efficiency of subsequent processing while maintaining the overall shape of the point cloud.

[0030] 4. Homogenization: The point cloud preprocessing unit 12 adjusts and optimizes the density distribution of the point cloud through interpolation, resampling and other technologies, making the point cloud more uniform in space and reducing sparse and redundant areas.

[0031] 5. Scale normalization: The point cloud preprocessing unit 12 unifies the scales of point cloud data acquired in different regions and at different times, eliminating scale differences caused by factors such as scanning distance and resolution, and facilitating the registration and fusion of point clouds.

[0032] After the above preprocessing steps, the point cloud preprocessing unit 12 converts the preliminary three-dimensional point cloud data into preprocessed point cloud data with higher quality, better data volume and more uniform distribution. These optimized point cloud data can not only more accurately and completely reflect the three-dimensional spatial information of the silo, but also greatly reduce the calculation burden of subsequent coordinate conversion, feature extraction and other links, and improve the real-time performance and stability of the system.

[0033] The coordinate conversion unit 13 is used to convert the pre-processed three-dimensional point cloud data from the local coordinate system of the scanner to the global unified coordinate system to obtain target three-dimensional point cloud data.

[0034] Specifically, the coordinate conversion unit 13 is indispensable for the following reasons: First, when the laser scanning unit 11 scans the silo at different times and locations, the local coordinate systems of the scanner used are often different, resulting in the lack of consistency in the spatial position and direction of the acquired point cloud data, which cannot be directly compared and fused. Secondly, the spatial position, dimensional parameters and material level height detection thresholds of the silo are usually defined in a global unified coordinate system. To achieve accurate material level height detection and overflow warning, the point cloud data must be unified in the same coordinate system for processing and analysis. Thirdly, coordinate conversion can eliminate the position error introduced by the scanner's own posture changes, improve the absolute accuracy of the point cloud data, and provide a more reliable spatial reference for subsequent feature extraction and semantic segmentation.

[0035] In order to achieve high-precision conversion from the scanner local coordinate system to the global unified coordinate system, the coordinate conversion unit 13 adopts the following key technologies: 1. Coordinate system calibration: The coordinate conversion unit 13 arranges a set of high-precision control points around the silo, and obtains the precise three-dimensional coordinates of these control points in the global unified coordinate system through measuring equipment such as a total station. Then, the feature matching algorithm is used to identify the feature points corresponding to the control points in the point cloud data, and establish the spatial transformation relationship between the scanner's local coordinate system and the global unified coordinate system.

[0036] 2. Rigid body transformation: The coordinate transformation unit 13 performs rigid body transformations such as translation and rotation on the pre-processed 3D point cloud data according to the spatial transformation relationship, and transforms it from the scanner local coordinate system to the global unified coordinate system. The rigid body transformation keeps the shape and topological structure of the point cloud data unchanged, and only changes its spatial position and direction.

[0037] 3. Error compensation: The coordinate transformation unit 13 takes into account factors such as the scanner's own posture changes and ranging errors, and introduces an error compensation model based on the rigid body transformation. Through methods such as polynomial fitting and neural networks, the converted point cloud data is refined and corrected to further improve the accuracy of coordinate transformation.

[0038] 4. Data optimization: The coordinate conversion unit 13 performs optimization processing such as deduplication and filtering on the converted point cloud data, removes redundant points and abnormal points introduced by the coordinate conversion process, and ensures the cleanliness and consistency of the generated target three-dimensional point cloud data.

[0039] After being processed by the coordinate conversion unit 13, the pre-processed three-dimensional point cloud data is uniformly converted to a global unified coordinate system to form high-precision, high-consistency target three-dimensional point cloud data. These point cloud data not only achieve seamless connection of data from different periods and different perspectives in spatial position and direction, but also significantly improve their absolute accuracy, laying a solid data foundation for subsequent applications such as material level detection and change analysis.

[0040] Visual recognition module 2, used to collect silo images using a camera, identify material types through image processing and machine learning algorithms, and obtain material type data; Specifically, point cloud segmentation is one of the core tasks of the data processing module. It divides the point cloud data into different sub-regions by spatial clustering and regional division. Each sub-region corresponds to a semantic entity in the silo space, such as a pile of materials, a silo wall, the ground, etc. In the process of point cloud segmentation, the data processing module comprehensively utilizes the geometric features of the point cloud (such as normal vectors, curvature, etc.) and spatial relationships (such as neighborhood relationships, topological structures, etc.) to achieve accurate segmentation of the point cloud through segmentation algorithms such as region growth, graph cuts, and deep learning. The segmented point cloud sub-regions have clear semantic identification, which provides a basis for subsequent spatial information extraction.

[0041] By analyzing the segmented point cloud sub-areas, the data processing module extracts the silo location information, that is, the spatial position and range of the silo in the point cloud coordinate system. By fitting and extracting the boundary of the silo wall area point cloud, the location, size and contour information of the silo can be accurately determined. This information provides a spatial reference for subsequent material level height detection, ensuring the position accuracy of the detection process.

[0042] In addition, the data processing module also extracts the rectangular frame information of the silo opening by analyzing the point cloud of the material pile area. The silo opening is a key feature of the material pile surface, and its position and shape are crucial to the material level detection. By performing plane fitting and edge extraction on the point cloud of the material pile area, the data processing module can accurately locate the position of the silo opening and fit the rectangular bounding box of the silo opening. The rectangular frame information provides important spatial constraints for the subsequent material level detection strategy formulation, ensuring that the detection process focuses on the key area of ​​the material pile.

[0043] Through point cloud segmentation and spatial information extraction, the data processing module realizes the semantic understanding and structured expression of the original point cloud data, and transforms the unstructured point cloud data into spatial information with clear semantics. This process fully mines the rich geometric and topological information contained in the point cloud data, laying a solid data foundation for the subsequent formulation of material level height detection strategies.

[0044] Based on the above embodiment, as an optional embodiment, the visual recognition module includes: an image acquisition unit, an image preprocessing unit and a material type recognition unit; An image acquisition unit, used for acquiring images of the silo using a camera; Specifically, the image acquisition unit, as the data source of the visual recognition module, has the main task of using the camera to collect real-time images of the silo. The image acquisition unit is introduced because there are many types of materials stored in the silo, and different materials have significant differences in visual characteristics such as color, shape, and texture. By analyzing and identifying the silo image, the specific type of material currently stacked in the silo can be accurately determined, providing key information for subsequent material level detection and early warning.

[0045] In practical applications, the image acquisition unit needs to reasonably select and arrange cameras based on factors such as the spatial size of the silo, lighting conditions, and material characteristics. Generally, the image acquisition unit uses an industrial-grade high-resolution digital camera and installs it in a fixed position above or on the side of the silo to obtain clear and comprehensive silo images. At the same time, in order to cope with complex environments such as insufficient light and dust obstruction inside the silo, the image acquisition unit will also be equipped with auxiliary equipment such as high-brightness LED fill lights and automatic cleaning devices to ensure the quality and stability of image data.

[0046] During the image acquisition process, the image acquisition unit automatically completes the image capture and storage according to the preset time interval and trigger conditions. In order to improve the efficiency of data transmission and processing, the image acquisition unit usually uses a high-speed industrial Ethernet interface to transmit the collected image data in real time to the image preprocessing unit and material type recognition unit for subsequent analysis and recognition.

[0047] Although the raw image data acquired by the image acquisition unit contains rich visual information, it is inevitably affected by factors such as illumination changes, viewing angle deviation, and image noise, and requires necessary preprocessing and optimization to improve image quality and recognition accuracy. Therefore, the introduction of the image preprocessing unit is very necessary. It converts the raw image data into standardized and normalized image features through a series of image processing algorithms, laying the foundation for the efficient operation of the material type recognition unit.

[0048] An image preprocessing unit, used for preprocessing the silo image to obtain a preprocessed silo image; Specifically, the main reasons for introducing the image preprocessing unit are as follows: First, the original silo images acquired by the image acquisition unit often have quality defects to varying degrees, such as uneven lighting, low contrast, blurred images, etc. These problems will directly affect the accuracy and reliability of material type recognition. Secondly, due to the complexity and variability of the silo environment, images acquired at different times and different viewing angles may have differences in size, color, direction, etc., and need to be unified and standardized to facilitate the extraction and comparison of image features. Thirdly, the original image data often contains a large amount of background information that is not related to material type recognition, such as silo walls, lighting equipment, etc. These redundant information will increase the computational complexity of the subsequent recognition algorithm and reduce recognition efficiency.

[0049] To address the above issues, the image preprocessing unit uses a series of image processing algorithms and strategies to optimize and enhance the silo image in many aspects, including the following steps: 1. Image correction: The image preprocessing unit uses the camera calibration parameters and distortion model to perform geometric correction on the silo image, eliminate the image distortion and deformation caused by the camera lens distortion, and restore the spatial geometric relationship of the image.

[0050] 2. Image enhancement: The image preprocessing unit adaptively adjusts the brightness distribution and contrast of the silo image through algorithms such as histogram equalization and contrast stretching, highlights the texture and edge features of the material in the image, and improves the visual quality and information content of the image.

[0051] 3. Image filtering: The image preprocessing unit uses Gaussian filtering, median filtering and other methods to suppress high-frequency noise and interference information in the silo image, smooth the image texture, and make the material area more uniform and continuous.

[0052] 4. Image segmentation: The image preprocessing unit uses threshold segmentation, region growing and other algorithms to segment the silo image into material area and background area, remove background information that is not related to material type identification, and reduce the amount of data and computational complexity of subsequent processing.

[0053] 5. Image normalization: The image preprocessing unit performs size normalization and color standardization on the silo image, unifying images from different sources and formats into a fixed size and color space to facilitate subsequent feature extraction and classification recognition.

[0054] After the above preprocessing steps, the image preprocessing unit converts the original silo image into a preprocessed image with higher quality, more obvious features and less interference. These optimized and standardized image data can not only reflect the visual characteristics of the materials in the silo more realistically and accurately, but also significantly reduce the computational complexity of subsequent material type recognition and improve the real-time and robustness of recognition.

[0055] The material type recognition unit is used to recognize the material type of the preprocessed silo image to obtain material type data.

[0056] Specifically, in order to achieve high-precision and high-real-time material type identification, the material type identification unit adopts the following key technologies: 1. Feature extraction: The material type recognition unit uses algorithms such as scale-invariant feature transform (SIFT) and histogram of oriented gradients (HOG) to extract key points and descriptors that can characterize the visual features of materials from the preprocessed silo images, construct a compact and robust feature representation, and lay the foundation for subsequent type classification.

[0057] 2. Deep learning: The material type recognition unit uses deep learning models such as convolutional neural networks (CNN) to perform end-to-end learning and classification of extracted image features. Through offline training of large-scale material image data, the CNN model can automatically learn and extract the essential characteristics of material types, and quickly and accurately determine the material category to which the current silo image belongs during online recognition.

[0058] 3. Multi-perspective fusion: The material type recognition unit comprehensively utilizes the multiple camera perspectives of the silo to perform multi-perspective feature extraction and classification recognition on the same material area, and optimizes the combination of recognition results from multiple perspectives through decision-level fusion algorithms, such as weighted voting and Bayesian fusion, to improve the accuracy and robustness of material type recognition.

[0059] 4. Incremental learning: The material type recognition unit introduces an incremental learning mechanism. By continuously accumulating and learning newly collected silo image data, the CNN model is updated and optimized online, so that it can adaptively adapt to the dynamic changes of material types in the silo and maintain high recognition accuracy and real-time performance.

[0060] Through the above technical means, the material type recognition unit can quickly and accurately identify the type of material currently stacked in the silo from the pre-processed silo image and generate real-time and dynamic material type data. These high-quality material type information provides important contextual basis for the material level detection module, enabling it to intelligently select and optimize the material level detection algorithm and threshold according to the stacking characteristics of different materials, thereby significantly improving the accuracy and reliability of the material level warning.

[0061] Data processing module 3, used to perform point cloud segmentation based on the target three-dimensional point cloud data to obtain the silo location information and the silo opening rectangular frame; Specifically, the formulation process of the material level height detection strategy is divided into two key steps: material level height threshold determination and detection strategy optimization. When determining the material level height threshold, the material level height detection strategy module fully considers the impact of material type on threshold selection. Due to differences in physical properties such as stacking density and natural repose angle, different materials have different safe material level height thresholds. By introducing material type data, the material level height detection strategy module can select appropriate thresholds from a pre-built threshold database for different materials, or dynamically generate thresholds through machine learning algorithms. This adaptive threshold determination method can effectively improve the sensitivity and accuracy of early warnings and reduce the risk of missed reports and false alarms.

[0062] On the basis of determining the material level height threshold, the material level height detection strategy module further optimizes the detection strategy. On the one hand, the module uses the silo location information and the silo opening rectangular frame information to accurately locate and constrain the detection area. By limiting the detection area to key parts of the material pile, such as the area around the silo opening, interference from other areas can be effectively reduced, and detection efficiency and accuracy can be improved. On the other hand, the module dynamically adjusts the parameters and strategies of the detection algorithm according to the characteristics of the material type. For example, for materials with large particles and loose stacking, a volume-based detection method can be used, while for materials with fine particles and dense stacking, a height-based detection method can be used. Through adaptive strategy selection and parameter optimization, the material level height detection strategy module can generate the optimal detection strategy for different types of materials to ensure the reliability and effectiveness of detection and early warning.

[0063] The optimization strategy generated by the material level height detection strategy module provides a key decision-making basis for subsequent real-time material level height detection. Based on this strategy, the detection module can select detection algorithms and parameters in a targeted manner, analyze the three-dimensional point cloud data of the material pile area, and accurately calculate the material level height value. When the material level height is detected to exceed the threshold, the system will promptly issue an early warning signal and trigger corresponding control measures, such as stopping feeding and starting unloading, to ensure the safety of the silo.

[0064] Based on the above embodiment, as an optional embodiment, the data processing module includes: a point cloud segmentation unit, a silo location extraction unit and a silo material opening identification unit; A point cloud segmentation unit is used to segment the target three-dimensional point cloud data using a clustering algorithm to obtain the silo location data; Specifically, point cloud segmentation is the first key link in data processing. Due to the huge amount of original point cloud data and the lack of structure, it is very difficult to directly extract the location of the silo from it. To this end, the point cloud segmentation unit uses a clustering algorithm to automatically classify the scattered point cloud data with spatial proximity and similar attributes together to form several independent point cloud clusters, each of which corresponds to an independent spatial entity. This similarity-based data reorganization process can quickly find the interface between the silo point cloud area and the point cloud of other objects, and automatically outline the general outline of the silo location. The implementation strategy of the clustering algorithm includes steps such as calculating the point cloud spatial distance matrix, constructing a point cloud topology map, iteratively optimizing the point cloud clusters, and fusing the semantic information of the point cloud. Through the quantitative analysis of spatial adjacency and attribute similarity, the original massive point cloud is reorganized into several point cloud clusters belonging to different physical objects, among which the silo point cloud area is initially separated. This bottom-up, local-to-global data organization form makes it possible to directly extract the location characteristics of the silo from the point cloud, greatly reducing the computational complexity of spatial positioning. At the same time, the independent point cloud clusters generated by segmentation also provide a data basis for silo morphology analysis and feature extraction.

[0065] A silo location extraction unit is used to extract spatial location information from silo location data to obtain silo location information; Specifically, the silo material opening recognition unit first trains an end-to-end silo material opening detection model based on the silo location data. The model adopts a deep convolutional neural network with an encoder-decoder architecture, takes the silo point cloud projection image as input, and directly outputs the probability that each pixel belongs to the silo material opening area through multi-scale feature extraction and cross-level feature fusion. Compared with the traditional two-stage detection method, this end-to-end model eliminates the tedious steps of candidate area generation and post-processing, greatly improving the real-time and robustness of silo material opening recognition. During model training, the silo material opening recognition unit collects a large amount of silo point cloud data of different material types and different silo types, and generates pixel-level silo material opening segmentation truth values ​​through manual annotation. Through the learning of massive data, the convolutional neural network has mastered the discriminative features of the silo material opening area on the point cloud image, such as texture and shape. The trained detection model has a strong generalization ability and can adapt to complex working conditions such as different shooting angles, lighting conditions, and partial occlusion.

[0066] The silo material opening identification unit is used to identify the silo material opening based on the silo location data to obtain the silo material opening position data, and to perform rectangular frame fitting based on the silo material opening position data to obtain a silo material opening rectangular frame.

[0067] For details, please refer to Figure 3, the bunker opening identification unit first takes the center of the bunker opening rectangular frame as the origin, and draws three concentric rectangular frames a, b, and c of different sizes to cover different parts of the bunker opening area. Taking into account the differences in the bulk cargo stacking morphology, the bunker opening identification unit sets different material level height thresholds for the three areas a, b, and c. Usually, the threshold of the central area a is the highest, followed by the peripheral area b, and the threshold of the outermost area c is the lowest. This differentiated threshold design is adapted to the natural morphology of bulk cargo stacking, taking into account the general rule that the material level in the central area is higher, and reserves buffer space for changes in the material level in the peripheral areas.

[0068] After determining the detection area and threshold, the silo material opening recognition unit further extracts the height information in each area from the silo's three-dimensional point cloud. Through the coordinate transformation and spatial division of the point cloud data, the point cloud height values ​​falling into the three rectangular boxes a, b, and c are extracted respectively, and the median of the height value of each area is calculated as the representative value of the material level height in the area. The median statistic has good robustness, can suppress the influence of noise and outliers, and objectively reflect the true height level of bulk cargo accumulation.

[0069] The bin port identification unit compares the representative value of the material level height in each area with the preset threshold to achieve real-time material level status monitoring and early warning. When the material level height in any area is detected to exceed the preset threshold, the bin port identification unit determines that the material level is abnormal and sends the alarm information to the control module in time, prompting the staff to take corresponding measures, such as suspending material feeding, speeding up material discharge, etc. Through real-time monitoring and early warning, the occurrence of unexpected events such as bin port blockage is effectively avoided, ensuring safe and smooth operation.

[0070] Taking into account the large differences in the accumulation and flow characteristics of different types of goods at the silo, the silo identification unit has also formulated differentiated detection strategies. For example, for goods with good fluidity, such as sand and gravel, the silo identification unit appropriately lowers the material level thresholds of the three areas to achieve a more sensitive early warning response; while for goods with poor fluidity, such as seeds, the silo identification unit appropriately increases the thresholds to reduce overly frequent alarms. In addition, the silo identification unit also dynamically adjusts the size ratios of the three areas a, b, and c according to the stacking angle characteristics of the goods to match the stacking forms of different goods. The formulation of differentiated detection strategies requires comprehensive consideration of multiple factors such as the physical characteristics of the goods and the operating process parameters. Through a large amount of measured data analysis and expert experience summary, the strategy is continuously optimized and improved in terms of refinement, personalization, and adaptability.

[0071] The material level height detection strategy module 4 is used to determine the material level height threshold based on the material type data, and formulate a detection strategy based on the material level height threshold, the silo location information and the silo opening rectangular frame to obtain the material level height detection strategy; Specifically, the detection module receives the optimized detection strategy from the material level height detection strategy module, and analyzes and calculates the three-dimensional point cloud data output by the data processing module according to the strategy to obtain an accurate material level height value.

[0072] The workflow of the detection module is as follows: First, the detection module performs spatial division and regional interception of the input three-dimensional point cloud data according to the silo location information and the silo opening rectangular frame information in the material level height detection strategy. Through this step, the detection module can accurately locate the key area where the material pile is located, eliminate interference from other areas, and reduce the complexity and time overhead of subsequent calculations. Then, the detection module preprocesses the intercepted material pile area point cloud data, including downsampling, filtering, and outlier removal operations, to further improve the quality and reliability of the point cloud data. On this basis, the detection module analyzes and calculates the preprocessed point cloud data according to the detection algorithm and parameters specified in the material level height detection strategy. Commonly used material level height detection algorithms include methods based on height histograms, methods based on surface fitting, and methods based on volume calculation. By selecting a detection algorithm suitable for the current material type and silo status, the detection module can accurately and efficiently calculate the height value of the material pile surface.

[0073] Based on the above embodiment, as an optional embodiment, the material level detection strategy module includes: a threshold determination unit and a strategy generation unit; A threshold determination unit, used to determine a material level height threshold based on material type data; Specifically, as the primary component of the material level detection strategy module, the core task of the threshold determination unit is to intelligently determine the threshold of the material level warning based on the material type data. The main reasons for introducing the threshold determination unit are as follows: First, different types of materials have significant differences in physical properties such as stacking density and repose angle, and the corresponding critical material level heights are also different. It is difficult to take into account the warning needs of different materials by adopting a unified material level height threshold, which can easily lead to problems such as untimely warnings or frequent false alarms. Secondly, there are different requirements for the storage capacity and turnover efficiency of different types of materials. It is necessary to dynamically adjust the standard of material level height warning according to the importance and timeliness of the materials to balance safety and economy. Thirdly, the frequent changes in material types make the preset thresholds invalid quickly, and an adaptive threshold determination mechanism is needed to cope with the dynamic and uncertainty of silo use.

[0074] In order to achieve accurate and dynamic material level height threshold determination, the threshold determination unit adopts the following strategies and algorithms: 1. Material property analysis: The threshold determination unit establishes a mapping relationship between material type and critical material level height based on material type data and combined with material stacking characteristic parameters such as bulk density, internal friction angle, particle size distribution, etc., to provide a theoretical basis and empirical rules for threshold determination.

[0075] 2. Historical data mining: The threshold determination unit makes full use of the historical operation data of the silo, including material type, material level height, warning records, etc. Through data mining and statistical analysis, it summarizes the optimal warning thresholds and safety margins for different types of materials to form a data-driven model for threshold determination.

[0076] 3. Real-time threshold optimization: The threshold determination unit tracks the material level changes and warning feedback of the silo in real time. Through Bayesian optimization, reinforcement learning and other algorithms, it adjusts and optimizes the warning threshold online so that it can dynamically adapt to the actual situation of the current silo and seek the best balance between safety and efficiency.

[0077] 4. Adaptive threshold strategy: The threshold determination unit formulates a multi-level threshold strategy for different levels of warning requirements. For general material level warnings, a relatively loose threshold standard is adopted to reduce the false alarm rate; while for critical material level warnings, a more stringent threshold standard is adopted to improve the timeliness and reliability of the warning.

[0078] Through intelligent calculation and dynamic optimization of the threshold determination unit, the material level detection strategy module can adaptively generate the optimal material level warning threshold for different types of materials. These tailored warning thresholds can not only effectively reduce the warning error caused by differences in material characteristics and improve the accuracy of material level warnings, but also make intelligent trade-offs between safety and efficiency according to actual needs, maximizing the economic and social benefits of material level warnings.

[0079] The strategy generation unit is used to formulate a detection strategy based on the material level height threshold, the silo location information and the silo opening rectangular frame to obtain the material level height detection strategy.

[0080] Specifically, material level detection involves many factors such as the spatial layout of the silo, material port distribution, and material stacking form. It is difficult to take into account the performance requirements under different working conditions by adopting a fixed and unified detection strategy, which can easily cause problems such as detection blind spots, precision loss, and computational waste. Therefore, it is imperative to dynamically and intelligently generate detection strategies based on the actual situation of the silo. Secondly, the complexity and variability of the production environment makes material level detection face many challenges such as false alarms, missed alarms, and lack of real-time performance. A flexible and robust strategy generation mechanism is needed to adaptively weigh various performance indicators to ensure the reliable operation of the system. Thirdly, the management personnel's needs for material level warnings are constantly changing, such as monitoring frequency, warning level, and regional division. The strategy generation unit needs to be able to respond to manual adjustment instructions and dynamically modify and optimize the detection strategy to meet the user's personalized needs.

[0081] In order to achieve accurate, efficient and flexible material level detection strategy generation, the strategy generation unit adopts the following strategies and algorithms: 1. Spatial perception and segmentation: The strategy generation unit makes full use of the location information of the silo, performs semantic segmentation and region of interest extraction on the three-dimensional space of the silo, automatically identifies the core area related to material level detection, and filters out irrelevant background such as the sky, ground, and obstacles, thereby focusing on the detection target and reducing the waste of computing resources.

[0082] 2. Material opening positioning and tracking: The strategy generation unit tracks and locates the main material openings and discharge openings in the silo in real time based on the rectangular frame data of the silo material opening, automatically generates a local coordinate system centered on the material opening, and converts the spatial reference of the material level height detection from the global coordinate system to the material opening coordinate system, thereby improving the pertinence and accuracy of the detection.

[0083] 3. Adaptive threshold mapping: The strategy generation unit maps the material level height threshold given by the threshold determination unit to the three-dimensional space of the silo and the local coordinate system of the material port, forming a spatial threshold surface and cutoff plane corresponding to the specific silo and material port, which is used to guide the threshold judgment and early warning triggering of the material level height detection module.

[0084] 4. Multi-objective trade-off optimization: The strategy generation unit comprehensively considers multiple performance indicators such as material level detection accuracy, real-time, stability, and computational efficiency, and establishes a multi-objective optimization model with detection accuracy, missed alarm rate, false alarm rate, response delay, resource occupancy, etc. as goals. Through intelligent search and decision-making algorithms, the optimal detection strategy parameter combination that takes into account various indicators is solved.

[0085] 5. Dynamic adjustment of strategies: The strategy generation unit responds to the control instructions and feedback from managers and dynamically adjusts the key parameters of the detection strategy, such as the shape and scale of the threshold surface, the detection frequency, the level of warning, the size and location of the area of ​​interest, etc., so that the detection strategy can flexibly adapt to changes in demand and always maintain the optimal system performance.

[0086] Through the intelligent, adaptive, and dynamic control of the strategy generation unit, a set of tailor-made material level height detection strategies is finally obtained. This strategy fully integrates environmental perception data such as material level height threshold, silo location information, and silo opening rectangular frame, and generates independent and detailed spatial threshold surfaces and regions of interest for each silo and opening. At the same time, the strategy achieves the best balance between multiple performance indicators such as accuracy, real-time, and computational efficiency, and can achieve dynamic optimization and adjustment according to manual control instructions. Based on the guidance of this intelligent strategy, the material level height detection module can stably, efficiently, and accurately implement material level height detection in a complex and dynamic environment, minimize the probability of risk events such as missed detection and false alarms, and improve the reliability and intelligence level of safety supervision.

[0087] The detection module 5 is used to detect the material level according to the material level detection strategy, generate material level data, and generate an early warning signal when the material level data exceeds a threshold value; Specifically, the control module receives the material level height data and warning signals from the detection module, and automatically generates control instructions according to preset control strategies and rules to coordinate the operation of the silo equipment to ensure that the material level height is always within a safe range.

[0088] The workflow of the control module is as follows: First, the control module receives and analyzes the material level data sent by the detection module in real time, and continuously monitors the material level status of the silo. When receiving the warning signal, the control module immediately starts the warning processing program, analyzes the severity and cause of the current material level exceeding the limit, and automatically generates corresponding control instructions according to the preset control strategies and rules.

[0089] When generating control instructions, the control module comprehensively considers multiple factors, such as the extent of the material level height exceeding the limit, the characteristics of the material type, the equipment status, etc., to formulate the optimal control strategy. For example, when the material level height exceeds the limit slightly, the control module may choose to reduce the feeding speed or suspend feeding to relieve the pressure on the pile; when the material level height exceeds the limit seriously, the control module may directly trigger the unloading operation to quickly transfer the materials in the silo to other safe areas; when the equipment failure causes the material level height to be abnormal, the control module will give priority to cutting off the power supply of related equipment to prevent the accident from expanding. The control module flexibly adjusts the control strategy according to the actual situation to ensure that normal operation is maintained to the maximum extent possible while ensuring safety.

[0090] After generating the control instructions, the control module transmits the instructions to the various execution units of the control system, such as feeders, unloaders, conveyors, etc. These execution units automatically adjust the operating parameters and states according to the received control instructions, and work together to complete the safety control of the silo. For example, the control module can control the material flow entering the silo by adjusting the feeding frequency and feeding amount of the feeder; and control the transfer and evacuation of materials by starting and stopping the unloader and conveyor. Through this automated control method, the material level of the silo can always be maintained below the safety threshold, effectively avoiding dangerous situations such as material overflow and silo damage.

[0091] The control module can not only implement emergency control according to the early warning signal, but also actively schedule and optimize the silo operation based on historical data and current status. By analyzing the changing trend of the material level height data, combined with factors such as production plan and equipment status, the control module can predict the material level height trend in the future, formulate control strategies in advance, and achieve smooth operation of the silo operation. In addition, the control module can also be interconnected with other management systems to achieve information sharing and linkage control. For example, the material level height data is shared with the planning and scheduling system to optimize the ship loading and unloading operation plan; the equipment status information is shared with the equipment management system to arrange equipment maintenance and overhaul in a timely manner.

[0092] Based on the above embodiment, as an optional embodiment, the detection module includes: a threshold determination module and a threshold judgment unit; A threshold determination module, used to determine a height threshold according to a material level height detection strategy; Specifically, in order to achieve accurate and dynamic height threshold determination, the threshold determination module adopts the following strategies and algorithms: Strategy analysis and mapping: The threshold determination module receives the detection strategy generated by the material level height detection strategy module, and extracts key information related to the height threshold through semantic analysis and logical mapping, such as material type, silo parameters, safety margin, warning level, etc., to provide a decision-making basis for threshold determination.

[0093] Knowledge base matching and reasoning: The threshold determination module matches and infers the extracted key information with the built-in security knowledge base, automatically obtains the standard threshold range, safety factor, warning level and other parameters under the corresponding material type and silo form, and forms the basic value range of the height threshold.

[0094] Data mining and learning: The threshold determination module fully mines historical operation data and uses machine learning algorithms, such as decision trees, support vector machines, neural networks, etc., to autonomously learn and summarize the optimal value rules of height thresholds under different working conditions from massive data, and dynamically correct and optimize the threshold range obtained by knowledge reasoning.

[0095] Adaptive threshold generation: Based on knowledge reasoning and data learning, the threshold determination module comprehensively considers the safety margin and warning level set by the management department, as well as the dynamic change trend of material accumulation, and adaptively generates an optimal height threshold that not only meets the objective requirements of safety management, but can also dynamically adapt to the current production scenario.

[0096] Collaboration at multiple time and space scales: The threshold determination module collaboratively generates a set of related height thresholds at different time scales (such as seconds, minutes, hours, etc.) and spatial scales (such as material ports, silos, material yards, etc.) based on the real-time and stability requirements of material level detection, forming a defense-in-depth threshold system to improve the reliability and flexibility of threshold judgment.

[0097] Through intelligent calculation and dynamic optimization of the threshold determination module, the detection module obtains an accurate, reliable, and adaptive height threshold, which fully absorbs the professional experience of the safety knowledge base and integrates the statistical laws of historical operation data. It can dynamically adapt to the adjustment and changes of the material level height detection strategy, which is crucial to improving the accuracy of subsequent threshold judgments and early warning decisions. At the same time, the coordination of thresholds at multiple time and space scales effectively avoids the one-sidedness and fragility of single threshold judgments, and provides strong guarantees for material level height warnings from multiple dimensions through three-dimensional threshold control across time and space.

[0098] The threshold judgment unit is used to compare the material level height data with the height threshold and generate an early warning signal when the material level height data exceeds the threshold.

[0099] Specifically, in order to achieve accurate and efficient threshold judgment and warning triggering, the threshold judgment unit adopts the following strategies and algorithms: Real-time data connection: The threshold judgment unit is seamlessly connected with the data acquisition system to obtain the material level height data collected by multiple source equipment such as hoppers, level meters, laser scanners, etc. in real time, ensuring that the data basis for judgment is always the latest and most complete.

[0100] Data cleaning and fusion: The threshold judgment unit performs intelligent cleaning and multi-source fusion processing on the collected material level height data, removes interference factors such as abnormal values ​​and noise, and improves the accuracy and reliability of the data. At the same time, it comprehensively utilizes the redundancy and complementary characteristics of multiple data sources to form a set of high-quality material level height data sets, providing a solid foundation for subsequent comparison and judgment.

[0101] Dynamic threshold matching: The threshold judgment unit receives the height threshold calculated by the threshold determination module in real time, and dynamically matches the corresponding multi-scale threshold according to the spatiotemporal attributes of the material level height data. This dynamic and sophisticated threshold matching mechanism can effectively cope with the dynamic adjustment of the threshold standard during the operation process, ensuring the real-time synchronization of the judgment standard and the data characteristics.

[0102] Multi-dimensional comparison decision: The threshold judgment unit comprehensively considers multiple key dimensions such as the absolute value, change rate, and duration of over-limit of the material level height data, and compares it with the corresponding height threshold in an all-round way to form a complete and rigorous judgment logic. When a certain dimension exceeds the threshold, it does not directly trigger an early warning. Instead, the final decision on whether to issue an early warning is made based on the comprehensive evaluation of the comparison results of each dimension, thereby improving the accuracy and robustness of the early warning judgment.

[0103] Generation and classification of early warning signals: When the material level height data is confirmed to exceed the threshold range, the threshold judgment unit immediately generates an early warning signal, and scientifically classifies the early warning signal according to factors such as the excess range, duration, and danger level, forming different levels of early warning prompts, such as warning, warning, alarm, danger, etc., to facilitate personnel to quickly understand and respond to dangerous situations.

[0104] The control module 6 is used to generate a discharge control strategy according to the warning signal after receiving the warning signal.

[0105] Specifically, first, the point cloud scanning module uses a laser scanner to obtain the three-dimensional point cloud data inside the silo, and comprehensively depicts the spatial distribution and morphological characteristics of the material pile; at the same time, the visual recognition module uses an industrial camera to collect video images of the silo surface, and uses an image recognition algorithm to detect the type and characteristics of the material. Next, the data processing module comprehensively utilizes the original data provided by the point cloud scanning module and the visual recognition module to perform data fusion and semantic extraction. Through the point cloud segmentation algorithm, the data processing module extracts the silo location information and the silo opening rectangular frame information from the point cloud data to determine the key areas and boundaries of the material pile; through the target detection and classification algorithm, the data processing module identifies the specific type of material from the video image and obtains the material type data.

[0106] Based on data processing, the material level height detection strategy module begins to play a role. The module first selects the appropriate material level height threshold from the preset threshold database based on the material type data, or uses the machine learning algorithm to dynamically generate the threshold to ensure that the threshold can reflect the safe stacking height of different materials. Then, the module combines the material level height threshold, the silo location information, and the silo opening rectangular frame information to formulate the optimal material level height detection strategy, clarify the detection area, method, and frequency, and provide a decision-making basis for subsequent real-time detection.

[0107] According to the guidance of the material level detection strategy, the detection module monitors the material level in the silo in real time. The detection module obtains the three-dimensional point cloud data from the data processing module, and calculates the real-time height of the material pile surface according to the key areas and detection algorithms determined by the material level detection strategy, and generates material level data. When the material level data exceeds the threshold, the detection module immediately sends an early warning signal to the control module to inform the abnormal material level.

[0108] On the basis of the above embodiment, as an optional embodiment, the material level height warning system based on point cloud and visual linkage also includes an intrusion alarm module; The intrusion alarm module is used to identify whether an intrusion object appears in the preset warning area in the silo image. When the intrusion object appears, an intrusion warning signal is issued, and a stop signal is generated to the operating equipment to control the operating equipment to stop operating.

[0109] Specifically, the core function of the intrusion alarm module is to monitor the silo and its surrounding areas in real time. Once a person or vehicle is found to have illegally invaded the preset warning area, an alarm will be immediately sounded and the gantry crane will be automatically stopped from operating, eliminating safety risks in the bud. The reason for setting up this module is mainly based on the following considerations: In the actual operation of the bulk gantry crane, a large amount of material often falls from high altitude when the hopper grabs and unloads goods. Once these scattered materials hit the personnel or vehicles in the operating area, it is very easy to cause serious safety accidents. Therefore, it is necessary to demarcate a safety warning zone around the hopper operating area, and strictly prohibit irrelevant personnel and equipment from entering, so as to fundamentally avoid the occurrence of such accidents.

[0110] The intrusion alarm module works specifically as follows: First, based on the movement trajectory of the hopper, the track characteristics of the scattered materials and other factors, a closed three-dimensional warning area is set in the silo and its surrounding area, and the spatial coordinate information of the area is input into the system. During the normal operation of the gantry crane, the intrusion alarm module continuously receives the real-time image data of the silo and the surrounding area collected by the visual recognition module, and performs real-time analysis and comparison of the occupancy of the corresponding area in the image based on the coordinate information of the preset warning area. Once an intruder (such as a person, vehicle, or debris) is found in the warning area within the silo image, the system will immediately issue an audible and visual alarm signal, and at the same time generate a stop signal and transmit it to the gantry crane control system to control the emergency braking of the hopper. The gantry crane can resume normal operation only after the intruder in the warning area is completely evacuated.

[0111] By integrating the intrusion alarm module into the material level height warning system, the present invention has established a "double insurance" mechanism for the safety control of bulk cargo gantry crane operations. On the one hand, the system uses advanced point cloud vision technology to perform real-time and dynamic material level height detection on the cargo hoisted by the hopper, and timely warns and adjusts the unloading strategy when the material level exceeds the limit to prevent the cargo from overflowing the silo and causing accidents; on the other hand, the system also monitors the illegal intrusion of personnel, vehicles, etc. at the operation site at all times, and kills the danger in the cradle through means such as alarm and linkage braking, weaving a dense and solid intelligent protection network for the life safety of front-line operators.

[0112] Based on the above embodiment, as an optional embodiment, the system further includes: a dirt alarm module; The dirt alarm module is used to generate corresponding alarm signals according to the degree of dirt.

[0113] In actual operations, not all bulk materials can be legally stored. Once certain dangerous and prohibited materials are mixed into the silo, they will not only contaminate other materials and cause serious cargo damage accidents, but are also more likely to induce secondary disasters such as fire, explosion, and poisoning that endanger personal and property safety. Traditional manual inspections make it difficult to identify incoming materials one by one, and there are large management and control blind spots. To solve this problem, this embodiment innovatively introduces material type identification into the material level height detection link to form a composite detection mechanism of material level and material properties. With the help of intelligent technologies such as computer vision and machine learning, it can automatically detect and warn of illegal intrusions of prohibited dangerous materials, thereby curbing safety hazards at the source.

[0114] In order to achieve accurate identification of material types and automatic warning, the detection module adopts the following strategies and algorithms: Material image acquisition: The detection module establishes a data link with the video surveillance system of the operation area to obtain high-definition camera image data near the material level detection point in real time. The original image is pre-processed by image processing algorithms such as enhancement, segmentation, and feature extraction to extract the local material image corresponding to each material point position, so as to prepare data for subsequent material type analysis.

[0115] Material type model construction: The detection module builds a set of fine-grained material type visual recognition models for common bulk materials, such as coal, ore, grain, chemical raw materials, etc. This model is based on a convolutional neural network. Through training and learning with a large number of labeled samples, it can automatically extract unique deep features from the visual features of the material such as color, texture, shape, etc., and form a stable material type discrimination capability.

[0116] Real-time material recognition and classification: The detection module inputs the collected material images into the trained recognition model, and obtains the probability distribution of the material type to which the image belongs through forward reasoning calculation. The material type with the highest probability is taken as the recognition result, and the recognition confidence score is given as the credibility of the type judgment. At the same time, the recognition results of multiple frames of images at the same material point are time-series fused to improve the stability of material type judgment.

[0117] Intrusion material comparison and warning: The detection module will match the real-time identified material type data with the system preset alarm material category library to determine whether the material is dangerous or prohibited. Once a suspected matching material type is found, an intrusion warning signal will be immediately generated, and the corresponding image evidence and hazard description will be attached and sent to the regulatory department for review and confirmation, and the on-site management personnel will be notified to take emergency measures.

[0118] Grading response of early warning signals: According to the degree of danger of intruding materials, identification confidence, intrusion volume and other factors, the early warning signals are scientifically graded to form early warning response mechanisms of different levels. For early warning signals with high danger and high confidence, the highest level of response process is initiated to strictly control the production order; for early warning signals with relatively low danger, conventional review and disposal processes are adopted to avoid excessive response and intervention.

[0119] Through the automatic material type identification and early warning function of the detection module, the safety of material storage will be raised to a new level. With accurate and real-time computer vision recognition technology, illegal intrusion of dangerous and prohibited materials can be intelligently and automatically discovered and warned, greatly reducing the difficulty of safety management.

[0120] Compared with simple height threshold warning, material type warning pays more attention to the substantial risk of materials, reflecting the refined safety concept of "taking measures according to materials". At the same time, this composite warning mechanism also greatly improves the comprehensiveness and effectiveness of safety warning.

[0121] On the other hand, the present invention also provides a material level height warning method based on point cloud and visual linkage, such as Figure 4 , the method comprising: S101, obtaining target three-dimensional point cloud data of the silo and the surrounding environment through a laser scanner; S102, using a camera to collect a silo image, identifying the material type through image processing and machine learning algorithms, and obtaining material type data; S103, performing point cloud segmentation based on the target three-dimensional point cloud data to obtain the location information of the silo and the rectangular frame of the silo opening; S104, determining a material level height threshold based on the material type data, and formulating a detection strategy based on the material level height threshold, the silo location information, and the silo opening rectangular frame to obtain a material level height detection strategy; S105, detecting the material level according to the material level detection strategy, generating material level data, and generating a warning signal when the material level data exceeds a threshold; S106, when the warning signal is received, a discharge control strategy is generated according to the warning signal.

[0122] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0123] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A material level height warning system based on point cloud and visual linkage, characterized in that: The system includes: a point cloud scanning module, a visual recognition module, a data processing module, a material level height detection strategy module, a detection module and a control module; The point cloud scanning module is used to obtain target three-dimensional point cloud data of the silo and the surrounding environment through a laser scanner; The visual recognition module is used to collect silo images using a camera, identify material types through image processing and machine learning algorithms, and obtain material type data; The data processing module is used to perform point cloud segmentation based on the target three-dimensional point cloud data to obtain the silo location information and the silo opening rectangular frame; The material level height detection strategy module is used to determine the material level height threshold based on the material type data, and formulate a detection strategy based on the material level height threshold, the silo location information and the silo opening rectangular frame to obtain a material level height detection strategy; The detection module is used to detect the material level according to the material level detection strategy, generate material level data, and generate an early warning signal when the material level data exceeds a threshold value; The control module is used to generate a discharge control strategy according to the warning signal after receiving the warning signal.

2. The system according to claim 1, characterized in that The point cloud scanning module includes: a laser scanning unit, a point cloud preprocessing unit and a coordinate conversion unit; The laser scanning unit is used to obtain preliminary three-dimensional point cloud data of the silo and its surrounding environment by emitting a laser beam to the silo and its surrounding environment through a laser scanner and receiving a returned reflected signal; The point cloud preprocessing unit is used to preprocess the preliminary three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data; The coordinate conversion unit is used to convert the preprocessed three-dimensional point cloud data from the scanner local coordinate system to the global unified coordinate system to obtain target three-dimensional point cloud data.

3. The system according to claim 1, characterized in that The data processing module includes: a point cloud segmentation unit, a silo location extraction unit and a silo opening identification unit; The point cloud segmentation unit is used to perform point cloud segmentation on the target three-dimensional point cloud data by using a clustering algorithm to obtain silo location data; The silo location extraction unit is used to extract spatial location information from the silo location data to obtain the silo location information; The silo material opening identification unit is used to perform silo opening identification based on the silo location data to obtain the silo material opening position data, and perform rectangular frame fitting based on the silo material opening position data to obtain the silo material opening rectangular frame.

4. The system according to claim 1, characterized in that The visual recognition module includes: an image acquisition unit, an image preprocessing unit and a material type recognition unit; The image acquisition unit is used to acquire the image of the silo using a camera; The image preprocessing unit is used to preprocess the silo image to obtain a preprocessed silo image; The material type identification unit is used to identify the material type of the preprocessed silo image to obtain the material type data.

5. The system according to claim 1, characterized in that The material level detection strategy module includes: a threshold determination unit and a strategy generation unit; The threshold determination unit is used to determine the material level height threshold based on the material type data; The strategy generating unit is used to formulate a detection strategy based on the material level height threshold, the silo location information and the silo opening rectangular frame to obtain the material level height detection strategy.

6. The system according to claim 1, characterized in that The detection module includes: a threshold determination module and a threshold judgment unit; The threshold determination module is used to determine the height threshold according to the material level height detection strategy; The threshold judgment unit is used to compare the material level height data with the height threshold, and generate the warning signal when the material level height data exceeds the threshold.

7. The system according to claim 1, characterized in that The system also includes an intrusion alarm module; The intrusion alarm module is used to identify whether an intrusion object appears in the preset warning area in the silo image. When the intrusion object appears, an intrusion warning signal is issued, and a stop signal is generated to the operating equipment to control the operating equipment to stop operating.

8. The system according to claim 1, characterized in that The system also includes: a scanner dirtiness alarm module; The scanner dirt alarm module is used to analyze the target three-dimensional point cloud data to obtain a dirt value, and determine the degree of dirt according to the dirt value and a preset dirt threshold.

9. The system according to claim 8, characterized in that The system also includes: a dirt alarm module; The dirt alarm module is used to generate a corresponding alarm signal according to the dirt degree.

10. A material level height warning method based on point cloud and visual linkage, characterized in that: The method comprises: Use laser scanner to obtain target 3D point cloud data of the silo and surrounding environment; Use the camera to collect silo images, identify material types through image processing and machine learning algorithms, and obtain material type data; Perform point cloud segmentation based on the target three-dimensional point cloud data to obtain the silo location information and the silo opening rectangular frame; Determine a material level height threshold based on the material type data, and formulate a detection strategy based on the material level height threshold, the silo location information, and the silo opening rectangular frame to obtain a material level height detection strategy; Detecting the material level according to the material level height detection strategy, generating material level height data, and generating an early warning signal when the material level height data exceeds a threshold value; After receiving the warning signal, a discharge control strategy is generated according to the warning signal.

Images