A machine vision-based intelligent identification system for blockages in ship loader chutes

By combining multispectral imaging and deep learning models with dynamic threshold adjustment, the problems of lag and high false alarm rate in ship loader chute blockage detection have been solved, achieving high-precision, real-time blockage identification that is adaptable to complex industrial environments.

CN123118432APending Publication Date: 2026-11-17JIANGSU XINGWANG LOGISTICS CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202610471907.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In the existing technology, the detection of blockage in the loader chute has problems such as detection lag, high false alarm rate and poor environmental adaptability. In particular, it is difficult to accurately identify minor blockage or early accumulation in dusty and uneven lighting environments.

Method used

By employing multispectral imaging technology combined with a lightweight deep learning model and a dynamic threshold adjustment mechanism, near-infrared and visible light images are acquired through a multispectral camera. Image quality is enhanced by nonlocal mean denoising and the CLAHE algorithm. Multimodal features are extracted using MobileNetV3 and 3D convolutional kernels, and the brightness and threshold of the light source are dynamically adjusted to achieve real-time congestion status recognition.

Benefits of technology

It achieves high-precision and robust chute blockage detection, reduces false detection rate, meets the real-time requirements of continuous ship loader operation, adapts to complex industrial environments, and improves detection accuracy and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118432A_ABST
    Figure CN123118432A_ABST
Patent Text Reader

Abstract

This invention discloses a machine vision-based intelligent identification system and method for blockage in a ship loader chute. The system comprises a hardware layer consisting of a multispectral camera, an embedded computing unit, a light source controller, and a communication module, and a software layer consisting of an image preprocessing module, a multimodal feature fusion network, a dynamic threshold adjustment module, and a blockage status decision module. The multispectral camera simultaneously acquires visible light and near-infrared images via a beam splitter. The image preprocessing module uses a non-local mean denoising algorithm for the near-infrared images and the CLAHE algorithm for band-specific differential preprocessing of the visible light images. The multimodal feature fusion network uses MobileNetV3 to extract static spatial features and a 3D convolutional network to extract dynamic temporal features. After fusion through an attention mechanism, the blockage probability and location are output. The dynamic threshold adjustment module generates a segmentation threshold in real time based on the illumination baseline and edge gradient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of industrial automation technology, specifically to a machine vision-based intelligent identification system for blockages in a ship loader chute. Background Technology

[0002] In port bulk cargo loading operations, the chute of the loader serves as a crucial channel for materials to move from the conveying system to the ship's hold, and its unobstructed flow directly impacts loading efficiency and operational safety. Traditional chute blockage detection primarily relies on manual inspection or contact sensors (such as pressure sensors and vibration sensors), which suffer from problems such as detection lag, high false alarm rates, and poor environmental adaptability. For example, manual inspection cannot cover the entire operation process in real time and is susceptible to interference from environmental factors such as dust and light; contact sensors are prone to signal distortion due to material adhesion or mechanical vibration, making it difficult to accurately identify minor blockages or early accumulation.

[0003] In recent years, machine vision technology has been increasingly applied to industrial inspection due to its advantages such as non-contact operation, high precision, and real-time performance. Existing technologies, including some patents, disclose material blockage detection schemes based on image processing, but these have the following limitations: Chinese patent CN202010123456.7 (publication date: May 10, 2020) proposes a belt conveyor blockage detection system based on a monocular camera, which identifies material accumulation through image grayscale threshold segmentation. However, this method is only applicable to planar belt conveyor scenarios and does not consider the complex lighting and material shape changes in the three-dimensional space inside the chute. It is also susceptible to interference from reflections and shadows, resulting in a high false detection rate.

[0004] Existing technologies still have unresolved issues such as image quality degradation caused by dust, uneven lighting, and material reflection inside the chute; they rely solely on grayscale, texture, or depth information without integrating multimodal data; and their algorithms are highly complex, making it difficult to implement high-speed inference on embedded devices and failing to meet the continuous operation requirements of ship loaders. Summary of the Invention

[0005] To address the problems in existing technologies, this invention provides a machine vision-based intelligent identification system for blockages in ship loader chutes, solving the problem of real-time monitoring and automated identification of material blockages inside chutes during bulk material loading operations. This system uses machine vision technology combined with image processing algorithms to achieve non-contact detection of abnormal conditions such as material accumulation and blockages within the ship loader chutes.

[0006] The technical solution adopted by the present invention to solve its technical problem is as follows: The present invention proposes a machine vision-based intelligent identification system for blockage of ship loader chute, which achieves high-precision, robust and real-time blockage status identification through multispectral imaging, lightweight deep learning model and dynamic threshold adjustment mechanism.

[0007] The overall system architecture includes a multispectral camera (including visible and near-infrared bands), an embedded computing unit (such as NVIDIA Jetson AGX Xavier), a light source controller (adjustable LED ring light source), and a communication module (5G / industrial Ethernet); it also includes an image preprocessing module, a multimodal feature fusion network, a dynamic threshold adjustment module, and a congestion state decision module.

[0008] By using a multispectral camera instead of a traditional monocular or binocular camera, the near-infrared band (850-950nm) is used to suppress dust reflection, and the visible light band (400-700nm) is used to preserve material texture details, thus solving the problem of image quality degradation in complex industrial environments.

[0009] Multispectral images are acquired synchronously by splitting the incident light into visible and near-infrared paths using a beam splitter prism. Two CMOS sensors simultaneously capture the images (frame rate ≥ 30fps) to avoid feature misalignment caused by timing errors. The ambient light intensity is monitored in real time by a light source controller, and the brightness of the LED ring light source is adjusted (range: 0-1000 lux) to ensure that the average gray value of the image is stable within a preset range (e.g., 120±20). The near-infrared image is denoised using a non-local mean denoising algorithm (NLM) to preserve material edge details while removing dust particle noise. The visible light image is enhanced with CLAHE (contrast-limited adaptive histogram equalization) to improve local contrast.

[0010] A dedicated denoising and enhancement algorithm was designed for the characteristics of visible light and near-infrared images to improve the accuracy of subsequent feature extraction.

[0011] MobileNetV3 is used as the backbone network. The input is a preprocessed visible light image, and the output is a 14×14×960 feature map (960 channels) to capture static features such as material outline and stacking angle. A 3D convolution kernel (size 3×3×3) is used to perform spatiotemporal joint convolution on 5 consecutive frames of near-infrared images to output a 14×14×256 feature map to extract dynamic features such as material flow velocity and displacement vector. The two branches of features are dynamically weighted through a spatial attention module (SE Block). After fusion, the feature map is processed by global average pooling and a fully connected layer to output the blockage probability (0-1) and the blockage location coordinates (x,y,w,h).

[0012] Knowledge distillation technology is adopted, where the soft targets of EfficientNet-B4 (teacher model) guide the training of MobileNetV3 (student model), and the loss function is the weighted sum of KL divergence and cross entropy with a weight ratio of 3:7; dust occlusion is randomly added to the training set to simulate actual working conditions, with the occlusion area accounting for 0-30%, so as to improve the anti-interference ability of the model; 3D convolution is used to compress the temporal dimension, and the attention mechanism is applied to dynamically fuse features, which increases the inference speed to 32fps while maintaining an accuracy of 98.7%.

[0013] Count the average brightness of the last 10 frames of visible light images to generate the lighting baseline L_base of the current working condition; apply the Sobel operator to the preprocessed image to extract horizontal and vertical gradients, and calculate the mean value G_edge of the material edge gradient; the blockage segmentation threshold T is determined by the following formula: T = α•L_base + β•G_edge + γ Wherein, α=0.4, β=0.5, and γ=10 are empirical parameters, which are determined by optimization through a genetic algorithm.

[0014] If the blockage probability P>0.9 and the blockage area S>0.2•A_total (A_total is the cross-sectional area of the chute), it is determined as a serious blockage; if 0.7<P≤0.9 and 0.1•A_total<S ≤ 0.2•A_total, it is determined as a mild blockage; otherwise, it is determined as a normal state.

[0015] The blockage level (0=normal, 1=mild, 2=serious) and position coordinates are sent to the PLC of the ship loader through the communication module, to trigger an audible and visual alarm or automatic shutdown.

[0016] Calibrate the parameters of the multispectral camera (such as white balance and exposure time), and train the multimodal feature fusion network; the camera collects images in real time and transmits them to the embedded unit; the image preprocessing module performs band-separated denoising and enhancement; the multimodal network outputs the blockage probability and position; the dynamic threshold module adjusts the segmentation threshold, and the auxiliary decision-making module determines the state; adjust the light source brightness or camera focal length according to the PLC control instruction to form closed-loop optimization. Beneficial Effects

[0017] Compared with existing technologies, this invention has the following significant advantages: Near-infrared and visible light images are acquired simultaneously using a beam-splitting prism, improving the edge clarity of near-infrared images by 60% compared to visible light; real-time adjustment of LED brightness based on a photoresistor stabilizes the average grayscale value of the image within the range of 120±20, reducing the detection error caused by illumination fluctuations from 18% to 5% compared to a fixed light source scheme; visible light images are enhanced through CLAHE, improving local contrast; the preprocessing module, through hardware beam splitting and software algorithm collaboration, directly solves the interference problem of dust and illumination in the background technology, providing high-quality input for subsequent feature extraction, thereby reducing the false detection rate; the dual-branch network, by simultaneously modeling static and dynamic features, directly solves the problem of poor adaptability of single features, while attention fusion further optimizes feature utilization efficiency, thereby improving detection robustness; multi-level decision-making, by quantifying the probability and location of blockages, directly solves the problem of excessive intervention in binary classification, while closed-loop feedback further optimizes system adaptability, thereby improving safety and operational efficiency; the software algorithm, after lightweight optimization, can run in real time on embedded platforms, meeting the needs of 24-hour continuous operation in industrial settings. Attached Figure Description

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0019] Figure 1 This is a schematic diagram of the overall system framework.

[0020] Figure 2 A schematic diagram of the optical path installation for the system.

[0021] Figure 3 This is a flowchart of the system process.

[0022] Figure 4 This is a diagram of the multimodal feature fusion network structure. Detailed Implementation

[0023] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0024] like Figures 1-4As shown, the present invention discloses a machine vision-based intelligent identification system for blockages in a ship loader chute. The system's hardware layer uses a multispectral camera as the image acquisition device. This camera separates the incident light path into two independent channels—one for the visible light band and one for the near-infrared band—using a precision beam-splitting prism. Images are simultaneously captured by two high-performance CMOS sensors. Specifically, the beam-splitting prism employs a multi-layer film design, forming optical thin films through alternating deposition of silicon dioxide and titanium dioxide. This achieves near-infrared anti-reflection at a wavelength of 850 nm, with reflectivity controlled below 0.3%, while maintaining a transmittance of over 92% in the visible light band (400-700 nm), ensuring a 50 / 50 split between visible and near-infrared light. This dual-channel design allows the system to simultaneously acquire detailed texture information of the material and near-infrared information resistant to dust interference, providing a rich data foundation for subsequent image processing.

[0025] The multispectral camera was carefully positioned 1.5 meters above the chute, with its lens axis at a 30-degree angle to the chute's centerline. This angle ensures full coverage of the chute's interior while preventing material from directly falling onto the lens and causing contamination. The camera is equipped with a high-resolution sensor of 2592×1944 pixels, operating at 14 frames per second to capture dynamic changes in material flow. To ensure strict time synchronization of the images acquired by the two sensors, the system employs a hardware triggering mechanism. A low-voltage differential signal interface connects the programmable logic controller (PLC) to the camera, keeping the trigger delay within 50 nanoseconds. The exposure start time difference between the two sensors does not exceed 100 microseconds. This synchronization precision ensures the accuracy of subsequent feature fusion.

[0026] In terms of light source configuration, the system uses an LED ring light source as the active lighting device. This light source integrates both white LEDs and near-infrared LEDs, with power reaching 8 watts and over 5 watts respectively. The brightness of the light source can be continuously adjusted within the range of 0 to 1000 lux. Through a precise current control circuit, the current ripple is controlled within 0.5% of the rated current (at a 100 kHz switching frequency). A 100 μF low equivalent series resistance tantalum capacitor is connected in parallel at the output terminal to stabilize the output. To adapt to different lighting environments, the system is equipped with a photoresistor sensor for ambient light monitoring. This sensor has a spectral response range covering 300 to 1100 nanometers, a resistance varying between 1 kΩ and 200 kΩ, and a sampling frequency set to 10 Hz. The sensor is installed to the side of the camera, at a 90-degree angle to the light source to avoid direct light interference. Through a piecewise linear correction formula, the original measurement error is reduced from ±15% to within 3%.

[0027] The system's computing core utilizes the NVIDIA Jetson AGX Xavier embedded computing platform, equipped with a 512-core Volta architecture GPU and 32GB of memory. It also integrates an 8-core ARM v8.2 64-bit CPU, with power consumption adjustable from 10 watts to 30 watts, ensuring computing performance while adapting to power constraints in industrial environments. The platform provides Gigabit Ethernet, USB 3.0, and general-purpose input / output interfaces for easy connection and communication with various industrial devices.

[0028] The software layer comprises four key components: an image preprocessing module, a multimodal feature fusion network, a dynamic threshold adjustment module, and a congestion state decision module. These modules work together to achieve a complete processing flow from the original image to congestion decision.

[0029] The image preprocessing module first performs a dual-modal image synchronous acquisition task. A programmable logic controller (PLC) outputs a 14 Hz pulse signal as a trigger source to synchronously control the visible light and near-infrared cameras. The exposure time is automatically adjusted according to ambient lighting conditions: 1 millisecond for the visible light channel and 2 milliseconds for the near-infrared channel to balance image brightness and motion blur. In the dynamic illumination compensation stage, a photoresistor collects the ambient light intensity L in real time, and the microcontroller adjusts the LED drive current according to a preset light-current mapping relationship. When the light intensity is below 400 lux, the drive current is set to 0.2 amps; when the light intensity is between 400 and 800 lux, the current is adjusted to 0.5 amps; and when the light intensity exceeds 800 lux, the current increases to 0.8 amps. This adaptive compensation mechanism ensures that, under 800 lux illumination, the average grayscale value of the compensated image is stable at around 118 with a standard deviation of 15, while the average grayscale value of the uncompensated image is as high as 240 with a standard deviation of 50, significantly improving image quality.

[0030] To address noise issues in dusty environments, the preprocessing module employs a band-specific differentiated processing strategy. For near-infrared images, a non-local mean denoising algorithm is used. This algorithm searches for similar image patches within a 21×21 pixel window, averages pixel values ​​within a 7×7 pixel similarity window, and sets the smoothing parameter to 10 times the noise standard deviation. This algorithm effectively removes dust particle noise while preserving details such as material edges to the greatest extent possible. For visible light images, a contrast-limited adaptive histogram equalization algorithm is used, with a cropping limit of 3.0 and a grid size of 8×8 pixels. This algorithm improves local image contrast by 40% and increases the structural similarity index from 0.62 to 0.85, significantly enhancing the distinction between material edges and the background.

[0031] The multimodal feature fusion network is the core of the system's intelligent recognition, employing a dual-branch architecture. The spatial feature extraction branch uses MobileNetV3-Large as its backbone network. It takes a pre-processed 224×224×3 visible light image as input and extracts static spatial features such as material contours and stacking angles through lightweight operations like depthwise separable convolution, outputting a 7×7×960-dimensional feature map. This branch is trained with a batch size of 16, an initial learning rate of 0.001, and a cosine annealing strategy. The AdamW optimizer is used for parameter updates. To accelerate convergence, the backbone network is frozen for the first five training epochs, training only the classification head.

[0032] The temporal feature extraction branch focuses on capturing the dynamic changes of materials. It receives five consecutive frames of 224×224 near-infrared images as input and performs spatiotemporal joint feature extraction using 3D convolutional kernels. This branch first extracts 16-dimensional features using a 3×3×3 3D convolutional kernel. After 2×2×2 max pooling, it applies another 3×3×3 convolutional kernel to extract 32-dimensional features, which are finally flattened into a 1×1×512-dimensional feature vector. During training, the batch size is set to 8, the learning rate is 0.0005, the Triplet loss function is used, and the margin is set to 1.0. To enhance the model's robustness to time-series changes, a temporal jitter data augmentation strategy is introduced during training, randomly discarding one frame from a series of frames and applying motion blur with kernels ranging from 3×3 to 7×7.

[0033] The features extracted from the two branches are intelligently fused in the attention fusion module. First, spatial and temporal features are concatenated into a 7×7×1472-dimensional joint feature set. Then, feature recalibration is performed through a compression activation module. This module first performs global average pooling on the feature map, then compresses the feature dimension from 1472 to 64 and back to 1472 through two fully connected layers. Finally, a sigmoid activation function is used to generate weight coefficients for each channel. In practical applications, the spatial feature weight is set to 0.7, and the temporal feature weight is set to 0.3. This weighting strategy fully utilizes the dominant role of static structural information while also considering the auxiliary judgment of dynamic changes. After global average pooling and fully connected layers, the fused features output a blocking probability value between 0 and 1, along with the coordinates of the blocking location.

[0034] To enhance the model's generalization ability, a variety of data augmentation strategies were employed during training. For visible light images, random rotations of -15 to +15 degrees and ±20% color jitter were applied; for near-infrared images, temporal jitter and motion blur were applied; simultaneously, bimodal images underwent synchronous 224×224 region random cropping to ensure the spatial correspondence between the two modalities. For knowledge distillation, the system used EfficientNet-B4 as the teacher model, whose output soft labels guided the training of the MobileNetV3 student model. The loss function employed a weighted sum of KL divergence and cross-entropy with a weight ratio of 3:7. This training strategy enabled the lightweight model to maintain over 95% accuracy while increasing inference speed to 32 frames per second.

[0035] The dynamic threshold adjustment module solves the problem of fixed thresholds failing under different lighting conditions. This module first calculates the lighting baseline by arithmetically averaging the average brightness of the most recent 10 visible light images, with an update frequency of 1 Hz. This sliding window strategy smooths out instantaneous fluctuations in lighting and reflects the stable lighting level under the current conditions. Then, the Sobel operator is applied to the preprocessed image to extract edge gradient information. Gradient components are calculated using horizontal convolution kernels [-1,0,1;-2,0,2;-1,0,1] and vertical convolution kernels [-1,-2,-1;0,0,0;1,2,1]. The gradient magnitude is obtained by taking the square root of the sum of squares, and the average gradient of all pixels is calculated as the edge gradient index.

[0036] Based on illumination baseline and edge gradient information, the system dynamically generates a blocking segmentation threshold using the formula T = 0.4 × L_base + 0.5 × G_edge + 10. The coefficients 0.4, 0.5, and 10 are optimized using a genetic algorithm on a large amount of experimental data. To prevent the threshold from failing under extreme conditions, a threshold constraint mechanism is implemented: when the calculated threshold is below 50, it is forcibly set to 50; when it is above 200, it is forcibly set to 200. This constraint ensures the effectiveness of the segmentation operation. Experiments show that under illumination intensity fluctuations of ±400 lux, this dynamic threshold mechanism reduces the segmentation error from ±35 gray levels to ±8 gray levels, significantly improving environmental adaptability.

[0037] The congestion status decision module makes a final judgment based on both the congestion probability and the congestion area. First, the congestion probability output by the network is initially screened. If the probability is less than 0.7, it is directly judged as a normal state; if the probability exceeds 0.7, it enters a further verification process. In the verification stage, a dynamic threshold is used to binarize the image, setting regions with pixel values ​​greater than the threshold to 1 and the rest to 0. Then, connected component analysis is used to extract the connected component with the largest area, and its area S and center coordinates are calculated. The final judgment rule is as follows: when the congestion probability exceeds 0.9 and the congestion area exceeds 20% of the chute's cross-sectional area, it is judged as a severe congestion, outputting level 2 and triggering an emergency stop of the ship loader; when the congestion probability is between 0.7 and 0.9 and the congestion area is between 10% and 20% of the cross-sectional area, it is judged as a mild congestion, outputting level 1 and triggering an audible and visual alarm; all other cases are judged as a normal state, outputting level 0. The decision results are sent to the programmable logic controller via the Modbus TCP protocol to achieve seamless integration with the ship loader control system. At the same time, the network decision threshold is fine-tuned by ±0.05 based on the feedback information to form a closed-loop optimization mechanism.

[0038] The overall system workflow is divided into three phases: initialization, operation, and feedback. The initialization phase completes the parameter calibration of the multispectral camera, including white balance adjustment and exposure time setting, as well as model loading for the multimodal feature fusion network. The operation phase executes the process cyclically in the order of image acquisition, preprocessing, feature extraction, threshold adjustment, and state decision-making. The total latency for processing each frame is controlled within 40 milliseconds, with an image acquisition frame rate of 30 frames per second, preprocessing time less than 10 milliseconds, and network inference time of 25 milliseconds, meeting the real-time requirements of continuous ship loader operation. The feedback phase adjusts the light source brightness or camera focal length according to instructions from the programmable logic controller (PLC) to continuously optimize system performance.

[0039] After practical testing, the system demonstrated excellent detection accuracy, achieving an overall accuracy of 98.7%, with a recall rate of 99.2% for severe blockages, 96.8% for minor blockages, and 99.5% for normal conditions. In terms of environmental adaptability, the system can operate stably within a dust concentration range of 0 to 100 mg / m³, a light intensity range of 50 to 1000 lux, and a temperature range of -20℃ to +60℃, fully meeting the harsh environmental requirements of port bulk cargo loading operations. Compared with existing technologies, this system effectively solves problems such as image quality degradation, limited feature extraction, insufficient real-time performance, and weak generalization ability in industrial environments through innovative technologies such as multispectral imaging, band-specific differentiated preprocessing, multimodal feature fusion, and dynamic threshold adjustment. It provides a high-precision, robust, and real-time intelligent solution for detecting blockages in ship loader chutes.

[0040] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0041] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A machine vision-based intelligent identification system for blockage in a ship loader chute, characterized in that: It includes a multispectral camera, an embedded computing unit, a light source controller and communication module, an image preprocessing module, a multimodal feature fusion network, a dynamic threshold adjustment module, and a congestion state decision module; The multispectral camera splits the incident light into two paths, visible light and near-infrared light, using a beam splitter, and images are simultaneously captured by two CMOS sensors. The image preprocessing module performs band-specific preprocessing on the acquired multispectral images. The multimodal feature fusion network includes spatial feature extraction and temporal feature extraction branches, used to extract static spatial features and dynamic temporal features and fuse them through an attention mechanism. The dynamic threshold adjustment module dynamically generates a segmentation threshold based on the illumination baseline and edge gradient. The congestion state decision module determines the congestion state based on the fused features and the dynamic threshold and outputs control commands.

2. The intelligent identification system for blockage of ship loader chute based on machine vision according to claim 1, characterized in that: The multispectral camera includes: a beam splitter prism for splitting incident light into visible light and near-infrared bands; a visible light CMOS sensor for capturing visible light images in the 400-700nm band; a near-infrared CMOS sensor for capturing near-infrared images in the 850-950nm band; and a synchronous triggering mechanism that ensures the exposure start time difference between the two sensors is ≤1ms through a hardware trigger signal.

3. The intelligent identification system for blockage of ship loader chute based on machine vision according to claim 1, characterized in that: The image preprocessing module's band-specific differential preprocessing includes using a non-local mean denoising algorithm for near-infrared images, with a search window size of 21×21 pixels and a similarity window size of 7×7 pixels; and using a contrast-limited adaptive histogram equalization (CLAHE) algorithm for visible light images, with a cropping threshold of 0.03 and a grid size of 8×8 pixels.

4. The intelligent identification system for blockage of ship loader chute based on machine vision according to claim 1, characterized in that: The multimodal feature fusion network includes a spatial feature extraction branch, uses MobileNetV3 as the backbone network, takes a preprocessed visible light image as input, and outputs a 14×14×960-dimensional feature map. The temporal feature extraction branch uses a 3D convolution kernel to perform spatiotemporal joint convolution on 5 consecutive frames of near-infrared images, outputting a 14×14×256-dimensional feature map. The attention fusion module dynamically weights the features of the two branches through the spatial attention module (SE Block), and after fusion, it outputs the blocking probability and blocking location coordinates through global average pooling and a fully connected layer. The 3D convolution kernel size of the temporal feature extraction branch is 3×3×3, the stride is 1×2×2, and the output feature map size is adjusted to 14×14, which is consistent with the spatial branch, through bilinear interpolation.

5. A machine vision-based intelligent identification system for blockage in a ship loader chute according to claim 1, characterized in that: The dynamic threshold adjustment module includes an illumination baseline calculation unit, which calculates the average brightness of the most recent 10 frames of visible light images and generates the current illumination baseline L_base. An edge gradient analysis unit, which applies a Sobel operator to the preprocessed image to extract horizontal and vertical gradients, and calculates the material edge gradient mean G_edge; a dynamic threshold generation unit, which dynamically generates a clogging segmentation threshold according to the formula T = α•L_base + β•G_edge + γ, where α=0.4, β=0.5, γ=10; the dynamic threshold generation unit is further provided with a threshold constraint condition: if T<50, it is forcibly set to 50; if T>200, it is forcibly set to 200.

6. A machine vision-based intelligent identification system for blockage in a ship loader chute according to claim 1, characterized in that: The determination rule of the clogging state decision module is: if the clogging probability P>0.9 and the clogging area S>0.2•A_total (A_total is the cross-sectional area of the chute), it is determined as severe clogging; if 0.7<P≤0.9 and 0.1•A_total<S≤0.2•A_total, it is determined as mild clogging; otherwise, it is determined as a normal state.

7. A machine vision-based intelligent identification system for blockage in a ship loader chute according to claim 1, characterized in that: The light source controller includes a photosensitive resistor sensor, configured to monitor the ambient light intensity in real time with a sampling frequency of 10Hz; an LED annular light source, which dynamically adjusts the brightness according to the light intensity, with an adjustment range of 0-1000lux; the brightness adjustment strategy is: if the light intensity L<300lux, the brightness of the light source is adjusted to the maximum 1000lux; if 300lux≤L≤700lux, the brightness of the light source is adjusted linearly; if L>700lux, the light source is turned off.

8. A machine vision-based intelligent identification method for blockage in a ship loader chute, characterized in that, The method includes the following steps: Step S1: Collect a visible light image and a near-infrared image synchronously by a multispectral camera; Step S2: Perform denoising processing on the near-infrared image by using a non-local means denoising algorithm, and perform enhancement processing on the visible light image by using a CLAHE algorithm; Step S3: Input the preprocessed visible light image into a spatial feature extraction branch to extract static spatial features; Input consecutive 5 frames of near-infrared images into a temporal feature extraction branch to extract dynamic temporal features; Step S4: Fuse the features of the two branches through an attention mechanism, and output the clogging probability and the coordinates of the clogging position; Step S5: Dynamically generate a segmentation threshold according to the average brightness of the latest 10 frames of visible light images and the edge gradient mean of the current frame; Step S6: Determine the clogging state and output a control instruction by integrating the clogging probability, the clogging area and the dynamic threshold; In step S3, the spatial feature extraction branch adopts a MobileNetV3 network, the temporal feature extraction branch adopts a 3D convolutional network, and the features of the two branches are weighted and fused through an SE module; determining the clogging state in step S6 specifically includes: if the clogging probability P>0.9 and the binarized segmented clogging area S>0.2•A_total, it is determined as severe clogging, and an emergency stop of the ship loader is triggered; if 0.7<P≤0.9 and 0.1•A_total<S≤0.2•A_total, it is determined as mild clogging, and an audible and visual alarm is triggered; otherwise, it is determined as a normal state.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program, when executed by a processor, implements the machine vision-based ship loader chute clogging intelligent identification method according to claim 8.

10. An embedded computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intelligent identification method for blockage of ship loader chute based on machine vision as described in claim 8.

Citation Information

Patent Citations

  • Mechanical product long-period fatigue test data processing and trend pre-judging method

    CN111307483A