Bulk grain in-out silo process and system

Abstract

This invention discloses a process and system for bulk grain entering and exiting silos. The method includes: acquiring multimodal image data of the grain dropping area; preprocessing the multimodal image data and inputting it into a dual-stream neural network for feature extraction and fusion, outputting a multimodal fusion feature vector; initially segmenting suspected impurity point cloud clusters and performing physical feature analysis and labeling; performing fine recognition and classification based on a semantic segmentation network, fusing the multimodal fusion feature vector and the suspected impurity point cloud clusters, and outputting a semantic segmentation mask; fusing the semantic segmentation result, the multimodal fusion feature vector, and the physical feature labeling, and outputting the impurity identification result; determining a cleaning scheme based on the impurity identification result; planning the unloading trolley's entry path into the silo; and executing the silo transfer task based on a dual-trigger mechanism. This method achieves multi-dimensional impurity identification, can accurately identify different impurity categories, and realizes precise screening and identification of foreign objects at the source of grain unloading, so as to carry out targeted impurity cleaning and achieve precise control of the quality of bulk grain entering the silo.

Description

Technical Field

[0001] This invention relates to the field of automated grain storage technology, specifically to a process and system for loading and unloading bulk grain into and out of silos. Background Technology

[0002] Silos are a typical type of warehouse for grain storage and logistics in my country. They are generally equipped with auxiliary grain loading and unloading facilities such as unloading stations and working towers. As the core facility for bulk grain storage, the efficiency and intelligence level of the loading and unloading process of silos are directly related to the overall efficiency of grain storage and logistics. To reduce grain turnover and improve the efficiency of grain entering and leaving silos from unloading, existing bulk grain silo entry and exit systems integrate cleaning and metering functions, such as the clean grain silo entry and exit system disclosed in CN213864488U. Although this system can perform pre-entry preparation processes such as cleaning and metering simultaneously with grain entering the silo, its level of automation is low and it has the following limitations: I. The cleaning function mainly relies on traditional physical cleaning methods such as magnetic separation and air separation. The level of intelligence is low, and it is impossible to identify and remove impurities in real time. When there are impurities that are not easy to separate or clean, it will affect the cleaning efficiency of bulk grain and cannot control and guarantee the cleaning quality of bulk grain. Second, the planning of bulk grain entering and transferring is relatively simple, relying on three-way valves and belt conveyors for bulk grain distribution and transportation. It can only achieve directional transportation of single or regular multi-target objectives. The distribution and transportation path of bulk grain is relatively fixed, lacking the dynamic coordination and intelligent scheduling capabilities of multiple transportation tasks. It is difficult to cope with complex and ever-changing operational tasks, and is prone to equipment congestion or resource idleness. Third, operations such as transferring grain to storage and storing bulk grain are triggered by fixed patterns or experience, lacking data-driven and adaptive adjustments based on the real-time status of the grain pile, making it difficult to achieve the best storage and preservation results. Therefore, the present invention provides a process method and system for bulk grain entering and exiting silos, which solves the problems of existing grain silo process systems being unable to control cleaning quality and lacking dynamic entry and flexible silo planning and turnover planning. Summary of the Invention

[0003] The purpose of this invention is to overcome the deficiencies of the prior art and provide a process and system for bulk grain entering and leaving silos that can perform image recognition of impurities, precise impurity removal, automatic silo entry planning, and silo relocation planning, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows: A process for loading and unloading bulk grain into and out of silos includes the following steps and contents: S1. During the unloading of bulk grain, image analysis methods are used to analyze impurities in the grain drop area: S11. Synchronously acquire multimodal image data of the grain drop area, including deformed images, visible light images, infrared thermal imaging, and depth information; S12. Preprocess the multimodal image data and perform the following feature extraction and analysis in parallel: S12a. Input the preprocessed multimodal image data into a dual-stream neural network to perform geometric feature extraction and thermal feature extraction, and then output a multimodal fusion feature vector. S12b: Based on the high-density point cloud data obtained after preprocessing, suspected impurity point cloud clusters are initially segmented by clustering algorithm, physical feature analysis and labeling are performed on the suspected impurity point cloud clusters, and the suspected impurity point cloud clusters are projected onto the synchronously acquired visible light image to generate a binary mask image of the corresponding image region. S13. Input the preprocessed multimodal image data into the semantic segmentation network for pixel-level fine recognition and classification. Using the binary mask image as spatial prior information, fuse the binary mask image with the multimodal fusion feature vector and inject it into the backbone of the semantic segmentation network. The semantic segmentation network finally outputs a semantic segmentation mask containing impurity categories and precise boundaries. S14. The semantic segmentation results, multimodal fusion feature vectors, and physical feature labels are fused together. The comprehensive confidence score is calculated through a weighted fusion model, and the impurity identification results are output. The weighted fusion model uses dynamic weights, which are adjusted according to the consistency between the semantic segmentation results and the physical feature labels. S2. Determine the cleaning plan before bulk grain is put into storage based on the impurity identification results; S3. Determine the target silo. Configure multiple unloading points on the main conveyor. Assign the unloading trolley's work tasks according to the unloading points and plan the unloading trolley's entry path from the main conveyor to the target silo. S4. Execute the transfer task based on the dual-trigger mechanism.

[0005] According to one aspect of this disclosure, the initial segmentation in step S12b is implemented based on the DBSCAN clustering algorithm, dynamically setting the neighborhood radius of the point cloud density. and minimum points Segmentation was used to obtain suspected impurity point cloud clusters; among them, , Based on the value, The dust concentration coefficient; physical characteristic analysis of suspected impurity point cloud clusters includes: Density feature analysis: Project the suspected impurity point cloud clusters onto a two-dimensional plane, use a sliding window to traverse the corresponding area, and count the average point cloud density or maximum point cloud density within the sliding window. If it is greater than the preset high density threshold, mark the suspected impurity point cloud clusters as impurity targets with high density features. Shape feature analysis: Calculate the shape factor of the binary mask image. For suspected impurity point cloud clusters with shape factors less than a preset proportional threshold, mark them as impurity targets with abnormal shape features.

[0006] According to one aspect of this disclosure, the physical characteristic analysis also includes: performing impurity attribute analysis using multispectral trapping technology; comparing the spectral response differences between synchronously acquired visible light images and infrared thermal images in the suspected impurity point cloud cluster region, and marking suspected impurity point cloud clusters that exceed the difference threshold as impurity targets with spectral anomalies.

[0007] According to one aspect of this disclosure, in step S12, the dual-stream neural network adopts a dual-stream CNN processing architecture, which includes a visual stream processing layer, a thermal imaging stream processing layer and a feature fusion layer, and its feature extraction method includes the following: (1) Input the two-dimensional feature map based on the high-density point cloud data into the visual stream processing channel. The visual stream processing layer extracts the geometric feature vector of the bulk grain surface based on the EfficientNetV2 network. (2) Input the infrared thermal image into the thermal imaging stream processing channel constructed based on the ConvNeXt network, and the thermal imaging stream processing layer extracts the temperature distribution feature vector of the grain surface. (3) The feature fusion layer fuses the geometric feature vector with the temperature distribution feature vector to form a multimodal fusion feature vector.

[0008] According to one aspect of this disclosure, after impurities are removed and metered, bulk grain is lifted to the main conveyor for transport. A sequence of unloading point combinations is generated based on an anti-stratification mode, and the entry path of the unloading trolleys for collaborative operations is dynamically planned. S31. Collect the status data of the main conveyor and the position coordinates of the unloading trolley. The unloading trolley is positioned using a hybrid method based on UWB and Gray code. S32. In response to the grain silo's inbound task, based on the process requirements for preventing grain pile stratification, a sequence of unloading point combinations containing multiple different unloading points is generated. Based on the resource bidding of each unloading trolley, with the goal of optimizing the overall operation efficiency, the inbound task and the unloading point combination sequence are decomposed and allocated to multiple unloading trolleys to form a collaborative task allocation scheme. S33. Plan the entry path for the unloading trolley according to the collaborative task allocation scheme; predict and detect spatiotemporal conflicts of different entry paths in real time; when a path conflict is detected, make a conflict resolution decision based on priority and plan a conflict resolution scheme for low priority tasks. S34. Monitor the load status of the main conveyor. When the load exceeds the preset load threshold, automatically adjust the upstream feeding speed and the conveying speed of the main conveyor.

[0009] According to one aspect of this disclosure, preprocessing fuses visible light images and depth information into RGB-D data, and inputs the RGB-D data into a semantic segmentation network for pixel-level recognition; multimodal fusion feature vectors are upsampled to generate a spatial attention heatmap, and a guide map is generated by fusing the binary mask image and the spatial attention heatmap element by element, and the guide map is injected into the backbone of the semantic segmentation network.

[0010] According to one aspect of this disclosure, the triggering mechanism in step S4 includes: periodic triggering and threshold triggering. Periodic triggering is timed by a preset silo turning cycle. Threshold triggering is triggered by monitoring the temperature and humidity of the grain in the silo. When the temperature exceeds a preset temperature threshold or a preset temperature gradient, the silo turning task is triggered. During the execution of the silo transfer task, the temperature drop rate of the source silo and the temperature rise rate of the target silo are monitored in real time. The current silo transfer task will be stopped under any of the following conditions: 1. The temperature of the source silo drops to the normal temperature range; 2. The target silo reaches the safety stock limit or the temperature rises to the temperature control limit; 3. The silo transfer task execution time reaches the silo transfer duration threshold.

[0011] According to one aspect of this disclosure, the unloading task is completed based on the main conveyor, and different cost calculation models are invoked to bid for resources for the unloading trolley according to the task type; When the task type is the aforementioned inbound task, the first cost model is invoked for calculation: Inbound task cost = Path length + Time+ Energy consumption; When the task type is the aforementioned warehouse transfer task, the second cost model is invoked for calculation: Warehouse transfer task cost = Path length + Device Status + Energy efficiency; in, , , , , and Cost weighting.

[0012] A bulk grain silo loading and unloading process system is provided to implement the aforementioned bulk grain silo loading and unloading process method. It includes a monitoring module, a central control unit, and, connected to the central control unit and arranged sequentially, a grain unloading module, a lifting module, a cleaning module, a metering module, a silo entry module, and a silo exit dispensing module. The central control unit incorporates an AI decision-making unit, a dynamic path planning unit, and a distributed collaborative control unit. The distributed collaborative control unit coordinates the operation of each module based on monitoring data from the monitoring module. The unloading module includes a hydraulic tilting platform, a receiving hopper, and a 3D vision perception unit. The 3D vision perception unit is arranged on the receiving hopper. The AI ​​decision unit identifies impurities based on the deformed images, depth information, visible light images, and infrared thermal imaging collected by the 3D vision perception unit, and calls the cleaning module to perform corresponding impurity removal operations based on the final impurity identification results. The cleaning module includes a cleaning robot arm, a manual cleaning unit, a blower, a magnetic separator, a vibrating cylinder, and a temporary impurity storage bin. The cleaning robot arm is set on the receiving hopper, and the lifting module is equipped with a cleaning channel. The blower, magnetic separator, and vibrating cylinder are set on the cleaning channel. The silo loading module includes a main conveyor, a loading conveyor line, and an unloading trolley. The loading conveyor line is horizontally erected on the top of the silo, connecting the main conveyor and all silos. The main conveyor is a scraper conveyor with multiple diversion unloading ports at the bottom. Each diversion unloading port is equipped with an electric gate independently controlled by a distributed collaborative control unit. The distributed collaborative control unit dynamically plans the opening and closing sequence of the diversion unloading ports based on real-time flow. The unloading trolley travels on the loading conveyor line according to the loading path issued by the dynamic path planning unit, transporting the unloaded material from the diversion unloading ports to the loading port at the top of the silo. The monitoring module includes grain condition monitoring, equipment status monitoring, and location coordinate monitoring; The discharge module connects to the discharge port and receiving hopper at the bottom of the silo. After impurity identification, the bulk grain and the grain transferred into the silo are initially cleaned by the impurity removal robotic arm and the manual cleaning unit. Then, the grain is lifted by the lifting module to the cleaning channel for flow cleaning, and after being measured by the metering module, it is transported into the silo by the silo entry module.

[0013] According to one aspect of this disclosure, the 3D visual perception unit includes: A light projection system used to project a specifically coded optical pattern onto the grain-falling area; A stereo vision camera array is used to acquire distorted images of optical patterns; A TOF depth camera is used to perceive a wide range of depth information in the grain dropping area; A high-resolution RGB camera was used to acquire visible light images of the surface texture of bulk grain. Thermal imaging cameras are used to collect infrared thermal images of the grain drop area.

[0014] Compared with the prior art, the present invention provides a process and system for loading and unloading bulk grain into and out of silos, which has the following beneficial effects: This method uses 3D visual perception units to perform AI visual recognition of impurities. It performs pixel-level segmentation and recognition based on semantic networks, and integrates temperature characteristics, shape characteristics, density characteristics and spectral characteristics to identify the physical characteristics of impurities, thus achieving multi-dimensional identification of impurities. It can accurately identify different types of impurities and achieve precise screening and identification of foreign objects at the source of grain unloading, so as to carry out targeted impurity cleaning, forming an active prevention and control mechanism, improving the quality of grain entering the warehouse, and achieving precise control of the quality of bulk grain entering the warehouse. This process method, based on the silo loading and unloading system, enables collaborative operation and intelligent scheduling of multiple unloading trolleys, effectively improving resource utilization. By controlling the generation of unloading combination sequences through an anti-stratification mode, it effectively solves the grading problem in the traditional automatic conveyor belt silo loading process. Dynamic silo turning based on real-time data of temperature and humidity inside the grain pile breaks the limitations of fixed-frequency silo turning, achieving scientific grain preservation and better storage results. Attached Figure Description

[0015] Figure 1 This is a flowchart of the bulk grain loading and unloading process disclosed in this invention.

[0016] Figure 2 This is a diagram illustrating the structure of the bulk grain loading and unloading silo process system disclosed in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely the best embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The term "embodiment" as used herein means that a particular method, step, or content described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0019] This embodiment provides a bulk grain loading and unloading silo process system and a bulk grain loading and unloading silo process method based on this system, such as... Figure 1 and Figure 2 As shown, the optimal implementation method for bulk grain loading and unloading from silos in this embodiment is described in conjunction with the process system, which includes the following steps and contents: S1. The bulk grain transport vehicle drives into the hydraulic tilting platform. The positioning sensor sends a positioning signal to the central control unit. The central control unit activates the vehicle fixing device to lock the bulk grain transport vehicle. At the same time, the hydraulic tilting platform drives the bulk grain transport vehicle to tilt and unload the bulk grain in batches into the receiving hopper directly below the hydraulic tilting platform. The receiving hopper uses a 3D vision perception unit to perceive and make decisions about the grain dropping area for each batch. S11, the 3D vision perception unit array is arranged above or around the receiving hopper, including a light projection system, a stereo vision camera array, a TOF depth camera, a high-resolution RGB camera, and a thermal imaging camera; the light projection system projects Gray code pattern structured light onto the grain dropping area, the stereo vision camera array acquires deformed images at a frequency of 30fps, and the TOF depth camera simultaneously acquires a large range of depth information of the bulk grain dropping area; the high-resolution RGB camera and the thermal imaging camera simultaneously acquire visible light images and infrared thermal images during the bulk grain unloading process; S12. Based on the principle of triangulation, utilizing the deformation differences of deformed images from different perspectives, the three-dimensional coordinates of each point on the surface of the loose grain are calculated using a stereo matching algorithm. The two-dimensional deformed image is converted into a discrete point set in three-dimensional space, i.e., an initial point cloud. The initial point cloud is then registered and fused with large-scale depth information to complete the TOF depth data in the edge region, generating a point cloud that covers the entire grain dropping area and has a projection density of no less than 100 points / cm² on the horizontal plane. 2 High-density point clouds are used to accurately reconstruct the three-dimensional shape of the grain pile; High-density point clouds are filtered to eliminate impulse noise and outliers during 3D reconstruction. Two-dimensional feature maps of the grain pile surface are generated based on the vertical projection of the high-density point cloud and local surface fitting for subsequent two-dimensional feature analysis. The high-density point cloud is divided into local regions and projected onto a two-dimensional plane, typically a horizontal plane. The generated two-dimensional projection point set is then smoothed by median filtering using a 3×3 sliding window for subsequent cluster analysis. Based on pre-calibrated camera parameters, the depth image and the visible light image are spatially registered to ensure that the pixels correspond strictly in geometric position. The registered depth information is then stitched together with the RGB three channels of the visible light image to generate RGB-D fusion data, which serves as the basis for subsequent multimodal analysis. The AI ​​decision-making unit performs multimodal feature extraction and fusion based on the above preprocessing: S12a. Employing a dual-stream convolutional neural network architecture, feature extraction and deep fusion are performed on information from different physical dimensions to form a unified multimodal feature representation; specifically: (1) The two-dimensional feature map representing the three-dimensional geometric shape of the surface of the grain drop area is used as the multi-channel image input to the visual stream processing channel. The visual stream processing layer is constructed based on the EfficientNetV2 network architecture. After multi-layer convolution, feature transformation and average pooling of the EfficientNetV2 network, a 512-dimensional geometric feature vector is output. This geometric feature vector encodes the overall and local geometric characteristics of the grain drop area. (2) Input the synchronously acquired infrared thermal image into the thermal imaging stream processing channel. The thermal imaging stream processing layer is built based on the ConvNeXt network architecture. The infrared thermal image is extracted by the ConvNeXt network convolution to extract the temperature distribution features and output a 256-dimensional temperature feature vector. This temperature feature vector encodes the temperature field features on the surface of the grain pile, the location and shape of the abnormal temperature zone, and other information. (3) The feature fusion layer receives the above geometric feature vector and temperature feature vector, and integrates the spatial geometric properties and thermophysical properties of the target through splicing operation, and outputs a multimodal fusion feature vector with 768-dimensional mixed features. S12b and the AI ​​decision-making unit use the DBSCAN clustering algorithm to perform density-based clustering of the two-dimensional projected point set, segmenting it into clusters of suspected impurity point clouds with abnormal distribution, and dynamically setting the neighborhood radius of the point cloud density. and minimum points Segmentation was used to obtain suspected impurity point cloud clusters; among them, , The base value is 5mm. The dust concentration coefficient was determined based on on-site measurements; physical properties were verified and marked for each suspected impurity point cloud cluster, including: (I) Density feature verification: The point cloud density of suspected impurity point cloud clusters within their projection range is statistically analyzed using a sliding window; the point cloud density of each sliding window is calculated as D=N / A, where A is the area of ​​the sliding window and N is the number of two-dimensional projection points within the sliding window. Anomaly detection: Compare the point cloud density D within the sliding window with a preset high-density threshold, which is typically set to 130 points / cm². 2 Alternatively, it can be based on the average density of the collected pure grain background. To perform dynamic settings, i.e. , The density standard deviation is the background of pure grain. The point cloud density distribution formed by the uniform grain flow is relatively consistent. The surface structure of stone particles is complex and opaque, forming point cloud clusters with abnormally high density distribution, which makes the local density significantly higher than the high density threshold. (II) Shape Feature Verification: The suspected impurity point cloud cluster is projected onto the synchronously acquired visible light image plane through camera calibration parameters. The set of pixel coordinates within the projection range of the cloud cluster is obtained. Based on the set of pixel coordinates, a connected binary image region is generated. A polygonal contour is constructed and filled to generate a binary mask image. The total area A and contour perimeter P of the binary mask image are calculated. Then, the shape factor F is calculated by substituting into the formula F=4πA / P². The shape factor F of a circular region is 1. The shape factor F of complex, slender or irregular shapes (such as ropes) will be smaller. When the shape factor F<0.65, the point cloud cluster is considered to be an impurity feature with an abnormal shape, different from common grain particles. The binary mask image and the shape abnormality marker are used as spatial prior information and input into the subsequent semantic segmentation network for spatial attention guidance and feature modulation. (III) For each suspected impurity point cloud cluster, locate its corresponding image region on the simultaneous acquisition of visible light images and infrared thermal imaging, extract the average spectral response value or texture features of the region in the visible light band and infrared band respectively, calculate the difference between the features of the two bands, and if the difference is greater than the preset difference threshold, the suspected impurity point cloud cluster is determined to be an impurity target with spectral abnormality. This method can be used to identify impurities that are similar in color to grain under visible light, but have temperature differences under infrared. S13. The AI ​​decision unit performs deep learning-based recognition, using the binary mask image generated by the multimodal fusion feature vector and shape anomaly marker as prior information to guide the semantic segmentation network. The specific method is as follows: using the above binary mask image as a spatial prior map, the 768-dimensional multimodal fusion feature vector is upsampled to the input image size through the decoder to generate a spatial attention heatmap, and the spatial prior map and the spatial attention heatmap are fused element by element to generate a guidance map. The semantic segmentation network employs a YOLO-based semantic segmentation model. RGB-D data is input into the four channels of the pre-trained YOLO semantic segmentation network for pixel-level fine-grained recognition and classification. A guide map is injected into the backbone of the YOLO semantic segmentation network, and spatial domain modulation is performed on the intermediate feature maps extracted by the backbone network to obtain segmentation feature maps containing spatial priors. This allows the backbone network to focus on suspected regions. The backbone network uses a MobileNetV3 variant with depthwise separable convolutions. The segmentation feature maps participate in the subsequent neck and head calculations of the YOLO semantic segmentation network, ultimately outputting a pixel-level semantic segmentation mask containing precise boundaries and category labels for "background," "grain," and multiple types of "impurities," as well as the pixel coordinates of the impurities. S14. The AI ​​decision-making unit performs multi-source information fusion decision-making based on semantic segmentation results, multimodal fusion feature vectors, and physical feature labels, and outputs highly reliable impurity determination results; specifically, it includes the following steps: S14a. For each suspected impurity region initially identified by the YOLO semantic segmentation network, extract the following evidence and perform normalization processing. Extract the category confidence score C of the region predicted by the YOLO semantic segmentation network. sem ; Extract the geometric feature vector of the region from the multimodal fusion feature vector, calculate its Euclidean or cosine path length with the preset normal grain surface geometric feature template, and map this path length to a geometric anomaly score C. geo Temperature feature vectors are extracted from the multimodal fusion feature vectors, and the average temperature of the region is compared with the temperature difference of the background grain pile, which is then mapped to a temperature difference score C. therm ; For suspected impurity regions marked by physical verification, regardless of whether they are high-density impurity regions, abnormal shape impurity regions, or spectral abnormal impurity regions, they are directly determined as valid impurities, and the physical verification score C for that region is then calculated. phy The physical verification score C is 1, or can be strengthened to be greater than 1. phy To maximize the contribution of physical evidence, the unlabeled version receives a physical verification score of C. phy =0; S14b, Adaptive Weight Fusion and Overall Confidence C total calculate: ; Initial weights are preset based on experience, and the weights are dynamically adjusted according to the consistency and conflict of evidence. , , and If the allocation is consistent, the impurity confidence of the suspected impurity region is increased. For example, when C sem High and C phy Set to 1, automatically increase and The weights; when C sem High but C geo With C therm If all values ​​are low, the value will automatically decrease. The weights; S14c, the preset decision threshold is 0.7, when C total If the value exceeds 0.7, it is considered a valid impurity. The category output by the YOLO semantic segmentation network is the primary factor, combined with the multimodal fusion feature vector extraction results and physical feature verification results. For example, if the YOLO semantic segmentation network classifies it as "organic impurity" and the physical verification indicates high density, then the final category is corrected to high-density organic impurity, i.e., gravel. If the YOLO semantic segmentation network classifies it as "unknown," and C... therm The high-level category is classified as "temperature abnormality impurity"; S14d: For valid impurities, output precise pixel-level boundaries, final category, spatial coordinates, and overall confidence level. The central control unit triggers an early warning based on this and sends the impurity coordinates to the cleaning system.

[0020] S2. For each batch of unloaded bulk grain, the optimal cleaning plan is automatically matched and executed based on multi-dimensional impurity identification results. The specific process is shown in the following example: The pre-defined rules for cleaning up built-in impurities and cleaning methods in the module are for C. total Impurities larger than 0.9 (such as stones or metal blocks) are identified based on their shape characteristics. Large, hard impurities that exceed a certain total area threshold and are physically marked as high-density are removed by a robotic arm that is positioned on the receiving hopper. For 0.7 < C total Impurities with a particle size <0.75 (such as soil clods, glass shards, and different types of grain) are semantically segmented as impurities. However, unknown impurities with insignificant physical characteristics are then classified and screened using a vibrating cylindrical screen. Furthermore, the vibration frequency, screen mesh size, and grain screening flow rate of the vibrating cylindrical screen can be adjusted based on the distribution density and size differences of the unknown impurities. For C total Metal impurities with a density greater than 0.8 and physical markings indicating high density, and thermal characteristics showing high thermal conductivity, such as iron filings and iron wires, are automatically adsorbed by starting the magnetic separator. The magnetic field strength of the magnetic separator can be automatically adjusted according to the size difference of the metal impurities. For C total If the impurities are greater than 0.7 and physically marked as having abnormal shapes (such as rope ends, hair, and plastic sheets), then the blower will be started to blow them away and separate them. For complex, ambiguous, or sticky impurities that the system cannot clearly match and classify, manual cleaning is carried out through a manual cleaning unit. The manual cleaning unit includes an alarm and a monitor with interactive commands. The monitoring interface highlights the area of ​​this type of impurity and alarms to provide manual reminders. After manual processing and confirmation, the alarm is deactivated by inputting a command.

[0021] Understandably, the above preset rules are only provided as exemplary data to more intuitively illustrate the matching between the identification results and the cleaning scheme. The actual weight allocation, impurity identification confidence, and matching rules between physical characteristics and the cleaning scheme are preset based on on-site experience.

[0022] S3. After being cleaned by the impurity removal robotic arm and manually, the batch of bulk grain in the receiving hopper is conveyed to the first-stage vertical elevator of the lifting module. The blower, magnetic separator, and vibrating cylinder are integrated on the cleaning channel, which is located between the first-stage and second-stage vertical elevators of the lifting module. The distributed collaborative control unit starts the corresponding cleaning equipment according to the cleaning plan settings, so that the bulk grain performs the corresponding impurity removal operations in sequence as it passes through the cleaning channel. The cleaned impurities are automatically separated into the impurity temporary storage bin. The metering module is set on the second-stage vertical elevator, and the metering data of the bulk grain is synchronized to the central control unit for storage, providing a basis for subsequent warehousing. Based on factors such as the type of bulk grain, the inventory status of the silos, or the predicted total amount of grain to be received, the target silos for the receiving process are determined. The receiving operation is achieved through the receiving module on the top of the silo. The receiving module includes a main conveyor, a receiving conveyor line, and an unloading trolley. The main conveyor is a scraper conveyor, which is erected on the top of the silo. Multiple independently controlled grain storage troughs are set on it. The scraper conveyor operates in a rotary manner and receives bulk grain from the discharge port of the secondary vertical elevator. Each grain storage trough is equipped with an independent electric gate for unloading. The receiving conveyor line includes a picking line and a receiving reciprocating branch line. The picking line is arranged in parallel directly below the feeding section of the main conveyor. The receiving reciprocating branch line connects to the picking line. The unloading trolley travels back and forth between the silo and the main conveyor via the receiving reciprocating branch line, thereby realizing the feeding of each silo. To reduce grain stratification during bulk grain collection and transportation, and to meet the technological requirements for preventing grain pile stratification, the unloading of grain storage troughs is not sequential. Instead, a distributed collaborative control unit dynamically plans the sequence of unloading point combinations based on real-time flow and a preset anti-stratification mode. This controls the opening and closing sequence of different electric gate combinations. The anti-stratification mode can employ a zigzag pattern or a random crossover pattern to allocate the unloading point sequence. The number of electric gates opened and closed each time matches the number of unloading trolleys performing the task. For example, in the random crossover pattern... There are a total of six grain storage troughs numbered sequentially. There are two unloading trolleys performing the task. The combination sequence of generating unloading point Pi is P1: (A1, A4) → P2: (A3, A6) → P3: (A2, A5). Then, according to the arrival of the two unloading trolleys, grain storage troughs A1 and A4 are opened sequentially for unloading, so that the bulk grain in the two grain storage troughs is mixed and put into the warehouse. The transport capacity of grain storage trough A6 is only half of the trough. After the transport capacity of grain storage trough A6 is completed, its electric gate is closed, and the electric gates of grain storage troughs A5 and A3 are opened sequentially to perform unloading. The bulk grain on the main conveyor is mixed into the silo to achieve grain property equalization. Based on the above unloading combination control allocation, the dynamic path planning unit allocates the operation tasks of the unloading trolley according to the unloading point. Taking the target silo number as the endpoint coordinate and the position of the unloading trolley performing the task as the starting coordinate, the unloading trolley plans the path for transporting the bulk grain into the silo: S31. Collect the status data of the main conveyor and the position coordinates of the unloading trolley. To improve the positioning accuracy of the unloading trolley, a hybrid positioning method combining ultra-wideband (UWB) positioning and Gray code positioning is adopted. By using an extended Kalman filter algorithm, the absolute coordinates of UWB and the incremental uniqueness of the Gray code encoder output are fused. After error correction, the high-precision fused coordinates and direction of the unloading trolley are output in real time. At the same time, during the operation of the silo module, the following status data are collected: conveyor current accuracy 0.1A, rotational speed (accuracy 1rpm), and bearing temperature (accuracy 0.1℃); the grain condition of the target silo is collected through distributed temperature and humidity sensors, including temperature (accuracy 0.1℃) and humidity (accuracy 1%RH). S32. Based on the above unloading point combination sequence, perform anti-hierarchical collaborative task allocation. The dynamic path planning unit uses a distributed task allocation contract network protocol to schedule the unloading trolleys. Each point Pi in the unloading point combination sequence is published as an atomic task. Each unloading trolley bids for resources for the atomic tasks. For the warehousing task, with the goal of optimizing global operation efficiency, the first cost model is called to calculate the cost: Calculation: Warehousing task cost = Path length + Time+ Energy consumption; cost weighting in some possible implementations. =0.6, =0.3, =0.1; where the cost of path length, time and energy consumption includes the unloaded operation of the unloading trolley from its current position to the picking point and the loaded operation from the picking point to the silo unloading point; the time needs to take into account the waiting time for picking up materials and the waiting time for unloading at the unloading point; With the goal of minimizing the total global cost, the subtasks of the atomic task are assigned to the most suitable unloading trolley through an optimization algorithm, forming a collaborative task allocation scheme from the unloading trolley to the picking point and then to the unloading point, and clarifying the work order sequence of each unloading trolley. S33. Based on the above collaborative task allocation scheme, considering the speed limit and turning radius constraints under the load of the unloading trolley, plan the operation path of each work order of the unloading trolley. Conflict Prediction and Resolution: In this embodiment, the inbound and return lines are parallel and connected. The unloading trolleys unload materials according to the order of arrival at the unloading point and return sequentially to retrieve materials. At the intersection of the inbound and return lines and the retrieval line, and at the retrieval line which serves as the main line, multi-vehicle conflicts may occur, resulting in time-space A. When the algorithm predicts a path intersection, it resolves the conflict based on priority. For example, if the priority of a fully loaded trolley is higher than that of an unloaded trolley, and there is a yielding point or yielding route, the trolley that yields will be inserted into the yielding point to wait for instructions or the trolley that yields will be instructed to detour via the yielding route, so that the unloading trolley with the priority sequence can pass smoothly. During the journey, the algorithm monitors the path in real time and increases the length of the safe yielding path by 0.7m. When a potential conflict is detected, the algorithm plans an yielding path for the trolley with the lower priority or sends a stop yielding instruction. S34. During the warehousing operation, monitor the load status of the main conveyor. When the status data shows that the conveyor load is greater than 85% of the preset threshold, the distributed collaborative control unit automatically adjusts the feeding speed of the upstream feeder (including the receiving hopper, vertical elevator and corresponding cleaning equipment, etc.) and the conveying speed of the main conveyor to prevent material accumulation.

[0023] S4. The silos execute the silo turning task based on a dual trigger mechanism: periodic trigger and threshold trigger. The periodic trigger is triggered by a preset silo turning cycle. Grain condition monitoring also includes temperature and humidity monitoring of other silos. The threshold trigger is triggered when the temperature and humidity monitoring data of the grain in the silo exceed a preset threshold or preset gradient. The grain transfer task is implemented based on the inbound route of the silo entry task, namely, transferring grain in the order of outbound dispensing module → receiving hopper → lifting module → cleaning module → metering module → inbound module. The outbound dispensing module includes a discharge device and an air-cushioned belt conveyor located at the discharge port at the bottom of the silo. The AI ​​decision unit identifies impurities in the transferred grain to determine whether cleaning is necessary. Impurities in the transferred grain are generally rotten, spoiled, or moldy grain. Therefore, the identification of impurities in the transferred grain focuses on spectral and temperature anomalies, significantly improving the overall confidence level calculation. weight and weight Significantly reduce geometric features The weight of the data is used to analyze the characteristics of the moldy areas in the grain being transferred to storage. If the mold is distributed in discrete points, it can be blown away by the blower of the cleaning module or the high-speed air spray valve array; if it is clustered mold, the impurity removal robotic arm is called to remove it precisely. The target silo for the grain transfer task is determined, and the grain is placed into the silo using the same method as in step S3. It's important to note that unlike the efficiency-first approach of grain placement, the goal of the grain transfer task is to ensure grain quality. Cost calculation prioritizes safety and quality. Therefore, when the task type is a grain transfer task, the second cost model is used to calculate the task cost: Grain transfer task cost = Path length + Device Status + Energy efficiency; cost weighting in some possible implementations. =0.4, =0.3, =0.3; During the execution of the warehouse transfer task, the temperature drop rate of the source warehouse transfer number and the temperature rise rate of the target warehouse transfer number are monitored in real time. The current grain transfer task will be terminated under any of the following conditions: 1. When the temperature of the source silo drops to the normal temperature range, the grain transfer task ends; 2. When the target silo reaches the safety stock limit or the temperature rises to the temperature control limit, the current grain transfer task is suspended. If the temperature of the source silo still has not dropped to the normal temperature range, a new target silo will be selected based on the grain condition distribution of the silos, and the grain transfer task will be re-planned; 3. When the execution time of the grain transfer task reaches the grain transfer duration threshold, the grain transfer task will be suspended, and the grain condition will be further analyzed.

[0024] Furthermore, the software product of the bulk grain silo loading and unloading process is stored in a storage medium, including several instructions to cause a computer device, such as, but not limited to, a personal computer, a server, or a network device, to store a computer program. When the computer program is executed by a processor, it can execute the bulk grain silo loading and unloading process of any embodiment of this application.

[0025] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A process for loading and unloading bulk grain into and out of silos, characterized in that, Includes the following steps and content: S1. During the unloading of bulk grain, image analysis methods are used to analyze impurities in the grain drop area: S11. Synchronously collect multimodal image data of the grain dropping area, including deformed images, visible light images, infrared thermal imaging, and depth information; S12. Preprocess the multimodal image data and perform the following feature extraction and analysis in parallel: S12a. Input the preprocessed multimodal image data into a dual-stream neural network to perform geometric feature extraction and thermal feature extraction, and then output a multimodal fusion feature vector. S12b: Based on the high-density point cloud data obtained after preprocessing, suspected impurity point cloud clusters are initially segmented by clustering algorithm, physical feature analysis and labeling are performed on the suspected impurity point cloud clusters, and the suspected impurity point cloud clusters are projected onto the synchronously acquired visible light image to generate a binary mask image of the corresponding image region. S13. Input the preprocessed multimodal image data into the semantic segmentation network for pixel-level fine recognition and classification. Using the binary mask image as spatial prior information, fuse the binary mask image and the multimodal fusion feature vector, and inject it into the backbone of the semantic segmentation network. The semantic segmentation network finally outputs a semantic segmentation mask containing impurity categories and precise boundaries. S14. The semantic segmentation result, the multimodal fusion feature vector, and the physical feature label are fused together, and the comprehensive confidence score is calculated through a weighted fusion model to output the impurity identification result. The weighted fusion model uses dynamic weights, which are adjusted according to the consistency between the semantic segmentation result and the physical feature label. S2. Determine the cleaning plan before bulk grain is put into storage based on the impurity identification results; S3. Determine the target silo. Configure multiple unloading points on the main conveyor. Assign the operation tasks of the unloading trolley according to the unloading points. Plan the entry path of the unloading trolley from the main conveyor to the target silo. S4. Execute the transfer task based on the dual-trigger mechanism.

2. The bulk grain loading and unloading process according to claim 1, characterized in that: The initial segmentation in step S12b is implemented based on the DBSCAN clustering algorithm, dynamically setting the neighborhood radius of the point cloud density. and minimum points The suspected impurity point cloud clusters are segmented and obtained; wherein, , Based on the value, The dust concentration coefficient; the physical characteristic analysis of the suspected impurity point cloud cluster includes: Density feature analysis: Project the suspected impurity point cloud cluster onto a two-dimensional plane, use a sliding window to traverse the corresponding area, and count the average point cloud density or maximum point cloud density within the sliding window. If it is greater than a preset high density threshold, mark the suspected impurity point cloud cluster as the impurity target with high density features. Shape feature analysis: Calculate the shape factor of the binary mask image, and mark the suspected impurity point cloud clusters with shape factors less than a preset ratio threshold as impurity targets with abnormal shape features.

3. The bulk grain loading and unloading process according to claim 2, characterized in that: The physical feature analysis also includes: using multispectral trapping technology to analyze impurity properties; comparing the spectral response differences between the synchronously acquired visible light image and the infrared thermal image in the suspected impurity point cloud cluster region; and marking the suspected impurity point cloud clusters that exceed the difference threshold as impurity targets with spectral anomalies.

4. The bulk grain loading and unloading process for silos according to claim 3, characterized in that: The visible light image and the depth information are fused into RGB-D data after the preprocessing. The RGB-D data is then input into the semantic segmentation network for pixel-level recognition. The multimodal fusion feature vector is upsampled to generate a spatial attention heatmap. The binary mask image and the spatial attention heatmap are fused element by element to generate a guide map. The guide map is then injected into the backbone of the semantic segmentation network.

5. The bulk grain loading and unloading process for silos according to claim 1, characterized in that: In step S12, the dual-stream neural network adopts a dual-stream CNN processing architecture, which includes a visual stream processing layer, a thermal imaging stream processing layer, and a feature fusion layer. Its feature extraction method includes the following: (1) Input the two-dimensional feature map converted from the high-density point cloud data into the visual stream processing channel. The visual stream processing layer extracts the geometric feature vector of the bulk grain surface based on the EfficientNetV2 network. (2) The infrared thermal image is input into the thermal imaging stream processing channel constructed based on the ConvNeXt network, and the thermal imaging stream processing layer extracts the temperature distribution feature vector of the grain surface. (3) The feature fusion layer fuses the geometric feature vector with the temperature distribution feature vector to form the multimodal fusion feature vector.

6. The bulk grain loading and unloading process for silos according to claim 1, characterized in that: After impurity removal and metering, the bulk grain is lifted to the main conveyor for transport. A sequence of unloading point combinations is generated based on an anti-stratification mode, and the entry path of the unloading trolleys working in coordination is dynamically planned. S31. Collect the status data of the main conveyor and the position coordinates of the unloading trolley. The unloading trolley is positioned using a hybrid method based on UWB and Gray code. S32. In response to the grain silo's loading task, according to the process requirements for preventing grain pile stratification, a sequence of unloading points with multiple different unloading points is generated. Based on the resource bidding of each unloading trolley, with the goal of optimizing global operation efficiency, the loading task and the unloading point combination sequence are decomposed and allocated to multiple unloading trolleys to form a collaborative task allocation scheme. S33. Plan the entry path for the unloading trolley according to the collaborative task allocation scheme; predict and detect spatiotemporal conflicts of different entry paths in real time; when a path conflict is detected, make a conflict resolution decision based on priority and plan a conflict resolution scheme for low priority tasks. S34. Monitor the load status of the main conveyor. When the load exceeds the preset load threshold, automatically adjust the upstream feeding speed and the conveying speed of the main conveyor.

7. The bulk grain loading and unloading process according to claim 6, characterized in that: The triggering mechanism for the grain transfer task includes: periodic triggering and threshold triggering. The periodic triggering is timed by a preset grain transfer cycle. The threshold triggering is triggered by monitoring the temperature and humidity of the grain in the silo. When the temperature exceeds a preset temperature threshold or a preset temperature gradient, the grain transfer task is triggered. During the execution of the silo transfer task, the temperature drop rate of the source silo and the temperature rise rate of the target silo are monitored in real time. The current silo transfer task shall be stopped under any of the following conditions:

1. The temperature of the source silo drops to the normal temperature range; 2. The target silo reaches the safety stock limit or the temperature rises to the temperature control limit; 3. The execution time of the silo transfer task reaches the silo transfer duration threshold.

8. The bulk grain loading and unloading process according to claim 7, characterized in that: The warehousing task is completed based on the main conveyor, and different cost calculation models are called according to the task type to conduct resource bidding for the unloading trolley; When the task type is the inbound task, the first cost model is invoked for calculation: Inbound task cost = Path length + Time+ Energy consumption; When the task type is the warehouse transfer task, the second cost model is invoked for calculation: Warehouse transfer task cost = Path length + Device Status + Energy efficiency; in, , , , , and Cost weighting.

9. A bulk grain loading and unloading silo process system, used to implement the bulk grain loading and unloading silo process method according to any one of claims 1 to 8, characterized in that: It includes a monitoring module, a central control unit, and unloading, lifting, cleaning, metering, warehousing, and dispensing modules connected to the central control unit and arranged sequentially. The central control unit incorporates an AI decision-making unit, a dynamic path planning unit, and a distributed collaborative control unit. The distributed collaborative control unit coordinates the operation of each module based on monitoring data from the monitoring module. The unloading module includes a hydraulic tilting platform, a receiving hopper, and a 3D vision perception unit. The 3D vision perception unit is arranged on the receiving hopper. The AI ​​decision unit identifies impurities based on the deformed image, depth information, visible light image, and infrared thermal imaging collected by the 3D vision perception unit, and calls the cleaning module to perform corresponding impurity removal operations according to the impurity identification results. The cleaning module includes a cleaning robot arm, a manual cleaning unit, a blower, a magnetic separator, a vibrating cylinder, and an impurity storage bin. The cleaning robot arm is mounted on the receiving hopper, and the lifting module is provided with a cleaning channel. The blower, the magnetic separator, and the vibrating cylinder are mounted on the cleaning channel. The silo loading module includes the main conveyor, the loading conveyor line, and the unloading trolley. The loading conveyor line is horizontally erected on the top of the silo, connecting the main conveyor and all the silos. The main conveyor is a scraper conveyor with multiple diversion unloading ports at its bottom. Each diversion unloading port is equipped with an electric gate independently controlled by the distributed collaborative control unit. The distributed collaborative control unit dynamically plans the opening and closing sequence of the diversion unloading ports based on real-time flow. The unloading trolley travels on the loading conveyor line according to the loading path issued by the dynamic path planning unit, transporting the unloaded material from the diversion unloading ports to the loading port at the top of the silo. The monitoring module includes grain condition monitoring, equipment status monitoring, and location coordinate monitoring; The discharge module is connected to the discharge port at the bottom of the silo and the receiving hopper. After the bulk grain and the grain transferred into the silo are identified for impurities, they are initially removed by the impurity removal robotic arm and the manual cleaning unit. Then, they are lifted by the lifting module to the cleaning channel for flow cleaning, and after being measured by the metering module, they are transported into the silo by the silo entry module.

10. The bulk grain loading and unloading silo process system according to claim 9, characterized in that: The 3D visual perception unit includes: A light projection system for projecting a specifically coded optical pattern onto the grain dropping area; A stereo vision camera array is used to acquire the deformed image of the optical pattern; A TOF depth camera is used to perceive a wide range of depth information in the grain dropping area; A high-resolution RGB camera is used to acquire the visible light image of the surface texture of the bulk grain; A thermal imaging camera is used to acquire infrared thermal images of the grain-falling area.

Images