Warehousing intelligent monitoring and checking integrated system and method based on multi-technology fusion

Abstract

The invention discloses a storage intelligent monitoring and checking integrated system and method based on multi-technology fusion, and particularly relates to the technical field of storage monitoring, the system obtains environmental parameters and cargo data in real time through a multi-source data acquisition module, carries out cargo identification and segmentation by using YOLOv7 and Mask R-CNN, and carries out storage monitoring and checking on the cargo. A three-dimensional model is constructed in combination with a Poisson surface reconstruction algorithm to calculate volume weight, an environment anomaly analysis module generates an early warning signal through dynamic threshold and influence level evaluation, and an intelligent decision module fuses data to realize quantity anomaly alarm and multi-level response and outputs a three-dimensional visual inventory report. According to the method, the problems of multiple blind areas, low checking efficiency, slow abnormal response and the like of traditional storage monitoring are solved, full coverage of environment monitoring, high accuracy of goods identification, low error of volume calculation and intelligence of abnormal handling are realized, and the automation and digitization levels of storage management are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of warehouse monitoring technology, and more specifically, to a system and method for integrating warehouse intelligent monitoring and inventory taking based on multi-technology integration. Background Art

[0002] As a core link in the logistics supply chain, warehousing undertakes key functions such as cargo storage, dispatching, and management. With the rapid development of e-commerce, smart manufacturing, and other fields, modern warehousing is evolving towards scale and intelligence, placing higher demands on accurate monitoring of the cargo storage environment, real-time inventory counting, and rapid response to abnormal situations.

[0003] Traditional warehouse management is time-consuming and labor-intensive due to manual inventory checks, limited monitoring range of static sensors, and a lack of three-dimensional visualization of cargo status. This makes it difficult to meet the development needs of smart logistics. Specific shortcomings include: The use of a fixed sensor network without a mobile data collection platform results in insufficient monitoring coverage and is unable to address the blind spot issue on shelves. It only realizes cargo quantity identification and lacks 3D modeling and volume calculation capabilities, which cannot meet the needs of warehouse capacity planning; The environmental abnormality alarm mechanism is single, the impact level assessment is not combined with the cargo category, and the emergency response lacks specificity.

[0004] To address the above problems, the present invention proposes a solution based on the integration of multiple technologies. Through collaborative collection between sensor networks and mobile platforms, deep learning target detection, three-dimensional reconstruction algorithms and intelligent decision-making systems, it realizes the integrated monitoring and inventory of storage environment and goods, so as to improve monitoring accuracy, inventory efficiency and abnormal response capabilities, and promote the transformation of warehouse management towards intelligent and digitalization. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides an integrated system and method for intelligent warehouse monitoring and inventory taking based on the fusion of multiple technologies to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solutions: an integrated warehouse intelligent monitoring and inventory system based on multi-technology integration, comprising: The multi-source data acquisition module includes a fixed sensor network and a mobile acquisition platform. The fixed sensor network is deployed at a fixed location in the warehouse and is used to collect environmental parameter data in real time, including temperature, humidity, smoke concentration, and door magnetic status. The mobile acquisition platform includes a rail-mounted pulley, a drive motor, and an onboard laser radar and binocular camera. It is used to move within the warehouse and collect cargo images and 3D point cloud data. The image processing and recognition module is connected to the multi-source data acquisition module and is used to perform the following operations: Performing target detection on the cargo image based on the YOLOv7 deep learning model to identify the cargo category, location, and quantity; Use instance segmentation algorithm to extract cargo outline and generate cargo segmentation mask; The 3D reconstruction and volume calculation module is connected to the image processing and recognition module and is used to: fusing the cargo segmentation mask with the three-dimensional point cloud data, and generating a three-dimensional model of the cargo using a Poisson surface reconstruction algorithm; Calculating the volume of the three-dimensional model and matching a preset density value according to the cargo category to estimate the cargo weight; The environmental anomaly analysis module is connected to the multi-source data acquisition module and is used to: Comparing the environmental parameter data with a preset threshold value, and generating an environmental abnormality signal if the threshold value is exceeded; Determine the impact of environmental abnormalities on the cargo based on the cargo type; The intelligent decision-making and alarm module is connected to the image processing and recognition module, the 3D reconstruction and volume calculation module, and the environmental anomaly analysis module, and is used to: Compare the quantity of goods with the inventory database and generate an alarm for abnormal quantity; Integrate environmental abnormal signals and impact levels to generate multi-level alarm instructions; Output cargo inventory report including quantity, volume, weight and environmental compliance assessment.

[0007] Preferably, the rail-type pulley of the mobile acquisition platform moves along a preset path through the drive motor, and the path covers all cargo stacking areas in the warehouse; the binocular camera collects cargo images at a frame rate of 30fps, and the lidar scans at a frequency of 10Hz to generate three-dimensional point cloud data with millimeter-level accuracy.

[0008] Preferably, the image processing and recognition module further includes: The occlusion compensation unit is used to predict the texture and shape of the occluded part based on the generative adversarial network (GAN) when there is occlusion in the cargo image, and to correct the cargo quantity recognition result; The category calibration unit is used to secondary verify the category of the goods through a support vector machine (SVM) classifier based on the structural characteristics of the three-dimensional model of the goods.

[0009] Preferably, in the three-dimensional reconstruction and volume calculation module, the Poisson surface reconstruction algorithm includes the following steps: Map the cargo segmentation mask to the 3D point cloud data and extract the point cloud boundary; Construct implicit surface functions through octree space partitioning; Gradient descent method is used to optimize the surface function and generate a smooth three-dimensional mesh model; Compute the convex hull volume based on mesh vertex coordinates.

[0010] Preferably, the intelligent decision-making and alarm module further includes: Dynamic threshold adjustment unit, used to automatically adjust environmental parameter thresholds according to cargo category: for cold chain cargo, a temperature threshold of -18°C to -22°C is used, and for dry cargo, a humidity threshold of <15% is used; A multimodal alarm unit is used to simultaneously trigger an audible and visual alarm, SMS notifications, and a pop-up visualization interface. The visualization interface overlays a 3D model of the cargo and a heat map of the abnormal environmental location.

[0011] A method for integrating intelligent warehouse monitoring and inventory based on multi-technology integration includes the following steps: S1: A fixed sensor network collects warehouse environmental parameter data in real time, while controlling a mobile acquisition platform to move along a preset path, capturing cargo images with a binocular camera, and collecting 3D point cloud data with a lidar. S2: Use the YOLOv7 model to detect objects in cargo images, identify cargo categories and locations, and generate cargo segmentation masks using an instance segmentation algorithm. S3: Fusion of cargo segmentation mask and 3D point cloud data, executing Poisson surface reconstruction algorithm to construct a 3D cargo model, calculate cargo volume, and estimate weight based on cargo category matching density values; S4: Compare the environmental parameter data with the preset threshold. If the threshold is exceeded, an environmental abnormality signal is generated and the impact level is determined based on the cargo type. S5: Compare the quantity of goods with the inventory database. If the deviation exceeds 5%, an abnormal quantity alarm is triggered. S6: Integrates environmental anomaly signals and quantity anomaly alarms to generate multi-level alarm instructions and output an inventory report containing cargo quantity, volume, weight, and environmental compliance assessment.

[0012] Preferably, the instance segmentation algorithm in step S2 adopts the Mask R-CNN model, and its loss function is: in is the classification loss, is the bounding box regression loss, It is the mask segmentation loss; when the cargo occlusion rate is greater than 30%, the occlusion compensation unit is activated to generate a complete mask.

[0013] Preferably, the Poisson surface reconstruction algorithm in step S3 includes: Downsample the point cloud data to a voxel grid to reduce computational complexity; Optimize the spatial distribution of point clouds through normal estimation and point cloud registration; Construct an octree hierarchy and set the depth to 10 to balance accuracy and efficiency; Solving the Poisson equation Generate isosurfaces where is the point cloud gradient field.

[0014] Preferably, the impact level in step S4 is divided into three levels: Level 1: If the environment is abnormal and the goods are perishable, a red alarm will be triggered immediately; Level 2: The environment is abnormal but the goods are storable, triggering a yellow alert; Level 3: Environmental parameters are approaching the threshold, triggering a blue prompt.

[0015] Preferably, the inventory report in step S6 is displayed in a three-dimensional visual form, including: A panoramic view of the warehouse, marking the location of goods and quantity deviations; Environmental parameter change curve with superimposed threshold line; Rotating view of the cargo 3D model, with volume and weight data marked; Alarm history log, abnormal events sorted by time.

[0016] Technical effects and advantages of the present invention: Through the coordinated work of fixed sensor networks and mobile collection platforms, the blind spot problem of traditional static monitoring is solved, the environmental monitoring coverage rate is improved, and blind-angle monitoring of the entire warehouse area is achieved. The high-frequency sampling and millimeter-level point cloud accuracy of the lidar and binocular camera ensure the real-time and accuracy of cargo data collection, providing a reliable foundation for subsequent analysis.

[0017] An image processing solution based on YOLOv7 and Mask R-CNN, combined with GAN occlusion compensation and SVM category calibration, improves the accuracy of cargo recognition in occluded scenarios, significantly increasing the success rate of inventory counting in complex stacking environments. The Poisson surface reconstruction algorithm fuses 2D masks and 3D point clouds to generate high-precision 3D models, reducing volume calculation errors and ensuring that weight estimation accuracy meets warehouse management requirements.

[0018] Through the dynamic threshold adjustment mechanism, differentiated environmental management is achieved for different cargo categories, reducing the false alarm rate and shortening the abnormal response time. The three-level impact level alarm is combined with the multimodal alarm method to ensure zero omissions of abnormal events and improve emergency response efficiency.

[0019] Through automated data collection and processing processes, the inventory time of the entire warehouse is shortened, which greatly reduces labor costs compared to traditional manual inventory. The three-dimensional visual inventory report integrates information such as cargo location, volume, and environmental parameters, supports real-time query and historical tracing, and improves the scientific nature and visual efficiency of warehouse management. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0021] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0022] The integrated warehouse intelligent monitoring and inventory system based on multi-technology integration provided by the present invention includes: The multi-source data acquisition module includes a fixed sensor network and a mobile acquisition platform. The fixed sensor network is deployed at a fixed location in the warehouse and is used to collect environmental parameter data in real time, including temperature, humidity, smoke concentration, and door magnetic status. The mobile acquisition platform includes a rail-mounted pulley, a drive motor, and an onboard laser radar and binocular camera. It is used to move within the warehouse and collect cargo images and 3D point cloud data. The image processing and recognition module is connected to the multi-source data acquisition module and is used to perform the following operations: Performing target detection on the cargo image based on the YOLOv7 deep learning model to identify the cargo category, location, and quantity; Use instance segmentation algorithm to extract cargo outline and generate cargo segmentation mask; The 3D reconstruction and volume calculation module is connected to the image processing and recognition module and is used to: fusing the cargo segmentation mask with the three-dimensional point cloud data, and generating a three-dimensional model of the cargo using a Poisson surface reconstruction algorithm; Calculating the volume of the three-dimensional model and matching a preset density value according to the cargo category to estimate the cargo weight; The environmental anomaly analysis module is connected to the multi-source data acquisition module and is used to: Comparing the environmental parameter data with a preset threshold value, and generating an environmental abnormality signal if the threshold value is exceeded; Determine the impact of environmental abnormalities on the cargo based on the cargo type; The intelligent decision-making and alarm module is connected to the image processing and recognition module, the 3D reconstruction and volume calculation module, and the environmental anomaly analysis module, and is used to: Compare the quantity of goods with the inventory database and generate an alarm for abnormal quantity; Integrate environmental abnormal signals and impact levels to generate multi-level alarm instructions; Output cargo inventory report including quantity, volume, weight and environmental compliance assessment.

[0023] In practice, the system consists of a multi-source data acquisition module, an image processing and recognition module, a 3D reconstruction and volume calculation module, an environmental anomaly analysis module, and an intelligent decision-making and alarm module. A fixed sensor network, deployed on the warehouse ceiling and shelves, includes temperature and humidity sensors, smoke sensors, and door magnetic sensors, sampling at a 1Hz rate. The mobile acquisition platform uses a track-mounted trolley equipped with a Basler acA1920 binocular camera and a Velodyne VLP-16 lidar, scanning at a 10Hz rate and driven by a stepper motor along an I-beam track. The image processing module runs the YOLOv7 model and the Mask R-CNN instance segmentation algorithm on a Jetson AGX Xavier edge device. The 3D reconstruction module uses the Open3D library to implement Poisson surface reconstruction, with an octree depth of 10. The environmental analysis module has a built-in dynamic threshold database, and the intelligent alarm module integrates a 120dB buzzer, RGB LED, and GSM SMS notifications.

[0024] Specifically, a fixed sensor network including temperature, humidity, smoke concentration sensors and door magnetic sensors is deployed at fixed locations in the warehouse to collect environmental parameter data in real time at a sampling frequency of 1 Hz. The mobile acquisition platform consists of a rail-mounted trolley, a drive motor, a lidar, and a binocular camera. The trolley moves along a preset path. The lidar scans at a frequency of 10Hz to generate 3D point cloud data, and the binocular camera captures cargo images at a frame rate of 30fps. The YOLOv7 deep learning model is then used to detect targets in the cargo images and identify the category, location, and quantity of the cargo. The instance segmentation algorithm is used to extract the cargo contour and generate a segmentation mask. The 3D reconstruction and volume calculation module fuses the cargo segmentation mask with the 3D point cloud data. The Poisson surface reconstruction algorithm is used to generate a 3D cargo model. The cargo weight is estimated by matching the cargo category with the preset density value. The environmental anomaly analysis module then compares the collected environmental parameter data with the preset threshold. If the threshold is exceeded, an environmental anomaly signal is generated. The impact level of the environmental anomaly on the cargo is determined based on the cargo category. Finally, the intelligent decision-making and alarm module compares the identified cargo quantity with the inventory database to generate a quantity anomaly alarm. The environmental anomaly signal and impact level are integrated to generate multi-level alarm instructions. The output is a cargo inventory report containing the quantity, volume, weight, and environmental compliance assessment. Through the coordinated collaboration of a fixed sensor network and a mobile data collection platform, comprehensive data collection of the warehouse environment and goods is achieved. The YOLOv7 model leverages the powerful feature extraction capabilities of deep learning to quickly and accurately identify goods. The instance segmentation algorithm further precisely outlines the goods. The Poisson surface reconstruction algorithm fuses 2D masks and 3D point cloud data to construct a 3D model of the goods, combining density values to estimate weight. The environmental anomaly analysis module assesses the impact of environmental anomalies based on threshold comparison and cargo characteristics. The intelligent decision-making and alarm module integrates multiple data sources to generate anomaly alarms and inventory reports, improving environmental monitoring coverage and avoiding blind spots associated with static monitoring. This improves inventory efficiency, reducing manual intervention through automated data collection and processing. Volume calculation errors are reduced, and 3D reconstruction technology enables more accurate volume estimation. Response time to anomalies is shortened, and a multi-level alarm mechanism ensures timely detection and resolution of issues.

[0025] The rail-type pulley of the mobile acquisition platform moves along a preset path through the drive motor, and the path covers all cargo stacking areas in the warehouse; the binocular camera collects cargo images at a frame rate of 30fps, and the lidar scans at a frequency of 10Hz to generate three-dimensional point cloud data with millimeter-level accuracy.

[0026] In specific implementation, the mobile acquisition platform's track-mounted trolley is driven by a drive motor and moves along a pre-set I-beam track path. This path covers all cargo storage areas within the warehouse, ensuring comprehensive scanning and addressing static monitoring blind spots. A binocular camera captures cargo images at a frame rate of 30fps, while a lidar scans at a frequency of 10Hz to generate millimeter-level precision 3D point cloud data. The camera and lidar synchronize data acquisition via timestamps, ensuring that the pre-set path design ensures that the mobile platform can comprehensively scan all cargo areas within the warehouse. The high frame rate of image acquisition and high-frequency lidar scanning ensure high temporal and spatial resolution of the acquired data. A timestamp synchronization mechanism ensures temporal alignment of image and point cloud data, facilitating subsequent data fusion processing and shortening warehouse-wide inventory counts. The efficient data acquisition frequency and coverage improve inventory efficiency. This reduces data registration errors, making the synchronously acquired data easier to fuse and supporting high-precision volume reconstruction.

[0027] The image processing and recognition module also includes: The occlusion compensation unit is used to predict the texture and shape of the occluded part based on the generative adversarial network (GAN) when there is occlusion in the cargo image, and to correct the cargo quantity recognition result; The category calibration unit is used to secondary verify the category of the goods through a support vector machine (SVM) classifier based on the structural characteristics of the three-dimensional model of the goods.

[0028] In specific implementation, the occlusion compensation unit adopts the Pix2Pix GAN network structure. The training dataset contains 5,000 occluded cargo images and complete annotations. When the Mask R-CNN detects an occlusion rate >30%, the GAN is triggered to generate predicted textures. The category calibration unit extracts the HOG features of the three-dimensional model, and the SVM classifier uses the RBF kernel function. The training data contains 200 categories of three-dimensional features of cargo. GAN is then used to learn the distribution pattern of cargo textures, reconstruct occluded areas, and improve the recognition accuracy of occluded scenes. The 2D image recognition results are cross-validated with 3D geometric features to reduce the cargo misclassification rate. The HOG descriptor is used to capture surface gradient features to enhance classification robustness, thereby improving the success rate of cargo inventory in complex stacking scenarios.

[0029] Specifically, when occlusion occurs in a product image and the occlusion rate exceeds 30%, the occlusion compensation unit based on the Pix2Pix GAN network architecture is activated. Using a trained dataset of 5,000 fully annotated images of occluded products, it predicts the texture and shape of the occluded portion and corrects the product quantity recognition results. The category calibration unit extracts HOG features based on the structural characteristics of the product's 3D model. Using an SVM classifier using an RBF kernel function and training data on 200 product categories, the product category is re-verified. This allows the GAN network to learn the texture distribution patterns of the product and predict the features of the occluded portion through a generative adversarial approach, thereby correcting for recognition errors caused by occlusion. The SVM classifier uses the HOG features of the 3D model to capture the gradient characteristics of the product surface and cross-validate the 2D image recognition results to improve the accuracy of category recognition. This improves the accuracy of product recognition in occluded scenarios and reduces the number of miscalculations caused by occlusion. This reduces the misclassification rate of goods. By combining 2D and 3D features, the robustness of classification is enhanced, improving the success rate of product counting in complex stacking scenarios.

[0030] In the three-dimensional reconstruction and volume calculation module, the Poisson surface reconstruction algorithm includes the following steps: Map the cargo segmentation mask to the 3D point cloud data, extract the point cloud boundary, and remove outliers; Construct implicit surface functions through octree space partitioning; Gradient descent method is used to optimize the surface function and generate a smooth three-dimensional mesh model; Compute the convex hull volume based on mesh vertex coordinates.

[0031] In specific implementation, the Poisson surface reconstruction algorithm uses the Poisson equation The system converts discrete point cloud data into continuous surfaces. Octree spatial partitioning balances detail preservation and computational efficiency. Gradient descent optimizes surface functions to generate smooth 3D models. Convex hull volume calculations based on mesh vertices enable volume estimation of irregular objects, improving volume calculation speed. The octree structure and optimization algorithm reduce computational complexity. This reduces errors in the volume of complex-shaped cargo. Precise 3D model construction ensures more accurate volume estimation and shortens reconstruction time.

[0032] The intelligent decision-making and alarm module also includes: Dynamic threshold adjustment unit, used to automatically adjust environmental parameter thresholds according to cargo category: for cold chain cargo, a temperature threshold of -18°C to -22°C is used, and for dry cargo, a humidity threshold of <15% is used; A multimodal alarm unit is used to simultaneously trigger an audible and visual alarm, SMS notifications, and a pop-up visualization interface. The visualization interface overlays a 3D model of the cargo and a heat map of the abnormal environmental location.

[0033] The visualization interface uses the Three.js engine, overlaying a 3D cargo model, an environmental heat map, and real-time alarm location markers. This dynamically adjusts the response level based on cargo sensitivity, triggering physical and digital notifications in parallel to ensure zero omissions. The heat map locates the source of anomalies, and the 3D model displays the affected cargo, reducing false alarms and improving the efficiency of all-in-one visualization analysis. During specific implementation, the dynamic threshold adjustment unit sets personalized environmental thresholds for different categories of goods based on the sensitivity differences of the goods to achieve differentiated management. The multimodal alarm unit triggers multiple alarm methods in parallel to ensure zero omissions of alarm information. The visual interface intuitively displays the abnormal location and impact range through three-dimensional models and heat maps, reducing the false alarm rate. The dynamic threshold is adjusted according to the characteristics of the goods, reducing unnecessary alarms and improving the efficiency of visual analysis. The intuitive interface display facilitates rapid positioning and handling of abnormalities, ensuring the timeliness of abnormal response.

[0034] like Figure 1 As shown, the present invention provides a method for integrating intelligent warehouse monitoring and inventory based on multi-technology integration, which includes the following steps: S1: Real-time warehouse environmental parameter data is collected through a fixed sensor network, while the mobile collection platform is controlled to move along a preset path, cargo images are collected through a binocular camera, and 3D point cloud data is collected through a lidar.

[0035] S2: Use the YOLOv7 model to perform object detection on cargo images, identify cargo categories and locations, and generate cargo segmentation masks through the instance segmentation algorithm.

[0036] The instance segmentation algorithm adopts the Mask R-CNN model, and its loss function is: in is the classification loss, is the bounding box regression loss, It is the mask segmentation loss; when the cargo occlusion rate is greater than 30%, the occlusion compensation unit is activated to generate a complete mask; Specifically, the classification loss, bounding box regression loss, and mask segmentation loss are trained collaboratively to achieve multi-task joint optimization and improve instance segmentation accuracy. In the case of severe occlusion, the GAN learns the texture and shape characteristics of the goods, predicts the occluded parts, and generates a complete mask to compensate for the segmentation error caused by occlusion. This improves the accuracy of the mask in occluded scenes and ensures accurate segmentation of goods even in complex stacking and occlusion situations, providing a reliable basis for subsequent 3D reconstruction and quantity counting.

[0037] S3: Fusion of cargo segmentation mask and 3D point cloud data, executing Poisson surface reconstruction algorithm to construct a 3D cargo model, calculate cargo volume, and estimate weight based on cargo category matching density values; The Poisson surface reconstruction algorithm includes: Downsample the point cloud data to a voxel grid to reduce computational complexity; Optimize the spatial distribution of point clouds through normal estimation and point cloud registration; Construct an octree hierarchy and set the depth to 10 to balance accuracy and efficiency; Solving the Poisson equation Generate isosurfaces where is the point cloud gradient field.

[0038] In specific implementation, voxel grid downsampling reduces the amount of data while retaining the features of the point cloud, lowering the computational burden; normal estimation and point cloud registration optimize the spatial distribution of the point cloud and improve the accuracy of three-dimensional reconstruction; the octree hierarchical structure realizes layered calculation, balancing accuracy and efficiency; the solution of the Poisson equation converts the point cloud into a continuous surface, generating an accurate three-dimensional model, improving the ability to retain small features, and making the three-dimensional model more accurately reflect the actual shape of the goods, thereby improving monitoring accuracy; the accurate three-dimensional model provides a reliable basis for volume calculation and weight estimation.

[0039] S4: Compare the environmental parameter data with the preset threshold. If the threshold is exceeded, an environmental abnormality signal is generated and the impact level is determined based on the cargo type. S5: Compare the quantity of goods with the inventory database. If the deviation exceeds 5%, an abnormal quantity alarm is triggered. The impact levels are divided into three levels: Level 1: If the environment is abnormal and the goods are perishable, a red alarm will be triggered immediately; Level 2: The environment is abnormal but the goods are storable, triggering a yellow alert; Level 3: Environmental parameters are approaching the threshold, triggering a blue prompt.

[0040] In specific implementation, when the cold chain temperature is >-18℃ or the paper humidity is >60%, a first-level warning will be issued through sound and light alarm + SMS + system shutdown; when the metal temperature is >50℃ or the electronic humidity is >45%, a second-level warning will be issued through yellow light warning + work order push; when the parameters are within the 5% buffer zone of the threshold, only a blue prompt will be marked on the web interface for a third-level warning; thus, the disposal intensity is matched according to the risk level, and a buffer mechanism is set to prevent frequent false alarms in the 5% buffer zone, thereby reducing cargo loss, improving emergency disposal efficiency, and reducing misoperation.

[0041] Specifically, different impact levels are defined based on the storage characteristics of the goods and the severity of environmental anomalies, enabling differentiated alarm and response strategies. Level 1 alarms target perishable goods and severe environmental anomalies, employing the most stringent response measures. Level 2 warnings target durable goods and minor environmental anomalies, employing more moderate response measures. Level 3 alerts address situations approaching thresholds, setting buffer zones to prevent frequent false alarms and thereby reducing cargo losses. Timely alarms and responses for perishable goods reduce the risk of deterioration due to environmental anomalies. This improves emergency response efficiency, ensuring appropriate measures are implemented at different levels, ensuring the rational allocation of resources. This reduces misoperation, as the buffer mechanism avoids frequent alarms and mishandling caused by small parameter fluctuations.

[0042] S6: Integrates environmental anomaly signals and quantity anomaly alarms to generate multi-level alarm instructions and output an inventory report containing cargo quantity, volume, weight, and environmental compliance assessment; The inventory report is presented in a three-dimensional visual form, including: A panoramic view of the warehouse, marking the location of goods and quantity deviations; Environmental parameter change curve with superimposed threshold line; Rotating view of cargo 3D model, with volume and weight data marked; Alarm history log, abnormal events sorted by time.

[0043] In practice, the inventory report is presented in a 3D visualization format, including: a panoramic overhead view of the warehouse, which can mark the location and quantity deviation of goods; a curve of environmental parameter changes, which can be superimposed with threshold lines; a rotating view of the goods 3D model, which is used to mark volume and weight data; and an alarm history log, which sorts abnormal events by time. Specifically, by leveraging visualization technologies like the Three.js engine, warehouse spatial information, cargo data, environmental parameters, and alarm logs are presented in intuitive 3D graphics and charts. A panoramic bird's-eye view facilitates an overall understanding of the warehouse layout and cargo quantity deviations; environmental parameter curves and threshold lines are compared to visually demonstrate environmental conditions; 3D model rotations and annotated data clearly display cargo volume and weight; and alarm history logs are sorted by time, making it easier to trace abnormal events, thereby improving data readability and comprehensibility. The visual display allows staff to quickly access key information. This facilitates data analysis and decision-making, and intuitive charts and models help identify potential issues and trends, improving the efficiency and scientific nature of warehouse management.

[0044] In summary, the present invention proposes an integrated method of intelligent warehouse monitoring and inventory based on multi-technology integration, which realizes the automation of the entire process from data collection, processing, analysis to decision-making and alarming through the collaborative work of multiple modules. YOLOv7 and MaskR-CNN are combined to realize the accurate identification and segmentation of goods, the Poisson surface reconstruction algorithm realizes three-dimensional modeling and volume calculation, the dynamic threshold and impact level assessment realize the intelligent analysis of environmental anomalies, and the multi-dimensional alarm and report generation realize the visualization of anomaly handling and inventory results, realizing the integration and intelligence of warehouse monitoring and inventory, and improving the overall operational efficiency. Through the integration of multiple technologies, the accuracy and reliability of data processing are improved, the loss of goods is reduced, the efficiency of emergency response is improved, and misoperation is reduced.

[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The integrated warehouse intelligent monitoring and inventory system based on multi-technology integration is characterized by: include: Multi-source data acquisition module, including fixed sensor networks and mobile acquisition platforms; The fixed sensor network is deployed at a fixed location in the warehouse and is used to collect environmental parameter data in real time, including temperature, humidity, smoke concentration, and door magnetic status. The mobile data collection platform includes a rail-mounted pulley, a drive motor, and an onboard laser radar and binocular camera, and is used to move within the warehouse and collect cargo images and three-dimensional point cloud data. The image processing and recognition module is connected to the multi-source data acquisition module and is used to perform the following operations: Performing target detection on the cargo image based on the YOLOv7 deep learning model to identify the cargo category, location, and quantity; Use instance segmentation algorithm to extract cargo outline and generate cargo segmentation mask; The 3D reconstruction and volume calculation module is connected to the image processing and recognition module and is used to: fusing the cargo segmentation mask with the three-dimensional point cloud data, and generating a three-dimensional model of the cargo using a Poisson surface reconstruction algorithm; Calculating the volume of the three-dimensional model and matching a preset density value according to the cargo category to estimate the cargo weight; The environmental anomaly analysis module is connected to the multi-source data acquisition module and is used to: Comparing the environmental parameter data with a preset threshold value, and generating an environmental abnormality signal if the threshold value is exceeded; Determine the impact of environmental abnormalities on the cargo based on the cargo type; The intelligent decision-making and alarm module is connected to the image processing and recognition module, the 3D reconstruction and volume calculation module, and the environmental anomaly analysis module, and is used to: Compare the quantity of goods with the inventory database and generate an alarm for abnormal quantity; Integrate environmental abnormal signals and impact levels to generate multi-level alarm instructions; Output cargo inventory report including quantity, volume, weight and environmental compliance assessment.

2. The integrated warehouse intelligent monitoring and inventory system based on multi-technology integration according to claim 1 is characterized in that: The rail-type pulley of the mobile acquisition platform moves along a preset path through the drive motor, and the path covers all cargo stacking areas in the warehouse; the binocular camera collects cargo images at a frame rate of 30fps, and the lidar scans at a frequency of 10Hz to generate three-dimensional point cloud data with millimeter-level accuracy.

3. The integrated warehouse intelligent monitoring and inventory system based on multi-technology integration according to claim 1 is characterized in that: The image processing and recognition module also includes: The occlusion compensation unit is used to predict the texture and shape of the occluded part based on the generative adversarial network (GAN) when there is occlusion in the cargo image, and to correct the cargo quantity recognition result; The category calibration unit is used to secondary verify the category of the goods through a support vector machine (SVM) classifier based on the structural characteristics of the three-dimensional model of the goods.

4. The integrated warehouse intelligent monitoring and inventory system based on multi-technology integration according to claim 1 is characterized in that: In the three-dimensional reconstruction and volume calculation module, the Poisson surface reconstruction algorithm includes the following steps: Map the cargo segmentation mask to the 3D point cloud data and extract the point cloud boundary; Construct implicit surface functions through octree space partitioning; Gradient descent method is used to optimize the surface function and generate a smooth three-dimensional mesh model; Compute the convex hull volume based on mesh vertex coordinates.

5. The integrated warehouse intelligent monitoring and inventory system based on multi-technology integration according to claim 1 is characterized in that: The intelligent decision-making and alarm module also includes: Dynamic threshold adjustment unit, used to automatically adjust environmental parameter thresholds according to cargo category: for cold chain cargo, a temperature threshold of -18°C to -22°C is used, and for dry cargo, a humidity threshold of <15% is used; A multimodal alarm unit is used to simultaneously trigger an audible and visual alarm, SMS notifications, and a pop-up visualization interface. The visualization interface overlays a 3D model of the cargo and a heat map of the abnormal environmental location.

6. A method for integrating intelligent warehouse monitoring and inventory based on multi-technology integration, characterized in that: The following steps are involved: S1: A fixed sensor network collects warehouse environmental parameter data in real time, while controlling a mobile acquisition platform to move along a preset path, capturing cargo images with a binocular camera, and collecting 3D point cloud data with a lidar. S2: Use the YOLOv7 model to detect objects in cargo images, identify cargo categories and locations, and generate cargo segmentation masks using an instance segmentation algorithm. S3: Fusion of cargo segmentation mask and 3D point cloud data, executing Poisson surface reconstruction algorithm to construct a 3D cargo model, calculate cargo volume, and estimate weight based on cargo category matching density values; S4: Compare the environmental parameter data with the preset threshold. If the threshold is exceeded, an environmental abnormality signal is generated and the impact level is determined based on the cargo type. S5: Compare the quantity of goods with the inventory database. If the deviation exceeds 5%, an abnormal quantity alarm is triggered. S6: Integrates environmental anomaly signals and quantity anomaly alarms to generate multi-level alarm instructions and output an inventory report containing cargo quantity, volume, weight, and environmental compliance assessment.

7. The integrated warehouse intelligent monitoring and inventory method based on multi-technology integration according to claim 6 is characterized in that: The instance segmentation algorithm in step S2 adopts the Mask R-CNN model, and its loss function is: in is the classification loss, is the bounding box regression loss, It is the mask segmentation loss; when the cargo occlusion rate is greater than 30%, the occlusion compensation unit is activated to generate a complete mask.

8. The integrated warehouse intelligent monitoring and inventory method based on multi-technology integration according to claim 6 is characterized in that: The Poisson surface reconstruction algorithm in step S3 includes: Downsample the point cloud data to a voxel grid to reduce computational complexity; Optimize the spatial distribution of point clouds through normal estimation and point cloud registration; Construct an octree hierarchy and set the depth to 10 to balance accuracy and efficiency; Solving the Poisson equation Generate an isosurface where is the point cloud gradient field.

9. The integrated warehouse intelligent monitoring and inventory method based on multi-technology integration according to claim 6 is characterized in that: The impact level in step S4 is divided into three levels: Level 1: If the environment is abnormal and the goods are perishable, a red alarm will be triggered immediately; Level 2: The environment is abnormal but the goods are storable, triggering a yellow alert; Level 3: Environmental parameters are approaching the threshold, triggering a blue prompt.

10. The integrated warehouse intelligent monitoring and inventory method based on multi-technology integration according to claim 6 is characterized in that: The inventory report in step S6 is displayed in a three-dimensional visual form, including: A panoramic view of the warehouse, marking the location of goods and quantity deviations; Environmental parameter change curve with superimposed threshold line; Rotating view of cargo 3D model, with volume and weight data marked; Alarm history log, abnormal events sorted by time.

Images