3D model real-time display system of box-type logistics system

Abstract

The invention discloses a 3D model real-time display system of a box-type logistics system, and the system comprises a data collection module which is used for collecting the position, motion state and environment information of a turnover box in real time; the data fusion processing module is used for preprocessing the acquired data by utilizing edge computing equipment and realizing multi-sensor data fusion based on Kalman filtering, particle filtering and a deep learning algorithm; the 3D digital twinning display module is used for constructing a high-precision 3D model of the interior of a hospital or a logistics scene based on a Unity 3D engine, and realizing dynamic rendering of the real-time position, the movement track and the state of the turnover box according to data fusion information; the intelligent anomaly detection and early warning module is used for automatically detecting anomaly and giving out early warning in a preset mode; and the data storage module is used for storing the whole-process logistics data by adopting a block chain technology, ensuring that the data cannot be tampered and realizing whole-process tracing. According to the system, full-process, real-time, dynamic and intelligent visual management of the box-type logistics system can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of hospital box logistics management systems, and more specifically to a 3D model real-time display system for a box logistics system. Background Art

[0002] In modern hospitals, the efficient transportation of drugs, medical devices, and other supplies is crucial for ensuring the quality of medical services. Traditional hospital logistics systems mostly rely on manual handling and simple conveyor belts, which have problems such as low efficiency, easy errors, and difficulty in real-time monitoring. With the expansion of hospital scale and the increase in medical needs, traditional logistics methods have been difficult to meet the requirements of efficient, accurate, and safe material transportation. Therefore, box logistics and pneumatic logistics have emerged; however, the box logistics system still has the following problems:

[0003] 1. Lack of real-time monitoring and visualization

[0004] Inaccurate location information: Existing systems usually rely on RFID or sensors for positioning, but these technologies may not provide high-precision location information in complex environments (such as multi-floor and multi-channel).

[0005] No intuitive display: Most systems lack intuitive display means such as 3D topology maps, resulting in managers having difficulty comprehensively understanding the real-time location and operating status of turnover boxes.

[0006] 2. Single and insufficient monitoring means

[0007] Lack of video monitoring: There is a lack of camera monitoring at key positions. Even if there is video monitoring, it is not integrated with the logistics system, making it impossible to view and trace in real time.

[0008] Insufficient anomaly detection: Existing anomaly detection means are relatively limited, mainly relying on sensor alarms, and it is impossible to detect and handle some complex anomalies (such as equipment failures and path blockages) in a timely manner.

[0009] 3. Imperfect anomaly handling mechanism

[0010] Single alarm method: Most systems only notify anomalies through a display screen or text message, lacking more intuitive reminder methods such as voice alarms and box flashing, resulting in a slow response speed.

[0011] Irregular handling process: When an anomaly occurs, there is a lack of standardized operating procedures, which easily causes chaos and affects the overall logistics efficiency.

[0012] 4. Poor data recording and traceability capabilities

[0013] Incomplete sending and receiving confirmation: The existing system has incomplete records of the sending and receiving times of the turnover boxes, making it difficult to achieve full traceability and increasing the risk of material loss or delay.

[0014] Scattered data management: The data management in different links is relatively independent, lacking a unified platform for centralized management and analysis, which affects the scientificity and accuracy of decision-making.

[0015] 5. Poor user experience

[0016] Complicated operation: The user interface of the existing system is not friendly enough, and the operation steps are cumbersome, increasing the workload of medical staff.

[0017] Unprompted feedback: The system feedback mechanism is not perfect, and users cannot quickly obtain support and solutions when encountering problems.

[0018] Therefore, in view of the above problems existing in the hospital box logistics, it is urgent for those skilled in the art to solve them. Summary of the Invention

[0019] In view of this, the present invention provides a 3D model real-time display system for a box logistics system, which comprehensively improves the efficiency, accuracy and safety of hospital logistics management through intelligent and visual technologies; and can solve the above technical problems.

[0020] In order to achieve the above object, the present invention adopts the following technical solutions:

[0021] An embodiment of the present invention provides a 3D model real-time display system for a box logistics system, including:

[0022] A data acquisition module, configured to collect the position, motion state and environmental information of the turnover box in real time through a variety of sensors arranged on the logistics track and nodes; the variety of sensors include: RFID, lidar, UWB and cameras;

[0023] A data fusion processing module, which preprocesses the collected data by using edge computing devices and realizes multi-sensor data fusion based on Kalman filtering, particle filtering and deep learning algorithms;

[0024] A 3D digital twin display module, which constructs a high-precision 3D model of the hospital interior or logistics scene based on the Unity3D engine, and realizes the dynamic rendering of the real-time position, motion trajectory and state of the turnover box according to the data fusion information;

[0025] An intelligent anomaly detection and warning module, which uses a deep learning model to perform real-time analysis on the data fusion information, automatically detects anomalies and issues warnings through a preset method; the preset method includes: voice alarm, graphic flashing, SMS and APP push;

[0026] The data storage and certification module uses blockchain technology to store and certify the whole-process logistics data, ensuring the data cannot be tampered with and realizing full-process traceability.

[0027] Furthermore, it also includes:

[0028] The multi-modal interaction module provides support for VR / AR interaction functions based on the 3D digital twin display module. Through VR glasses or AR devices, managers can achieve an immersive experience of the logistics site.

[0029] The voice assistant module receives voice commands in natural language and queries or locates the status of the corresponding turnover box.

[0030] Furthermore, the data fusion and processing module also includes:

[0031] The dynamic perception unit is used to real-time scan the environmental changes, locate the target objects in the environment through lidar and visual positioning cameras, and update the 3D digital twin display module.

[0032] The video surveillance and analysis unit is used to determine whether the appearance of the turnover box is intact through machine vision, and determine whether to blur the facial information of the people appearing in the video according to the viewing permission.

[0033] Furthermore, the 3D digital twin display module also includes:

[0034] The perspective switching unit is used to support zooming and free perspective switching, and real-time present the dynamic distribution of the transfer boxes and the path planning effect.

[0035] The node identification unit is used to display the real-time monitoring screen identification of the sorting station, buffer area, and elevator entrances and exits, and retrieve the real-time monitoring video after being clicked.

[0036] Furthermore, the 3D digital twin display module also includes:

[0037] The path prediction visualization unit is used to mark the planned path of the turnover box, the estimated arrival time, and the congestion risk area in the model.

[0038] Furthermore, the intelligent anomaly detection and warning module includes:

[0039] The delay detection unit is used to detect whether there is a delay in the turnover box arriving at the target station.

[0040] The path deviation detection unit is used to detect whether the turnover box deviates from the planned transfer path.

[0041] The alarm unit is used to trigger local edge alarms and global warnings when there is a transfer delay and / or path deviation; the alarm methods include voice prompts, flashing of relevant boxes in the 3D interface, text messages, and APP push.

[0042] Furthermore, the intelligent anomaly detection and early warning module further includes:

[0043] A device failure prediction unit for predicting device failures for real-time data using a trained LSTM model.

[0044] Furthermore, the data evidence storage module includes:

[0045] A blockchain unit for writing the sending / receiving time of the turnover box, the operator's identity, and the hash value of the device status into the blockchain;

[0046] A query unit for providing an audit interface for authorized users to query the encrypted records of the entire process.

[0047] Furthermore, the data evidence storage module further includes:

[0048] A data analysis and report generation unit for statistically analyzing the stored logistics data, generating various statistical reports, and assisting management personnel in optimizing decisions.

[0049] Furthermore, it further includes:

[0050] A site status information display module for displaying the site name, number, bin status, and remarks indicating abnormal or normal; and visualizing the global load status through a heat map and a traffic curve.

[0051] Through the above technical solutions, compared with the prior art, the present invention has the following technical effects:

[0052] By integrating new technologies such as multi-sensor data acquisition, digital twin modeling, intelligent anomaly detection and early warning, edge computing, and blockchain evidence storage, the full-process, real-time, dynamic, and intelligent visualization management of the box-type logistics system is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0054] Figure 1 It is a structural block diagram of the 3D model real-time display system of the box-type logistics system provided by the present invention.

[0055] Figure 2 It is an architecture diagram of the 3D model real-time display system of the box-type logistics system provided by the present invention.

[0056] Figure 3 Block diagram of the data fusion processing module provided by the present invention.

[0057] Figure 4 3D digital twin display effect diagram provided by the present invention.

[0058] Figure 5 Block diagram of the 3D digital twin display module provided by the present invention.

[0059] Figure 6 Block diagram of the intelligent anomaly detection and warning module provided by the present invention.

[0060] Figure 7 Block diagram of the data storage and certification module provided by the present invention. Detailed implementation manners

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] An embodiment of the present invention discloses a 3D model real-time display system for a box-type logistics system, which adopts a hierarchical architecture design and includes a perception layer, a data layer, a service layer, and an application layer, as Figure 1 shown:

[0063] Perception layer: Deploy sensors such as RFID tags, lidar, UWB base stations, and cameras in the hardware structure of the hospital box-type logistics to collect environmental and equipment data of the logistics track and nodes in real time.

[0064] Data layer: Implement data preprocessing and fusion through edge computing devices, and combine blockchain nodes to implement data storage and certification. By deploying edge computing units at key monitoring nodes, low-latency data processing and preliminary anomaly determination are realized to ensure the system response speed and stability.

[0065] Service layer: Build a digital twin model based on the Unity3D engine, and combine a deep learning model for anomaly detection and prediction.

[0066] Application layer: Provide 3D visualization interfaces, multi-modal interaction interfaces, and data analysis report functions.

[0067] Referring to Figure 2 shown, the 3D model real-time display system of the box-type logistics system includes the following modules:

[0068] 1. Data acquisition module, which is used to collect the position, motion state and environmental information of the turnover box in real time through a variety of sensors set on the logistics track and nodes; the variety of sensors include: RFID, lidar, UWB and cameras.

[0069] Sensor layout and installation:

[0070] Install RFID readers at the logistics track and key nodes (such as turns, sorting stations, buffer areas, transplant machines, hoist entrances and exits, etc.) to achieve identity identification and location punching of each turnover box. For example: The RFID tag adopts a dual-frequency design (125kHz low-frequency wake-up + 920MHz high-frequency communication), and the reader is deployed every 5 meters at the track nodes to achieve accurate identification of the turnover box ID.

[0071] Deploy lidar (LiDAR), Velodyne VLP-16, with an installation height of 2.5m and a horizontal scanning angle of 270°, to generate a point cloud map (resolution 5cm), which is used to scan the surrounding environment and static obstacles such as track racks and walls to ensure high-precision positioning within complex geometric structures.

[0072] Use ultra-wideband (UWB) positioning equipment for high-precision distance measurement, which is suitable for multi-floor and multi-channel scenarios. The UWB positioning base stations are arranged according to the TDoA principle, and 4 base stations are deployed at the corridor intersections to form a positioning grid, with a time synchronization accuracy of ±10ns, achieving a three-dimensional space positioning error < 15cm.

[0073] Install high-definition cameras at the nodes (hoist entrances and exits) and corridors, which are used for video surveillance and can also assist in visual positioning to extract environmental image information. The camera uses Hikvision DS-2CD3326D double-ball cameras, and a fish-eye lens (horizontal field of view 180°) is deployed in the sorting area, and H.265 coding is used to reduce the bandwidth.

[0074] Data acquisition communication: All sensor data is uploaded to the edge computing device through a low-power wireless network (such as ZigBee, LoRa or an Internet of Things gateway based on the MQTT protocol) to ensure low latency and stability of data transmission.

[0075] The synchronization mechanism uses the Network Time Protocol (NTP) or the Precision Time Protocol (PTP) to stamp a unified timestamp on each sensor based on the server system, which is convenient for subsequent data fusion.

[0076] 2. Data fusion processing module, which uses the edge computing device to preprocess the collected data and realizes multi-sensor data fusion based on Kalman filtering, particle filtering and deep learning algorithms, which can improve the positioning accuracy and robustness in complex environments.

[0077] Refer to Figure 3Specifically, it includes:

[0078] Edge preprocessing unit: First, filter the original data (such as mean filtering and Kalman filtering preprocessing) and remove outliers to reduce the impact of noise. Data normalization ensures that different sensor data have a unified dimension, facilitating the input of subsequent deep learning models.

[0079] Multi-sensor data fusion unit: Use the Kalman filter to achieve continuous state estimation and smoothly predict the movement trajectory of the turnover box; combine the particle filter method to process non-Gaussian distributed noise data and adapt to dynamic environmental changes; for example, the data of each channel can be further fused by deploying a deep learning model (such as the combination of a convolutional neural network and a time series model) to output high-precision position information and state judgment results.

[0080] Among them, the Kalman filter algorithm for the smooth prediction of the movement trajectory of the turnover box is as follows:

[0081] (1) Model definition: Assume that the turnover box moves in a uniform straight line on the horizontal plane (X-Y plane), and the state vector is defined as:

[0082]

[0083] Where: p x , p y represent the positions on the X and Y axes (unit: meter); v x , v y represent the velocities on the X and Y axes (unit: meter per second).

[0084] Adopt a uniform motion model, and the state transition equation is:

[0085] x k = F·x k-1 + w k w k represents the process noise.

[0086] The state transition matrix F is:

[0087]

[0088] Among them, Δt is the sensor sampling interval (example value: Δt = 0.1 second).

[0089] Process noise: Assume that the acceleration perturbation follows a Gaussian distribution, and the process noise covariance matrix Q is:

[0090]

[0091] Where, σ a is the standard deviation of the acceleration noise (example value: σ a = 0.2m / s2 )。

[0092] (2) Measurement model

[0093] Observed variable: Assume that the UWB sensor is used to measure the position, and the observation vector is:

[0094]

[0095] Observation matrix H: Only observe the position and ignore the speed:

[0096]

[0097] Measurement noise covariance R: According to the UWB positioning accuracy (example error: ±0.15 m):

[0098]

[0099] (3) Kalman filtering process

[0100] Initialization: Initial state : Set p according to the first measurement value x , p y , speed v x , v y = 0.

[0101] Initial covariance P0:

[0102]

[0103] (Assume the initial position uncertainty is 10 m and the speed uncertainty is 1 m / s).

[0104] Iterative steps:

[0105] 1) Prediction stage:

[0106] State prediction:

[0107]

[0108] Covariance prediction:

[0109]

[0110] 2) Update stage:

[0111] Calculate the Kalman gain:

[0112]

[0113] State correction:

[0114]

[0115] Covariance Update:

[0116]

[0117] (4) Physical Meaning and Tuning of Parameters

[0118] 4.1 Process Noise Covariance Q: Reflects the tolerance of the model to acceleration disturbances. If the turnover box moves with frequent speed changes (such as sudden stops and accelerations), it is necessary to increase σ a . Tuning method: The standard deviation of acceleration can be calculated through historical data.

[0119] 4.2 Measurement Noise Covariance R: Directly depends on the sensor accuracy. When fusing lidar (error ±0.05 meters), it is necessary to adjust R to:

[0120]

[0121] 4.3 Covariance Initialization P0: When the initial uncertainty is large (such as cold start), it is necessary to increase the diagonal values of P0 to accelerate convergence.

[0122] (5) Extension of Multi-Sensor Fusion

[0123] If UWB and lidar are used simultaneously:

[0124] Observation Vector Extension:

[0125]

[0126] Observation Matrix:

[0127]

[0128] Measurement Noise R:

[0129]

[0130] (6) Output Results

[0131] Smoothing Position: : The position after filtering, with noise suppression.

[0132] Velocity Estimation: : Used to predict future trajectories.

[0133] Uncertainty Measure: Covariance Matrix P k The diagonal elements reflect the estimation errors of position and velocity.

[0134] Finally, through Kalman filtering, the measurement noise is effectively suppressed, the trajectory is smoothed, and the velocity estimation is stable, meeting the requirements of the hospital logistics system for high-precision real-time monitoring.

[0135] Particle Filtering Process:

[0136] 1) Particle initialization: Generate N = 1000 particles, and the initial positions follow a Gaussian distribution N(μ0, ∑0), where μ0 is the initial measurement value, and ∑0 = diag([0.5 2 , 0.5 2 ).

[0137] 2) State prediction: Each particle propagates according to the motion model, and process noise w k ~N(0, Q k ) is added.

[0138] 3) Weight calculation: Calculate the particle weights according to the observation value z k and normalize them.

[0139] 4) Resampling: Use systematic resampling to retain high-weight particles and avoid degeneracy.

[0140] 5) State estimation: Output the particle mean as the final position estimate.

[0141] Particle filtering is used to handle non-Gaussian noise (such as multipath interference), runs in parallel with Kalman filtering, and the results are weighted and fused.

[0142] The dynamic perception unit combines lidar and visual positioning cameras to collect environmental change data in real time, ensuring that the 3D model can be updated in a timely manner, such as new obstacles, channel changes, etc.;

[0143] Use image recognition algorithms to identify target objects in the environment and map their geometric positions into the 3D model to achieve dynamic environmental updates.

[0144] The video surveillance and analysis unit applies machine vision algorithms (such as convolutional neural networks) to analyze the video stream collected by the camera to detect abnormalities such as damage and deformation of the appearance of the turnover box. In addition, regarding the protection of personnel privacy, YOLOv5s is used to detect the face area, and the differential privacy algorithm is used to perform Gaussian blur (σ = 3.0) on the ROI area to occlude the real-time facial information of unauthorized viewers to ensure privacy protection.

[0145] 1) YOLOv5s face detection

[0146] Model configuration:

[0147] Input resolution: 640×640, and the anchor boxes are optimized for faces (size 16×16 to 64×64).

[0148] Training dataset: WIDER FACE (annotate the face area, and data augmentation includes random cropping and rotation).

[0149] Inference parameters:​

[0150] Confidence threshold: 0.6, NMS threshold: 0.4.

[0151] After detecting a human face, the ROI area is cropped to 64×64 pixels.

[0152] 2) Differential privacy blurring

[0153] Gaussian blur parameter: kernel size

[0154] 15×15, standard deviation σ = 3.0.

[0155] Permission control: Based on the RBAC model, blurring is automatically triggered when unauthorized personnel view.

[0156] 3. 3D digital twin display module, which constructs a high-precision 3D model of the hospital interior or logistics scenario based on the Unity3D engine, and realizes the dynamic rendering of the real-time position, movement trajectory and status of the turnover box according to the data fusion information; the display effect is as Figure 4 shown. Based on the basic models of each ward and pharmacy in the hospital, by constructing a virtual model of the logistics system, it realizes real-time data interaction with the actual equipment and environment, forming a closed-loop monitoring and management mode of virtual-real interconnection; among them, DT represents the transfer machine and LT represents the elevator; the upper area in the figure is a clickable button for displaying the corresponding data information or display effect.

[0157] Refer to Figure 5 shown, the 3D digital twin display module specifically includes:

[0158] 3D model construction unit, based on the Unity3D engine, constructs a high-precision three-dimensional model of the hospital or logistics center through laser scanning data and CAD drawings; all key nodes and paths are pre-calibrated in the model and matched with the actual physical dimensions.

[0159] Dynamic rendering unit, according to the real-time data after fusion processing, updates the position, movement trajectory and status identification of the turnover box in the 3D scene in real time through the built-in script of Unity3D. Apply particle effects or custom Shaders to visually highlight abnormal states.

[0160] Viewpoint switching unit, used for multi-angle dynamic display, supporting interactive switching of zooming, rotation, and free viewpoints (including horizontal view, top view, and oblique view); for example, users can adjust the viewpoint by dragging the mouse or gesture operation to observe the dynamic distribution and path planning effect of the transfer box in real time; the server dynamically loads 3D model data with the corresponding resolution according to the current viewpoint to ensure smooth rendering.

[0161] Node identification unit, which accesses the video surveillance footage into the 3D model, embeds camera graphic identifiers at key nodes (such as sorting stations, buffer areas, and elevator entrances and exits), and the identification unit is connected to the corresponding cameras in real time; after the user clicks on the identifier, the real-time video stream is called through the interface (for example, transmitted via the RTSP protocol), and the surveillance video is displayed in a pop-up window to achieve the linkage between the video and the 3D scene. And when the transport box passes through the camera surveillance range, the corresponding camera graphic identifier is highlighted or prominently displayed in a distinguishable color.

[0162] Path prediction visualization unit, which highlights the planned path of the transfer box, and based on historical transport data and real-time status, predicts the expected arrival time of the transfer box through regression analysis or machine learning models (such as random forest, neural network) and displays it on one side of the transfer box model; in addition, at the same time, it identifies potential congestion risk areas according to real-time load data and intuitively marks them on the 3D model using color gradients (such as red indicating high risk); this unit is dynamically updated based on data to ensure the timeliness and accuracy of the displayed information.

[0163] 4. Intelligent anomaly detection and warning module, which uses a deep learning model to perform real-time analysis on the data fusion information, automatically detects anomalies and issues warnings through preset methods; the preset methods include: voice alarm, graphic flashing, SMS, and APP push.

[0164] Refer to Figure 6 Specifically include:

[0165] Delay detection unit, which is used to evaluate whether there is a delay in the transfer box reaching the target station according to the detection results of the path prediction visualization unit; by recording the scheduled arrival time and the actual arrival time of the transfer box in real time, the delay time is calculated by comparison; when the delay time exceeds the preset threshold, the abnormal event is automatically recorded.

[0166] Path deviation detection unit, which is used to detect whether the transfer box deviates from the planned transmission path; the system generates a standard transmission path during the planning stage, and obtains the actual path through real-time data collection, and uses the Euclidean distance and trajectory similarity algorithm to judge the deviation degree; when the deviation degree exceeds the tolerance range, it is regarded as an anomaly.

[0167] Equipment failure prediction unit, which uses a trained LSTM model to predict equipment failures for real-time data. Key operating parameters (such as vibration, temperature, current, etc.) of the equipment (such as transmission machinery, conveyor belts, lifting equipment) are collected to form time series data;

[0168] Using a deep learning model based on Long Short-Term Memory Network (LSTM), train the device operation data. The input layer of the model receives multi-dimensional sensor data, and the hidden layer adopts a multi-layer LSTM structure (which can be combined with Dropout to prevent overfitting), and the output predicts the failure probability within a future period of time;

[0169] LSTM model of the intelligent anomaly detection and warning module:

[0170] Model structure:

[0171] Input layer: 8-dimensional time series data (vibration, temperature, current, voltage, rotational speed, load, noise, operation duration), time step T = 60.

[0172] Hidden layer:

[0173] Double-layer LSTM (64 units in each layer, Dropout = 0.3).

[0174] Fully connected layer (32 nodes, ReLU).

[0175] Output layer: Sigmoid activation, output failure probability p ∈ [0, 1].

[0176] Training parameters:

[0177] Loss function: Binary Cross-Entropy.

[0178] Optimizer: Nadam (learning rate = 0.0005).

[0179] Batch size: 32, early stopping strategy (patience = 10).

[0180] Threshold setting: p ≥ 0.85 triggers an early warning.

[0181] During the online real-time prediction process, when the failure probability exceeds the set threshold, trigger a device failure early warning and record the relevant data for subsequent maintenance and preventive measure formulation.

[0182] Alarm unit, used to trigger local edge alarms and global early warnings when there are transmission delays and / or path deviations; alarm methods include voice prompts, flashing of relevant boxes in the 3D interface, text messages, and APP push. Early warning method: The system has a built-in preset alarm strategy: when the anomaly score exceeds the set threshold, immediately trigger an early warning through the following methods:

[0183] Path deviation detection algorithm process:

[0184] 1) Trajectory similarity calculation:

[0185] Use Dynamic Time Warping (DTW) to compare the difference between the planned path P and the actual path Q.

[0186] Similarity score: Threshold S th = 0.7.

[0187] 2) Euclidean distance threshold: A vertical distance greater than 0.5 m between the real-time position and the planned path is regarded as deviation.

[0188] Voice alarm: Play a preset voice message through the on-site built-in speaker;

[0189] Graphic flashing: In the 3D display interface, the relevant turnover box model is highlighted by flashing or color gradient;

[0190] SMS and APP push: Send SMS and push notifications to the mobile phones of relevant management personnel through the background server to ensure multi-channel transmission of warning information.

[0191] 5. Data storage and evidence module, using blockchain technology to store the whole-process logistics data for evidence, ensuring that the data cannot be tampered with and realizing the whole-process traceability.

[0192] Refer to Figure 7 as shown, specifically including:

[0193] Blockchain unit: Each piece of logistics data (including the sending / receiving time of the turnover box, the identity of the operator, the equipment status, etc.) generates a 256-bit hash value using the national secret SM3 after data fusion processing, and calls the blockchain API to write it into the blockchain network; the blockchain uses a permissioned chain based on the consensus mechanism, and each logistics event (sending / receiving) is uploaded to the chain in real time with a delay < 500 ms. Ensure the authenticity and immutability of the data, and the whole process is traceable.

[0194] Query unit: Provide a Web-based query interface for authorized users, and use encryption authentication technology (such as OAuth2) to ensure secure access; users can retrieve the corresponding blockchain storage records according to conditions such as time, node, operator, etc., to realize the whole-process data audit. Authorized users can obtain historical logistics data and corresponding storage records through the query unit and generate reports to assist decision-making.

[0195] Data analysis and report generation unit: The system regularly extracts logistics data from the blockchain and the background database, and statistics indicators such as transportation timeliness, delay times, equipment failure frequencies, etc.;

[0196] Use data visualization tools (such as Echarts, D3.js) to generate various statistical reports such as heat maps, flow curves, bar charts, etc. for management personnel to analyze and make decisions; it supports custom report templates to meet the data analysis needs of different users.

[0197] 6. Multimodal Interaction Module, which develops interaction functions supporting VR / AR using the Unity3D engine, is compatible with VR head-mounted devices such as Oculus and HTC Vive, as well as mobile-based AR applications; through VR glasses or AR devices, managers can achieve an immersive experience of the logistics site;

[0198] Through the immersive 3D scene display, managers can freely browse the logistics status in the virtual environment, achieving close viewing and real-time operation.

[0199] In addition, gesture recognition and touch input can be integrated to enable intuitive interaction on different terminals.

[0200] 7. Voice Assistant Module, by deploying speech recognition and natural language processing modules, supports users to input query instructions through Mandarin speech, such as "Query the status of the box in Channel 3" or "Locate the position of a certain box"; after the backend parses the instructions, it automatically calls the database and real-time data interface, returns the corresponding status information, and marks the positioning result on the 3D interface.

[0201] 8. Site Status Information Display Module, which integrates the site information database and obtains the site name, number, box position status, and remarks information in real time; in the 3D digital twin model, the status of each site is displayed using an overlay, and normal and abnormal statuses are distinguished by different colors and icons.

[0202] Global load visualization, generates a heat map based on real-time traffic data, and intuitively displays the logistics load in each area using color gradients (such as blue to red); uses a traffic curve graph to show the historical and current traffic change trends to assist managers in optimizing resource scheduling.

[0203] For example:

[0204] Taking the in-hospital box logistics system of a certain hospital as an example, the implementation process and key technology realization of each module are described in detail:

[0205] 1. Overall System Architecture

[0206] Hardware Deployment:

[0207] Install RFID readers, lidar, UWB locators, and high-definition cameras at each key logistics node within the hospital;

[0208] Configure edge computing devices (with GPU acceleration capabilities) in corridors and sorting areas for preprocessing and real-time data analysis;

[0209] Deploy cloud platform servers in the data center to build data fusion, deep learning model training, and blockchain networks;

[0210] Configure local speakers, display terminals, and mobile terminals to form an omni-channel early warning system.

[0211] Software platform:

[0212] The front-end uses the Unity3D engine to develop a 3D digital twin display platform, integrating VR / AR interaction modules and perspective switching functions;

[0213] The back-end adopts a microservices architecture to achieve data collection, fusion, anomaly detection, alarm, and blockchain data storage and proof. Services communicate with each other through RESTful API or MQTT;

[0214] The database and the blockchain network jointly store and manage logistics data, and provide Web query and report generation interfaces.

[0215] 2. Implementation of data collection and fusion

[0216] The data collected by sensors is transmitted to edge computing devices in an encrypted manner through a dedicated gateway;

[0217] Edge devices use pre-deployed Kalman filtering and particle filtering algorithms to smooth the data and normalize the data at the same time;

[0218] The processed data is uploaded to the cloud platform through a secure channel. The deep learning model (deployed in the cloud or at the edge) fuses the information of each sensor in real time and outputs the accurate position information and status judgment of the turnover box.

[0219] 3. Implementation of 3D digital twin and interaction

[0220] Use Unity3D combined with laser scanning data to build a 3D model of the hospital interior, and pre-calibrate all stations and paths; the front-end program calls the cloud platform interface in real time to map the data fusion results into the 3D model to realize the dynamic update of the turnover box movement trajectory;

[0221] Users can view real-time monitoring videos, station status, and path prediction information through the PC side, tablet, or VR / AR terminal, using the perspective switching unit and node identification unit.

[0222] 4. Implementation of intelligent anomaly detection and early warning

[0223] The deep learning module uses trained CNN and LSTM networks to perform anomaly scoring on the collected data in real time; when transportation delays, path deviations, or equipment anomalies are detected, the edge device immediately triggers a local alarm and notifies the cloud platform for global early warning;

[0224] The system selects the corresponding alarm method according to the exception level: voice prompts are played by on-site speakers; the target box in the 3D interface flashes or highlights; at the same time, alarm information is notified to the duty personnel through text messages and APP push;

[0225] The fault prediction module outputs a prediction value according to the LSTM model, and pushes a fault risk warning to the equipment maintenance personnel in advance to ensure that maintenance measures are taken in advance.

[0226] 5. Implementation of data evidence preservation, analysis and report generation

[0227] After each piece of logistics data processed by fusion is calculated by digest, it is written into the blockchain through the blockchain unit to form an immutable evidence record;

[0228] Authorized users can retrieve blockchain data through the query unit to achieve full-process data auditing;

[0229] The data analysis module regularly extracts evidence data and background logs, generates heat maps, traffic curves, and statistical reports, providing data support for management to optimize scheduling and decision-making.

[0230] 6. Implementation of terminal display and user interaction

[0231] The site status information display module reads the site status data from the database in real time and superimposes it on the 3D model for display;

[0232] Users can not only adjust the viewing angle through touch and mouse operations through the multi-modal interaction module, but also use the voice assistant module to achieve quick query and positioning;

[0233] The system supports cross-platform synchronization to ensure that consistent real-time logistics status displays can be obtained on PC terminals, mobile terminals, and VR / AR terminals.

[0234] The usage scenario is described as follows: emergency drug transportation in the hospital operating room:

[0235] A certain tertiary hospital operating room suddenly needs to call a special anticoagulant drug. The pharmacy loads the drug into a turnover box (number BX-202503108001) and transports it from the 3rd floor pharmacy to the 8th floor operating room through the box-type logistics system. The full process of the system is as follows:

[0236] (1) Data acquisition stage

[0237] RFID trigger: When the turnover box is placed on the track, the Impinj R420 reader (operating frequency 902-928 MHz) at the pharmacy exit reads the RFID tag (NTAG 216) of the box and records the shipping time (2025-03-10 14:05:23.456)

[0238] UWB Positioning: After the box enters the track, four Decawave TREK1000 base stations deployed in the corridor calculate the three-dimensional coordinates in real time through the TDoA algorithm (X = 32.15m, Y = 15.78m, Z = 3.2m ± 0.1m).

[0239] LiDAR Synchronization: The SICK TIM571 LiDAR scans the track area at a frequency of 25Hz.

[0240] (2) Data Fusion Processing

[0241] Technical Implementation:

[0242] Multi-Sensor Calibration: The edge computing node (NVIDIA Jetson AGX Xavier) runs the following algorithm:

[0243] Time Alignment: Synchronize the clocks of each sensor through the PTP protocol (error < 1ms);

[0244] Kalman Filtering: Fuse the UWB coordinates and the LiDAR point cloud data to correct the UWB multipath interference caused by metal shelves;

[0245] Privacy Processing: When passing by the nurse station, the Hikvision camera triggers face blurring processing: YOLOv5 detects 3 face regions; apply Gaussian blur with σ = 3.0 to the ROI region;

[0246] (3) 3D Digital Twin Display

[0247] Dynamic Rendering: The Unity3D engine displays the moving trajectory of the turnover box (blue light band effect) in the floor model according to the fused coordinate data; automatic perspective switching: when the box approaches the elevator, you can click on the camera icon to display the real-time monitoring screen;

[0248] Congestion Warning: When the system detects that 3 boxes are queuing in the transfer area on the 5th floor, trigger:

[0249] Mark the red warning area in the 3D model; display that the estimated arrival time changes from 2 minutes to 3 minutes and 15 seconds;

[0250] (4) Anomaly Detection and Handling

[0251] Anomaly Event: When the turnover box stalls on the 4th floor track for more than 90 seconds (preset threshold):

[0252] LSTM Prediction: Analyze the motor current fluctuation data (sampling frequency 10kHz) to judge possible blockage of the conveyor belt;

[0253] Multi-Level Alarm:

[0254] Local alarm: The stagnant position flashes frequently on the 3D interface (Shader to achieve the Glow effect);

[0255] Global early warning: Push to the maintenance team via text message (Content: "4F_East_Conveyor is suspected of jamming, Priority: High");

[0256] Emergency handling: For example, update the recommended path in the 3D model (guided by green dashed arrows);

[0257] (5) Blockchain evidence storage

[0258] When the turnover box arrives at the operating room, generate an evidence storage data packet and call the smart contract to write it into the blockchain; Auditors can query the complete transportation record through the hospital intranet to verify whether the data hash values match.

[0259] Technical effect comparison

[0260]

[0261] This example fully demonstrates the entire process from drug storage to exception handling. Through the synergistic effect of multi-sensor fusion, edge computing optimization, 3D real-time rendering, and blockchain evidence storage, it verifies the innovative value of the present invention in improving hospital logistics efficiency and ensuring transportation safety.

[0262] The 3D model real-time display system of the box-type logistics system provided by the present invention:

[0263] Real-time performance and accuracy: The combination of multi-sensor fusion and edge computing realizes high-precision and low-latency data acquisition and processing, ensuring the real-time dynamic display of the 3D model.

[0264] Intelligent early warning: The combination of deep learning and traditional filtering algorithms accurately detects anomalies such as delays, path deviations, and equipment failures, ensuring the safety of logistics transportation.

[0265] Interaction experience: Through the VR / AR and voice interaction modules, provide an immersive user experience, reduce the operation complexity, and improve the management efficiency.

[0266] Data security and traceability: The blockchain evidence storage mechanism ensures the immutability of data, constructs a fully transparent traceability system, and provides a reliable basis for subsequent analysis and decision-making.

[0267] Multi-dimensional visualization: Utilize technologies such as heat maps, traffic curves, and path prediction to achieve an intuitive display of the global load status, assisting in optimizing scheduling and resource allocation.

[0268] The present invention can achieve high-precision real-time monitoring of the in-hospital or in-logistics-center box-type logistics system, which helps improve transportation efficiency and safety; intelligent anomaly detection and multimodal alarms effectively shorten the response time and reduce the risks of logistics delays and material losses; the application of digital twin and VR / AR technologies significantly improves the user's management experience and decision-making efficiency.

[0269] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method part.

[0270] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A real-time display system for the 3D model of a box-type logistics system, characterized in that, Including: A data acquisition module, which is used to collect the position, motion state and environmental information of the turnover box in real time through a variety of sensors set on the logistics track and nodes; The variety of sensors include: RFID, lidar, UWB and cameras; A data fusion processing module, which uses edge computing devices to preprocess the collected data and realizes multi-sensor data fusion based on Kalman filtering, particle filtering and deep learning algorithms; A 3D digital twin display module, which constructs a high-precision 3D model of the hospital interior or logistics scenario based on the Unity3D engine, and realizes the dynamic rendering of the real-time position, motion trajectory and state of the turnover box according to the data fusion information; An intelligent anomaly detection and warning module, which uses a deep learning model to analyze the data fusion information in real time, automatically detects anomalies and issues warnings through preset methods; the preset methods include: voice alarm, graphic flashing, SMS and APP push; A data certification module, which uses blockchain technology to certify the whole-process logistics data, ensures that the data cannot be tampered with, and realizes the whole-process traceability.

2. The real-time display system for the 3D model of a box-type logistics system according to claim 1, characterized in that It also includes: A multi-modal interaction module, which provides support for VR / AR interaction functions based on the 3D digital twin display module. Through VR glasses or AR devices, managers can realize an immersive experience of the logistics site; A voice assistant module, which receives voice commands in natural language and queries or locates the status of the corresponding turnover box.

3. The real-time display system of the 3D model of a box-type logistics system according to claim 1, characterized in that, The data fusion processing module also includes: A dynamic perception unit, which is used to scan the environmental changes in real time and locate the target objects in the environment through lidar and visual positioning cameras, and update the 3D digital twin display module; A video surveillance and analysis unit, which is used to determine whether the appearance of the turnover box is intact through machine vision, and determine whether to blur the facial information of the personnel appearing in the video according to the viewing permission.

4. The real-time display system of the 3D model of a box-type logistics system according to claim 1, characterized in that, The 3D digital twin display module also includes: A perspective switching unit, which is used to support zooming and free perspective switching, and presents the dynamic distribution of the transfer boxes and the path planning effect in real time; A node identification unit, which is used to display the real-time monitoring screen identification of the sorting station, buffer area, and elevator entrance and exit, and calls up the real-time monitoring video after being clicked.

5. The real-time display system for the 3D model of a box-type logistics system according to claim 1, characterized in that, The 3D digital twin display module also includes: A path prediction visualization unit, which is used to mark the planned path, estimated arrival time and congestion risk area of the turnover box in the model.

6. The real-time display system for the 3D model of a box-type logistics system according to claim 1, characterized in that, The intelligent anomaly detection and warning module includes: A delay detection unit, which is used to detect whether there is a delay in the turnover box arriving at the target station; A path deviation detection unit, which is used to detect whether the turnover box deviates from the planned transfer path; An alarm unit, which is used to trigger local edge alarms and global warnings when there is a transfer delay and / or path deviation; the alarm methods include voice prompts, flashing of relevant boxes in the 3D interface, SMS and APP push.

7. The real-time display system of the 3D model of a box-type logistics system according to claim 1, characterized in that, The intelligent anomaly detection and warning module also includes: A device failure prediction unit, which uses a trained LSTM model to predict device failures for real-time data.

8. The real-time display system for the 3D model of a box-type logistics system according to claim 1, characterized in that, The data certification module includes: A blockchain unit, which is used to write the sending / receiving time of the turnover box, the identity of the operator and the hash value of the device state into the blockchain; A query unit for providing an audit interface for authorized users to query full-process encrypted records.

9. The real-time display system of the 3D model of a box-type logistics system according to claim 8, characterized in that, The data storage and evidence module further includes: A data analysis and report generation unit that statistically analyzes the stored logistics data, generates various statistical reports, and assists management personnel in optimizing decisions.

10. The real-time display system of the 3D model of a box-type logistics system according to claim 1, characterized in that, It further includes: A site status information display module for displaying the site name, number, bin status, and remarks indicating whether it is abnormal or normal; And visualizes the global load status through a heat map and a traffic curve.

Images