A method and system for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses

Abstract

The present application discloses a method and system for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses, which relates to the field of intelligent warehousing and automation control technology. The disclosed method and system for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses solve the problems of low target recognition accuracy and real-time obstacle avoidance of dynamic obstacles in low temperature environments through multi-sensor fusion and dynamic feature calibration technology, combined with an adaptive path planning algorithm, and can improve the operating efficiency and safety of cold chain warehouses.

Description

Technical Field

[0001] The present application relates to the field of intelligent warehousing and automated control technology, and in particular to a method and system for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses. Background Art

[0002] Cold chain warehouses, characterized by low temperatures, high humidity, and complex, dynamic environments, such as dense metal shelves and mixed operations involving personnel and forklifts, place extremely high demands on target recognition and path planning technologies. Conventional RFID recognition technology, which achieves contactless identification by attaching tags to pallets, faces numerous challenges in cold chain warehouse environments. The strong reflection interference of metal shelves and the reduced sensitivity of tags due to low temperatures significantly increase the tag miss rate and make it impossible to detect the position changes of dynamic objects in real time. Meanwhile, single-vision recognition algorithms, while capable of detecting objects through camera images, suffer from significant drawbacks in cold chain scenarios. Frost easily forms on the camera surface, resulting in blurred images and a significant loss of texture detail. Furthermore, low temperatures can slow model inference speed, significantly reducing detection performance for small objects such as pallets. Furthermore, while static path planning algorithms can generate globally optimal paths, they lack the ability to detect dynamic obstacles, such as moving personnel and temporarily stacked goods, leading to frequent interruptions during forklift operations and severely impacting warehouse efficiency. These technical defects are further magnified in the special environment of cold chain warehouses, forming a technical bottleneck in which insufficient target recognition accuracy and weak dynamic obstacle avoidance capabilities constrain each other.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method and system for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses, aiming to improve the target recognition accuracy and dynamic obstacle avoidance capability in the low temperature environment of cold chain warehouses.

[0005] To achieve the above objectives, the present application proposes a method for realizing identification of people, forklifts, and pallets based on remote control of cold chain warehouses, the method comprising:

[0006] The 3D camera is used to obtain RGB-D image data of the cold chain warehouse environment, the laser point cloud data of the cold chain warehouse is obtained through the lidar, the posture data of the forklift forks is obtained through the wire encoder, and the IMU position data of the forklift body is obtained through the forklift's built-in IMU sensor;

[0007] Performing timestamp synchronization and coordinate system transformation on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data to generate spatiotemporally aligned fusion data;

[0008] The fused data is input into the LightFocus-YOLO model, a multi-scale feature map is extracted through the backbone network reconstructed by the LightFocus-YOLO model, and the multi-scale feature map is dynamically weighted and multi-scale feature fused to generate multi-scale fused feature data;

[0009] Perform bounding box regression and classification on the multi-scale fusion feature data, and output object detection result data including people, forklifts, and pallets;

[0010] Generating a dynamic occupancy grid map based on the target detection result data, and planning a forklift path in real time according to the dynamic occupancy grid map to determine the forklift's movement path;

[0011] The step of inputting the fused data into the LightFocus-YOLO model and extracting a multi-scale feature map through the backbone network reconstructed by the LightFocus-YOLO model includes:

[0012] Input the fused data as the original feature map into the LightFocus-YOLO model, and split the original feature map into a first subspace and a second subspace in a ratio of 3:1 according to the channel dimension, wherein the first subspace performs partial convolution to extract local feature data, and the second subspace retains the original channel data;

[0013] Performing cross-channel interaction between the local feature data of the first subspace and the global feature data of the second subspace through a 1×1 convolution kernel to generate fused feature map data;

[0014] The fused feature map data is gradually extracted at multiple network depths to extract multi-scale feature maps; the multi-scale feature maps include a shallow feature map with a resolution of 80×80, a mid-level feature map with a resolution of 40×40, and a deep feature map with a resolution of 20×20, corresponding to the detection tasks of pallets, personnel, and forklifts, respectively.

[0015] In one embodiment, the step of performing timestamp synchronization and coordinate system transformation on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data to generate spatiotemporally aligned fusion data includes:

[0016] Performing timestamp matching on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data, and unifying the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data into a forklift body coordinate system using a quaternion rotation matrix based on the heading angle in the IMU pose data and the forklift fork lift height in the posture data;

[0017] Voxelization is performed on the laser point cloud data to generate 3D grid map data of a preset resolution. The RGB-D image data is mapped to the forklift body coordinate system through coordinate transformation and spatially aligned with the 3D grid map data to generate the fused data.

[0018] In one embodiment, the step of dynamically allocating weights to the multi-scale feature map includes:

[0019] Grouping the multi-scale feature maps to obtain grouped feature maps of each scale;

[0020] Horizontal global average pooling and vertical global average pooling are performed on the grouped scale feature maps along the spatial dimension to generate direction-sensitive feature data corresponding to each scale feature map;

[0021] Perform 3×3 depth-separable convolution on the grouped scale feature maps to output the fine-grained feature data corresponding to each scale feature map;

[0022] The direction-sensitive feature data is Hadamard-multiplied with the fine-grained feature data of the corresponding scale, and then the space-to-channel dimension interaction is performed through matrix multiplication to generate 80×80, 40×40, and 20×20 cross-dimensional fused feature data;

[0023] Batch normalization and Sigmoid activation function processing are performed on the cross-dimensional fused feature data of each scale in sequence to generate the corresponding dynamic attention mask, and each mask is multiplied element-by-element with the multi-scale feature map of the corresponding scale to obtain the recalibrated feature map data; the recalibrated feature map data includes an 80×80 recalibrated shallow feature map, a 40×40 recalibrated middle feature map, and a 20×20 recalibrated deep feature map.

[0024] In one embodiment, performing multi-scale feature fusion on the feature map includes:

[0025] Downsampling the 80×80 recalibrated shallow feature map to a resolution of 40×40 by bilinear interpolation, and performing adaptive spatial feature fusion with the 40×40 recalibrated middle feature map to generate first fused feature data;

[0026] Downsampling the 40×40 recalibrated middle-layer feature map to a resolution of 20×20 using a 4×4 convolution kernel, and performing a residual connection with the 20×20 recalibrated deep-layer feature map to generate second fused feature data;

[0027] The 80×80 recalibrated shallow feature map is directly passed to the output layer through cross-layer skip connections, and superimposed with the first fused feature data and the second fused feature data to generate multi-scale fused feature data.

[0028] In one embodiment, the step of performing bounding box regression and classification on the multi-scale fusion feature data and outputting object detection result data including a person, a forklift, and a pallet includes:

[0029] Perform feature extraction of 3×3 convolution kernels on the multi-scale fusion feature data to generate detection feature map data with resolutions of 80×80, 40×40, and 20×20;

[0030] Map each detection feature map data to the target detection parameter space through a 1×1 convolution kernel, and output prediction data including bounding box coordinates, target category and confidence level;

[0031] The non-maximum suppression algorithm is used to deduplicate the predicted data, and the predicted data is filtered based on the preset confidence level to output target detection result data including people, forklifts and pallets.

[0032] In one embodiment, the step of generating a dynamic occupancy grid map based on the target detection result data, and planning a forklift path in real time according to the dynamic occupancy grid map to determine the forklift's movement path includes:

[0033] Based on the target detection result data, extract the real-time position coordinate data of the person, forklift and pallet, and generate a dynamic occupancy grid map by voxel processing in combination with the fused data;

[0034] Marking dynamic obstacle areas in the dynamic occupancy grid map with probability values ​​greater than a preset probability threshold;

[0035] Use A The algorithm generates initial global path data on a dynamic occupancy grid map and calculates the pallet grab priority weight based on the heading angle from the IMU pose data and the forklift fork lift height from the posture data.

[0036] The initial global path data is locally optimized using a rolling temporal control algorithm with a time window of preset length, and the laser point cloud data is integrated in real time to avoid dynamic obstacle areas and prioritize the pallet path with the highest priority weight to determine the forklift's movement path.

[0037] In one embodiment, the method further comprises:

[0038] Monitor the image quality of 3D cameras;

[0039] When it is detected that the image clarity is lower than a preset clarity threshold due to frost on the camera lens of the 3D camera, historical data of the recalibrated feature map data is obtained;

[0040] The currently acquired calibrated feature map data and the historical data of the recalibrated feature map data are weightedly calculated to obtain compensated feature map data to compensate for image distortion caused by frosting.

[0041] In one embodiment, the method further comprises:

[0042] Use LiDAR to scan a hemispherical area with a preset radius around the forklift in real time to generate dynamic obstacle point cloud data;

[0043] When a human target is detected in the dynamic obstacle point cloud data and continues to exist for more than a preset first time period, a warning signal is triggered and the maximum speed of the forklift is limited to below the preset speed;

[0044] When it is detected that the human target in the dynamic obstacle point cloud data disappears after the preset second time period, the forklift resumes its normal operating speed and the warning signal is lifted.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a system for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses. The system includes: a memory, a processor and a program for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses stored on the memory and runnable on the processor. The program for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses is configured to implement the steps of the method for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses.

[0046] This application provides a method and system for realizing the recognition of people, forklifts and pallets based on remote control of cold chain warehouses. Through multi-sensor fusion and dynamic feature calibration technology, combined with an adaptive path planning algorithm, it solves the problems of low target recognition accuracy and real-time obstacle avoidance in low-temperature environments, and can improve the operating efficiency and safety of cold chain warehouses. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 A flowchart illustrating an embodiment of a method for realizing identification of people, forklifts, and pallets based on remote control of a cold chain warehouse according to the present application;

[0050] Figure 2 For this application Figure 1 Detailed flow diagram of step S200;

[0051] Figure 3 For this application Figure 1 A detailed flow chart of step S300 provided in an embodiment;

[0052] Figure 4 For this application Figure 1 A detailed flow chart of another embodiment of step S300 is provided;

[0053] Figure 5 For this application Figure 1 A detailed flow chart of another embodiment of step S300 is provided;

[0054] Figure 6 For this application Figure 1 Detailed flowchart of step S400;

[0055] Figure 7 For this application Figure 1 Detailed flowchart of step S500;

[0056] Figure 8 A flowchart illustrating another embodiment of the method for realizing identification of people, forklifts, and pallets based on remote control of cold chain warehouses provided in this application;

[0057] Figure 9 A flowchart illustrating another embodiment of the method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses provided in this application;

[0058] Figure 10 This is a structural diagram of an embodiment of a system for realizing human, forklift and pallet identification based on remote control of a cold chain warehouse.

[0059] Description of Figure Numbers:

[0060] 10. Memory; 20. Processor.

[0061] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] The technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0063] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0064] Existing technologies face significant challenges in target recognition and path planning due to the low temperature and high humidity of cold chain warehouses. The densely packed metal shelves cause significant interference from radio frequency signal reflections, leading to tag misses with traditional RFID technology. Single-vision algorithms experience image blur when cameras are frosted, and model inference speed decreases in low temperatures, leading to increased pallet misses. Static path planning methods are unable to cope with dynamic obstacles, increasing forklift interruptions. Existing technologies struggle to achieve high-precision target recognition and real-time path adjustments in complex, dynamic environments.

[0065] To address the above problems, we consider adopting a complementary multimodal sensor approach to enhance data reliability in response to the image distortion problem caused by low-temperature frost in cold chain warehouses. We explore a method for generating spatiotemporal aligned fusion data to meet the real-time perception needs of dynamic obstacles. We design a feature fusion mechanism based on dynamic weight allocation to address the problem of insufficient multi-target recognition accuracy. We study a rolling optimization strategy based on dynamic grid maps to address the problem of insufficient real-time performance in path planning.

[0066] Therefore, this application proposes a method for realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses. Figure 1 The method includes steps S100 to S500, wherein:

[0067] Step S100: Acquire RGB-D image data of the cold chain warehouse environment using a 3D camera, acquire laser point cloud data of the cold chain warehouse using a lidar, acquire posture data of the forklift forks using a wire encoder, and acquire IMU posture data of the forklift body using an IMU sensor built into the forklift;

[0068] Step S200, performing timestamp synchronization and coordinate system transformation on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data to generate spatiotemporally aligned fusion data;

[0069] Step S300: Input the fused data into the LightFocus-YOLO model, extract multi-scale feature maps through the backbone network reconstructed by the LightFocus-YOLO model, and perform dynamic weight assignment and multi-scale feature fusion on the multi-scale feature maps to generate multi-scale fused feature data;

[0070] Step S400 , performing bounding box regression and classification on the multi-scale fusion feature data, and outputting target detection result data including people, forklifts, and pallets;

[0071] Step S500 : generating a dynamic occupancy grid map based on the target detection result data, and performing real-time planning on a forklift path according to the dynamic occupancy grid map to determine a movement path of the forklift.

[0072] In this embodiment, RGB-D image data refers to stereoscopic vision data containing color and depth information, which can be obtained using Microsoft Kinect or Intel RealSense series 3D cameras to capture the three-dimensional contour features of the target object. Laser point cloud data refers to a set of discrete three-dimensional coordinate points generated by laser radar scanning, which can be collected using a Velodyne HDL-32E radar device and used to construct the geometric structure of the environment. A wire encoder refers to a device that measures the lifting height of a fork using a linear displacement sensor. A SICK DFS60 series encoder can be used to obtain the operating status of the fork. IMU pose data refers to motion parameters containing accelerometer and gyroscope information. An Xsens MTi-30 series sensor can be used to track the motion trajectory of the forklift body. Timestamp synchronization refers to clock alignment of data from different acquisition devices. It can be implemented using the PTP precision clock protocol to ensure temporal consistency of multi-source data. Dynamic weight allocation refers to a mechanism that adjusts the importance of channels based on the spatial position of the feature map. It can be implemented through an attention mechanism and is used to enhance the feature response of key areas.

[0073] Specifically, multi-sensor data is aligned in time and space to form fused data in a unified coordinate system, eliminating measurement deviations between devices. The LightFocus-YOLO model performs hierarchical feature extraction on the fused data, enhancing the details of the pallet edge in shallow features and capturing the overall outline of the forklift in deep features. The dynamic weight allocation mechanism adjusts the weights of each channel based on the directional sensitivity and spatial position of the feature map to improve the ability to detect small targets. Multi-scale feature fusion integrates features at different levels of abstraction through cross-layer connections to establish an association between the local features of the target and the global semantics. The dynamic occupancy grid map updates obstacle location information in real time, and the rolling time domain control algorithm optimizes the path within a preset time window to achieve a balance between dynamic obstacle avoidance and pallet grabbing priority decisions.

[0074] Compared with existing technologies, multimodal sensor fusion overcomes the reliability issues of single visual sensors in frosty environments, and the spatiotemporal alignment mechanism addresses the accuracy issues of collaborative processing of multi-source data. Dynamic weight allocation replaces traditional fixed-weight convolution, enhancing the model's ability to recognize occluded objects in cold chain environments. Furthermore, feature map data is gradually extracted at multiple network depths, effectively improving the detection accuracy of objects of varying sizes compared to single-scale detection. A dynamic grid map combined with a rolling optimization algorithm significantly improves the real-time performance of routing in complex environments compared to static path planning.

[0075] Through the above technical solutions, this application effectively reduces the impact of cold chain warehouses' low temperature and high humidity on target recognition, improving the accuracy of pallet, personnel, and forklift detection. A dynamic occupancy grid map reflects environmental changes in real time, reducing forklift operation interruptions. A multi-scale feature fusion mechanism enhances the model's sensitivity to small targets, reducing the risk of missed pallet detection. The path planning algorithm balances global optimization with local obstacle avoidance requirements, improving cold chain warehouse operational efficiency.

[0076] In one possible implementation, reference Figure 2 , step S200 includes steps S210 to S220, wherein:

[0077] Step S210, performing timestamp matching on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data, and unifying the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data into a forklift body coordinate system using a quaternion rotation matrix based on the heading angle in the IMU pose data and the forklift fork lift height in the posture data;

[0078] Step S220 , voxelizing the laser point cloud data to generate 3D grid map data of a preset resolution, mapping the RGB-D image data to the forklift body coordinate system through coordinate transformation, and spatially aligning the data with the 3D grid map data to generate the fused data.

[0079] In this embodiment, timestamp matching refers to aligning the data collected by multi-source sensors in chronological order. Specifically, it can be achieved by hardware trigger synchronization or software interpolation algorithm to ensure that the data collected by different sensors under the same time reference can be associated. Quaternion rotation matrix refers to a mathematical tool for coordinate system conversion. Specifically, the rotation relationship between the forklift body coordinate system and the world coordinate system can be calculated through sensor calibration parameters to eliminate the coordinate offset caused by differences in the installation positions of different sensors. Voxelization processing refers to the three-dimensional grid division of laser point cloud data. Specifically, the point cloud can be downsampled and spatially aggregated using a cubic grid to generate regularized spatial structure data. Spatial alignment refers to unifying sensor data from different coordinate systems to the same reference system. Specifically, the RGB-D image data can be mapped to the forklift body coordinate system through the coordinate transformation matrix, and matched voxel by voxel with the voxelized 3D grid map to achieve spatial consistency of multimodal data.

[0080] Specifically, in cold chain warehouse scenarios, the temporal and spatial consistency of multi-source sensor data directly impacts target detection accuracy due to the dynamic motion of forklifts, personnel, and pallets. Timestamp matching eliminates sensor acquisition delays, ensuring synchronous processing of laser point clouds, RGB-D images, and posture data at the same moment. The quaternion rotation matrix unifies multi-source data into the forklift's coordinate system based on the lift height of the forklift's forks and the heading angle of the IMU, avoiding spatial misalignment caused by coordinate system differences. Voxelization converts high-density laser point clouds into low-resolution 3D grid maps, reducing computational complexity while preserving key obstacle outlines. After the RGB-D image is mapped to the forklift's coordinate system through coordinate transformation, it is spatially aligned with the 3D grid map, allowing the visual information and lidar data to complement each other in the spatial dimension, enhancing the perception of low-texture pallets and dynamic targets.

[0081] Compared with existing technologies, traditional methods usually only process data from a single sensor. For example, they use visual data to independently generate target detection results, which results in a significant decrease in detection accuracy under low-temperature frost or metal reflection interference. The multi-sensor fusion solution in the existing technology lacks an effective time synchronization mechanism and does not consider the impact of the forklift fork posture change on the coordinate system, resulting in spatial deviation during data fusion. This solution dynamically adjusts the coordinate system during the forklift movement by matching the quaternion rotation matrix with the timestamp, solving the problem of spatiotemporal misalignment of multi-sensor data caused by movement. At the same time, the combination of voxel processing and spatial alignment enables lidar and visual data to complement each other in a unified spatial framework, overcoming the limitations of a single sensor in complex environments.

[0082] Through the above technical solution, this application solves the problem of fusion error caused by time asynchrony and inconsistent coordinate systems in multi-source sensor data in cold chain warehouses, and improves the robustness of dynamic target detection. By aligning visual data with the laser point cloud in the coordinate system of the forklift itself, it can accurately identify the outline of pallets and the location of personnel under low-light conditions, reducing false detections caused by reflections from metal shelves or frosting on the camera. In addition, the spatiotemporally aligned fusion data provides highly consistent environmental perception input for subsequent path planning, thereby reducing the risk of collision between the forklift and dynamic obstacles.

[0083] In one possible implementation, reference Figure 3 , the step S300 includes steps S310A to S330A, wherein:

[0084] Step S310A, inputting the fused data as the original feature map into the LightFocus-YOLO model, splitting the original feature map into a first subspace and a second subspace in a 3:1 ratio according to the channel dimension, wherein the first subspace performs partial convolution to extract local feature data, and the second subspace retains the original channel data;

[0085] Step S320A: performing cross-channel interaction between the local feature data of the first subspace and the global feature data of the second subspace through a 1×1 convolution kernel to generate fused feature map data;

[0086] In step S330A, the fused feature map data is gradually extracted at multiple network depths to extract a multi-scale feature map; the multi-scale feature map includes a shallow feature map with a resolution of 80×80, a mid-layer feature map with a resolution of 40×40, and a deep feature map with a resolution of 20×20, corresponding to the detection tasks of pallets, personnel, and forklifts, respectively.

[0087] In this embodiment, splitting by the channel dimension at a ratio of 3:1 refers to dividing the number of channels of the input feature map into two parts in proportion, which can be achieved by using a channel splitting operation. Local feature extraction is performed by retaining some channels, and the remaining channels maintain global information transmission, thereby balancing computational efficiency and feature integrity. Cross-channel interaction refers to information exchange between different channels, which can be achieved by using a 1×1 convolution kernel, and enhancing the complementarity between local features and global features by adjusting the channel dimension weights. Features are gradually extracted at multiple network depths, which can be achieved by using multi-stage convolution and pooling operations, and gradually expanding the receptive field to adapt to the detection needs of targets of different sizes.

[0088] Specifically, the original feature map is divided into two subspaces. The first subspace captures detailed texture features through local convolution operations with a kernel size of 3×3, while the second subspace retains the original channel data to avoid loss of high-frequency information. Subsequently, the features of the two subspaces are fused across channels using 1×1 convolutions to generate a new feature map that combines local details with global context. The fused feature map is then passed through a hierarchical convolutional structure, generating a high-resolution feature map in the shallow layer to capture the tiny outlines of the pallet, a medium-resolution feature map in the middle layer to locate medium-sized objects such as personnel, and a low-resolution feature map in the deep layer to identify the overall structure of the forklift.

[0089] Compared with existing technologies, traditional object detection models suffer from feature redundancy in low-temperature environments, leading to slower inference speeds. Furthermore, single-scale features are unable to account for objects of varying sizes. This solution reduces the computational complexity of invalid features by dynamically segmenting channels and interacting across channels. Furthermore, a layered abstraction strategy enables multi-scale feature adaptation, effectively reducing model complexity and improving small object detection accuracy.

[0090] Through the above technical solution, this application solves the problem of image blur caused by frosting on cameras in cold chain warehouses. By retaining the original channel data, it avoids detail loss and uses layered feature fusion to improve the robustness of pallet, personnel, and forklift detection. The multi-scale feature matching strategy can adapt to the detection needs of targets of different sizes, reduce missed detections and false detections caused by target size differences, and improve recognition accuracy in complex dynamic environments.

[0091] In one possible implementation, reference Figure 4 , the step S300 further includes steps S310B to S350B, wherein:

[0092] Step S310B, grouping the multi-scale feature maps to obtain grouped scale feature maps;

[0093] Step S320B: performing horizontal global average pooling and vertical global average pooling on each grouped scale feature map along the spatial dimension to generate direction-sensitive feature data corresponding to each scale feature map;

[0094] Step S330B: performing a 3×3 depthwise separable convolution on each grouped scale feature map, and outputting fine-grained feature data corresponding to each scale feature map;

[0095] Step S340B: perform Hadamard product on the direction-sensitive feature data and the fine-grained feature data of the corresponding scale, and then perform matrix multiplication to interact from space to channel dimensions to generate 80×80, 40×40, and 20×20 cross-dimensional fused feature data;

[0096] Step S350B, perform batch normalization and Sigmoid activation function processing on the cross-dimensional fused feature data of each scale in sequence to generate a corresponding dynamic attention mask, and multiply each mask by the multi-scale feature map of the corresponding scale element by element to obtain the recalibrated feature map data; the recalibrated feature map data includes an 80×80 recalibrated shallow feature map, a 40×40 recalibrated middle feature map, and a 20×20 recalibrated deep feature map.

[0097] In this embodiment, group processing refers to dividing feature maps of different resolutions into independent processing units. Specifically, channel segmentation can be used to treat 80×80, 40×40, and 20×20 feature maps as separate processing groups. Through group processing, differentiated feature optimization can be performed for the detection needs of targets of different scales. Horizontal global average pooling and vertical global average pooling refer to global statistical calculations along the width and height directions of the image, respectively. Specifically, feature vectors compressed to 1×W and H×1 along the spatial dimension can be used to capture the spatial distribution characteristics of the target in the horizontal and vertical directions. 3×3 depthwise separable convolution refers to decomposing the standard convolution into a combination of depthwise convolution and pointwise convolution. Specifically, each input channel can be convolved separately and then the channels can be merged through 1×1 convolution to extract fine-grained texture features while reducing the amount of calculation. Hadamard product refers to the multiplication operation of corresponding matrix elements. Specifically, the fusion of directional attention and local features can be achieved by element-by-element multiplication of the direction-sensitive feature vector and the convolution feature map. Matrix multiplication for cross-dimensional interaction maps spatial features to channel dimensions. Specifically, this involves multiplying the spatial feature matrix with the channel weight matrix to establish a correlation between space and channels. Batch normalization and sigmoid activation function processing standardize and nonlinearly transform the fused feature data. Specifically, a normalization layer stabilizes the training process, and a sigmoid function generates attention weights in the range of 0-1, dynamically enhancing key features.

[0098] Specifically, in the low-temperature environment of cold chain warehouses, image blur caused by camera frost can weaken target edge features. By grouping multi-scale feature maps and independently optimizing the 80×80 shallow feature map, the 40×40 mid-level feature map, and the 20×20 deep feature map, this approach can adapt to the detection needs of objects of varying sizes, such as pallets, people, and forklifts. Horizontal and vertical global pooling operations statistically analyze feature distributions from the horizontal and vertical directions, respectively. For example, for a horizontally stacked array of pallets, horizontal pooling can enhance the regularity of the pallet arrangement; for vertically raised forks, vertical pooling can capture height variations. Depthwise separable convolutions retain the advantages of the 3×3 convolution kernel's receptive field while reducing the number of parameters. For example, when processing 40×40 mid-level features, detailed information about a person's outline can be extracted while minimizing computational resources. Hadamard products are used to fuse directionally sensitive features with fine-grained features. For example, multiplying the pallet's horizontal distribution features with its metal edge features can enhance the response strength of stacked pallets. Matrix multiplication enables cross-dimensional interaction from space to channels. For example, combining the spatial position weights of a pallet region with channel features can improve the feature activation values ​​of the pallet recognition channel. The resulting dynamic attention mask adaptively enhances the features of relatively intact target regions in frosted images. For example, dynamic weighting can be used to suppress noise interference and enhance effective features in pallet corners blurred by frosting.

[0099] Compared with existing technologies, traditional methods usually adopt fixed-weight feature fusion strategies, such as directly performing weighted summation on multi-scale features, which cannot dynamically adjust the importance of features according to environmental changes. However, this solution uses direction-sensitive feature extraction and cross-dimensional interaction mechanisms. For example, when detecting pallets with partial distortion due to frost, it can automatically enhance the weight of the effective area based on the horizontal distribution characteristics, which improves the robustness to blurred targets compared to fixed fusion methods. The attention mechanism in existing technologies mostly uses a single channel or spatial attention, such as calculating the attention weight only through the channel dimension. However, this solution uses spatial direction statistics and cross-dimensional interaction, such as simultaneously capturing the horizontal distribution characteristics of the pallet and the vertical movement characteristics of the fork, to achieve more refined feature calibration.

[0100] Through the above technical solution, this application effectively solves the problem of missed pallet detection caused by insufficient feature extraction in the low-temperature frosting environment of cold chain warehouses. It enhances the effective feature expression of fuzzy areas through a dynamic weight distribution mechanism, and improves the ability to capture the edge features of metal pallets and small-sized human targets. During the lifting and lowering process of the forklift fork, the vertical feature enhancement can accurately identify the relative position relationship between the fork and the pallet, avoiding the pallet positioning deviation caused by the fork occlusion. The cross-dimensional fusion of direction-sensitive features and fine-grained features enables the model to distinguish between real targets and interference contours generated by the reflection of metal shelves, reducing the false detection rate in dynamic environments.

[0101] In one possible implementation, reference Figure 5 , the step S300 further includes steps S310C to S330C, wherein:

[0102] Step S310C: downsampling the 80×80 recalibrated shallow feature map to a resolution of 40×40 by bilinear interpolation, and performing adaptive spatial feature fusion with the 40×40 recalibrated middle feature map to generate first fused feature data;

[0103] Step S320C: downsample the 40×40 recalibrated middle-layer feature map to a resolution of 20×20 using a 4×4 convolution kernel, and perform a residual connection with the 20×20 recalibrated deep-layer feature map to generate second fused feature data;

[0104] In step S330C, the 80×80 recalibrated shallow feature map is directly transferred to the output layer through a cross-layer skip connection, and superimposed with the first fused feature data and the second fused feature data to generate multi-scale fused feature data.

[0105] In this embodiment, bilinear interpolation downsampling refers to the process of reducing the resolution of the feature map by a linear interpolation method, which can be specifically implemented by calculating the new pixel value by weighted average between pixels. Adaptive spatial feature fusion refers to dynamically adjusting the fusion weight according to the spatial position of feature maps of different resolutions, which can be specifically implemented by calculating the correlation matrix between feature maps using an attention mechanism. This method can eliminate the spatial misalignment between features of different scales. Residual connection refers to the operation of transferring the original features through direct cross-layer connection, which can be specifically implemented by merging the downsampled features with the deep features using element-level addition. This method can avoid the problem of gradient disappearance during training. Cross-layer skip connection refers to a mechanism for transferring shallow features directly to the output of the network, which can specifically be implemented by retaining the original feature details using feature map splicing. This method can enhance the positioning accuracy of small target detection.

[0106] Specifically, in the low temperature and high humidity environment of the cold chain warehouse, the surface of the pallet is prone to frost, resulting in blurred texture. After the resolution of the shallow features is reduced through bilinear interpolation downsampling operations, it can be fused with the 40×40 recalibrated middle-level feature map. During the fusion process, the adaptive spatial feature fusion mechanism dynamically adjusts the fusion area according to the posture of the forklift forks, effectively eliminating the feature dislocation caused by the movement of the forklift. When generating the second fusion feature, the residual connection retains the macroscopic features of the forklift outline during the downsampling process, avoiding the distortion of the forklift shape caused by excessive abstraction of the deep network. Finally, the original shallow features are directly transferred to the output end through cross-layer jump connections, so that the texture information of the pallet surface weakened by frost can be fully utilized, significantly improving the recognition accuracy of pallets in low temperature environments.

[0107] Compared with existing technologies, traditional multi-scale fusion methods use fixed-weight feature superposition, which is susceptible to high-frequency noise interference in cold chain warehouses with dense metal shelves. Adaptive spatial feature fusion can suppress feature interference caused by shelf reflections by dynamically adjusting fusion weights. Existing technologies often miss small targets due to insufficient cross-layer feature transfer, while the cross-layer skip connection mechanism achieves the direct superposition of shallow detail features and deep semantic features, which can effectively solve the problem of missed detection of pallets due to surface frost. Conventional path planning algorithms rely on single-layer feature detection results, while this solution constructs a dynamic grid map by fusing feature data at multiple scales, which improves the accuracy of obstacle positioning and optimizes the real-time performance of the forklift's obstacle avoidance path.

[0108] Through the above technical solution, this application can solve the problem of pallet texture blurring caused by frosting on equipment in cold chain warehouses, enhance small target detection capabilities through multi-scale feature fusion, and reduce the pallet missed detection rate. At the same time, the cross-layer feature transfer mechanism is used to retain the original environmental details, improve the positioning accuracy of dynamic obstacles in the occupancy grid map, provide more reliable environmental perception data for forklift path planning, and ultimately achieve efficient collaborative operation of people, forklifts, and pallets in the complex environment of cold chain warehouses.

[0109] In one possible implementation, reference Figure 6 , the step S400 further includes steps S410 to S430, wherein:

[0110] Step S410, performing feature extraction of 3×3 convolution kernels on the multi-scale fusion feature data to generate detection feature map data with resolutions of 80×80, 40×40, and 20×20;

[0111] Step S420 , mapping each detection feature map data to the target detection parameter space through a 1×1 convolution kernel, and outputting prediction data including bounding box coordinates, target category, and confidence level;

[0112] In step S430 , a non-maximum suppression algorithm is used to perform deduplication processing on the predicted data, and the predicted data is filtered based on a preset confidence level to output target detection result data including people, forklifts, and pallets.

[0113] In this embodiment, bounding box regression refers to predicting the position and size of the target object through convolution operations. Specifically, a 3×3 convolution kernel can be used to extract spatial features from multi-scale fused feature data to capture target geometric information at different resolutions, thereby improving the positioning accuracy of small target pallets. Target classification refers to categorizing the detected targets. Specifically, a 1×1 convolution kernel can be used to map the feature map to an output channel containing category probabilities to achieve multi-category parallel discrimination. The non-maximum suppression algorithm refers to eliminating redundant results of overlapping detection frames. Specifically, the bounding box with the highest confidence can be selected by calculating the intersection-over-union threshold to solve the problem of false detection in dense target detection. Pre-set confidence screening refers to setting the confidence threshold according to the actual scenario requirements. Specifically, the threshold can be dynamically adjusted to balance the missed detection rate and the false detection rate to meet the detection reliability requirements in the complex environment of cold chain warehouses.

[0114] Specifically, multi-scale fused feature data is processed with a 3×3 convolution kernel, which can extract semantic information at different scales while preserving spatial details. For example, an 80×80 resolution feature map is suitable for capturing small object details on pallets, a 40×40 resolution feature map is suitable for locating medium-scale objects such as personnel, and a 20×20 resolution feature map is suitable for identifying large-scale objects such as forklifts. Subsequently, a 1×1 convolution kernel compresses the feature maps at each scale into a unified detection parameter space, generating bounding box coordinate offsets, class probabilities, and confidence scores. The non-maximum suppression algorithm compares the intersection-over-union ratio of adjacent detection boxes to retain the prediction with the highest confidence, thus avoiding repeated detection of the same object. Finally, candidate boxes are filtered based on a preset confidence threshold to eliminate low-confidence noisy detection results, ensuring the high reliability of the output results even in low-temperature and high-humidity environments.

[0115] Compared with existing technologies, traditional target detection methods typically use a single-scale feature map for regression and classification, making it difficult to simultaneously detect targets of varying sizes, resulting in a high rate of missed detection for small targets such as pallets. Furthermore, existing methods use a fixed intersection-over-union threshold for deduplication, which is unsuitable for the complex scenarios of densely stacked pallets in cold chain warehouses, significantly increasing the false detection rate. This solution effectively improves the detection accuracy of targets of varying sizes through multi-scale feature fusion and a dynamic threshold screening mechanism. It also incorporates the adaptive optimization of a non-maximum suppression algorithm to reduce the impact of environmental interference on detection results.

[0116] Through the above technical solution, this application can significantly reduce the missed detection rate of small target pallets in low temperature and high humidity environments, and suppress false detection caused by reflections from metal shelves or frosting on cameras, thereby providing more accurate obstacle location information for dynamic path planning and improving the safety and efficiency of forklift operations in complex cold chain environments.

[0117] In one possible implementation, reference Figure 7, the step S500 further includes steps S510 to S540, wherein:

[0118] Step S510: extracting real-time position coordinate data of the person, forklift, and pallet based on the target detection result data, and generating a dynamic occupancy grid map by voxelization processing in combination with the fused data;

[0119] Step S520, marking a dynamic obstacle area in the dynamic occupancy grid map with a probability value greater than a preset probability threshold;

[0120] Step S530, using A The algorithm generates initial global path data on a dynamic occupancy grid map and calculates the pallet grab priority weight based on the heading angle from the IMU pose data and the forklift fork lift height from the posture data.

[0121] In step S540, the initial global path data is locally optimized using a rolling temporal control algorithm with a time window of a preset length, and the laser point cloud data is integrated in real time to avoid dynamic obstacle areas and prioritize the pallet path with the highest priority weight to determine the forklift's movement path.

[0122] In this embodiment, the dynamic occupancy grid map refers to dividing the warehouse environment into fixed-size grid cells, with each grid cell representing the probability of being occupied by an obstacle through a probability value. Specifically, the three-dimensional space can be discretized into uniform cubic cells through voxelization, and the grid state is updated in real time based on the target detection results and sensor data, thereby achieving spatial marking of dynamic obstacles. Voxelization converts continuous point cloud data into a discrete grid structure, such as using 0.1-meter-resolution cubic cells to divide the space, which helps reduce computational complexity and improve map update efficiency. The algorithm is a path planning method based on heuristic search. Specifically, it searches for the global optimal path in the grid map by evaluating the path cost function. For example, the coordinates of the forklift's current position and the target pallet are used as the starting and end points, and the initial path is generated by combining the path length and the turning cost. The rolling temporal control algorithm refers to repeatedly optimizing the local path within a limited time window. For example, a 5-second time window is used to adjust the global path in segments, and obstacle information scanned by the lidar is integrated in real time to dynamically correct the forklift's direction of travel to avoid collisions. The pallet grabbing priority weight is calculated by the fork lifting height and the heading angle deviation. For example, when the fork is in a high position and the heading angle deviation is less than 5 degrees, the current pallet is determined to be a high-priority target, ensuring that the path planning prioritizes approaching the target area.

[0123] Specifically, the dynamic occupancy grid map updates obstacle location information in real time by fusing target detection results with multi-sensor data. When the target detection module identifies a person or a moving forklift, its coordinate data is converted into an obstacle marking area in the grid map. The algorithm generates an initial global path based on a preset path cost function, incorporating pallet priority weights into the path selection criteria. A rolling horizon control algorithm makes local adjustments to the global path within a fixed time window. For example, when the LiDAR detects temporarily stacked cargo, the current path segment is replanned to avoid the additional obstacle while prioritizing adjacent paths for high-priority pallets. During this process, fork posture data and IMU pose data are used to calculate pallet grab feasibility, ensuring that the forklift's mechanical state matches the task requirements during path optimization.

[0124] Compared to existing technologies, traditional static path planning methods rely solely on fixed map information and are unable to cope with dynamic obstacles caused by the movement of personnel or temporary storage of goods within cold chain warehouses. This solution uses a dynamic occupancy grid map to update environmental information in real time, combined with rolling time domain control to achieve local path optimization, effectively solving the problem of dynamic obstacle avoidance. Furthermore, while existing technologies typically determine the order in which pallets are picked based on preset rules, this solution introduces a priority weighting mechanism that incorporates fork posture and heading angle into the decision-making process, making path planning more tailored to actual operating conditions.

[0125] Through the above technical solutions, this application can sense dynamic obstacles in real time and update path planning, reducing forklift operation interruptions caused by personnel movement or cargo stacking. The priority weight mechanism ensures that high-feasibility pallet grabbing tasks are prioritized, improving material handling efficiency within cold chain warehouses. The combination of rolling time domain control and local path optimization enhances the ability to respond to sudden obstacles and reduces collision risks while ensuring global path optimality.

[0126] In one possible implementation, reference Figure 8 The method further includes steps S610 to S630, wherein:

[0127] Step S610, monitoring the image quality of the 3D camera;

[0128] Step S620 , when it is detected that the image clarity is lower than a preset clarity threshold due to frost on the camera head of the 3D camera, historical data of the recalibrated feature map data is obtained;

[0129] Step S630 , performing weighted calculation on the currently acquired calibrated feature map data and the historical data of the recalibrated feature map data to acquire compensated feature map data to compensate for image distortion caused by frosting.

[0130] In this embodiment, the historical data of the recalibrated feature map data refers to the feature map data from multiple past time points retained in a time series. Specifically, this data can be stored in a circular buffer or time window queue to provide a historical reference when image quality degrades. Weighted calculation refers to linearly combining the current feature map data with the historical feature map data in a preset ratio. Specifically, this can be achieved using a sliding average or exponential weighting method, balancing the influence of current and historical data by dynamically adjusting the weight coefficients.

[0131] Specifically, when the image clarity of a 3D camera decreases due to frost caused by low temperatures, the system monitors image quality indicators (such as edge sharpness and contrast) in real time to determine whether they are below a preset threshold. After detecting an image quality anomaly, the system retrieves the recalibrated feature map data of several recent frames from the storage module and performs a weighted fusion with the feature map of the current frame. For example, during the weighted calculation process, the feature map of the current frame can be assigned a higher weight to retain immediate information, while the feature map of the historical frame can be assigned a lower weight to suppress noise interference. By dynamically adjusting the weight ratio, the loss of texture details caused by frost is compensated, and ultimately compensated feature map data that is closer to the real scene is generated.

[0132] Compared with existing technologies, traditional methods typically rely solely on the current frame data for processing when the camera is frosted, resulting in a significant decrease in target detection accuracy. This solution effectively reduces the impact of noise in single-frame data by introducing the spatiotemporal correlation of historical feature map data, thereby improving the robustness of target recognition in low-temperature frosting scenarios.

[0133] Through the above technical solution, this application solves the problem of image distortion caused by frost in 3D cameras in cold chain warehouses. By fusing historical and current feature map data, it effectively restores target details covered by the frost layer, reduces the missed detection rate of pallets and personnel, and ensures the continuous and stable operation of the forklift path planning system in complex low-temperature environments.

[0134] In one possible implementation, reference Figure 9 The method further includes steps S710 to S730, wherein:

[0135] Step S710: Scanning a hemispherical area with a preset radius around the forklift in real time using a laser radar to generate dynamic obstacle point cloud data;

[0136] Step S720: When it is detected that a human target in the dynamic obstacle point cloud data persists for more than a preset first time period, a warning signal is triggered and the maximum speed of the forklift is limited to below a preset speed;

[0137] Step S730 : When it is detected that the human target in the dynamic obstacle point cloud data disappears for a preset second time period, the forklift is restored to a normal operating speed and the warning signal is released.

[0138] In this embodiment, the hemispherical area of ​​a preset radius refers to the scanning range extending forward and to the sides of the forklift. This range can be specifically set using the LiDAR's field of view and maximum detection range parameters. For example, a scanning radius of 5 meters covers the potential collision zone during forklift operation. Dynamic obstacle point cloud data refers to point cloud information containing 3D coordinates collected in real time by the LiDAR. Clustering algorithms can be used to extract the position and outline of moving objects for human identification.

[0139] Specifically, during the operation of the forklift, the lidar scans the surrounding environment at a fixed frequency and generates point cloud data. When a dynamic target that conforms to the contours of the human body is detected in the point cloud data, the system starts timing. If the target persists for more than a first duration (e.g., 0.5 seconds), it is determined that there is a risk of human presence, triggering an audible and visual alarm and limiting the maximum speed of the forklift to a safe threshold (e.g., 0.5 m / s). When the system detects that the target has disappeared, it must wait for a second duration (e.g., 3 seconds) to confirm that no one has re-entered the danger zone before the speed limit can be lifted and normal operation can be resumed.

[0140] In some embodiments, the warning signal can be provided by a combination of an onboard buzzer and a warning light, while also uploading the forklift status information to a central control system. Speed ​​limiting is achieved by adjusting the motor controller output power to ensure the forklift completes the obstacle avoidance maneuver in low-speed mode.

[0141] Compared with existing technologies, existing solutions rely solely on static obstacle maps for path planning and are unable to handle sudden human movement in real time. This solution uses dynamic point cloud data to identify human targets in real time. Incorporating a duration determination mechanism, it effectively distinguishes between brief transits and long-term stays, avoiding operation interruptions caused by false triggers.

[0142] Through the above technical solution, this application achieves a balance between personnel safety protection and operational efficiency in the dynamic environment of cold chain warehouses. Through real-time monitoring and speed classification control, it can minimize interference with forklift operations while ensuring personnel safety, solving the problems of high operation interruption rates and large response delays caused by the inability to avoid dynamic obstacles in traditional solutions.

[0143] The present application provides a method for realizing the recognition of people, forklifts and pallets based on remote control of cold chain warehouses. The method obtains RGB-D image data of the cold chain warehouse environment through a 3D camera, obtains laser point cloud data of the cold chain warehouse through a lidar, and obtains the posture data of the forklift fork through a wire encoder; obtains the IMU posture data of the forklift body through the IMU sensor built into the forklift; then performs timestamp synchronization and coordinate system transformation on the RGB-D image data, the laser point cloud data, the posture data and the IMU posture data to generate spatiotemporal aligned fusion data; then inputs the fusion data into the LightFocus-YOLO model, extracts multi-scale feature maps through the backbone network reconstructed by the LightFocus-YOLO model, and performs dynamic weight allocation and multi-scale feature fusion on the multi-scale feature maps to generate multi-scale fusion feature data; then performs bounding box regression and classification on the multi-scale fusion feature data, and outputs target detection result data including people, forklifts and pallets; finally, a dynamic occupancy grid map is generated based on the target detection result data, and the forklift path is planned in real time according to the dynamic occupancy grid map to determine the forklift's movement path. In this way, this application solves the problems of low target recognition accuracy and real-time obstacle avoidance of dynamic obstacles in low-temperature environments through multi-sensor fusion and dynamic feature calibration technology, combined with adaptive path planning algorithm, and can improve the operating efficiency and safety of cold chain warehouses.

[0144] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method of realizing the identification of people, forklifts and pallets based on remote control of cold chain warehouses in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0145] This application also provides a system for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses. Figure 10 The system includes: a memory 10, a processor 20, and a program for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses, which is stored in the memory 10 and can be run on the processor 20. The program for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses is configured to implement the steps of the method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses.

[0146] The system for identifying people, forklifts, and pallets based on remote control of cold chain warehouses provided in this application adopts the method for identifying people, forklifts, and pallets based on remote control of cold chain warehouses in the above-mentioned embodiment, which can improve the target recognition accuracy and dynamic obstacle avoidance capability in the low-temperature environment of cold chain warehouses. Compared with the existing technology, the beneficial effects of the system for identifying people, forklifts, and pallets based on remote control of cold chain warehouses provided in this application are the same as the beneficial effects of the method for identifying people, forklifts, and pallets based on remote control of cold chain warehouses provided in the above-mentioned embodiment, and the other technical features of the system for identifying people, forklifts, and pallets based on remote control of cold chain warehouses are the same as the features disclosed in the above-mentioned embodiment method, and are not further described here.

[0147] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses, characterized in that: The method includes: The 3D camera is used to obtain RGB-D image data of the cold chain warehouse environment, the laser point cloud data of the cold chain warehouse is obtained through the lidar, the posture data of the forklift forks is obtained through the wire encoder, and the IMU position data of the forklift body is obtained through the forklift's built-in IMU sensor; Performing timestamp synchronization and coordinate system transformation on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data to generate spatiotemporally aligned fusion data; The fused data is input into the LightFocus-YOLO model, a multi-scale feature map is extracted through the backbone network reconstructed by the LightFocus-YOLO model, and the multi-scale feature map is dynamically weighted and multi-scale feature fused to generate multi-scale fused feature data; Perform bounding box regression and classification on the multi-scale fusion feature data, and output object detection result data including people, forklifts, and pallets; Generating a dynamic occupancy grid map based on the target detection result data, and planning a forklift path in real time according to the dynamic occupancy grid map to determine the forklift's movement path; The step of inputting the fused data into the LightFocus-YOLO model and extracting a multi-scale feature map through the backbone network reconstructed by the LightFocus-YOLO model includes: Input the fused data as the original feature map into the LightFocus-YOLO model, and split the original feature map into a first subspace and a second subspace in a ratio of 3:1 according to the channel dimension, wherein the first subspace performs partial convolution to extract local feature data, and the second subspace retains the original channel data; Performing cross-channel interaction between the local feature data of the first subspace and the global feature data of the second subspace through a 1×1 convolution kernel to generate fused feature map data; The fused feature map data is gradually extracted at multiple network depths to extract multi-scale feature maps; the multi-scale feature maps include a shallow feature map with a resolution of 80×80, a mid-level feature map with a resolution of 40×40, and a deep feature map with a resolution of 20×20, corresponding to the detection tasks of pallets, personnel, and forklifts, respectively.

2. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to claim 1, characterized in that: The step of performing timestamp synchronization and coordinate system transformation on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data to generate spatiotemporally aligned fusion data includes: Performing timestamp matching on the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data, and unifying the RGB-D image data, the laser point cloud data, the posture data, and the IMU pose data into a forklift body coordinate system using a quaternion rotation matrix based on the heading angle in the IMU pose data and the forklift fork lift height in the posture data; Voxelization is performed on the laser point cloud data to generate 3D grid map data of a preset resolution. The RGB-D image data is mapped to the forklift body coordinate system through coordinate transformation and spatially aligned with the 3D grid map data to generate the fused data.

3. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to claim 1, characterized in that: The step of dynamically allocating weights to the multi-scale feature map comprises: Grouping the multi-scale feature maps to obtain grouped feature maps of each scale; Horizontal global average pooling and vertical global average pooling are performed on the grouped scale feature maps along the spatial dimension to generate direction-sensitive feature data corresponding to each scale feature map; Perform 3×3 depth-wise separable convolution on the grouped scale feature maps to output the fine-grained feature data corresponding to each scale feature map; The direction-sensitive feature data is Hadamard-multiplied with the fine-grained feature data of the corresponding scale, and then the space-to-channel dimension interaction is performed through matrix multiplication to generate 80×80, 40×40, and 20×20 cross-dimensional fused feature data; Batch normalization and Sigmoid activation function processing are performed on the cross-dimensional fused feature data of each scale in sequence to generate the corresponding dynamic attention mask, and each mask is multiplied element-by-element with the multi-scale feature map of the corresponding scale to obtain the recalibrated feature map data; the recalibrated feature map data includes an 80×80 recalibrated shallow feature map, a 40×40 recalibrated middle feature map, and a 20×20 recalibrated deep feature map.

4. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to claim 3, characterized in that: Performing multi-scale feature fusion on the feature map includes: Downsampling the 80×80 recalibrated shallow feature map to a resolution of 40×40 by bilinear interpolation, and performing adaptive spatial feature fusion with the 40×40 recalibrated middle feature map to generate first fused feature data; Downsampling the 40×40 recalibrated middle-layer feature map to a resolution of 20×20 using a 4×4 convolution kernel, and performing a residual connection with the 20×20 recalibrated deep-layer feature map to generate second fused feature data; The 80×80 recalibrated shallow feature map is directly passed to the output layer through cross-layer skip connections, and superimposed with the first fused feature data and the second fused feature data to generate multi-scale fused feature data.

5. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to claim 4, characterized in that: The step of performing bounding box regression and classification on the multi-scale fusion feature data and outputting target detection result data including a person, a forklift, and a pallet comprises: Perform feature extraction of 3×3 convolution kernels on the multi-scale fusion feature data to generate detection feature map data with resolutions of 80×80, 40×40, and 20×20; Map each detection feature map data to the target detection parameter space through a 1×1 convolution kernel, and output prediction data including bounding box coordinates, target category and confidence level; The non-maximum suppression algorithm is used to deduplicate the predicted data, and the predicted data is filtered based on the preset confidence level to output target detection result data including people, forklifts and pallets.

6. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to claim 1, characterized in that: The step of generating a dynamic occupancy grid map based on the target detection result data, and planning a forklift path in real time according to the dynamic occupancy grid map to determine the forklift's movement path includes: Based on the target detection result data, extract the real-time position coordinate data of the person, forklift and pallet, and generate a dynamic occupancy grid map by voxel processing in combination with the fused data; Marking dynamic obstacle areas in the dynamic occupancy grid map with probability values ​​greater than a preset probability threshold; Use A The algorithm generates initial global path data on a dynamic occupancy grid map and calculates the pallet grab priority weight based on the heading angle from the IMU pose data and the forklift fork lift height from the posture data. The initial global path data is locally optimized using a rolling temporal control algorithm with a time window of preset length, and the laser point cloud data is integrated in real time to avoid dynamic obstacle areas and prioritize the pallet path with the highest priority weight to determine the forklift's movement path.

7. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to any one of claims 1 to 6, characterized in that: The method further comprises: Monitor the image quality of 3D cameras; When it is detected that the image clarity is lower than a preset clarity threshold due to frost on the camera lens of the 3D camera, historical data of the recalibrated feature map data is obtained; The currently acquired calibrated feature map data and the historical data of the recalibrated feature map data are weightedly calculated to obtain compensated feature map data to compensate for image distortion caused by frosting.

8. The method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses according to any one of claims 1 to 6, characterized in that: The method further comprises: Use LiDAR to scan a hemispherical area with a preset radius around the forklift in real time to generate dynamic obstacle point cloud data; When a human target is detected in the dynamic obstacle point cloud data and continues to exist for more than a preset first time period, a warning signal is triggered and the maximum speed of the forklift is limited to below the preset speed; When it is detected that the human target in the dynamic obstacle point cloud data disappears after the preset second time period, the forklift resumes its normal operating speed and the warning signal is lifted.

9. A system for identifying people, forklifts, and pallets based on remote control of cold chain warehouses, characterized in that: The system includes: a memory, a processor, and a program for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses, which is stored in the memory and can be run on the processor. The program for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses is configured to implement the steps of the method for realizing identification of people, forklifts and pallets based on remote control of cold chain warehouses as described in any one of claims 1 to 8.

Images