Digital raster map generation method and system based on deep learning

Abstract

The application discloses a kind of digital raster map generation method and system based on deep learning, which comprises: obtaining the actual position coordinate data grid data set M1 of each camera corresponding original ground data set H, assigning coordinates to the grid without actual position coordinate data, to obtain the original grid data set U of camera shooting, aggregate the grid coordinates and grid label of original grid data set U of camera shooting, establish grid coordinate data set K;Based on the grid coordinates and grid type of grid coordinate data set K, a warehouse static raster map is generated;Based on the improved YOLOv8 algorithm model, the real-time running state information of all forklifts in the warehouse is obtained based on the original grid data set U of camera shooting and the warehouse static raster map, and a dynamic raster map is generated based on the real-time running state information, which helps to solve the high cost problem faced by the prior art in intelligent transformation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for generating digital raster maps based on deep learning. Background Technology

[0002] In the current development of the warehousing industry, many warehousing companies are still trapped in the predicament of traditional operating models. On the one hand, many companies still rely on traditional forklifts for cargo handling and adopt floor-stacking cargo management methods, which not only consume a lot of manpower but also lack scientific cargo storage layout, resulting in low efficiency in searching and handling. On the other hand, although the trend of intelligent warehouse transformation is becoming increasingly prominent, and advanced equipment such as intelligent forklifts and intelligent racking have great potential to improve operational efficiency and optimize management processes, companies face high cost barriers. Purchasing an intelligent forklift costs at least hundreds of thousands of yuan and sometimes even millions of yuan. The construction and deployment of intelligent racking and supporting systems are also very expensive. The high upfront investment and subsequent maintenance costs deter many warehousing companies, and intelligent upgrade plans are repeatedly shelved.

[0003] Meanwhile, digital raster maps (DRGs), as graphic data presented in raster form through digital scanning and computer processing, have great application value in the field of smart warehousing, providing accurate digital models for warehouse spaces.

[0004] Therefore, there is an urgent need for a technology that focuses on the existing equipment and sites of warehousing enterprises, combines advanced image recognition technology to generate digital grid maps, and then unlocks diverse application scenarios such as intelligent forklift navigation, intelligent warehouse operation scheduling and management, and warehouse digital twins. This aims to solve the current problems of high cost and low efficiency in the intelligent transformation of traditional warehousing enterprises in a low-cost and high-efficiency manner. Summary of the Invention

[0005] In view of this, the present invention proposes a method and system for generating digital grid maps based on deep learning, which can generate static and dynamic grid maps at low cost using existing warehousing equipment and sites, improve warehouse operation efficiency, and help traditional warehousing enterprises achieve intelligent management.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A deep learning-based method for generating digital raster maps includes:

[0008] The warehouse is divided into areas and cameras are installed and fixed. The original ground image dataset H before the small flag is placed and the ground image dataset F after the small flag is placed are obtained for each camera.

[0009] Based on the improved YOLOv8 algorithm model, the grids of the images in the ground image dataset F are identified. The pixel region of each grid in the image is obtained and a unique number is generated. The grids are marked as lane grids or parking space grids based on their color.

[0010] For each camera, the difference between the original ground image dataset H and the ground image dataset F before and after placing the flag is calculated to determine the pixel area and coordinate data of the flag.

[0011] Record the set of grid numbers, the set of adjacent grid numbers, and the set of adjacent relationships for each image in the ground image dataset F;

[0012] The pixel regions of each grid are summed and compared with the pixel regions of the images in the original ground image dataset H to obtain at least one blank pixel region. The blank pixel region is defined as a new grid and a unique number is generated. The adjacency relationship of the new grid is determined. The coordinate data of the flags whose pixel regions are contained in the blank pixel regions are obtained. The coordinate data of the flags is used as the actual position coordinate data of the grid corresponding to the blank pixel region. All the coordinate data of the flags are assigned to the corresponding grids to obtain all grids, grid numbers, grid adjacency relationships and actual position coordinate data of some grids in the original ground dataset H captured by each camera.

[0013] Obtain the grid dataset M1 with actual position coordinate data for each camera's original ground dataset H. Assign coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the camera. Aggregate the grid coordinates and grid labels of the original grid dataset U captured by the camera. Remove duplicate grid coordinate data and grid labels from the aggregated data to establish a grid coordinate dataset K. Establish a relationship between the grid coordinate dataset K and the original grid dataset U captured by the camera.

[0014] A static raster map of the warehouse is generated based on the grid coordinates and grid types of the grid coordinate dataset K. A static raster map of each camera is generated based on the association between the grid coordinate dataset K and the original grid dataset U captured by each camera.

[0015] Based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera, and the static grid map of the warehouse, the real-time operating status information of all forklifts in the warehouse is obtained, and a dynamic grid map is generated based on the real-time operating status information.

[0016] Based on the above technical solution, the present invention can be further improved as follows:

[0017] Optionally, the improved YOLOv8 algorithm model includes an input terminal, a backbone network, a Neck terminal, and an output terminal;

[0018] A channel prior convolutional attention module is added to the backbone network. The channel prior convolutional attention module includes a channel attention mechanism and a spatial attention mechanism connected in sequence. The channel attention mechanism generates a channel attention graph, and the spatial attention mechanism generates a spatial attention graph.

[0019] The Neck module includes a bottleneck module, which replaces the convolution module in the bottleneck module with DCNv3.

[0020] The spatial attention map is received by DCNv3 for channel blending. The channel blending result is obtained by multiplying the channel blending structure element-wise with the channel prior to obtain the feature representation of the image content.

[0021] Optionally, the step of generating a channel attention map through a channel attention mechanism and generating a spatial attention map through spatial attention includes:

[0022] Channel attention is calculated using formula (1);

[0023] Formula (1);

[0024] In the formula, For channel attention, F is a given feature map F. Perform average pooling on the feature map F. To perform max pooling on the feature map F, For multilayer perceptrons, For activation functions;

[0025] Spatial attention is calculated using formula (2);

[0026] Formula (2);

[0027] In the formula, For spatial attention, For depthwise separable convolution; For the first Branches.

[0028] Optionally, the step of receiving spatial attention maps via DCNv3 for channel mixing, obtaining channel mixing results, and multiplying the channel mixing structure element-wise with the channel priors to obtain feature representations of the image content includes:

[0029] The feature representation of the image content is calculated using formula (3);

[0030] Formula (3);

[0031] In the formula, For the feature representation of image content, For cluster number, For the dimension, for the first Group, For the first Group-independent projection weights, For the first Group 1 The modulation scalar of each sampling point Input feature maps for slices, For the current pixel, For grid sampling locations, For the first Group grid sampling location The corresponding offset.

[0032] Optionally, the step of obtaining the grid dataset M1 with actual position coordinate data for each camera's original ground dataset H, and assigning coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the cameras, includes:

[0033] For coordinates ( , Given cell M11, obtain the left neighboring cell of cell M11, and determine whether the left neighboring cell has coordinates. If it does, do nothing; otherwise, define ( -1, Let M be the coordinates of the left adjacent cell, and record the left adjacent cell in dataset M2;

[0034] Similarly, obtain the right neighbor, top neighbor, and bottom neighbor of cell M11, and complete the assignment of the actual position coordinate data of the cell in the cell dataset M1 to the neighboring cells;

[0035] The actual position coordinates of the cells in the newly generated dataset M2 are assigned to adjacent cells. The assigned cells are recorded in dataset M3. This process continues until the coordinates of the cells in the original ground images of each camera are assigned. All cell numbers, cell coordinates, cell labels, the pixel regions corresponding to the cells, and the camera numbers in the shooting area of ​​each camera are obtained to obtain the original cell dataset U captured by the cameras.

[0036] Optionally, the step of obtaining real-time operating status information of all forklifts in the warehouse based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera, and the static grid map of the warehouse, and generating a dynamic grid map based on the real-time operating status information, includes:

[0037] The pixel region of each forklift in the current camera image and the marked number of the forklift's heading are obtained by improving the YOLOv8 algorithm model;

[0038] The pixel region of the forklift in the current camera image is compared with the pixel region of the corresponding grid in the original grid dataset U captured by the current camera. If a grid is matched, the pixel region value of the matched grid is recorded to obtain the pixel region to be compared.

[0039] If multiple cells are matched, the pixel regions of the multiple cells are merged to obtain the pixel region to be compared.

[0040] Based on the pixel regions to be compared, the coordinate data and heading data of each forklift are obtained;

[0041] A dynamic grid map is generated based on the current static grid map of the warehouse, as well as the coordinate data and heading data of each forklift.

[0042] Optionally, obtaining the coordinate data of each forklift based on the pixel region to be compared includes:

[0043] Calculate the forklift's x-coordinate using formula (4);

[0044] Formula (4);

[0045] In the formula, Let x be the x-coordinate of the forklift. The x-coordinate of the pixel region to be compared. The ratio of the overlapping area to the distance to the left. The ratio of the overlapping area to the distance to the right;

[0046] Calculate the forklift's longitudinal coordinate using formula (5);

[0047] Formula (5);

[0048] In the formula, Let the forklift's vertical coordinate be... The vertical coordinate of the pixel region to be compared is... This represents the ratio of the overlapping area to the distance above. This represents the ratio of the overlapping area to the distance to the bottom edge.

[0049] A deep learning-based digital raster map generation system includes:

[0050] The data acquisition module is used to divide the warehouse into areas and install and fix cameras, and to acquire the original ground image dataset H before the small flag is placed and the ground image dataset F after the small flag is placed for each camera.

[0051] The grid recognition module is used to identify grids in the ground image dataset F based on the improved YOLOv8 algorithm model. It obtains the pixel area of ​​each grid in the image and generates a unique number. Based on the grid color, it marks the grid as a lane grid or a storage location grid.

[0052] The difference calculation module is used to calculate the difference between the original ground image dataset H and the ground image dataset F corresponding to each camera before and after placing the flag, and to determine the pixel area and coordinate data of the flag.

[0053] The recording module is used to record the set of grid numbers, the set of adjacent grid numbers, and the set of adjacent relationships for images in the ground image dataset F;

[0054] The actual position coordinate data acquisition module is used to sum the pixel regions of each grid, compare the difference with the pixel regions of the image in the original ground image dataset H, obtain at least one blank pixel region, define the blank pixel region as a new grid and generate a unique serial number, determine the adjacency relationship of the new grid, obtain the flag coordinate data of the flag pixel region contained in the blank pixel region, use the flag coordinate data as the actual position coordinate data of the grid corresponding to the blank pixel region, and assign all the flag coordinate data to the corresponding grid to obtain the actual position coordinate data of all grids, grid serial numbers, grid adjacency relationships and some grids of the original ground dataset H captured by each camera;

[0055] The coordinate assignment module is used to obtain the grid dataset M1 with actual position coordinate data in the original ground dataset H corresponding to each camera, assign coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the camera, aggregate the grid coordinates and grid labels of the original grid dataset U captured by the camera, deduplicate the grid coordinate data and grid labels of the aggregated data to establish the grid coordinate dataset K, and establish the association between the grid coordinate dataset K and the original grid dataset U captured by the camera.

[0056] The static map generation module is used to generate a warehouse static raster map based on the grid coordinates and grid types of the grid coordinate dataset K, and to generate a static raster map for each camera based on the association between the grid coordinate dataset K and the original grid dataset U captured by each camera.

[0057] The dynamic map generation module is used to obtain real-time operating status information of all forklifts in the warehouse based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera and the static grid map of the warehouse, and to generate a dynamic grid map based on the real-time operating status information.

[0058] An electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method described herein.

[0059] A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method when executed by a processor.

[0060] The present invention has the following advantages:

[0061] The deep learning-based digital raster map generation method in this invention can generate maps using existing warehouse space and basic equipment such as conventional cameras, without the need for large-scale purchase of new and expensive smart hardware. This effectively reduces the high cost of hardware investment for traditional warehousing companies to move towards intelligent management, enabling more warehousing companies to embark on the path of intelligent transformation at a lower cost.

[0062] The deep learning-based digital raster map generation method in this invention, with the help of an improved YOLOv8 algorithm model, can accurately identify grids in ground images, accurately distinguish between lane grids and storage location grids, meticulously process blank pixel areas to transform them into new grids and reasonably determine their relevant attributes and coordinates. The static raster map generated through a rigorous data processing process can highly restore the actual layout of the warehouse, providing an accurate digital spatial reference for subsequent management.

[0063] The deep learning-based digital grid map generation method in this invention generates a dynamic grid map based on the generated static grid map and the acquisition of real-time forklift operating status information. This allows for real-time monitoring of forklift dynamics and cargo flow, enabling intelligent navigation, intelligent operation scheduling and management, optimizing forklift travel paths, reducing cargo handling time, avoiding operational conflicts, significantly improving overall warehouse operation efficiency, and enhancing the smoothness and orderliness of warehouse operations.

[0064] The deep learning-based digital grid map generation method in this invention promotes the transformation of traditional warehousing enterprises from traditional management models to digital twin intelligent management models, making it easier for enterprises to better monitor inventory, allocate resources, and make planning decisions, thereby improving the refinement and intelligence of warehousing management and enhancing market competitiveness. Attached Figure Description

[0065] For illustrative and not limiting purposes, the present invention will now be described in conjunction with embodiments and accompanying drawings, wherein:

[0066] Figure 1 This is a flowchart illustrating the deep learning-based digital raster map generation method according to an embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram of the main components of the deep learning-based digital raster map generation system in an embodiment of the present invention;

[0068] Figure 3 A first schematic diagram illustrating the division of a warehouse area and the installation and fixing of cameras in an embodiment of the present invention;

[0069] Figure 4 A second schematic diagram illustrating the division of the warehouse area and the installation and fixing of cameras in an embodiment of the present invention;

[0070] Figure 5 This is a schematic diagram of the server environment configuration used in the experiment of this invention embodiment;

[0071] Figure 6 This is a schematic diagram of the model-improved ablation test in an embodiment of the present invention;

[0072] Figure 7 This is a schematic diagram comparing the results of the warehouse forklift detection model in an embodiment of the present invention;

[0073] Figure 8 This is a schematic diagram of a static grid map in an embodiment of the present invention;

[0074] Figure 9 This is a schematic diagram of the forklift roof marking design in an embodiment of the present invention;

[0075] Figure 10 This is a schematic diagram of a dynamic grid map in an embodiment of the present invention;

[0076] Figure 11 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0077] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0078] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0079] It should be noted that, where there is no conflict, the embodiments and features of the present invention can be combined with each other. The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0080] Figure 1 This is a flowchart illustrating the deep learning-based digital raster map generation method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the deep learning-based digital raster map generation method provided in this embodiment of the invention includes the following steps S101 to S108.

[0081] S101, Divide the warehouse into areas and install and fix the cameras, and obtain the original ground image dataset H before the small flag is placed and the ground image dataset F after the small flag is placed for each camera.

[0082] Specifically, such as Figure 3 As shown, the warehouse is divided into zones and cameras are installed and fixed in place: For the camera installed on the top of the warehouse, For storage locations, the adjacent storage locations Lanes for AGVs and other forklifts to travel on;

[0083] The camera installation will be determined based on factors such as floor height, area, and environment. Installation requirements are: the area captured by all cameras must include all warehouse goods, each driveway, and other areas. The shooting angle of each camera will be determined and fixed to obtain effective images for generating a digital twin of the warehouse.

[0084] The entire warehouse area is marked with grids and storage locations, with different colors used to distinguish storage locations, driveways, etc.; the left and bottom edges of the grids are marked as follows. Figure 3 The numbers are used to generate coordinates for each area's grid.

[0085] S102, based on the improved YOLOv8 algorithm model, identify the grids in the ground image dataset F, obtain the pixel region of each grid in the image and generate a unique number, and mark the grid as lane grid or parking space grid based on the grid color.

[0086] Specifically, in real-world warehouse applications, the quality of acquired images is often low due to uncontrollable factors such as lighting, angle, and surrounding environment, leading to difficulties in identifying some targets. These issues typically include: image blurring caused by changes in lighting conditions; occlusion of objects due to densely packed targets caused by shooting angle problems; and inconsistent target size and shape due to target movement and acquisition distance. To address these problems, this invention improves the algorithm based on YOLOv8n. The improvements are mainly reflected in two aspects: First, a Channel Prior Convolutional Attention (CPCA) module is added to the backbone network. CPCA is an attention mechanism used to enhance feature representation and dynamically allocate attention weights. CPCA combines channel attention and spatial attention, effectively extracting spatial relationships and preserving channel priors through multi-scale depthwise separable convolutional modules. Second, DCNv3 (Deformable Convolutional Networks v3) extends traditional convolutional neural networks, aiming to improve the flexibility and accuracy of feature extraction. DCNv3 introduces deformable convolutions, enabling convolutional kernels to adapt to different shapes and scales, thus better capturing non-rigid objects and complex structures in images. In each convolutional layer, DCNv3 enhances the adaptability of the convolutional kernels by learning adaptive sampling locations, thereby improving robustness to object deformation and the network's expressive power. This method performs exceptionally well in image detection and segmentation tasks.

[0087] In warehouse image recognition applications, issues such as image blurring or target occlusion may occur due to changes in lighting environment or angle, increasing the difficulty of image recognition. Channel Prior Convolutional Attention (CPCA) has the ability to focus on information channels and important regions. By using multi-scale deep convolution modules, spatial relationships can be effectively extracted while preserving prior channels.

[0088] The overall structure of CPCA (Concurrent Perceptron Alignment) consists of the sequential placement of channel attention (CA) and spatial attention (SA). Spatial information from the feature map is aggregated by the channel attention through operations such as average pooling and max pooling. Subsequently, the spatial information is processed and added through a shared MLP (Multilayer Perceptron) to generate the channel attention map. The channel prior is obtained by element-wise multiplication of the input features and the channel attention map. The channel prior is then fed into a deep convolutional module to generate the spatial attention map. The convolutional module receives the spatial attention map for channel blending. Finally, the refined features are obtained as the output by element-wise multiplication of the channel blending result with the channel prior. The channel blending process helps to enhance the representation of features.

[0089] The input is an intermediate feature map mapping First, a one-dimensional channel attention map is calculated using CA. Secondly, Input features Element-wise multiplication yields a refined feature with channel attention. Then, the 3D spatial attention map is calculated using SA. Finally, and Element-wise multiplication yields output features The formula for calculating overall attention is as follows:

[0090]

[0091]

[0092] In the formula: This indicates element-wise multiplication; Indicates channel attention; Represents spatial attention; F represents the given feature map; This indicates the output feature map.

[0093] The primary function of CA (Channel Attention Map) is to generate channel attention maps by aggregating channel relationships in the feature maps. In this process, average pooling and max pooling are used to aggregate spatial information from the feature maps. This spatial information is then processed and added through a shared multilayer perceptron to generate the channel attention maps.

[0094] Channel attention is calculated using formula (1);

[0095] Formula (1);

[0096] In the formula, For channel attention, F is a given feature map F. Perform average pooling on the feature map F. To perform max pooling on the feature map F, For multilayer perceptrons, For activation functions;

[0097] The primary function of the spatial attention module (SA) is to extract spatial feature information. In this architecture, a multi-scale structure is employed to enhance the ability of convolutional operations to capture spatial relationships. Channel blending is performed by a 1×1 convolution at the tail of the spatial attention module.

[0098] Spatial attention is calculated using formula (2);

[0099] Formula (2);

[0100] In the formula, For spatial attention, For depthwise separable convolution; For the first Branches.

[0101] Because ordinary convolutions use regular kernels, they cannot effectively adapt to geometric changes in objects, such as rotation, symmetry, and scaling, resulting in poor feature extraction. Their ability to model geometric transformations primarily comes from extensive data augmentation, large model capacity, and some simple, handcrafted modules (e.g., max pooling for small translation invariance). A key challenge in visual recognition is adapting to geometric changes in object scale, pose, viewpoint, and part deformation, or model geometry transformations; fixed-size convolutional kernel operations cannot effectively achieve this.

[0102] In warehouse recognition applications, most targets are not fixed in size and shape due to motion, sampling distance, and other factors. Therefore, traditional convolutional methods are not very effective for feature extraction in images with deformation or varying target sizes. Variable convolution, by predicting offsets for sampling points and adaptively changing the sampling position, can improve the model's ability to handle irregular shapes and deformations. Furthermore, the receptive field size can adaptively change according to the target size, effectively solving the problems of traditional convolution in handling deformed images.

[0103] This invention uses DCNv3 (deformable convolution v3) to replace the convolutional module in the bottleneck module of the YOLOv8 model. As an extension of the DCN series, DCNv3 has the following three advantages:

[0104] This operator compensates for the shortcomings of regular convolution in long-range dependencies and adaptive spatial aggregation; compared with common attention-based operators such as MHSA and closely related deformable attention, this operator inherits the inductive bias of convolution, making the model more efficient with less training data and shorter training time.

[0105] This operator is based on sparse sampling and has higher computational and memory efficiency than previous methods such as MHSA and reparameterized large kernels.

[0106] For the current pixel The feature representation of the image content is calculated using formula (3);

[0107] Formula (3);

[0108] In the formula, For the feature representation of image content, For cluster number, For the dimension, for the first Group, For the first Group-independent projection weights, For the first Group 1 The modulation scalar of each sampling point Input feature maps for slices, For the current pixel, For grid sampling locations, For the first Group grid sampling location The corresponding offset.

[0109] The dataset used in this invention is the image dataset detected by the camera in the experimental warehouse. To solve the problem of forklift classification and detection in complex environments, a dataset of 2957 images in 13 categories was constructed, including images in different scenarios such as angles, lighting, distances, and object intrusions, after manual selection and annotation. This dataset can ultimately provide basic data for tasks such as forklift image classification and object detection. The images are randomly divided into training, validation, and test sets.

[0110] The experimental setup used an Ubuntu 20.04 server operating system, a single NVIDIA RTX A5000 graphics card, and PyTorch 1.11.0 as the deep learning framework. Detailed configuration is as follows: Figure 5 As shown.

[0111] After 300 training epochs on both the original and improved models, the original model, able to load pre-trained weights, initially performed well and converged in approximately 200 epochs. However, the improved model, due to its altered structure, could only load a portion of the pre-trained weights, effectively requiring retraining and significantly more epochs to converge. Comparing the two curves reveals that the improved model, by introducing CPCA attention and replacing traditional convolutions with DCNv3, improved its mAP50 score by 3.7 percentage points, from 0.669 to 0.706.

[0112] mAP is an important performance indicator, referring to the average precision (AP). For each class, its AP value can be calculated as the area under the curve of precision versus recall across all samples in that class, segmented at different thresholds. The average of the AP values ​​for all classes is then the mAP value. The relevant formula is as follows:

[0113]

[0114]

[0115]

[0116]

[0117] To verify the effectiveness of the improvements made in this invention, the following ablation experiments were performed on the YOLOv8n model. Figure 6 It can be seen that the designed DCNv3_C2f Block improves the model mAP value by 3.7 points while reducing the model size by 0.4MB, achieving a dual improvement in model accuracy and speed.

[0118] The improved model of this invention was compared with the target detection models YOLOv3-tiny, YOLOv5, YOLOv6, and YOLOv8n. The results verified that the improved model of this invention outperforms these models in both accuracy and speed, and also has fewer parameters and lower inference time. This further demonstrates the effectiveness of the proposed improved method. Applying the improved forklift recognition model to forklift detection in a real warehouse significantly improved both forklift recognition accuracy and computational efficiency. Figure 7 As shown.

[0119] To verify the detection results of the algorithm in different environments in a real warehouse, images with different backgrounds were selected from the experimental test set. It was found that the improved algorithm can accurately identify forklifts under complex environments such as different lighting, angles, and backgrounds, and can also identify the direction of the forklift's front end. The improved algorithm can accurately identify forklifts and their specific information, meeting the needs of practical application scenarios.

[0120] S103, calculate the difference between the original ground image dataset H and the ground image dataset F corresponding to each camera before and after placing the flag, and determine the pixel area and coordinate data of the flag.

[0121] Specifically, such as Figure 4 As shown, small flags of different colors are placed in each grid area of ​​the warehouse. The coordinates of each flag's location are recorded, aligned with the numbers on the left and bottom. A dataset F (F1, F2... Fn) of ground images with attached flags, captured by each camera, is obtained.

[0122] S104 records the set of grid numbers, the set of adjacent grid numbers, and the set of adjacent relationships for the images in the ground image dataset F.

[0123] S105, sum the pixel regions of each grid cell, compare the difference with the pixel regions of the images in the original ground image dataset H, obtain at least one blank pixel region, define the blank pixel region as a new grid cell and generate a unique serial number, determine the adjacency relationship of the new grid cell, obtain the coordinate data of the flags whose pixel regions are contained in the blank pixel regions, use the flag coordinate data as the actual position coordinate data of the grid cell corresponding to the blank pixel region, and assign all the flag coordinate data to the corresponding grid cells to obtain all grid cells, grid serial numbers, grid adjacency relationships and actual position coordinate data of some grid cells in the original ground dataset H captured by each camera.

[0124] S106. Obtain the grid dataset M1 with actual position coordinate data for each camera's original ground dataset H. Assign coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the camera. Aggregate the grid coordinates and grid labels of the original grid dataset U captured by the camera. Remove duplicate grid coordinate data and grid labels from the aggregated data to establish a grid coordinate dataset K. Establish a relationship between the grid coordinate dataset K and the original grid dataset U captured by the camera.

[0125] Specifically, for coordinates ( , Given cell M11, obtain the left neighboring cell of cell M11, and determine whether the left neighboring cell has coordinates. If it does, do nothing; otherwise, define ( -1, Let M be the coordinates of the left adjacent cell, and record the left adjacent cell in dataset M2;

[0126] Similarly, obtain the right neighbor, top neighbor, and bottom neighbor of cell M11, and complete the assignment of the actual position coordinate data of the cell in the cell dataset M1 to the neighboring cells;

[0127] The actual position coordinates of the cells in the newly generated dataset M2 are assigned to adjacent cells. The assigned cells are recorded in dataset M3. This process continues until the coordinates of the cells in the original ground images of each camera are assigned. All cell numbers, cell coordinates, cell labels, the pixel regions corresponding to the cells, and the camera numbers in the shooting area of ​​each camera are obtained to obtain the original cell dataset U captured by the cameras.

[0128] S107. Generate a static raster map of the warehouse based on the grid coordinates and grid types of the grid coordinate dataset K. Generate a static raster map of each camera based on the association between the grid coordinate dataset K and the original grid dataset U captured by each camera.

[0129] Specifically, the static grid map of the warehouse is the foundation for the generation of dynamic grid maps and warehouse digital twins. The static grid map generation in this invention is based on drawing grids and storage location areas in an empty warehouse, acquiring ground image information captured by each camera, and finally generating the coordinate information of the grids in the area captured by each camera according to the grid map generation algorithm. The actual length and width of each grid are defined according to the actual situation, which can then be used for applications such as generating forklift driving routes based on scheduling tasks; for example... Figure 8 As shown, the light-colored grid represents lane grids, and the dark-colored grid represents parking space grids.

[0130] S108 uses an improved YOLOv8 algorithm model, a camera-captured original grid dataset U, and a static grid map of the warehouse to obtain real-time operating status information of all forklifts in the warehouse, and generates a dynamic grid map based on the real-time operating status information.

[0131] Specifically, the pixel area of ​​each forklift in the current camera image and the marked number of the forklift's heading are obtained by improving the YOLOv8 algorithm model;

[0132] Because warehouse forklifts often look similar, this invention, to ensure accurate and effective identification, fixes an arrow pointing in the direction of the forklift's direction, along with a number or symbol representing the uniqueness of each forklift, to the roof of each forklift. This allows for more accurate identification of each forklift with minimal resource costs. Figure 9As shown, each forklift in the warehouse has a pattern on its roof that clearly distinguishes different numbers or other symbols to identify each forklift; at the same time, the roof is marked with an arrow pattern to identify the direction the forklift is traveling or parking.

[0133] The pixel region of the forklift in the current camera image is compared with the pixel region of the corresponding grid in the original grid dataset U captured by the current camera. If a grid is matched, the pixel region value of the matched grid is recorded to obtain the pixel region to be compared.

[0134] If multiple cells are matched, the pixel regions of the multiple cells are merged to obtain the pixel region to be compared.

[0135] Based on the pixel regions to be compared, the coordinate data and heading data of each forklift are obtained;

[0136] Based on the current static raster map of the warehouse, as well as the coordinates and heading data of each forklift, a dynamic raster map is generated, such as... Figure 10 As shown.

[0137] Calculate the forklift's x-coordinate using formula (4);

[0138] Formula (4);

[0139] In the formula, Let x be the x-coordinate of the forklift. The x-coordinate of the pixel region to be compared. The ratio of the overlapping area to the distance to the left. The ratio of the overlapping area to the distance to the right;

[0140] Calculate the forklift's longitudinal coordinate using formula (5);

[0141] Formula (5);

[0142] In the formula, Let the forklift's vertical coordinate be... The vertical coordinate of the pixel region to be compared is... This represents the ratio of the overlapping area to the distance above. This represents the ratio of the overlapping area to the distance to the bottom edge.

[0143] Figure 2 This is a schematic diagram illustrating the main components of the deep learning-based digital raster map generation system according to an embodiment of the present invention. Figure 2As shown, the deep learning-based digital grid map generation system 1 provided in this embodiment of the invention includes a data acquisition module 10, a grid recognition module 20, a difference calculation module 30, a recording module 40, an actual location coordinate data acquisition module 50, a coordinate assignment module 60, a static map generation module 70, and a dynamic map generation module 80.

[0144] The data acquisition module 10 is used to divide the warehouse into areas and install and fix the cameras, and to acquire the original ground image dataset H before the small flags are placed and the ground image dataset F after the small flags are placed for each camera.

[0145] The grid recognition module 20 is used to identify the grids in the ground image dataset F based on the improved YOLOv8 algorithm model, obtain the pixel area of ​​each grid in the image and generate a unique number, and mark the grid as a lane grid or a storage location grid based on the grid color.

[0146] The difference calculation module 30 is used to calculate the difference between the original ground image dataset H and the ground image dataset F corresponding to each camera before and after placing the flag, and to determine the pixel area and coordinate data of the flag.

[0147] The recording module 40 is used to record the set of grid numbers, the set of adjacent grid numbers, and the set of adjacent relationships of the images in the ground image dataset F;

[0148] The actual position coordinate data acquisition module 50 is used to sum the pixel regions of each grid, compare the difference with the pixel regions of the image in the original ground image dataset H, obtain at least one blank pixel region, define the blank pixel region as a new grid and generate a unique serial number, determine the adjacency relationship of the new grid, obtain the flag coordinate data of the flag pixel region contained in the blank pixel region, use the flag coordinate data as the actual position coordinate data of the grid corresponding to the blank pixel region, and assign all the flag coordinate data to the corresponding grid to obtain the actual position coordinate data of all grids, grid serial numbers, grid adjacency relationships and some grids of the original ground dataset H captured by each camera;

[0149] The coordinate assignment module 60 is used to obtain the grid dataset M1 with actual position coordinate data in the original ground dataset H corresponding to each camera, assign coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the camera, aggregate the grid coordinates and grid labels of the original grid dataset U captured by the camera, deduplicate the grid coordinate data and grid labels of the aggregated data to establish a grid coordinate dataset K, and establish an association between the grid coordinate dataset K and the original grid dataset U captured by the camera.

[0150] The static map generation module 70 is used to generate a warehouse static raster map based on the grid coordinates and grid type of the grid coordinate dataset K, and to generate a static raster map for each camera based on the association between the grid coordinate dataset K and the original grid dataset U captured by each camera.

[0151] The dynamic map generation module 80 is used to obtain real-time operating status information of all forklifts in the warehouse based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera and the static grid map of the warehouse, and to generate a dynamic grid map based on the real-time operating status information.

[0152] Figure 11 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 11 As shown, the electronic device 90 includes: a processor 901, a memory 902, and a bus 903;

[0153] The processor 901 and the memory 902 communicate with each other via the bus 903.

[0154] The processor 901 is used to call program instructions in the memory 902 to execute the methods provided in the above-described method embodiments, and to execute the methods provided in the embodiments of the present invention.

[0155] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions, which cause a computer to execute the method provided in this embodiment of the invention.

[0156] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0157] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating digital raster maps based on deep learning, characterized in that, include: The warehouse is divided into areas and cameras are installed and fixed. The original ground image dataset H before the small flag is placed and the ground image dataset F after the small flag is placed are obtained for each camera. Based on the improved YOLOv8 algorithm model, the grids of the images in the ground image dataset F are identified. The pixel region of each grid in the image is obtained and a unique number is generated. The grids are marked as lane grids or parking space grids based on their color. For each camera, the difference between the original ground image dataset H and the ground image dataset F before and after placing the flag is calculated to determine the pixel area and coordinate data of the flag. Record the set of grid numbers, the set of adjacent grid numbers, and the set of adjacent relationships for each image in the ground image dataset F; The pixel regions of each grid are summed and compared with the pixel regions of the images in the original ground image dataset H to obtain at least one blank pixel region. The blank pixel region is defined as a new grid and a unique number is generated. The adjacency relationship of the new grid is determined. The coordinate data of the flags whose pixel regions are contained in the blank pixel regions are obtained. The coordinate data of the flags is used as the actual position coordinate data of the grid corresponding to the blank pixel region. All the coordinate data of the flags are assigned to the corresponding grids to obtain all grids, grid numbers, grid adjacency relationships and actual position coordinate data of some grids in the original ground dataset H captured by each camera. Obtain the grid dataset M1 with actual position coordinate data for each camera's original ground dataset H. Assign coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the camera. Aggregate the grid coordinates and grid labels of the original grid dataset U captured by the camera. Remove duplicate grid coordinate data and grid labels from the aggregated data to establish a grid coordinate dataset K. Establish a relationship between the grid coordinate dataset K and the original grid dataset U captured by the camera. A static raster map of the warehouse is generated based on the grid coordinates and grid types of the grid coordinate dataset K. A static raster map of each camera is generated based on the association between the grid coordinate dataset K and the original grid dataset U captured by each camera. Based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera, and the static grid map of the warehouse, the real-time operating status information of all forklifts in the warehouse is obtained, and a dynamic grid map is generated based on the real-time operating status information.

2. The method for generating digital raster maps based on deep learning according to claim 1, characterized in that, The improved YOLOv8 algorithm model includes an input terminal, a backbone network, a Neck terminal, and an output terminal. A channel prior convolutional attention module is added to the backbone network. The channel prior convolutional attention module includes a channel attention mechanism and a spatial attention mechanism connected in sequence. The channel attention mechanism generates a channel attention graph, and the spatial attention mechanism generates a spatial attention graph. The Neck module includes a bottleneck module, which replaces the convolution module in the bottleneck module with DCNv3. The spatial attention map is received by DCNv3 for channel blending. The channel blending result is obtained by multiplying the channel blending structure element-wise with the channel prior to obtain the feature representation of the image content.

3. The method for generating digital raster maps based on deep learning according to claim 2, characterized in that, The generation of channel attention maps via channel attention mechanism and spatial attention maps via spatial attention mechanism includes: Channel attention is calculated using formula (1); Formula (1); In the formula, For channel attention, F is a given feature map F. Perform average pooling on the feature map F. To perform max pooling on the feature map F, For multilayer perceptrons, For activation functions; Spatial attention is calculated using formula (2); Formula (2); In the formula, For spatial attention, For depthwise separable convolution; For the first Branches.

4. The method for generating digital raster maps based on deep learning according to claim 2, characterized in that, The process of receiving spatial attention maps via DCNv3 for channel mixing, obtaining channel mixing results, and then multiplying the channel mixing structure element-wise with the channel priors to obtain feature representations of the image content includes: The feature representation of the image content is calculated using formula (3); Formula (3); In the formula, For the feature representation of image content, For cluster number, For the dimension, for the first Group, For the first Group-independent projection weights, For the first Group 1 The modulation scalar of each sampling point Input feature maps for slices, For the current pixel, For grid sampling locations, For the first Group grid sampling location The corresponding offset.

5. The method for generating digital raster maps based on deep learning according to claim 1, characterized in that, The process of obtaining the original ground dataset H corresponding to each camera, which contains a grid dataset M1 with actual position coordinate data, and assigning coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the cameras, includes: For coordinates ( , Given cell M11, obtain the left neighboring cell of cell M11, and determine whether the left neighboring cell has coordinates. If it does, do nothing; otherwise, define ( -1, Let M be the coordinates of the left adjacent cell, and record the left adjacent cell in dataset M2; Similarly, obtain the right neighbor, top neighbor, and bottom neighbor of cell M11, and complete the assignment of the actual position coordinate data of the cell in the cell dataset M1 to the neighboring cells; The actual position coordinates of the cells in the newly generated dataset M2 are assigned to adjacent cells. The assigned cells are recorded in dataset M3. This process continues until the coordinates of the cells in the original ground images of each camera are assigned. All cell numbers, cell coordinates, cell labels, the pixel regions corresponding to the cells, and the camera numbers in the shooting area of ​​each camera are obtained to obtain the original cell dataset U captured by the cameras.

6. The method for generating digital raster maps based on deep learning according to claim 1, characterized in that, The process of obtaining real-time operating status information of all forklifts in the warehouse based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera, and the static grid map of the warehouse, and generating a dynamic grid map based on the real-time operating status information, includes: The pixel region of each forklift in the current camera image and the marked number of the forklift's heading are obtained by improving the YOLOv8 algorithm model; The pixel region of the forklift in the current camera image is compared with the pixel region of the corresponding grid in the original grid dataset U captured by the current camera. If a grid is matched, the pixel region value of the matched grid is recorded to obtain the pixel region to be compared. If multiple cells are matched, the pixel regions of the multiple cells are merged to obtain the pixel region to be compared. Based on the pixel regions to be compared, the coordinate data and heading data of each forklift are obtained; A dynamic grid map is generated based on the current static grid map of the warehouse, as well as the coordinate data and heading data of each forklift.

7. The method for generating digital raster maps based on deep learning according to claim 6, characterized in that, The process of obtaining forklift coordinate data based on the pixel regions to be compared includes: Calculate the forklift's x-coordinate using formula (4); Formula (4); In the formula, Let x be the x-coordinate of the forklift. The x-coordinate of the pixel region to be compared. The ratio of the overlapping area to the distance to the left. The ratio of the overlapping area to the distance to the right; Calculate the forklift's longitudinal coordinate using formula (5); Formula (5); In the formula, Let the forklift's vertical coordinate be... The vertical coordinate of the pixel region to be compared is... This represents the ratio of the overlapping area to the distance above. This represents the ratio of the overlapping area to the distance to the bottom edge.

8. A system for generating digital raster maps based on deep learning, characterized in that, include: The data acquisition module is used to divide the warehouse into areas and install and fix cameras, and to acquire the original ground image dataset H before the small flag is placed and the ground image dataset F after the small flag is placed for each camera. The grid recognition module is used to identify grids in the ground image dataset F based on the improved YOLOv8 algorithm model. It obtains the pixel area of ​​each grid in the image and generates a unique number. Based on the grid color, it marks the grid as a lane grid or a storage location grid. The difference calculation module is used to calculate the difference between the original ground image dataset H and the ground image dataset F corresponding to each camera before and after placing the flag, and to determine the pixel area and coordinate data of the flag. The recording module is used to record the set of grid numbers, the set of adjacent grid numbers, and the set of adjacent relationships for images in the ground image dataset F; The actual position coordinate data acquisition module is used to sum the pixel regions of each grid, compare the difference with the pixel regions of the image in the original ground image dataset H, obtain at least one blank pixel region, define the blank pixel region as a new grid and generate a unique serial number, determine the adjacency relationship of the new grid, obtain the flag coordinate data of the flag pixel region contained in the blank pixel region, use the flag coordinate data as the actual position coordinate data of the grid corresponding to the blank pixel region, and assign all the flag coordinate data to the corresponding grid to obtain the actual position coordinate data of all grids, grid serial numbers, grid adjacency relationships and some grids of the original ground dataset H captured by each camera; The coordinate assignment module is used to obtain the grid dataset M1 with actual position coordinate data in the original ground dataset H corresponding to each camera, assign coordinate values ​​to the remaining grids without actual position coordinate data to obtain the original grid dataset U captured by the camera, aggregate the grid coordinates and grid labels of the original grid dataset U captured by the camera, deduplicate the grid coordinate data and grid labels of the aggregated data to establish the grid coordinate dataset K, and establish the association between the grid coordinate dataset K and the original grid dataset U captured by the camera. The static map generation module is used to generate a warehouse static raster map based on the grid coordinates and grid types of the grid coordinate dataset K, and to generate a static raster map for each camera based on the association between the grid coordinate dataset K and the original grid dataset U captured by each camera. The dynamic map generation module is used to obtain real-time operating status information of all forklifts in the warehouse based on the improved YOLOv8 algorithm model, the original grid dataset U captured by the camera and the static grid map of the warehouse, and to generate a dynamic grid map based on the real-time operating status information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images