A grid-based yard scheduling method based on video GIS fusion

Abstract

This invention relates to the field of intelligent yard management and industrial scheduling technology, and discloses a grid-based yard scheduling method based on video-GIS fusion. The method first divides the yard into square grids and dynamically updates the grid status using visual recognition technology to reflect changes in cargo. Next, based on operational information, a path search algorithm generates a conflict-free initial route from the entrance to the work point. If the route is obstructed during vehicle travel, a new route is replanned in real time. Furthermore, the method includes a grid-level conflict avoidance mechanism, which predicts and warns of potential conflicts between multiple vehicles using an early warning grid set, significantly improving the safety and efficiency of yard operations.

Description

Technical Field

[0001] This invention relates to the field of intelligent yard management and industrial scheduling technology, specifically to a grid-based yard scheduling method based on video GIS fusion. Background Technology

[0002] In traditional bulk cargo yard operations (such as coal, ore, and grain), cargo handling and vehicle scheduling heavily rely on human experience, resulting in low efficiency and a high risk of errors. This is particularly evident when using driverless vehicles, where issues such as untimely updates to stacking status information and inefficient loading and unloading scheduling become especially pronounced. While some yards have introduced automated equipment with the development of automation technology, numerous challenges remain in route planning.

[0003] Existing technologies mostly rely on static maps for route planning, which makes it difficult to reflect the impact of dynamic changes in cargo within the yard on the route in real time. This results in a mismatch between the planned route and the actual operating environment, increasing operational risks and costs.

[0004] In addition, traditional methods lack effective conflict avoidance mechanisms when multiple vehicles are working together, which can easily lead to traffic accidents and affect the efficiency and safety of yard operations. Summary of the Invention

[0005] To address the aforementioned technical shortcomings, the purpose of this invention is to provide a grid-based yard scheduling method based on video GIS fusion, thereby solving the problem of low efficiency in yard operation scheduling and conflict avoidance.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In a first aspect, the present invention provides a grid-based yard scheduling method based on video GIS fusion, the method comprising: Step S1: Divide the yard operation area into square grids with equal side lengths, and assign an initial status label to each grid; the status label includes idle status and occupied status; Step S2: Access the yard monitoring video stream, parse the video stream using a visual recognition algorithm, identify the outline and location of goods in the yard, match the recognition results with grid coordinates, and dynamically update the status labels of the corresponding grids; Step S3: Based on the yard operation information, mark the status label of the target grid as the operation point; with the operation point as the target, based on the latest status labels of all grids, use the path search algorithm to calculate the conflict-free travel trajectory from the entrance grid to the operation point and generate the initial operation route; Step S4: Send the initial work route to the work vehicle; during the vehicle's journey, if the status label of any grid in the initial work route changes to occupied, then take the grid where the vehicle is currently located as the starting point and the work point as the target, recalculate and send out a new work route based on the latest status label.

[0007] Preferably, in one possible implementation of the first aspect, step S1, the process of dividing the yard operation area into a square grid, includes: Based on the map data of the storage yard, extract the boundaries of all passable areas within the work area and establish a unified Cartesian coordinate system; In a Cartesian coordinate system, the side length of the square grid is determined based on the minimum turning radius and standard vehicle width of vehicles operating in the yard. Using the origin of the Cartesian coordinate system as a reference, the entire yard operation area is divided into multiple square grids with equal side lengths, and each square grid is assigned a globally unique coordinate identifier. During initialization, all grids are traversed. If a grid contains an area marked as a fixed obstacle on the map, the grid's status label is initialized to occupied; otherwise, it is initialized to idle.

[0008] Preferably, in one possible implementation of the first aspect, step S2, the process of parsing the video stream using a visual recognition algorithm, includes: Access the real-time video stream from the surveillance cameras deployed in the yard, and extract key frame images from the video stream at fixed time intervals; Each keyframe image is preprocessed, including perspective transformation correction and illumination normalization. The preprocessed image is input into the trained deep neural network model, and the deep neural network model outputs the boundary coordinates of the area where the goods are located in the image and its confidence score. For recognition results with a confidence level higher than a set threshold, the vertex coordinates in the Cartesian coordinate system are calculated to obtain the outline and location information of each item.

[0009] Preferably, in one possible implementation of the first aspect, the process of dynamically updating the state label of the corresponding grid includes: Based on the coordinates of the cargo vertices obtained by the visual recognition algorithm, calculate the physical area covered by the cargo outline; Calculate the spatial geometric relationship between the physical area and the square grid of the yard operation area to determine all grids that are completely covered or partially overlapped by the outline of the cargo; If a grid is completely covered or partially overlapped by any cargo outline, the status label of that grid is updated to occupied. For a grid whose status label is occupied, continuously detect whether the corresponding goods have been moved away. When it is detected that the goods have left and the grid no longer intersects with any goods outline, update its status label to idle.

[0010] Preferably, in one possible implementation of the first aspect, step S3, the initial job route generation process includes: After obtaining the grid coordinates of the target work point, the entry grid is used as the starting node for path search; A path search algorithm is used to search for paths. The path search algorithm is subject to hard constraints to avoid all grids with state labels in an occupied state, and searches are performed in a gridded map. The path search algorithm outputs a series of ordered coordinate sequences from the starting node to the target node, which are formed by connecting the center points of adjacent grids in sequence. This sequence constitutes the initial path.

[0011] Preferably, in one possible implementation of the first aspect, the path search algorithm employs A algorithm; A When the algorithm searches in the gridded map, it treats the center point of each free state grid as a reachable path node; Maintain a cost value for each path node, which consists of the sum of the actual movement cost from the starting node to the path node and the estimated cost from the path node to the destination node; The actual movement cost is calculated based on the Euclidean distance of the vehicle moving from one grid center point to an adjacent grid center point, while the estimated cost is the Euclidean distance from the node to the target node. In each iteration of the path search, the node with the lowest replacement value is selected from all unprocessed path nodes as the current expansion node; Check all adjacent grids of the current extended node. If the status label of an adjacent grid is idle, add its corresponding path node to the set to be evaluated and calculate its cost. Repeat the selection and expansion process until the path node corresponding to the target work point is selected as the current expansion node. At this point, backtrack from the target work point node to the starting node to obtain an ordered coordinate sequence.

[0012] Preferably, in one possible implementation of the first aspect, step S4, the process of recalculating and issuing the replanning operation route, includes: The status of the grid along the initial work route is monitored in real time as the vehicle travels along the initial work route. When the status label of any grid cell changes from idle to occupied, local path replanning is triggered. Using the grid where the vehicle is currently located as the new starting node for path search, and keeping the target work point unchanged, a replanned work route from the current location to the target work point is calculated based on the latest global grid status label and using the same path search algorithm as in step S3. After the replanned operation route is generated, it is sent to the operation vehicle through the vehicle-to-ground communication network to replace the interrupted initial operation route.

[0013] Preferably, in one possible implementation of the first aspect, the method further includes a grid-level conflict avoidance step: Based on the real-time location and speed of the work vehicle, the dispatch system dynamically calculates the number of idle state grids in front of and behind it, forming a warning grid set for the vehicle, and marks all grids in the set as warning state. At the same time, a global list containing a set of real-time warning grids for all vehicles in motion is continuously maintained; For any two work vehicles that have potential path intersections, compare the vehicle's early warning grid set in real time; When an intersection is detected between two warning grid sets, it is determined that there is a risk of driving conflict, and a conflict warning signal is sent to the vehicle to provide a warning.

[0014] In a second aspect, the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a gridded yard scheduling method based on video GIS fusion as described in the first aspect.

[0015] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a gridded yard scheduling method based on video GIS fusion as described in the first aspect.

[0016] The beneficial effects of this invention are as follows: This invention achieves real-time perception and dynamic modeling of the yard operation environment through GIS maps and visual recognition technology, ensuring the accuracy and real-time performance of yard scheduling.

[0017] Using gridded maps and A The algorithm improves the efficiency and quality of path search, reducing vehicle travel time and energy consumption.

[0018] Meanwhile, the grid-level conflict avoidance mechanism can predict and warn of potential conflicts in multiple workshops in real time, which significantly improves the safety of yard operations.

[0019] Overall, this invention improves the automation and intelligence level of yard operations, reduces labor costs, and enhances the operational competitiveness of yards. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This application provides a flowchart of a grid-based yard scheduling method based on video-GIS fusion. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: As Figure 1 As shown, this invention provides a grid-based yard scheduling method based on video-GIS fusion, comprising: Step S1: Divide the yard operation area into square grids with equal side lengths, and assign an initial status label to each grid; the status label includes idle status and occupied status.

[0024] In this embodiment, the original high-precision map data of the yard is first obtained based on the yard GIS, including the boundary coordinates of all roads, fixed buildings, equipment and facilities, and inaccessible areas within the operating area. Based on this map data, the boundary lines of all areas accessible to operating vehicles are extracted, and a unified Cartesian coordinate system covering the entire yard operating area is established. The origin of this coordinate system is set at a fixed corner point in the operating area, and the X-axis and Y-axis are parallel to the main roads of the port area.

[0025] Next, determine the grid size of the coordinate system and the grid side length. The physical dimensions and maneuverability of the main vehicles used in the yard were taken into consideration. Specifically, the side length... Simultaneously satisfy , , in, This represents the minimum safe turning radius of the operating vehicle. S represents the standard width of the vehicle, and S is a preset safety margin. The side length is determined by the above formula. This ensures that the vehicle has sufficient space to perform steering maneuvers within any available grid cell, and guarantees that the vehicle maintains a safe distance from grid boundaries when moving between adjacent grid cells. In this embodiment... The value is 5.0 meters.

[0026] After determining the coordinate system and grid size, using the origin of the coordinate system as a reference, along the X and Y axes, with a step size... The entire yard operation area is divided into sections, generating a series of sections with side lengths of... A square grid. Each grid cell is represented by the coordinates of its lower left vertex in a Cartesian coordinate system. To provide a unique identifier, where and These integer indices together form the globally unique coordinate identifier of the grid. .

[0027] After geometric partitioning, each mesh is assigned an initial state label, which is divided into two types: idle and occupied. The initialization process iterates through all meshes. For each grid cell, it is determined whether there are areas pre-marked as fixed obstacles in the high-precision map within its area, such as building foundations, fixed crane bases, permanent roadblocks, etc. Specifically, if a grid cell contains areas marked as fixed obstacles in the map, the grid cell is determined to be occupied, its status label is initialized to occupied, and the grid cell is marked in red on the system interface. For grid cells that do not contain any fixed obstacle areas, their status labels are initialized to idle.

[0028] Step S2: Access the yard monitoring video stream, parse the video stream using a visual recognition algorithm, identify the outline and location of goods in the yard, match the recognition results with grid coordinates, and dynamically update the status labels of the corresponding grids.

[0029] In this embodiment, real-time video streams from monitoring cameras deployed at key locations in the yard are accessed. These cameras cover the entire work area. The system synchronously extracts one frame as a keyframe from each video stream at fixed time intervals; in this embodiment, it does so every 5 seconds.

[0030] For each extracted keyframe image, a standardized preprocessing workflow is performed. First, perspective transformation correction is applied. Due to differences in camera installation position and angle, the original image exhibits perspective distortion. Pre-calibrated camera parameters are invoked, including the camera intrinsic matrix and the extrinsic matrix relative to the Cartesian coordinate system of the storage yard. Using these parameters, an inverse perspective transformation is performed on the input image, projecting the image onto the top plane corresponding to the two-dimensional Cartesian coordinate system of the storage yard. This eliminates geometric distortion caused by the viewing angle, enabling a linear mapping between the pixel coordinates of the goods in the image and their physical world coordinates.

[0031] Next, illumination normalization is performed. The storage yard operating environment is affected by the alternation of day and night and weather changes, resulting in significant differences in illumination conditions. A white balance algorithm based on adaptive histogram equalization and gray world assumption is used to process the corrected image, suppressing the interference of uneven illumination, shadows and color cast on the recognition algorithm, and improving the robustness of image quality.

[0032] The preprocessed image is input into a trained deep neural network model to identify the contours of all goods in the image. In this embodiment, the deep neural network model used in the visual recognition algorithm is a YOLOv7-based object detection architecture. The network structure consists of three main parts: Backbone, Neck, and Head. The Backbone part uses an efficient layer aggregation network, including multiple CBS modules, ELAN modules, and MP convolutional downsampling modules, to extract multi-level feature maps from the input image. The Neck part uses a path aggregation network and a spatial pyramid pooling structure to achieve deep fusion and enhancement of feature maps at different scales, while capturing detailed features of large container stacks and small bulk cargo. The Head part contains three detection heads with different resolutions, corresponding to large, medium, and small scale cargo targets, respectively. Each detection head outputs the coordinates of the predicted bounding box, confidence score, and pixel-level segmentation mask, and the resulting cargo contours are described by the segmentation mask.

[0033] The training process of the deep neural network model is as follows. First, a training dataset is constructed. The dataset contains tens of thousands of labeled images of yard operation scenarios, covering different time periods, weather conditions, cargo types, and stacking states. The annotation information for each image includes the vertex coordinates of the segmentation mask of the cargo instance and its category label. Before training, Mosaic data augmentation, random affine transformation, and color space perturbation are applied to the dataset to improve the model's generalization ability.

[0034] Model training employs an end-to-end approach, with the loss function consisting of a weighted average of three parts: bounding box regression loss, confidence loss, and segmentation mask loss. The bounding box regression loss uses CIoU loss to measure the overlap and alignment between the predicted and ground truth boxes; the confidence loss uses binary cross-entropy loss; and the segmentation mask loss combines Dice loss and binary cross-entropy loss to optimize pixel-level classification accuracy. The optimizer chosen is AdamW, with an initial learning rate of 0.001, and a cosine annealing learning rate scheduling strategy is applied.

[0035] During inference, the preprocessed image undergoes forward propagation through the network. The three detector heads output multiple sets of prediction results. Non-maximum suppression is performed on all these prediction results to filter out redundant prediction boxes with excessive overlap, retaining prediction results with a confidence score higher than a set threshold of 0.85. Each retained result contains a set of vertex pixel coordinates of a cargo segmentation mask and its confidence score.

[0036] For each recognition result with a confidence level higher than 0.85, the system performs coordinate mapping. Based on the camera calibration parameters, a homography transformation matrix is ​​established from the image pixel coordinate system to the yard plane rectangular coordinate system. The physical world coordinates of each vertex pixel coordinate of the recognized segmentation mask are calculated in the yard plane rectangular coordinate system using this homography transformation matrix. The polygonal region formed by connecting a series of vertex coordinates in the physical world is the actual physical area covered by the cargo outline.

[0037] After obtaining the physical outline of the goods, the status labels of the corresponding grids are dynamically updated. Specifically, for each set of vertex physical coordinates obtained through a visual recognition algorithm, the continuous physical region covered by the polygonal outline is calculated. Then, the spatial geometric relationship between this physical region and the square grid map of the yard operation area is calculated. For each grid, its region is represented as a rectangle defined by the coordinates of its lower left and upper right vertices. By determining whether the grid rectangle intersects with the goods outline polygon, all grids completely covered or partially overlapped by the goods outline are identified. If overlap exists, the grid is considered occupied.

[0038] If a grid is completely covered or partially overlapped by any cargo outline polygon, the status label of that grid is updated to occupied.

[0039] For grids whose status labels have been marked as occupied, the dynamics of their occupancy sources are continuously monitored. When the visual recognition algorithm continuously detects that a certain item has been removed by a work vehicle, and in subsequent keyframe analysis, the outline polygon coordinates of the item no longer intersect with the occupied grid area, and after confirmation for two consecutive frames, the grid no longer intersects with any item outline polygon, the system determines that the item has left, the grid returns to a passable state, and the status label of the grid is updated back to an idle state.

[0040] Step S3: Based on the yard operation information, mark the status label of the target grid as the operation point; with the operation point as the target, based on the latest status labels of all grids, use the path search algorithm to calculate the conflict-free travel trajectory from the entrance grid to the operation point and generate the initial operation route.

[0041] In this embodiment, a work instruction is received from the yard scheduling system. This work instruction is generated automatically by the system reading work information or by the worker selecting it. It includes the target location of the work and marks the grid in green on the system interface according to the target location of the work.

[0042] First, determine the physical coordinates of the target location. Based on the pre-established mapping relationship, it is transformed to a unified Cartesian coordinate system, and then calculated. , Determine the unique grid coordinates. Subsequently, a "Work Point" attribute label is added to the global grid state map table for this grid. The starting point for path searching is the entry grid where the work vehicle is currently located, and its coordinates... Provided by the vehicle positioning system.

[0043] The path search algorithm used is A The algorithm evaluates the function Guiding the search direction, among which Represents the distance from the starting node to the current node. The actual cost, Represents starting from the current node The estimated cost to reach the target node.

[0044] The algorithm defines the center point of the grid with a status of "idle" or "working point" as a walkable path node. Each node is a data structure containing the following fields: grid coordinates. Parent node pointer, actual cost Estimated Costs Total cost The algorithm maintains two data structures: an "open list" to store discovered but yet-to-be-evaluated nodes, implemented as a priority queue using a minimum binary heap; and a "closed list" to store nodes that have been evaluated, implemented using a hash table.

[0045] In the initialization phase of path search, the starting node is... Add to open list, set actual cost Calculate its distance to the target node Heuristic cost This embodiment uses Euclidean distance as the heuristic function because it is acceptable and consistent in grid environments, ensuring that the shortest path is found. The calculation formula is as follows: Thus, the total cost of the starting node is obtained. .

[0046] The algorithm then enters the main iteration loop. In each iteration, a character is popped from the top of the binary heap of the open list. The node with the smallest value is selected as the current expansion node. First, determine Are the coordinates consistent with If they are equal, it indicates that the target has been found, the algorithm terminates successfully, and enters the path backtracking phase; if they are not equal, then... Moving a component to the closed list indicates that it has been optimally processed.

[0047] Next, expand All adjacent nodes are connected using a four-neighborhood model, meaning each grid cell can be connected to adjacent grid cells in four directions. For each adjacent direction, its grid coordinates are calculated. Query the global grid status table. If the status label of an adjacent grid is "occupied," ignore the node. Only consider it as a candidate successor node if the status is "idle" or "working point." .

[0048] For each feasible successor node, calculate the sequence from the starting node via... The actual cost of moving to this node The movement cost is based on the Euclidean distance between the grid centers. , For the current expansion node The actual cost.

[0049] Then, the heuristic cost from the successor node to the target node is calculated. The calculation formula is the same as above, and the estimated total cost of this node is... .

[0050] Next, check Existence state: like If a node is neither in the open list nor the closed list, it is a newly discovered node, and its parent node pointer is set to... and set its , , Then insert the node into the binary heap of the open list.

[0051] like If it's already in the open list, compare the newly calculated one. Compared to the original nodes in the open list Value, if A smaller value indicates that a better path to the node has been found. At this point, update the node's data in the open list: reset its parent node to [value]. and update its g Recalculate Value. Due to the node's The value may become smaller, requiring adjustment of its position in the binary heap. This is an operation of "lowering node priority", which is achieved in this embodiment by floating the node up in the heap.

[0052] like In this closed list, to address the extreme case where path costs may change in a dynamic environment, this embodiment employs a "dynamic weighting" and "reopening" strategy. When the conditions are met... When the cost of the new path is significantly lower than that of the original path, remove this node from the closed list and use the updated path. , The new parent node information is then used to re-add it to the open list for re-evaluation. This is a small positive threshold; in this embodiment, it is set to 0.1. This avoids frequent replanning due to minor cost fluctuations.

[0053] The iteration ends after evaluating all feasible successor nodes of the current node. The algorithm then executes iteratively, retrieving the next node from the open list. The node with the smallest value is expanded until the target node is retrieved or the open list is empty. If the open list is empty, the search fails, and there is no reachable path from the starting point to the target node.

[0054] The path search is complete when the target node is successfully retrieved. Start by backtracking backwards along the parent node pointers to... This yields a reverse sequence of nodes from the starting point to the ending point. Reversing this sequence gives an ordered sequence of grid coordinates from the entry grid to the work point grid. ,in , This sequence is a discrete grid representation of the planned initial path.

[0055] Before the path is issued, a static conflict check is performed. The initially planned paths are then traversed. All covered grids are snapshotted and compared with the global grid status table after the planning is completed to ensure that the status of each grid on the path is "idle" or "work point" at that moment. If the verification passes, an initial work route data packet containing information such as path ID, target point coordinates, and waypoint list is sent to the designated work vehicle. If the verification fails, it means that an obstacle entered the planned path during the planning calculation, and the current planning fails, triggering a replanning based on the latest environmental information.

[0056] Step S4: Send the initial work route to the work vehicle; during the vehicle's journey, if the status label of any grid in the initial work route changes to occupied, then take the grid where the vehicle is currently located as the starting point and the work point as the target, recalculate and send out a new work route based on the latest status label.

[0057] In this embodiment, after the initial work route is generated and passes static conflict verification, the system sends a route data packet to the target work vehicle via the vehicle-to-ground communication network. This data packet uses a binary encoding format and contains a unique identifier for the path sequence, the grid coordinates of the target work point, and a waypoint sequence consisting of a series of grid center point coordinates. After successfully receiving and parsing the data packet, the work vehicle's control system generates a smooth local trajectory based on the waypoint sequence and controls the vehicle to begin traveling along the initial work route.

[0058] During vehicle operation, the system monitors the initial work route executed by the vehicle in real time. The system continuously outputs a global grid state table, showing the state of each covered grid. A table is maintained by the system. The corresponding grid status monitoring list is updated at a fixed frequency (once every 3 seconds in this embodiment). Each grid coordinate in the list is traversed to query its real-time status label in the latest global grid status table. As long as the status labels of all grids along the path remain "idle" or "operational," the vehicle will continue along the predetermined route.

[0059] When the monitoring process detects the initial work route When the status label of any grid changes from idle to occupied, the system triggers local path replanning. The determination of status changes is real-time. When the global grid status table completes the update of the occupied status of a certain grid, and the coordinates of that grid exist in the grid sequence of the currently active path, the replanning trigger is activated.

[0060] Path replanning starts from the vehicle's current physical location, uses pre-defined coordinate transformation relationships to determine its coordinates in a unified Cartesian coordinate system, and calculates the coordinates of its corresponding grid, using that grid as the starting node for the new path search. Target work point grid Remain unchanged. Obtain the latest global grid state table snapshot at the current moment, which reflects the distribution of all static obstacles and the most recently identified dynamic cargo obstacles.

[0061] The path search algorithm from step S3 is used for replanning. The algorithm uses... Starting from the node, For the target node, recalculate a conflict-free path within the search space defined by the latest grid state table. .exist Before deployment, the system performs an immediate conflict verification to ensure that all grids on the new path are in an idle or operational state at the time of verification. After successful verification, the system generates a replanning operation route data packet, which is then deployed to the operation vehicle to replace the interrupted initial operation route.

[0062] Example 2: This invention provides a grid-based yard scheduling method based on video GIS fusion, which also includes a grid-level conflict avoidance step, which runs synchronously during the operation of the work vehicles while they are traveling on the planned route, and is used to predict in real time whether there will be cross-traffic or head-on conflicts between multiple work vehicles.

[0063] Specifically, firstly, a dedicated real-time early warning grid is dynamically constructed for each operating vehicle, based on its location coordinates. With grid side length The mapping relationship is used to determine the current grid coordinates of the vehicle. Let this grid be the center grid.

[0064] The early warning grid set is constructed to cover the areas in front of and behind the vehicle's direction of travel, addressing various scenarios such as vehicles moving forward, reversing, and merging. The number of grids in the set is a dynamic value determined by a preset look-ahead time. Looking back Real-time vehicle speed and grid side length The number of grid cells to be covered in front of the vehicle's direction of travel is determined jointly. Through formula The number of grids that need to be covered behind the vehicle in the direction of travel. Through formula Calculation. Where, the symbol... This represents the floor function. In this embodiment, the lookahead time... The time is set to 3 seconds to allow for braking or avoidance distance ahead; the time for looking back is also considered. Set to 1.5 seconds, this system protects the rear of the vehicle from rear-end collisions. The system uses a central grid... Based on this, along the vehicle's work route (initial work route) Or replan the work route ), sequentially search the grids in front and behind, until the cumulative number of free grids added in the forward direction reaches 100. The cumulative number of free grids added forward has reached The resulting warning grid set represents the vehicle's immediate occupation and short-term reservation of surrounding road resources at the current moment. The system marks all grids within this warning grid set with a "warning" attribute in the global grid status table to indicate to other vehicles that the grid area has been dynamically reserved.

[0065] Create and maintain a global early warning grid list, which is a centralized, dynamic data structure used to aggregate the status of all on-the-go work vehicles. Each item in the list corresponds to a work vehicle, and the record must include at least the vehicle's unique identifier. and its corresponding real-time early warning grid set. Once a work vehicle completes its task, its record will be removed from the global early warning grid list, and its corresponding "early warning" marker in the global grid status table will be cleared.

[0066] Conflict risk detection is achieved by periodically comparing the warning grid sets of different vehicles in the global warning grid list. The system iterates through the global warning grid list at a fixed period; in this embodiment, it does so every 1 second. For any two different operating vehicles in the list... and Obtain the current early warning grid set. and Then calculate the intersection of the two sets. If the intersection If the value is not empty, meaning there exists at least one grid coordinate belonging to both sets, then the system determines it to be a vehicle. and There is a potential risk of traffic collisions, and the physical location where a collision occurs is defined by the area corresponding to all grid coordinates in the intersection.

[0067] When a conflict risk is detected, a structured conflict warning signal data packet is generated and sent to the work vehicle and dispatch system. The work vehicle adjusts its operation based on the conflict warning signal data packet, and the staff monitors the situation based on the warning information.

[0068] Example 3: This example provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method involved in Example 1.

[0069] Example 4: This example provides a computer program product containing instructions, including a memory, a processor, and a program stored in the memory and executable on the processor, which, when run on a computer, causes the computer to perform the steps of the method involved in Example 1.

[0070] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A grid-based yard scheduling method based on video-GIS fusion, characterized in that, The method includes: Step S1: Divide the yard operation area into square grids with equal side lengths, and assign an initial status label to each grid; the status label includes idle status and occupied status; Step S2: Access the yard monitoring video stream, parse the video stream using a visual recognition algorithm, identify the outline and location of goods in the yard, match the recognition results with grid coordinates, and dynamically update the status labels of the corresponding grids; Step S3: Based on the yard operation information, mark the status label of the target grid as the operation point; with the operation point as the target, based on the latest status labels of all grids, use the path search algorithm to calculate the conflict-free travel trajectory from the entrance grid to the operation point and generate the initial operation route; Step S4: Send the initial work route to the work vehicle; during the vehicle's journey, if the status label of any grid in the initial work route changes to occupied, then take the grid where the vehicle is currently located as the starting point and the work point as the target, recalculate and send out a new work route based on the latest status label.

2. The grid-based yard scheduling method based on video-GIS fusion as described in claim 1, characterized in that, In step S1, the process of dividing the yard operation area into a square grid includes: Based on the map data of the storage yard, extract the boundaries of all passable areas within the work area and establish a unified Cartesian coordinate system; In a Cartesian coordinate system, the side length of the square grid is determined based on the minimum turning radius and standard vehicle width of vehicles operating in the yard. Using the origin of the Cartesian coordinate system as a reference, the entire yard operation area is divided into multiple square grids with equal side lengths, and each square grid is assigned a globally unique coordinate identifier. During initialization, all grids are traversed. If a grid contains an area marked as a fixed obstacle on the map, the grid's status label is initialized to occupied; otherwise, it is initialized to idle.

3. The grid-based yard scheduling method based on video GIS fusion as described in claim 1, characterized in that, In step S2, the process of parsing the video stream using a visual recognition algorithm includes: Access the real-time video stream from the surveillance cameras deployed in the yard, and extract key frame images from the video stream at fixed time intervals; Each keyframe image is preprocessed, including perspective transformation correction and illumination normalization. The preprocessed image is input into the trained deep neural network model, and the deep neural network model outputs the boundary coordinates of the area where the goods are located in the image and its confidence score. For recognition results with a confidence level higher than a set threshold, the vertex coordinates in the Cartesian coordinate system are calculated to obtain the outline and location information of each item.

4. The grid-based yard scheduling method based on video GIS fusion as described in claim 3, characterized in that, The process of dynamically updating the status label of the corresponding grid includes: Based on the coordinates of the cargo vertices obtained by the visual recognition algorithm, calculate the physical area covered by the cargo outline; Calculate the spatial geometric relationship between the physical area and the square grid of the yard operation area to determine all grids that are completely covered or partially overlapped by the outline of the cargo; If a grid is completely covered or partially overlapped by any cargo outline, the status label of that grid is updated to occupied. For a grid whose status label is occupied, continuously detect whether the corresponding goods have been moved away. When it is detected that the goods have left and the grid no longer intersects with any goods outline, update its status label to idle.

5. A grid-based yard scheduling method based on video-GIS fusion as described in claim 1, characterized in that, In step S3, the initial work route generation process includes: After obtaining the grid coordinates of the target work point, the entry grid is used as the starting node for path search; A path search algorithm is used to search for paths. The path search algorithm is subject to hard constraints to avoid all grids with state labels in an occupied state, and searches are performed in a gridded map. The path search algorithm outputs a series of ordered coordinate sequences from the starting node to the target node, which are formed by connecting the center points of adjacent grids in sequence. This sequence constitutes the initial path.

6. The grid-based yard scheduling method based on video GIS fusion as described in claim 5, characterized in that, The path search algorithm uses A algorithm; A When the algorithm searches in the gridded map, it treats the center point of each free state grid as a reachable path node; Maintain a cost value for each path node, which consists of the sum of the actual movement cost from the starting node to the path node and the estimated cost from the path node to the destination node; The actual movement cost is calculated based on the Euclidean distance of the vehicle moving from one grid center point to an adjacent grid center point, while the estimated cost is the Euclidean distance from the node to the target node. In each iteration of the path search, the node with the lowest replacement value is selected from all unprocessed path nodes as the current expansion node; Check all adjacent grids of the current extended node. If the status label of an adjacent grid is idle, add its corresponding path node to the set to be evaluated and calculate its cost. Repeat the selection and expansion process until the path node corresponding to the target work point is selected as the current expansion node. At this point, backtrack from the target work point node to the starting node to obtain an ordered coordinate sequence.

7. The grid-based yard scheduling method based on video-GIS fusion as described in claim 1, characterized in that, In step S4, the process of recalculating and issuing the replanned work route includes: The status of the grid along the initial work route is monitored in real time as the vehicle travels along the initial work route. When the status label of any grid cell changes from idle to occupied, local path replanning is triggered. Using the grid where the vehicle is currently located as the new starting node for path search, and keeping the target work point unchanged, a replanned work route from the current location to the target work point is calculated based on the latest global grid status label and using the same path search algorithm as in step S3. After the replanned operation route is generated, it is sent to the operation vehicle through the vehicle-to-ground communication network to replace the interrupted initial operation route.

8. The grid-based yard scheduling method based on video GIS fusion as described in claim 1, characterized in that, The method also includes a grid-level collision avoidance step: Based on the real-time location and speed of the work vehicle, the dispatch system dynamically calculates the number of idle state grids in front of and behind it, forming a warning grid set for the vehicle, and marks all grids in the set as warning state. At the same time, a global list containing a set of real-time warning grids for all vehicles in motion is continuously maintained; For any two work vehicles that have potential path intersections, compare the vehicle's early warning grid set in real time; When an intersection is detected between two warning grid sets, it is determined that there is a risk of driving conflict, and a conflict warning signal is sent to the vehicle to provide a warning.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the gridded yard scheduling method based on video GIS fusion as described in any one of claims 1-8.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the gridded yard scheduling method based on video GIS fusion as described in any one of claims 1-8.

Images