Intelligent positioning and trajectory monitoring method and system for container yard

Abstract

The present invention discloses an intelligent positioning and trajectory monitoring method and system for a container yard, belonging to the field of image processing technology. The method comprises: a camera records a scene image at a preset angle in real time and generates a scene video; a container location is set in a yard map; a reach stacker obtains the container number based on the camera installed on it; the target storage location of the container is determined from the container yard, and a vehicle planning path is generated; the reach stacker in the scene video is identified, and the reach stacker is associated with its own license plate number, container number and vehicle planning path; the reach stacker is tracked based on the scene video shot by multiple cameras, and the actual movement path of the reach stacker is generated. If the actual movement path does not match the vehicle planning path, the reach stacker is marked as abnormal. The present invention can comprehensively track the trajectory of the container and identify abnormal trajectories therein.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition technology, and in particular relates to an intelligent positioning and trajectory monitoring method and system for a container yard. Background Art

[0002] Traditional container yard management relies on manual operations and simple monitoring systems, such as manually recording container locations and using basic security surveillance cameras. These methods are not only time-consuming and labor-intensive, but also prone to errors, making it difficult to meet the needs of modern logistics parks for efficient and precise management.

[0003] With the rapid development of computer vision and artificial intelligence technologies, image recognition-based container tracking technology has gradually emerged. This technology automatically identifies and tracks containers by installing high-definition cameras in the container yard to capture images and intelligently analyze them using algorithms such as deep learning. For example, Chinese patent publication number CN115690061A discloses a vision-based method for detecting container trucks at container terminals. This method first captures image data and then uses image processing and deep learning methods to identify and track targets such as the truck head, containers, and the truck's carriage.

[0004] The above method only monitors the containers and vehicles in the yard from the perspective of the yard business process. However, when the trajectory of the vehicle or container is abnormal, the above solution cannot perform accurate detection, resulting in incomplete monitoring. Summary of the Invention

[0005] To solve the above problems, the present invention provides an intelligent positioning and trajectory monitoring method and system for a container yard, so as to achieve tracking and identification of container trajectories.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention proposes an intelligent positioning and trajectory monitoring method for a container yard, comprising:

[0007] Cameras are placed at multiple preset locations in the yard. Each camera records the scene from a preset angle in real time and generates a scene video.

[0008] Establishing a yard map, and setting container stacking locations and corresponding first stacking attributes in the yard map;

[0009] The reach stacker obtains the container number of the container based on a camera installed on the reach stacker, and obtains the second stacking attribute of the container based on the container number;

[0010] Obtaining the first stacking attribute corresponding to the second stacking attribute to determine a target container location in a container yard, and generating a vehicle planning path based on a starting position of the reach stacker and the target container location;

[0011] Acquire the scene video at the starting position, identify the reach stacker in the scene video, and associate the reach stacker with its own license plate number, the container number, and the planned vehicle path;

[0012] The reach stacker is tracked based on the scene video captured by multiple cameras to generate an actual moving path of the reach stacker. If the actual moving path does not match the planned vehicle path, the reach stacker is marked as abnormal.

[0013] Furthermore, tracking the reach stacker includes the following steps:

[0014] Splitting the scene video into multiple first static images, performing spatial filtering processing on each of the first static images to obtain a second static image, and establishing an image recognition model, wherein the image recognition model includes a preprocessing layer, a convolution layer, and a fully connected layer, the convolution layer includes multiple convolution kernels, each of the convolution kernels has a different kernel size, and one of the convolution kernels is selected as a default processing kernel;

[0015] Inputting the second static image into the image recognition model in sequence, the image recognition model performing a convolution operation on the second static image based on the default processing kernel to extract image features therein, and inputting the image features into a fully connected layer for image classification to detect the reach stacker in the second static image;

[0016] After the front gantry is detected in the second static image of the Nth frame, the target area of ​​the front gantry is located in the second static image, and the first size of the target area is obtained, and the first size is adjusted to the second size. After the image recognition model receives the second static image of the N+1th frame, the preprocessing layer maps the center of gravity of the target area from the Nth frame to the second static image of the N+1th frame, and generates an extended area of ​​the second size with the mapped center of gravity as the center. A target kernel is selected from the convolution kernel to perform a convolution operation on the extended area to continue detecting the front gantry in the second static image of the N+1th frame.

[0017] Furthermore, the preprocessing layer marks the pixels outside the extended area as mask values ​​in the second static image of the N+1th frame, obtains the third size of the second static image, calculates the ratio of the third size to the second size, and if the ratio is greater than a first threshold, selects a convolution kernel with a kernel size greater than the default processing kernel as the target kernel; if it is equal to the first threshold, selects the default processing kernel as the target kernel; if it is less than the first threshold, selects a convolution kernel with a kernel size smaller than the default processing kernel as the target kernel.

[0018] Furthermore, detecting the reach stacker in the second static image of the N+1th frame includes the following steps:

[0019] The image recognition model includes a return layer and a cross-correlation layer. The return layer extracts the first feature map corresponding to the target area in the second static image of the Nth frame output by the default processing core and the second feature map of the second static image of the N+1th frame output by the target core. The cross-correlation layer compares the first feature map and the second feature map, locates the overlapping area corresponding to the first feature map in the second feature map, and determines the overlapping area as the position of the front hanger in the second static image of the N+1th frame.

[0020] Furthermore, after locating the overlapping area, the target area is captured in the second static image of the Nth frame as the first image, and the overlapping area is captured in the second static image of the N+1th frame as the second image. The distribution characteristics of the pixel points in the first image and the second image are calculated, and the distribution characteristics include the variance and average of the pixel values ​​of all pixel points in each direction. The confidence is calculated based on the distribution characteristics. When the confidence is greater than a second threshold, the tracking result is judged to be correct.

[0021] Furthermore, adjusting the first size to the second size includes the following steps:

[0022] A plurality of adjustment ratios are set, each adjustment ratio corresponds to a distance value, a corresponding adjustment ratio is selected based on the distance value and the travel direction of the reach stacker, and the first size is adjusted to the second size based on the adjustment ratio.

[0023] Furthermore, the cameras are divided into a first type and a second type. The cameras of the first type are distributed at the boundary of the yard, and the cameras of the second type are located inside the yard. The cameras of the second type define the cameras around themselves as perception cameras and define themselves as linkage cameras. When the second static image taken by the linkage camera does not include the front loader, the second static image taken by the linkage camera is no longer input into the image recognition model. When the perception camera detects that the front loader is moving toward the linkage camera, the second static image taken by the linkage camera is re-input into the image recognition model.

[0024] Furthermore, comparing the actual moving path with the vehicle planned path includes the following steps:

[0025] The planned vehicle path includes a sequence table of the reach stacker passing through cameras. If the order in which the reach stacker actually passes through the cameras is different from that in the sequence table, it is determined that the actual movement path does not match the planned vehicle path.

[0026] Furthermore, the first stacking attribute and the second stacking attribute include size, weight, and volume of the container.

[0027] The present invention also provides an intelligent positioning and trajectory monitoring system for a container yard, which is used to implement the above-mentioned intelligent positioning and trajectory monitoring method for a container yard. The system includes:

[0028] The recording unit includes cameras placed at multiple preset locations in the yard, each camera recording a scene from a preset angle in real time and generating a scene video;

[0029] A planning unit has a yard map established therein, in which container stacking locations and corresponding first stacking attributes are set in the yard map. The reach stacker obtains the container number based on a camera installed on the reach stacker, obtains the second stacking attribute of the container based on the container number, obtains the first stacking attribute corresponding to the second stacking attribute, determines a target storage location for the container in the container yard, and generates a vehicle planning path based on the start position of the reach stacker and the target location.

[0030] an association module, acquiring the scene video at the starting position, identifying the reach stacker in the scene video, and associating the reach stacker with its own license plate number, the container number, and the planned vehicle path;

[0031] The judgment unit tracks the reach stacker based on the scene video captured by multiple cameras, generates an actual movement path of the reach stacker, and marks the reach stacker as abnormal if the actual movement path does not match the planned vehicle path.

[0032] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0033] The present invention realizes the full-length trajectory tracking of the container from the yard entrance to the target location by combining camera monitoring. The image recognition system can accurately read the container number, thereby obtaining its second stacking attributes, such as size, weight and volume. This information is matched with the container stacking location and its first stacking attribute set in the yard map, so that the target location of the container can be quickly determined. Then, based on the starting position of the front loader and the position of the target location, the vehicle planning path is generated. Through camera monitoring and intelligent recognition technology, the front loader can be operated in real time and the actual moving path can be generated. This full-length trajectory tracking method not only improves the management efficiency of the container yard, but also ensures the accurate and rapid positioning of the container, providing strong support for the management and transshipment of containers in the logistics park.

[0034] Based on container trajectory tracking, the present invention compares the reach stacker's actual movement path with the preset vehicle planned path. If a discrepancy is detected, the reach stacker is immediately marked as abnormal. This anomaly identification mechanism can promptly detect and address potential violations or safety hazards, such as misplaced or illegally moved containers. Simultaneously, the method can provide yard managers with real-time anomaly information, allowing them to take timely intervention measures to ensure yard safety and order. Furthermore, by correlating the image features of suspicious vehicles with information such as the container number and license plate number, it can provide strong evidence support for subsequent investigations and actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flowchart of the steps of the intelligent positioning and trajectory monitoring method for container yards according to the present invention;

[0036] Figure 2 The figure is a schematic structural diagram of the intelligent positioning and trajectory monitoring system for container yards according to the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0038] It is understood that the terms "first," "second," etc., used herein may be used to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script without departing from the scope of this application.

[0039] like Figure 1 As shown, a method for intelligent positioning and trajectory monitoring for a container yard includes:

[0040] S1: Cameras are placed at multiple preset locations in the yard. Each camera records the scene from a preset angle in real time and generates a scene video.

[0041] S2: Create a yard map, and set container stacking locations and corresponding first stacking attributes in the yard map.

[0042] S3: The reach stacker obtains the container number based on the camera installed on the reach stacker, and obtains the second stacking attribute of the container based on the container number.

[0043] The first stacking attribute and the second stacking attribute include the size, weight, and volume of the container.

[0044] Specifically, cameras can be set up inside the yard, at boundary points, and other locations. Each camera monitors the scene at a fixed angle by default. A yard map is then created based on the actual situation of the yard, and the locations where containers are stacked are marked on the yard map. Each location has a first stacking attribute, such as container size, type, and volume. In addition, the second stacking attributes of the containers loaded on the reach loader also include container size, type, and volume. When the reach loader reaches the front of the container to be moved, it reads the container number through the image recognition system, and obtains the second stacking attribute of the container from the database based on the container number; it then associates the container number with its own license plate number.

[0045] S4: Acquire the first stacking attribute corresponding to the second stacking attribute to determine a target container location in the container yard, and generate a vehicle planning path based on the starting position of the reach stacker and the target storage location.

[0046] S5: Obtain the scene video at the starting position, identify the reach stacker in the scene video, and associate the reach stacker with its own license plate number, container number, and vehicle planned path.

[0047] S6: Track the reach stacker based on the scene video captured by multiple cameras and generate the actual movement path of the reach stacker. If the actual movement path does not match the vehicle's planned path, the reach stacker is marked as abnormal.

[0048] A first stacking attribute identical to the second stacking attribute is obtained to determine the location where the container should be placed, i.e., the target location. A vehicle planning path is then generated based on the reach stacker's starting position and the location of the target storage location. The vehicle planning path is the minimum movement path between the two points. A camera with a shooting angle facing the reach stacker captures a real-time image of the reach stacker. Intelligent recognition technology is used to obtain the reach stacker's position and image feature information in the scene video. The reach stacker's image feature information is then associated with the previously acquired container number, license plate number, and vehicle planning path. During subsequent tracking, the container number and license plate number can be annotated in the tracking box of the reach stacker.

[0049] The present invention then uses cameras at multiple positions and angles in the yard to continuously monitor the travel trajectory of the reach loader, that is, the actual moving path. When the actual moving path deviates from the preset vehicle planning path, the reach loader is marked as abnormal, and the reach loader is continuously marked when it appears in subsequent scene videos, thereby facilitating timely manual intervention.

[0050] The present invention realizes the full-length trajectory tracking of the container from the yard entrance to the target location by combining camera monitoring. The image recognition system can accurately read the container number, thereby obtaining its second stacking attributes, such as size, weight and volume. This information is matched with the container stacking location and its first stacking attribute set in the yard map, so that the target location of the container can be quickly determined. Then, based on the starting position of the front loader and the position of the target location, the vehicle planning path is generated. Through camera monitoring and intelligent recognition technology, the front loader can be operated in real time and the actual moving path can be generated. This full-length trajectory tracking method not only improves the management efficiency of the container yard, but also ensures the accurate and rapid positioning of the container, providing strong support for the management and transshipment of containers in the logistics park.

[0051] Based on container trajectory tracking, the present invention compares the reach stacker's actual movement path with the preset vehicle planned path. If a discrepancy is detected, the reach stacker is immediately marked as abnormal. This anomaly identification mechanism can promptly detect and address potential violations or safety hazards, such as misplaced or illegally moved containers. Simultaneously, the method can provide yard managers with real-time anomaly information, allowing them to take timely intervention measures to ensure yard safety and order. Furthermore, by correlating the image features of suspicious vehicles with information such as the container number and license plate number, it can provide strong evidence support for subsequent investigations and actions.

[0052] It is particularly noteworthy that the present invention can comprehensively track container trajectories and identify abnormal trajectories therein.

[0053] In this embodiment, tracking the reach stacker includes the following steps:

[0054] Split the scene video into multiple first static images, perform spatial filtering on each first static image to obtain a second static image, and establish an image recognition model. The image recognition model includes a preprocessing layer, a convolution layer, and a fully connected layer. The convolution layer includes multiple convolution kernels, each of which has a different kernel size. One of the convolution kernels is selected as the default processing kernel.

[0055] The second static image is sequentially input into the image recognition model. The image recognition model performs a convolution operation on the second static image based on a default processing kernel to extract image features therein. The image features are input into a fully connected layer for image classification to detect the reach stacker in the second static image.

[0056] After the front gantry is detected in the second static image of the Nth frame, the target area of ​​the front gantry is located in the second static image, and the first size of the target area is obtained, and the first size is adjusted to the second size. After the image recognition model receives the second static image of the N+1th frame, the preprocessing layer maps the center of gravity of the target area from the Nth frame to the second static image of the N+1th frame, and generates an extended area of ​​the second size with the mapped center of gravity as the center. The target kernel is selected from the convolution kernel to perform a convolution operation on the extended area to continue detecting the front gantry in the second static image of the N+1th frame.

[0057] Among them, the preprocessing layer marks the pixels outside the expanded area as mask values ​​in the second static image of the N+1th frame, obtains the third size of the second static image, calculates the ratio of the third size to the second size, and if the ratio is greater than the first threshold, selects the convolution kernel with a kernel size larger than the default processing kernel as the target kernel; if it is equal to the first threshold, selects the default processing kernel as the target kernel; if it is less than the first threshold, selects the convolution kernel with a kernel size smaller than the default processing kernel as the target kernel.

[0058] After the camera sends the recorded scene video to the server, the server splits the scene video. When splitting, it can be split according to the number of recorded frames of the scene video, or it can be split at intervals of a certain number of frames to balance computing resources and tracking efficiency. After obtaining the first static image, it is first subjected to spatial filtering to reduce image noise and improve the accuracy of feature extraction. Spatial filtering methods include low-pass filtering, high-pass filtering, band-pass filtering, etc. An image recognition model is established in the server. The image recognition model in this embodiment is an improved CNN convolutional neural network model. Unlike traditional convolutional neural networks, the present invention adds a preprocessing layer to the convolutional neural network. The preprocessing layer can select a suitable convolution kernel for feature processing according to the size of the image.

[0059] Specifically, the image recognition model in this embodiment has a convolutional layer and a fully connected layer. The convolutional layer is used to extract deep features of the image, while the fully connected layer is responsible for mapping the features to different categories. The convolutional layer has three layers, each with three convolution kernels, namely 3*3, 5*5, and 7*7. The 5*5 convolution kernel is set as the default processing kernel. After obtaining the second static image, the second static image is input into the image recognition model in chronological order. The image recognition model uses the default processing kernel to extract features from the second static image to obtain image features, and the image features are input into the fully connected layer for classification judgment. When the fully connected layer detects the presence of a front gantry in the second static image, the location area of ​​the front gantry is marked in the corresponding second static image, that is, the target area. The first size of the target area is then obtained. The first size includes the length and width of the target area. Here, it is assumed that the front gantry appears for the first time in the second static image of the Nth frame.

[0060] This embodiment also adaptively increases or decreases the size of the first dimension based on the distance between the reach stacker and the camera. Using increase as an example, the first dimension is gradually increased as the reach stacker approaches the camera. Specifically, after receiving the second static image of frame N+1, the coordinates of the center of gravity of the target area in the second static image of frame N are obtained. Since the camera angle in this embodiment is fixed, the center of gravity can be directly mapped to the second static image of frame N+1. An extended region is then generated based on the mapped center of gravity. The extended region has the second dimension. Subsequently, in the second static image of frame N+1, the pixel values ​​of the pixels in the non-extended region are marked as a mask value, such as 0, so that the image recognition model does not perform feature extraction on the pixels in the masked region. By restricting feature extraction to the extended region, the reach stacker can be more efficiently located, reducing the waste of computing resources. Furthermore, the second dimension is enlarged compared to the first dimension, so that pixels in the area where the reach stacker is located are not mistakenly marked as a mask. In particular, if multiple reach stackers are simultaneously in the second static image, multiple extended regions will be generated.

[0061] In a CNN deep learning network, a larger convolution kernel size results in faster convolution and fewer extracted features, leading to faster image recognition. Conversely, a smaller convolution kernel size results in more accurate capture of image details. The preprocessing layer obtains the third dimension of the second static image in frame (N+1) and calculates the ratio of the third dimension to the second dimension. If the ratio is greater than a first threshold, it indicates that the expanded area occupies a majority of the second static image in frame (N+1). In this case, a convolution kernel with a larger kernel size is selected for convolution, such as a 7*7 kernel in each layer as the target kernel. In other embodiments, if there are multiple convolution kernels per layer, a selection rule can be set to select a larger kernel size for convolution at each layer, rather than necessarily selecting the largest kernel. If the ratio is equal to the first threshold, it indicates that the expanded area has changed little compared to the original size, and the kernel size does not need to be adjusted. If the ratio is less than the first threshold, it indicates that the expanded area is small, and a smaller kernel size is required to improve the recognition accuracy of the reach stacker.

[0062] In this embodiment, detecting the reach stacker in the second static image of the N+1th frame includes the following steps:

[0063] The image recognition model includes a return layer and a cross-correlation layer. The return layer extracts the first feature map corresponding to the target area in the second static image of the Nth frame output by the default processing core and the second feature map of the second static image of the N+1th frame output by the target core. The cross-correlation layer compares the first feature map and the second feature map, locates the overlapping area corresponding to the first feature map in the second feature map, and determines the overlapping area as the position of the front hanger in the second static image of the N+1th frame.

[0064] After the traditional method, the target kernel extracts the features of the second static image of the N+1th frame and inputs it into the fully connected layer for analysis to determine the specific position of the front gantry. In order to improve the recognition efficiency, the present invention sets a return layer and a cross-correlation layer in the image recognition model, wherein the return layer is used to extract the feature map output by the convolution kernel of the last convolution layer in the convolutional neural network. Specifically, the relevant library of TensorFlow can be called to visualize the feature map, or a visualize statement can be added to the code to obtain the feature map. Here, the first feature map of the target area in the second static image of the Nth frame and the second feature map of the entire third static image are obtained. Then, the first feature map and the second feature map are compared through the cross-correlation layer, so as to locate the same part as the first feature map in the second feature map. The cross-correlation layer can be constructed based on the Siamese twin model, for example. Due to the efficient positioning of the cross-correlation layer, the position of the front gantry can be quickly determined.

[0065] In this embodiment, after locating the overlapping area, the target area is captured in the second static image of the Nth frame as the first image, and the overlapping area is captured in the second static image of the N+1th frame as the second image. The distribution characteristics of the pixel points in the first image and the second image are calculated. The distribution characteristics include the variance and average of the pixel values ​​of all pixels in each direction. The confidence is calculated based on the distribution characteristics. When the confidence is greater than the second threshold, the tracking result is judged to be correct.

[0066] During the target tracking process, the present invention also uses this step to verify the tracking results, so that the recognition results of the image recognition model can be continuously evaluated during the use of the present invention. When deviations are found in multiple recognition results, the system reminds relevant personnel to optimize the image recognition model.

[0067] Specifically, after obtaining the overlapping area, the target area is intercepted in the second static image of the Nth frame as the first image, and the overlapping area is intercepted in the second static image of the N+1th frame as the second image, and then the confidence between the first image and the second image is calculated. During the specific calculation, the first image and the second image are first adjusted to the same size, and then the average value and variance of the pixel values ​​of each row of pixels and each column of pixels in the first image and the second image are calculated. In other embodiments, more directions can also be added, such as diagonal directions. When there are first-row pixels in the first image and second-row pixels in the second image, the difference between the variances of the first-row pixels and the second-row pixels, and the difference between the average values ​​are all within the preset range, then it is judged that the first-row pixels and the second-row pixels have the same distribution characteristics, and the comparison process of the pixels in the column direction is the same; finally, the confidence is calculated according to the following formula, and the specific formula is: Where P is the confidence, D is the number of rows and columns with the same distribution features in the first image and the second image, L is the number of rows in the first image, and W is the number of columns in the first image.

[0068] In this embodiment, adjusting the first size to the second size includes the following steps:

[0069] Multiple adjustment ratios are set, each adjustment ratio corresponds to a distance value, a corresponding adjustment ratio is selected based on the distance value and the travel direction of the reach stacker, and the first size is adjusted to the second size based on the adjustment ratio.

[0070] For example, the adjustment ratios include 0.8, 0.9, 1, 1.1, and 1.2, corresponding to distance values ​​of 10, 20, 30, 40, and 50 meters, respectively. If the reach loader is detected as being 10 meters from the camera and moving toward it in the second static image of the Nth frame, the adjustment ratio is set to 0.9. If it is moving away from the camera, the adjustment ratio is set to 0.8. The vehicle's heading can be obtained through linkage with the vehicle's Beidou positioning system or based on detections from surrounding cameras.

[0071] In this embodiment, cameras are divided into a first type and a second type. The first type of cameras are distributed at the boundary of the yard, and the second type of cameras are located inside the yard. The second type of camera defines the cameras around it as perception cameras and defines itself as a linkage camera. When the second static image taken by the linkage camera does not include a front loader, the second static image taken by the linkage camera is no longer input into the image recognition model. When the perception camera detects that there is a front loader moving toward the linkage camera, the second static image taken by the linkage camera is re-input into the image recognition model.

[0072] Since the first type of cameras are distributed at the border, it is necessary to monitor each frame of the video in real time to detect abnormal situations in a timely manner. Since the second type of cameras are distributed inside the yard, there must be cameras of the first or second type around them. Therefore, they can be defined as linkage cameras, and the surrounding cameras are defined as perception cameras. The perception cameras include cameras of the first and second types. When the linkage camera detects that there are no more front loaders in the scene video, it stops inputting the second static image it takes into the image recognition model to reduce the workload of the server. In other embodiments, personnel detection can also be added. When there are no people or vehicles, the input into the image recognition model is stopped. When the surrounding perception cameras detect that there is a front loader moving towards the linkage camera, the scene video taken by the linkage camera is split and input into the image recognition model to avoid missing the detection of the front loader.

[0073] In this embodiment, comparing the actual moving path with the vehicle planned path includes the following steps:

[0074] The vehicle's planned path includes a table of the order in which the reach stacker passes the cameras. If the order in which the reach stacker actually passes the cameras is different from that in the table, it is determined that the actual movement path does not match the vehicle's planned path.

[0075] After the cameras are arranged, each camera is numbered 1, 2, 3, etc. The generated vehicle planning path includes a list of the order of cameras to be passed. For example, from the train unloading point to the target container location, it is necessary to pass through cameras numbered 1, 3, and 6 in sequence. If it is found that the order of cameras passed by the reach loader is 1, 2, and 3, it is judged that its driving path deviates from the predetermined planned path.

[0076] like Figure 2 As shown, the present invention also provides an intelligent positioning and trajectory monitoring system for a container yard, which is used to implement the above-mentioned intelligent positioning and trajectory monitoring method for a container yard. The system includes:

[0077] The recording unit includes cameras placed at multiple preset locations in the yard, each camera recording a scene from a preset angle in real time and generating a scene video;

[0078] The planning unit has a yard map built in it, and sets the container stacking locations and corresponding first stacking attributes in the yard map. The reach stacker obtains the container number based on its own camera, obtains the container's second stacking attribute based on the container number, and obtains the first stacking attribute corresponding to the second stacking attribute to determine the target storage location of the container in the container yard. The vehicle planning path is generated based on the reach stacker's starting position and the target location.

[0079] The association module obtains the scene video at the starting position, identifies the reach stacker in the scene video, and associates the reach stacker with its own license plate number, container number, and vehicle planned path;

[0080] The judgment unit tracks the reach stacker based on scene videos captured by multiple cameras and generates an actual movement path of the reach stacker. If the actual movement path does not match the vehicle's planned path, the reach stacker is marked as abnormal.

[0081] It should be understood that the various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent positioning and trajectory monitoring method for a container yard, characterized in that: include: Cameras are placed at multiple preset locations in the yard. Each camera records the scene from a preset angle in real time and generates a scene video. Establishing a yard map, and setting container stacking locations and corresponding first stacking attributes in the yard map; The reach stacker obtains the container number of the container based on a camera installed on the reach stacker, and obtains the second stacking attribute of the container based on the container number; Obtaining the first stacking attribute corresponding to the second stacking attribute to determine a target container location in a container yard, and generating a vehicle planning path based on a starting position of the reach stacker and the target container location; Acquire the scene video at the starting position, identify the reach stacker in the scene video, and associate the reach stacker with its own license plate number, the container number, and the planned vehicle path; Tracking the reach stacker based on the scene video captured by multiple cameras to generate an actual movement path of the reach stacker, and marking the reach stacker as abnormal if the actual movement path does not match the planned vehicle path; Tracking the reach stacker includes the following steps: Splitting the scene video into multiple first static images, performing spatial filtering processing on each of the first static images to obtain a second static image, and establishing an image recognition model, wherein the image recognition model includes a preprocessing layer, a convolution layer, and a fully connected layer, the convolution layer includes multiple convolution kernels, each of the convolution kernels has a different kernel size, and one of the convolution kernels is selected as a default processing kernel; Inputting the second static image into the image recognition model in sequence, the image recognition model performing a convolution operation on the second static image based on the default processing kernel to extract image features therein, and inputting the image features into a fully connected layer for image classification to detect the reach stacker in the second static image; After the front gantry is detected in the second static image of the Nth frame, the target area of ​​the front gantry is located in the second static image, and the first size of the target area is obtained, and the first size is adjusted to the second size. After the image recognition model receives the second static image of the N+1th frame, the preprocessing layer maps the center of gravity of the target area from the Nth frame to the second static image of the N+1th frame, and generates an extended area of ​​the second size with the mapped center of gravity as the center. A target kernel is selected from the convolution kernel to perform a convolution operation on the extended area to continue detecting the front gantry in the second static image of the N+1th frame.

2. The method according to claim 1, characterized in that The preprocessing layer marks the pixels outside the extended area as mask values ​​in the second static image of the N+1th frame, obtains the third size of the second static image, calculates the ratio of the third size to the second size, and if the ratio is greater than a first threshold, selects a convolution kernel with a kernel size greater than the default processing kernel as the target kernel; if it is equal to the first threshold, selects the default processing kernel as the target kernel; if it is less than the first threshold, selects a convolution kernel with a kernel size smaller than the default processing kernel as the target kernel.

3. The method according to claim 1 or 2, characterized in that Detecting the reach stacker in the second static image of the N+1th frame includes the following steps: The image recognition model includes a return layer and a cross-correlation layer. The return layer extracts the first feature map corresponding to the target area in the second static image of the Nth frame output by the default processing core and the second feature map of the second static image of the N+1th frame output by the target core. The cross-correlation layer compares the first feature map and the second feature map, locates the overlapping area corresponding to the first feature map in the second feature map, and determines the overlapping area as the position of the front hanger in the second static image of the N+1th frame.

4. The method according to claim 3, characterized in that After locating the overlapping area, the target area is captured in the second static image of the Nth frame as the first image, and the overlapping area is captured in the second static image of the N+1th frame as the second image. The distribution characteristics of the pixel points in the first image and the second image are calculated, and the distribution characteristics include the variance and average of the pixel values ​​of all pixels in each direction. The confidence is calculated based on the distribution characteristics. When the confidence is greater than a second threshold, the tracking result is judged to be correct.

5. The method according to claim 3, characterized in that Adjusting the first size to the second size comprises the following steps: A plurality of adjustment ratios are set, each adjustment ratio corresponds to a distance value, a corresponding adjustment ratio is selected based on the distance value and the travel direction of the reach stacker, and the first size is adjusted to the second size based on the adjustment ratio.

6. The method according to claim 1, characterized in that Cameras are divided into a first type and a second type. The cameras of the first type are distributed at the boundary of the yard, and the cameras of the second type are located inside the yard. The cameras of the second type define the cameras around them as perception cameras and define themselves as linkage cameras. When the second static image taken by the linkage camera does not include the reach loader, the second static image taken by the linkage camera is no longer input into the image recognition model. When the perception camera detects that the reach loader is moving toward the linkage camera, the second static image taken by the linkage camera is re-input into the image recognition model.

7. The method according to claim 1, characterized in that Comparing the actual moving path with the vehicle planned path includes the following steps: The planned vehicle path includes a sequence table of the reach stacker passing through cameras. If the order in which the reach stacker actually passes through the cameras is different from that in the sequence table, it is determined that the actual movement path does not match the planned vehicle path.

8. The method according to claim 3, characterized in that The first stacking attribute and the second stacking attribute include size, weight, and volume of the container.

9. An intelligent positioning and trajectory monitoring system for a container yard, for implementing the method according to any one of claims 1 to 8, characterized in that: include: The recording unit includes cameras placed at multiple preset locations in the yard, each camera recording a scene from a preset angle in real time and generating a scene video; The planning unit has a yard map established therein, and sets container stacking locations and corresponding first stacking attributes in the yard map. The reach stacker obtains the container number based on a camera installed on the reach stacker, obtains the second stacking attribute of the container based on the container number, obtains the first stacking attribute corresponding to the second stacking attribute, determines the target location of the container in the container yard, and generates a vehicle planning path based on the starting position of the reach stacker and the target location. an association module, acquiring the scene video at the starting position, identifying the reach stacker in the scene video, and associating the reach stacker with its own license plate number, the container number, and the planned vehicle path; The judgment unit tracks the reach stacker based on the scene video captured by multiple cameras, generates an actual movement path of the reach stacker, and marks the reach stacker as abnormal if the actual movement path does not match the planned vehicle path.

Images