Harbor transportation equipment control method and system in mixed traffic scene

Abstract

This invention relates to the field of automated terminal technology, specifically providing a method and system for controlling terminal transportation equipment in mixed traffic scenarios. The method includes: acquiring location data of transportation equipment, which may be manned or unmanned; filtering one or more target location data points within a pre-stored intersection location range from the location data of multiple transportation equipment; generating scheduling instructions for manned transportation equipment based on the target location data, assuming normal operation of unmanned transportation equipment; and sending the scheduling instructions to corresponding displays within the intersection location range. This invention fully leverages the flexibility of manned transportation equipment, eliminating the need for machine learning to find optimal solutions and significantly reducing computational load.

Description

Technical Field

[0001] This invention belongs to the field of automated terminals, specifically relating to a method and system for controlling terminal transportation equipment in mixed traffic scenarios. Background Technology

[0002] In the process of automating manned terminals, scenarios where manned and unmanned transport equipment operate together are unavoidable. However, in such mixed environments, their coordinated operation faces significant challenges. Unmanned transport equipment relies on automation technology, while manned transport equipment depends on the real-time judgment and operation of the driver. In terminal operations, a high degree of coordination between the two is essential; otherwise, it will not only cause chaos in horizontal transport traffic at the terminal and increase the probability of safety accidents, but also disrupt operational flow, reduce overall operational efficiency, and negatively impact the safety and efficiency of terminal operations.

[0003] At intersections where different transport routes meet, manned and unmanned transport vehicles often encounter driving conflicts. While unmanned transport vehicles rely on automated perception systems such as lidar and visual sensors, blind spots created by stacked containers in the port environment, interference from metal structures with radar signals, and the low reflectivity of vehicles on the side significantly increase the difficulty of obstacle recognition. This makes intersections prone to safety accidents.

[0004] Currently, some collaborative control methods utilize machine learning to solve for the highest transportation efficiency, thereby generating optimal scheduling schemes for manned and unmanned transportation equipment. However, this approach has significant drawbacks. It ignores the fundamental differences between manned and unmanned transportation equipment, employs a unified scheduling method for path planning, resulting in massive computational demands and excessively high hardware costs. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method and system for controlling port transportation equipment in mixed traffic scenarios to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a method for controlling terminal transportation equipment in a mixed traffic scenario, comprising: Acquire location data of transportation equipment, which may be manned or unmanned transportation equipment; Filter out one or more target location data within a pre-stored intersection location range from the location data of multiple transportation devices; Based on the principle of ensuring the normal operation of unmanned transportation equipment, dispatch instructions for manned transportation equipment are generated according to the target location data. The dispatch command is sent to the corresponding display within the intersection location range.

[0007] In one optional implementation, acquiring the location data of the transportation equipment includes: Obtain the location reported by the transportation equipment, which is generated by the positioning chip of the transportation equipment; Acquire video streams from multiple cameras, extract images containing the transportation equipment from the video streams, and obtain the capture time of the images; The target image with the latest shooting time is selected, and the positioning position of the transportation equipment is corrected according to the latest shooting time and the fixed position of the camera to which the target image belongs.

[0008] In an optional implementation, acquiring video streams from multiple cameras, extracting images containing the transport equipment from the video streams, and obtaining the capture time of the images, includes: The video stream is converted into a sequence of image frames, and each image in the sequence is labeled with its corresponding capture time. The image is preprocessed, including grayscale conversion, noise reduction, and resizing. Using the object detection algorithm in HyperLPR3, the license plate location is identified from the preprocessed image, and the bounding box of the license plate is generated based on the license plate location; The driving lane of the corresponding transportation equipment is determined based on the license plate location. Extract the local image corresponding to the bounding box from the overall image; The local image is segmented into individual characters, and then each character is recognized by a character recognition model. The identified characters are combined in sequence to form a complete license plate number; Save the driving lane and shooting time corresponding to the same license plate number in the same image as key-value pairs, and generate license plate number tags for the key-value pairs.

[0009] In one optional implementation, the target image with the latest capture time is selected, and the positioning position of the transport device is corrected based on the latest capture time and the fixed position of the camera to which the target image belongs, including: Save the key-value pairs corresponding to the same license plate number to the same array, and filter out the latest shooting time from the array; Obtain the video stream to which the latest key-value pair belongs, wherein the latest key-value pair is a key-value pair that includes the latest shooting time; The fixed location of the camera to which the video stream belongs is determined based on the source IP of the video stream to which the latest key-value pair belongs; Calculate the speed of the transport vehicle to which the license plate number of the latest key-value pair belongs, based on the video stream to which the latest key-value pair belongs; Obtain the location and timestamp of the transportation equipment, and calculate the time difference between the timestamp and the latest shooting time. The offset distance is obtained by multiplying the time difference by the driving speed. The simulated positioning location is determined by using the fixed position and offset distance of the corresponding camera; The location of the transportation equipment is corrected based on the simulated location.

[0010] In one optional implementation, filtering one or more target location data points within a pre-stored intersection location range from location data of multiple transportation devices includes: Multiple intersection location ranges are pre-stored. The intersection location range is the coordinate range of the intersection monitoring area. The intersection monitoring area is bounded by the farthest shooting distance position of the cameras facing multiple directions at the intersection. The location data of multiple transportation devices are matched with the location range of each intersection to obtain one or more target location data corresponding to the location range of each intersection.

[0011] In one optional implementation, based on the principle of normal operation of unmanned transportation equipment, dispatch instructions for manned transportation equipment are generated according to target location data, including: The target location data within the same intersection location range are divided into a first type of location data and a second type of location data. The first type of location data is the location data of unmanned transportation equipment, and the second type of location data is the location data of manned transportation equipment. The occupied lanes and the time of occupation within the location range of the intersection are determined based on the first type of data; The second type of location data corresponding to the occupied lane is determined, and a dispatch instruction is generated for the manned transportation equipment corresponding to the second type of location data. The dispatch instruction includes the license plate number of the manned transportation equipment, a no-passage status reminder, and the duration of the no-passage. The duration of the no-passage is positively correlated with the time the lane is occupied.

[0012] In an optional implementation, determining the occupied lanes and their occupation time within the intersection location range based on the first type of data includes: The driving lane, driving speed, and driving direction of the unmanned transport equipment are determined based on the first type of location data generated at multiple times. Mark the lane where the unmanned transport equipment is located as an occupied lane; Based on the latest location, speed, and direction of the unmanned transport equipment, calculate the required time for the unmanned transport equipment to leave the intersection location range, and set the required time as the occupied time.

[0013] In an optional implementation, determining a second type of location data corresponding to the occupied lane, and generating a scheduling instruction for the manned transportation equipment corresponding to the second type of location data, including: Based on the second type of location data generated at multiple times, predict the target lane for the corresponding manned transport equipment after it enters the intersection location range. The target lane is the lane where the manned transport equipment passes the center of the intersection and leaves the intersection location range. If the target lane is not an occupied lane, then the manned transport equipment is allowed to pass normally by default. If the target lane is occupied, then determine whether there is a candidate lane parallel to the occupied lane according to the road traffic rules: If not, a dispatch instruction is generated prohibiting the passage of the manned transport equipment; If so, the status of the candidate lane is obtained. If the candidate lane is not occupied, the manned transport equipment is allowed to pass normally by default. If the candidate lane is occupied, a dispatch instruction is generated to prohibit the manned transport equipment from passing.

[0014] In an optional implementation, the method further includes: Confirm that the display time of the dispatch instruction for manned transport equipment has reached the occupied time of the occupied lane; Remove the scheduling command from the display's data list.

[0015] Secondly, the present invention provides a control system for terminal transportation equipment in a mixed traffic scenario, comprising: Cameras are used to collect video data of the monitored area. The cameras are deployed at multiple monitoring nodes and intersections along the road. A display screen is used to show dispatch instructions, which include license plate number, a no-passage status reminder, and the duration of the no-passage restriction; The control terminal is communicatively connected to the camera and the display, and is also communicatively connected to manned and unmanned transportation equipment in a mixed traffic scenario. The control terminal executes a control program, which includes: The data acquisition module is used to acquire the location data of the transportation equipment, which may be manned or unmanned. The data matching module is used to filter out one or more target location data within a pre-stored intersection location range from the location data of multiple transportation devices; The instruction generation module is used to generate dispatch instructions for manned transportation equipment based on target location data, with the principle of normal operation of unmanned transportation equipment as the guideline. The display output module is used to send the dispatching instructions to the corresponding displays within the intersection location range.

[0016] The beneficial effects of this invention are that the control method and system for dock transportation equipment in mixed traffic scenarios provided by this invention, through global monitoring of the positions of manned and unmanned transportation equipment, and then filtering out manned and unmanned transportation equipment near intersections through data matching, with the principle of ensuring the normal operation of unmanned transportation equipment, only needs to issue dispatch instructions to the display at the corresponding intersection for manned transportation equipment with driving conflicts, so as to prompt the manned transportation equipment to avoid the unmanned transportation equipment. This control method fully utilizes the flexibility of manned transportation equipment, does not require the use of machine learning to find the optimal solution, and greatly reduces the amount of computation.

[0017] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart of the intersection scheduling method according to an embodiment of the present invention.

[0021] Figure 3 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of a display screen at the first moment of a first application scenario of a method according to an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of a second moment display screen in a first application scenario of a method according to an embodiment of the present invention.

[0024] Figure 6 This is a schematic diagram of a display screen illustrating a second application scenario of a method according to an embodiment of the present invention.

[0025] Figure 7 This is a schematic diagram of the first moment of a third application scenario of the method of one embodiment of the present invention.

[0026] Figure 8This is a schematic diagram of the second moment of a third application scenario of the method of one embodiment of the present invention.

[0027] Figure 9 This is a schematic diagram of a third moment of a third application scenario of a method according to an embodiment of the present invention.

[0028] Figure 10 This is a schematic diagram of the fourth moment of a third application scenario of a method according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0031] The key terms used in this invention will be explained below.

[0032] HyperLPR3: A high-performance license plate recognition framework. HyperLPR3's core technology is based on deep learning. In the license plate detection stage, a convolutional neural network-based object detection algorithm is used to quickly and accurately locate the license plate region in the input image. Next, a series of preprocessing and adjustments are performed on the license plate region, including image enhancement, resizing, and grayscale conversion, making it more suitable for subsequent character recognition operations. In the character recognition stage, deep learning technology is used, with a convolutional neural network-based character classifier recognizing characters. Finally, the recognized characters are combined according to the license plate sequence to generate a complete license plate number.

[0033] In HyperLPR3, object detection algorithms refer to algorithms used to quickly and accurately locate and identify specific targets (in this case, license plates) in images or videos.

[0034] Common algorithm types and their application in HyperLPR3: Algorithms based on Convolutional Neural Networks (CNNs), such as the YOLO (You Only Look Once) series and Faster R-CNN, are utilized in Hyper-LPR3. These algorithms leverage their architectural features, employing multiple convolutional and pooling layers to automatically extract license plate features. Taking YOLO as an example, it divides an image into multiple grids, each responsible for predicting whether a license plate is present, as well as the plate's location and category. This enables rapid license plate detection in images, achieving end-to-end object detection.

[0035] Other deep learning-based improved algorithms: HyperLPR3 improves and optimizes some classic algorithms to meet specific needs of license plate detection. For example, it incorporates attention mechanisms to make the algorithm focus more on regions related to the license plate in the image and ignore irrelevant information, thereby improving the accuracy and efficiency of detection. Alternatively, it uses multi-scale feature fusion methods to comprehensively utilize image features at different scales to better detect license plates of different sizes.

[0036] The dock transportation equipment control method in the mixed traffic scenario provided in this embodiment of the invention is executed by a computer device, and correspondingly, the dock transportation equipment control system in the mixed traffic scenario runs in the computer device.

[0037] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The executing entity can be a port transportation equipment control system in a mixed traffic scenario. Depending on different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted.

[0038] like Figure 1 As shown, the method includes: S1. Obtain the location data of the transportation equipment, which may be a manned or unmanned transportation equipment.

[0039] Location data is acquired by installing different types of positioning modules on manned and unmanned transportation equipment. Unmanned transportation equipment can be equipped with a high-precision lidar positioning system, which emits laser beams and receives reflected light to create a real-time map of the surrounding environment, thereby accurately calculating its own position with centimeter-level precision. Manned transportation equipment can utilize a combination of GPS (Global Positioning System) and inertial navigation system. GPS provides approximate geographical location information, while the inertial navigation system, when GPS signals are weak (such as indoors or in obstructed areas), measures the equipment's acceleration and angular velocity using accelerometers and gyroscopes to calculate changes in position, ensuring the continuity and accuracy of location data. These positioning modules transmit the location data to the central control system at a set frequency (e.g., 10 times per second) in a standard coordinate format (such as the WGS84 coordinate system).

[0040] S2. Filter out one or more target location data within a pre-stored intersection location range from the location data of multiple transportation devices.

[0041] The system pre-stores the location ranges of various intersections within the dock, defined as polygonal regions, each composed of a series of coordinate points. Upon receiving the location data of the transport equipment, the system employs spatial query algorithms, such as the ray casting or corner casting methods. The ray casting method involves emitting a ray in a specific direction from the location of the transport equipment to be determined, counting the number of intersections between the ray and the boundary of the intersection polygon. If the number of intersections is odd, the point is within the polygon, meaning the transport equipment is within the intersection's location range. The corner casting method calculates the sum of the angles between the vector from the first vertex of the polygon to each subsequent vertex and the vectors from the point to be determined to these vertices. If the sum of these angles is 360 degrees, the point is within the polygon. This method quickly filters out the location data of transport equipment within the intersection's location range, using this data as the target location data.

[0042] S3. Based on the principle of normal operation of unmanned transportation equipment, generate dispatch instructions for manned transportation equipment according to target location data.

[0043] First, the system analyzes the movement trajectories and speeds of both unmanned and manned transport vehicles within the target location data. Based on collision avoidance algorithms, if a collision between the manned and unmanned transport vehicles is predicted at a future moment, the system prioritizes the normal operation and right-of-way of the unmanned transport vehicles. For example, if an unmanned transport vehicle is moving at a constant speed along a predetermined route towards an intersection, while the speed and direction of the manned transport vehicle might cause a conflict, the system will calculate a stop instruction for the manned transport vehicle at the intersection. The stop instruction will specify the stopping location and waiting time.

[0044] S4. Send the dispatch instruction to the corresponding display within the intersection location range.

[0045] The generated dispatch instructions are sent to displays located within the intersection area via wireless communication networks, such as Wi-Fi or 4G / 5G networks. These displays are installed in prominent locations, such as traffic signs at the intersection or on nearby control towers. Reliable communication protocols, such as TCP (Transmission Control Protocol), are used when sending instructions to ensure the complete and error-free transmission of the instruction data. Upon receiving the instructions, the displays show textual or graphical dispatch information, such as arrows indicating the direction manned transport equipment should travel, or numerical displays showing the required waiting time, so that drivers can clearly and accurately understand and execute the dispatch instructions.

[0046] In one embodiment of the present invention, based on step S1, the following will provide a non-limiting description of the specific implementation of an embodiment.

[0047] S101. Obtain the location reported by the transportation equipment, wherein the location is generated by the positioning chip of the transportation equipment.

[0048] Transportation equipment can be equipped with various types of positioning chips, such as GPS (Global Positioning System) chips and BeiDou positioning chips. These chips calculate their own location information by receiving satellite signals. For example, a GPS chip receives signals from at least four GPS satellites and uses the principle of triangulation to calculate the latitude and longitude coordinates of the transportation equipment on Earth.

[0049] The positioning chip sends the calculated location information to the data receiver at preset time intervals (e.g., every second). The data is encapsulated in a specific protocol format, such as the NMEA (National Marine Electronics Association) protocol, which specifies the data transmission format and content, including latitude, longitude, speed, and time. The transportation equipment then sends the encapsulated data to a designated server or monitoring center via a wireless communication module (such as GPRS, 4G, or 5G).

[0050] After receiving the location data, the receiving end verifies the data. Verification includes checking data integrity (whether the data was lost or corrupted during transmission) and validity (whether the latitude and longitude values ​​are within a reasonable range). For example, the range of latitude and longitude values ​​is -180° to 180° (longitude) and -90° to 90° (latitude). If the received data exceeds this range, it is considered invalid.

[0051] S102. Acquire video streams captured by multiple cameras, extract images containing the transportation equipment from the video streams, and obtain the shooting time of the images.

[0052] S102.1 converts the video stream into a sequence of image frames and marks the corresponding capture time for each image in the sequence of image frames.

[0053] Multiple cameras located in different positions transmit captured video streams to a server via a network (such as Ethernet, Wi-Fi, etc.). The server uses video stream receiving software (such as FFmpeg) to receive and process these video streams. FFmpeg is a powerful open-source multimedia processing tool that supports various video formats and encoding standards, such as H.264 and MPEG-4.

[0054] The frame rate of the video stream is set according to actual needs, such as the common frame rates of 25fps (frames per second) or 30fps. A higher frame rate results in smoother video, but also a larger amount of data is processed. The server extracts image frames from the video stream according to the set frame rate, converting the video stream into a sequence of image frames. When extracting each frame, the capture time is recorded. The capture time can be obtained from the camera's system clock, accurate to the millisecond level. The capture time is associated with the corresponding image frame and stored in a database or file for easy later querying and use.

[0055] S102.2 Preprocess the image, the preprocessing including grayscale conversion, noise reduction, and size adjustment.

[0056] Grayscale conversion: Converting a color image to a grayscale image reduces the amount of data while preserving the image's main features. A common grayscale conversion method is the weighted average method, which assigns different weights (e.g., R: 0.299, G: 0.587, B: 0.114) to the red, green, and blue channels based on their influence on human vision, and then calculates the grayscale value of each pixel. The formula is: Gray = 0.299 * R + 0.587 * G + 0.114 * B.

[0057] Noise Reduction: Images may be affected by noise during acquisition and transmission, impacting subsequent target detection and recognition. Common noise reduction methods include Gaussian filtering and median filtering. Gaussian filtering smooths the image and reduces noise by weighted averaging of each pixel and its neighborhood. Median filtering replaces the value of each pixel with the median value of its neighborhood pixels, effectively removing salt-and-pepper noise.

[0058] Resizing: To ensure consistency and efficiency in subsequent processing, images need to be resized to a uniform size. A suitable size can be selected based on actual needs, such as 640x480 pixels. Image scaling algorithms (such as bilinear interpolation) are used to scale the image, resizing it to the specified size while maintaining its aspect ratio.

[0059] S102.3 uses the object detection algorithm in HyperLPR3 to identify the license plate position from the preprocessed image and generate the license plate bounding box based on the license plate position.

[0060] The object detection algorithm in Hyper-LPR3 is based on deep learning technology, typically employing a convolutional neural network (CNN) architecture. This algorithm is trained using a large amount of license plate image data to learn the features and patterns of license plates. In practical applications, the preprocessed image is input into the trained object detection model, which outputs the location information of the license plate in the image, usually represented by a bounding box determined by the coordinates of its top-left and bottom-right corners.

[0061] Deploy a pre-trained HyperLPR3 object detection model on the server and use a deep learning framework (such as TensorFlow or PyTorch) for model inference. Input the pre-processed image into the model, which performs convolution and pooling operations to extract image features and uses these features to determine the presence and location of a license plate. During inference, the model outputs a confidence score, representing the credibility of the detected license plate. Only when the confidence score exceeds a certain threshold (e.g., 0.8) is a valid license plate considered detected.

[0062] Based on the license plate location information output by the model, a bounding box for the license plate is generated. The coordinates of the bounding box can be obtained through simple calculations, such as converting the center point coordinates and width and height information output by the model into the coordinates of the top left and bottom right corners. The generated bounding box is then drawn on the original image for easy viewing and processing later.

[0063] S102.4 Determine the driving lane of the corresponding transportation equipment based on the license plate location.

[0064] When installing the camera, lane lines in the shooting area are detected and marked beforehand. Image processing algorithms (such as the Hough transform) can be used to detect lane lines. The Hough transform is a classic algorithm for detecting straight lines in an image. It determines the position of the lane line by transforming the image space to a parameter space and finding the parameters of the line in the parameter space.

[0065] The lane in which the transport equipment is located is determined based on the location information of the license plate and the lane lines. This can be achieved by comparing the coordinates of the center point of the license plate with the position of the lane lines. For example, if the center point of the license plate is located between two lane lines and is closer to one of the lane lines, the transport equipment is considered to be in that lane.

[0066] S102.5 Extract the local image corresponding to the bounding box from the overall image.

[0067] Based on the coordinates of the generated license plate bounding box, a local image containing the license plate is cropped from the preprocessed overall image. Using a cropping function provided by an image processing library (such as OpenCV), the coordinates of the top-left and bottom-right corners of the cropping region can be specified to extract the local image of the license plate from the overall image.

[0068] Further processing of the cropped portion of the image, such as adjusting brightness and contrast, can improve the clarity of the license plate characters. Methods like histogram equalization can be used to enhance image contrast, making the license plate characters more clearly legible.

[0069] S102.6 performs character segmentation on the local image, dividing it into individual characters, and then uses a character recognition model to recognize each character.

[0070] License plate character segmentation is the process of separating the characters in a license plate image into individual characters. One method is projection-based segmentation, which calculates the horizontal and vertical projections of the license plate image and determines the character boundaries based on the peaks and troughs of the projection curves. Alternatively, deep learning methods, such as character segmentation models based on convolutional neural networks, can be used to learn character segmentation rules through training.

[0071] A pre-trained character recognition model is used to identify individual characters after segmentation. This model can be based on a convolutional neural network (CNN) or a recurrent neural network (RNN). The segmented character image is input into the character recognition model, which outputs the recognition result for each character. To improve recognition accuracy, a multi-model fusion approach can be used, combining the results of multiple different character recognition models for a comprehensive judgment.

[0072] S102.7 Combines the identified characters in sequence to form a complete license plate number.

[0073] Based on the order of character segmentation, the identified individual characters are combined from left to right to form a complete license plate number. During this combination process, the format and rules of the license plate need to be considered, such as the length of the license plate number and the character type (numbers, letters, etc.).

[0074] The combined license plate number is verified to check if it conforms to the license plate coding rules. The legality of the license plate number is verified by querying the license plate coding database. If the recognition result does not conform to the rules, the recognition is considered a failure, requiring re-recognition or manual intervention.

[0075] S102.8 Save the driving lane and shooting time corresponding to the same license plate number in the same image as a key-value pair, and generate a license plate number tag for the key-value pair.

[0076] The driving lane and shooting time corresponding to the same license plate number in the same image are stored in the database as key-value pairs, with the license plate number as the key and the driving lane and shooting time as the values. This data can be stored using a relational database (such as MySQL) or a non-relational database (such as MongoDB).

[0077] A unique license plate number tag is generated for each key-value pair to facilitate subsequent data querying and management. The tag can be generated using a hash algorithm, taking the license plate number, driving lane, and shooting time as input to generate a fixed-length hash value as the tag.

[0078] S103. Select the target image with the latest shooting time, and correct the positioning position of the transportation equipment according to the latest shooting time and the fixed position of the camera to which the target image belongs.

[0079] S103.1 Save the key-value pairs corresponding to the same license plate number to the same array, and filter out the latest shooting time from the array.

[0080] Retrieve all key-value pairs corresponding to the same license plate number from the database and save these key-value pairs into an array. Each element in the array contains information such as the license plate number, the lane in which the vehicle traveled, and the time the photo was taken.

[0081] Iterate through the array, compare the capture times for each key-value pair, and find the latest capture time. You can use timestamps for comparison; a timestamp is a number representing a specific point in time, with larger timestamps indicating later times.

[0082] S103.2 Obtain the video stream to which the latest key-value pair belongs, wherein the latest key-value pair is a key-value pair that includes the latest shooting time.

[0083] Based on the latest shooting time and license plate number, the corresponding key-value pair is retrieved from the database, and the video stream information to which that key-value pair belongs is obtained. The video stream information includes the source IP address of the video stream, the video file path, etc.

[0084] Based on the source IP address or file path of the video stream, retrieve the video stream belonging to the latest key-value pair from the corresponding camera or storage device. Use video stream receiving software (such as FFmpeg) to acquire and process the video stream.

[0085] S103.3 Determine the fixed location of the camera to which the video stream belongs based on the source IP of the video stream to which the latest key-value pair belongs.

[0086] The system pre-establishes a mapping relationship between camera IP addresses and fixed locations, storing this information in a database or configuration file. Based on the source IP address of the video stream corresponding to the latest key-value pair, the mapping relationship is queried to determine the fixed location of the camera associated with that video stream. The fixed location of the camera can be represented using latitude and longitude coordinates or relative position.

[0087] Verify the location of the found cameras to ensure accuracy. This can be done through manual inspection or comparison with other known location information.

[0088] S103.4 Calculate the speed of the transport equipment to which the license plate number of the latest key-value pair belongs, based on the video stream to which the latest key-value pair belongs.

[0089] Extract two adjacent frames from the video stream containing the latest key-value pair. Calculate the displacement of the transport equipment by comparing its positional changes in the two frames. Feature matching algorithms (such as SIFT and SURF) can be used to match feature points of the transport equipment in the two frames to determine its positional changes.

[0090] The speed of the transport equipment is calculated based on the time interval between two image frames and the displacement of the transport equipment. The speed calculation formula is: Speed ​​= Displacement / Time Interval. To improve the accuracy of the speed calculation, a multi-frame analysis method can be used, analyzing multiple sets of adjacent image frames and taking the average value as the final speed.

[0091] S103.5 Obtain the location and timestamp of the transportation equipment, and calculate the time difference between the timestamp and the latest shooting time.

[0092] Retrieve the location and timestamp of the transportation equipment from the database. The timestamp indicates the time when the transportation equipment reported its location.

[0093] The location timestamp and the latest capture time are converted into a unified time format, and then the time difference between the two is calculated. The time difference can be expressed in seconds or milliseconds.

[0094] S103.6 Calculate the product of the time difference and the driving speed to obtain the offset distance.

[0095] Based on the time difference and travel speed, calculate the offset distance of the transport equipment during this period. The formula for offset distance is: Offset distance = Time difference * Travel speed. The offset distance represents the distance traveled by the transport equipment from the positioning time to the latest shooting time.

[0096] S103.7 Determine the simulated positioning position using the fixed position and offset distance of the corresponding camera.

[0097] The direction of travel of the transport equipment is determined based on its direction of travel in the video stream and the camera's installation location. This direction can also be determined by analyzing changes in the transport equipment's position within adjacent frames.

[0098] Based on the fixed position, offset distance, and direction of travel of the camera, the simulated location of the transportation equipment is calculated. Trigonometric functions and vector operations are used to calculate the latitude and longitude coordinates of the simulated location.

[0099] S103.8 Correct the positioning position of the transportation equipment based on the simulated positioning position.

[0100] A weighted average method is used to correct the positioning of the transportation equipment. The simulated positioning position and the original positioning position are weighted and averaged according to certain weights to obtain the corrected positioning position. The weights are determined based on the accuracy of the simulated positioning position and the reliability of the original positioning position.

[0101] The corrected location is verified to check its reasonableness. The accuracy of the corrected location is verified by comparing it with other relevant information (such as map data, the locations of other transportation equipment, etc.). If the corrected location does not match the actual situation, it needs to be corrected again or manual intervention is required.

[0102] This approach can improve the positioning accuracy of transportation equipment without adding hardware, thereby improving the accuracy of subsequent data analysis.

[0103] In one embodiment of the present invention, based on step S2, the following will provide a non-limiting description of the specific implementation of an embodiment.

[0104] S201. Pre-store multiple intersection location ranges, wherein the intersection location range is the coordinate range of the intersection monitoring area, and the intersection monitoring area is bounded by the farthest shooting distance position of multiple cameras facing the intersection.

[0105] Based on the actual needs of the dock or related scenarios, select cameras with suitable shooting angles and maximum shooting distances. For example, wide-angle high-definition cameras can be used at dock intersections, with horizontal shooting angles of 120°-180°, vertical shooting angles of 60°-90°, and maximum shooting distances of 50-100 meters. Install these cameras in suitable locations at the intersection, such as traffic light poles or monitoring towers, ensuring coverage of vehicles approaching from all directions.

[0106] Determine the specific parameters of each camera through actual testing or by consulting the camera product manual, including shooting angle, focal length, and maximum shooting distance. For example, a certain camera may have a maximum shooting distance of 80 meters and a shooting angle of 150° horizontally and 70° vertically. Simultaneously, record the installation location coordinates of each camera (using either the WGS84 global geographic coordinate system or a custom local coordinate system).

[0107] Using the camera's installation location as the origin, a mathematical model is established based on the camera's shooting angle and maximum shooting distance to calculate the boundary of the monitored area. For intersections with cameras facing multiple directions, the coverage range of each camera needs to be considered comprehensively. For example, for an intersection with cameras installed facing four directions (east, south, west, and north), the boundary of the monitored area of ​​each camera is calculated separately, and then these areas are merged.

[0108] In real-world scenarios, intersections may contain obstacles such as buildings and trees, which can affect the camera's field of view. Therefore, it is necessary to identify and analyze these obstacles and correct the boundaries of the monitored area. The location and height information of obstacles can be obtained through LiDAR scanning or manual on-site surveys, and the portions obscured by obstacles can be excluded when calculating the boundaries of the monitored area.

[0109] The calculated coordinates of the boundary points of the intersection monitoring area are converted to a unified coordinate system (such as WGS84 or a local coordinate system) and recorded. Geographic Information System (GIS) software or a custom coordinate transformation program can be used to perform the coordinate transformation. For example, the relative coordinates calculated through a mathematical model can be converted to WGS84 latitude and longitude coordinates.

[0110] The coordinates of the monitored area for each intersection are stored in a database for later querying and use. The database can be a relational database (such as MySQL or Oracle) or a non-relational database (such as MongoDB), and each intersection is assigned a unique identifier for easy management and retrieval.

[0111] S202. Match the location data of multiple transportation devices with the location range of each intersection to obtain one or more target location data corresponding to the location range of each intersection.

[0112] Transportation equipment (manned and unmanned) is equipped with positioning modules, such as GPS and BeiDou positioning systems. These positioning modules are connected to the central control system via network interfaces (such as GPRS, 4G, and 5G) to obtain real-time location data of the transportation equipment. Location data is usually represented in coordinate form, such as latitude and longitude coordinates.

[0113] The acquired location data of transportation equipment is preprocessed, including data cleaning and format conversion. Data cleaning mainly removes invalid data (such as data with coordinate values ​​outside the reasonable range) and duplicate data; format conversion unifies the coordinate formats output by different positioning systems to the same format as the intersection location range coordinates.

[0114] Spatial query algorithms (such as R-Tree algorithm and quadtree algorithm) are used to match transportation equipment location data with intersection location ranges. These algorithms can efficiently handle spatial data query and matching problems. For example, the R-Tree algorithm is a tree-like data structure used for indexing multidimensional spatial data. It can organize spatial data into a hierarchical structure and quickly find objects that intersect with a specified area by traversing the tree.

[0115] Based on the actual data volume and system performance requirements, the selected spatial query algorithm is optimized. For example, the intersection location range can be pre-divided, dividing a large area into multiple smaller sub-regions, reducing the number of regions that need to be compared in each query and improving matching efficiency.

[0116] Traverse the location data of transportation equipment: Read the location data of each transportation equipment from the database in sequence and use it as the query point.

[0117] The selected spatial query algorithm compares the query point with the location ranges of various intersections stored in the database. If the query point falls within the location range of a certain intersection, the location data of the transportation equipment is considered to match the location range of that intersection.

[0118] For each successfully matched transportation equipment location data, it is recorded in the target location data list for the corresponding intersection location range. This ultimately yields one or more target location data points for each intersection location range.

[0119] In one embodiment of the present invention, based on step S3, the following will provide a non-limiting description of the specific implementation of an embodiment.

[0120] S301. Divide the target location data within the same intersection location range into a first type of location data and a second type of location data. The first type of location data is the location data of unmanned transportation equipment, and the second type of location data is the location data of manned transportation equipment.

[0121] When transportation equipment is deployed, its positioning module is assigned specific identification information, which distinguishes between unmanned and manned transportation equipment. For example, in a vehicle management system, unmanned transportation equipment is assigned a unique identifier starting with "U-", while manned transportation equipment is assigned a unique identifier starting with "H-".

[0122] When the positioning system uploads location data, it also uploads the device's identifier. After receiving target location data within the same intersection location range, the system classifies the data into Category I (unmanned transportation equipment) and Category II (manned transportation equipment) location data based on the prefix of the identifier.

[0123] The system reads data from a database storing target location data and uses a programming language (such as Python) to write a classification script. It iterates through each location data entry, extracts device identifiers, and classifies the data based on the identifier prefix.

[0124] The categorized data is stored in different data structures or database tables for quick access and processing in subsequent steps. For example, the first category of location data is stored in a table named "unmanned_vehicle_data", and the second category of location data is stored in a table named "manned_vehicle_data".

[0125] S302. Determine the occupied lanes and the occupied time within the location range of the intersection based on the first type of data.

[0126] S302.1 Determine the driving lane, driving speed, and driving direction of the unmanned transport equipment based on the first type of location data generated at multiple times.

[0127] The system pre-stores the location range information of each lane at the intersection, which is represented in the form of geographic coordinates or relative positions.

[0128] For the first type of location data across multiple time points, the location of the unmanned transport vehicle is matched with the lane location range. For example, a spatial query algorithm (such as the R-Tree algorithm) is used to determine the lane the vehicle is in at each time point. If the vehicle's location remains within a certain lane range over a period of time, that lane is determined to be its driving lane.

[0129] The travel speed is calculated based on the location data at different times and the time interval. Assuming the unmanned transport device's position is (x1, y1) at time t1 and (x2, y2) at time t2, the travel speed is... .

[0130] To improve the accuracy of speed calculation, multiple sets of time data can be averaged. For example, select data from five consecutive time points, calculate the speed between adjacent time points, and then take the average as the final driving speed.

[0131] The direction of travel is determined by comparing the position changes of the unmanned transport vehicle at different times. The angle between the position vectors of adjacent times is calculated; for example, the vector from position (x1, y1) to (x2, y2) is... The direction of travel is determined by the direction angle of the vector.

[0132] The driving direction can be divided into several main directional intervals (such as east, south, west, and north), and the driving direction of the unmanned transport equipment can be determined according to the calculated directional angle.

[0133] S302.2 marks the driving lane where the unmanned transport equipment is located as an occupied lane.

[0134] Maintain a lane status table to record the occupancy status of each lane. Once the lane for the unmanned transport equipment is determined, mark the corresponding lane record in the lane status table as "occupied".

[0135] The tagging information can include the identifier of the device being used, the start time of use, etc., for subsequent management and querying.

[0136] S302.3 Based on the latest location, speed, and direction of the unmanned transport equipment, calculate the required time for the unmanned transport equipment to leave the intersection location range, and set the required time as the occupied time.

[0137] Determine the distance d between the unmanned transport device and the boundary of the intersection location range. Based on the latest positioning location and the boundary coordinates of the intersection location range, calculate the shortest distance from the device to the boundary using a distance calculation formula (such as the Euclidean distance formula).

[0138] Then, based on the driving speed v and distance d, the required time t = d / v is calculated. Considering the possible deceleration and turning during actual driving, a safety factor k can be appropriately added, resulting in the final required time t. f =kt.

[0139] The calculated required time is used as the occupied time of the occupied lane, and the corresponding record in the lane status table is updated.

[0140] S303. Determine the second type of location data corresponding to the occupied lane, and generate a dispatch instruction for the manned transportation equipment corresponding to the second type of location data. The dispatch instruction includes the license plate number of the manned transportation equipment, a no-passage status reminder, and the duration of the no-passage, wherein the duration of the no-passage is positively correlated with the occupation time.

[0141] S303.1 Based on the second type of location data generated at multiple times, predict the target lane for the corresponding manned transport equipment after it enters the intersection location range. The target lane is the lane where the manned transport equipment passes the center of the intersection and leaves the intersection location range.

[0142] Single-direction lanes: If the system determines that the entered lane only allows straight-ahead travel, it directly designates the straight-ahead lane as the target lane. It queries the database for the straight-ahead lane numbers at the intersection and records them as target lane information. For example, if the system queries the "lane_rules" table for the lane numbers that allow straight-ahead travel at the intersection, and the query results for lane 2 and lane 3, then these two lanes are designated as target lanes.

[0143] Right-turn lane: When entering a right-turn lane, the system determines the corresponding right-turn lane as the target lane based on the intersection layout and lane rules. This may involve querying lane connection information of adjacent intersections or road segments. For example, by querying the "intersection_connections" table, the system obtains the connection relationship between the right-turn lane and the next road segment lane, and determines the corresponding lane number after the right turn.

[0144] Lanes allowing straight and left turns: For lanes allowing straight and left turns, the system lists both the straight lane and the corresponding left-turn lane as possible target lanes. Similarly, the system queries the database to find the lane numbers that allow straight and left turns at the intersection. For example, querying the "lane_rules" table reveals lanes 1 and 4 that allow straight and left turns, and these two lanes are then included as the set of possible target lanes.

[0145] Lanes allowing straight and right turns: If the entered lane allows both straight and right turns, the system designates the straight lane and the corresponding right-turn lane as the target lanes. The system then queries the database to retrieve the corresponding lane numbers. For example, if the query finds that lanes 3 and 5 allow both straight and right turns, these two lanes are selected as the target lane set.

[0146] Based on the destination location of the manned transport equipment, its target lane can be further identified: (1) The driver of a manned transport vehicle can input the destination location information of the transport through an in-vehicle navigation system or a mobile application. This information is transmitted to the control terminal in the form of geographical coordinates (such as latitude and longitude).

[0147] (2) The system verifies the received endpoint location data, checking the legality and validity of the coordinates. At the same time, the endpoint location coordinates are converted to the same coordinate system as the intersection lane data for subsequent analysis and calculation.

[0148] (3) Use path planning algorithms (such as Dijkstra's algorithm, A* algorithm, etc.) to plan an optimal travel route based on the current location and destination location of the transportation equipment. These algorithms take into account factors such as road topology and traffic flow to calculate the shortest or fastest path.

[0149] (4) Based on the planned route and the previously predicted set of target lanes, select the target lanes that meet the route requirements. For example, if the planned route shows that a right turn is required to reach the destination, and the previously predicted set of target lanes includes both straight lanes and right-turn lanes, then select the right-turn lane as the final target lane.

[0150] (5) During the operation of the transport equipment, changes in traffic conditions or temporary adjustments by the driver may require real-time updates to the route planning and target lane. The system will periodically acquire the equipment's location data and re-plan the route and select the target lane to ensure the accuracy and effectiveness of the dispatch instructions.

[0151] (6) Perform an intersection operation between the target lane set of the manned transport equipment and the planned path to obtain the target lane. For example, if the lane to which the vehicle enters allows straight and right turns, and the planned path is to go straight at the intersection, then the target lane of the manned transport equipment is the straight lane.

[0152] S303.2 If the target lane is not an occupied lane, then the manned transport equipment is allowed to pass normally by default.

[0153] The predicted target lane is compared with the occupied lanes in the lane status table. If the target lane is marked as "unoccupied", the manned transport equipment is considered to be able to pass normally.

[0154] Record the device's traffic information, such as license plate number, passage time, and lane, for subsequent traffic statistics and analysis.

[0155] S303.3 If the target lane is an occupied lane, determine whether there is a candidate lane parallel to the occupied lane according to the road driving rules.

[0156] The road layout and driving rules information of the intersection are stored in advance, including the number of lanes and the parallel relationship between lanes.

[0157] When the target lane is occupied, candidate lanes parallel to the occupied lane are found based on this rule information. For example, at a three-lane intersection, if the middle lane is occupied, it is determined whether the left and right lanes are candidate lanes.

[0158] S303.4 If not, then generate a dispatch instruction to prohibit the passage of the manned transport equipment.

[0159] If no candidate lane exists parallel to the occupied lane, the system generates a prohibition order. The order includes the license plate number of the manned transport vehicle, a prohibition status reminder (such as "The lane ahead is occupied, please wait"), and the duration of the prohibition.

[0160] The duration of the no-passage period is determined based on the time the occupied lane has been occupied, and it can usually be set to be the same as or slightly longer than the occupied time to ensure safety.

[0161] S303.5 If yes, then obtain the status of the candidate lane. If the candidate lane is not occupied, then the manned transport equipment is allowed to pass normally by default. If the candidate lane is occupied, then generate a dispatch instruction to prohibit the manned transport equipment from passing.

[0162] The system queries the candidate lane status information from the lane status table.

[0163] If the candidate lane is marked as "unoccupied," it is assumed that manned transport equipment can pass through the candidate lane normally. A passage prompt message is sent to the equipment driver, including the suggested candidate lane.

[0164] If the candidate lane is also occupied, a no-entry order is generated, similar to the above, including the license plate number, a no-entry status reminder, and the duration of the no-entry order. Additionally, some extra information, such as the estimated waiting time, can be provided to help drivers understand the situation.

[0165] In one embodiment of the present invention, based on step S4, the following will provide a non-limiting description of the specific implementation of an embodiment.

[0166] Establish a stable and reliable network architecture within the dock or related operational area to provide network connectivity for displays located at intersections. A combination of wired networks (such as Ethernet) and wireless networks (such as Wi-Fi, 4G / 5G) can be used to ensure that the displays can receive dispatch instructions in real time. For example, for displays permanently installed at intersections, Ethernet connections should be prioritized to ensure stable data transmission; for some mobile or temporarily set-up displays, wireless networks can be used.

[0167] Each display is assigned a unique identifier and registered with the control terminal. During registration, the display's location information (associated with the intersection's location range), network address (IP address), etc., are recorded. The control terminal uses this identifier to identify and manage different displays, ensuring that dispatch commands are accurately sent to the appropriate displays.

[0168] Define a unified data format for dispatch instructions, such as JSON. Dispatch instructions include key information such as the license plate number of manned transport equipment, a status reminder indicating that passage is prohibited, and the duration of the prohibition.

[0169] To ensure the security and integrity of scheduling instructions, the instruction data is encrypted before transmission. Symmetric encryption algorithms (such as AES) or asymmetric encryption algorithms (such as RSA) can be used to encrypt the data. A secure transmission protocol (such as HTTPS) is then used to send the encrypted scheduling instructions to the display.

[0170] After receiving the dispatch command, the display sends a response message to the control terminal to confirm successful reception. The response message includes the display's identifier, reception time, and other information. If the control terminal does not receive a response message from the display within a certain time (e.g., 5 seconds), it considers the command transmission to have failed and will retransmit. A maximum of three retransmissions are allowed. If the retransmission still fails, an error message is recorded and the administrator is notified for manual intervention.

[0171] Based on the above embodiments, in order to further improve the display data mechanism of the display, a feasible method for managing the display content of the display is provided, specifically including: S601. Confirm that the display time of the dispatch instruction for manned transport equipment has reached the occupied time of the occupied lane.

[0172] The control terminal and all displays use Network Time Protocol (NTP) for time synchronization to ensure their clocks are in sync. Through regular time synchronization with an NTP server (such as the NTP server of the National Time Service Center), the error is controlled within milliseconds.

[0173] When the display receives a dispatch instruction, it records the current local time as the display start time. At the same time, it stores the duration of the no-passage restriction in the dispatch instruction (i.e., the time the occupied lane is occupied) locally.

[0174] An internal timer is set in the monitor to monitor the time at regular intervals (e.g., 1 second). During each monitoring, the difference between the current time and the start time of the display is calculated and compared with the duration of the no-passage period.

[0175] When the calculated time difference is equal to or exceeds the duration of the no-passage rule, it is considered that the display time of the scheduling instruction has reached the occupied time, and the corresponding processing operation is triggered.

[0176] S602. Remove the scheduling instruction from the display data list of the monitor.

[0177] The display internally maintains a display data list to store the currently displayed scheduling instructions. This list can be implemented using data structures such as queues or linked lists, facilitating the addition, deletion, and querying of data.

[0178] When a dispatch instruction is displayed on the screen, the relevant information of the instruction (such as license plate number, display start time, etc.) is stored in the display data list, and a unique identifier is assigned to each instruction.

[0179] When it is determined that the display time of a scheduling instruction has reached the occupied time, the monitor searches for the instruction in the display data list based on the instruction's unique identifier and deletes it from the list.

[0180] After a scheduling instruction is deleted, the monitor updates its screen display, removing any information related to the instruction to ensure that the displayed information always reflects a valid scheduling instruction. Simultaneously, a deletion confirmation message is sent to the central scheduling system, notifying it that the instruction has been removed from the monitor.

[0181] In other embodiments of the present invention, when it is confirmed that a lane is occupied, it is not limited to unmanned transportation equipment, but can also be the identification of targets such as signs prohibiting passage, roadblocks, and people in the lane.

[0182] In one specific implementation, driverless and manually driven trucks operate in a mixed manner. Dispatch instructions need to be generated by dynamically correcting vehicle positioning errors and combining this with intersection occupancy status to avoid lane conflicts.

[0183] Step 1, Vehicle localization and image collaborative correction: Data collection: The transport equipment (truck) is equipped with a GPS chip and reports its location (such as coordinate point A) every 30 seconds.

[0184] Deploy multiple cameras at the dock intersection (for example, the camera with the IP address 192.168.1.10 covers the east-west lane), and collect the video stream at 25fps.

[0185] License plate recognition and data association: Decompose the video stream into single-frame images, and mark the timestamp for each frame (such as 2025-02-21 14:30:00.500).

[0186] Grayscale the image (using the OpenCV cvtColor method) and reduce noise (Gaussian filter, kernel size 3×3).

[0187] Use the YOLOv5 model of HyperLPR3 to detect the license plate bounding box (confidence threshold 0.8), and intercept the local license plate image.

[0188] Recognize the license plate characters (such as "Lu AXXXXX") through the CRNN model, and associate the lane where the vehicle is located (for example, lane 2) and the shooting time.

[0189] Suppose the reported time of a truck's GPS is 14:30:00, and the positioning coordinates are (X1, Y1), but the latest shooting time of the corresponding camera at this time point is 14:30:00.500, indicating that the vehicle is actually located at (X2, Y2).

[0190] Calculate the time difference: 0.5 seconds × vehicle speed (such as 10m / s) = 5 meters offset distance.

[0191] According to the fixed position of the camera (known coordinates (X c , Y c )), and the offset direction (east-west), the corrected simulated positioning coordinates are (X2 + 5, Y2).

[0192] Step 2, intersection area matching and lane occupancy analysis: Store the intersection position range (for example, rectangular area: longitude 121.400 - 121.405, latitude 31.200 - 31.205), and the boundary is determined by the farthest distance covered by the camera (such as 50 meters).

[0193] When the driverless truck (license plate "Lu AXXXXX") enters the intersection area and continuously occupies lane 3 for more than 10 seconds: According to its driving speed (8m / s) and the distance from the intersection exit (80 meters), calculate the required time: 80 / 8 = 10 seconds.

[0194] Mark lane 3 as the "occupied lane", and the duration is the current time + 10 seconds (released at 14:30:10, for example).

[0195] Step 3, intelligent scheduling instruction generation: It is predicted that the manually driven truck (license plate "Lu BXXXXX") will drive into the occupied lane 3: The system detects that the parallel lane 4 is not occupied and generates the instruction: "Please switch to lane 4 for passage".

[0196] If there is no available parallel lane (such as at a single-lane intersection): Generate a no-passage instruction: "The front lane is occupied and it is expected to be passable after 14:30:10".

[0197] In this embodiment, by integrating GPS positioning and video spatio-temporal data, the vehicle positioning error is reduced from an average of 15 meters to less than 2 meters. At the same time, dynamic lane occupancy prediction improves the intersection passing efficiency by 40% and reduces the need for manual intervention by 75%.

[0198] Please refer to Figure 2 , in this embodiment, an application process is provided, including the following steps: Step 1: When the horizontal transportation equipment drives into the detection range of the intersection display screen, the information acquisition device of the intersection display screen real-time acquires the current on-site information and sends it to the information processing module.

[0199] Step 2: The information processing module processes the on-site information, removes the noise of the on-site information through information processing methods such as filters, and sends it to the information output module.

[0200] Step 3: The information output module sends the noise-removed on-site information to the equipment control system ECS.

[0201] Step 4: The traffic planning module of the equipment control system ECS comprehensively processes the noise-removed on-site information, other information, the position, speed and other information of the loading and unloading equipment, and through comprehensive calculation, obtains the passing instruction for the horizontal transportation equipment at the intersection.

[0202] Step 5: The instruction acceptance module accepts the passing instruction for the horizontal transportation equipment sent by the equipment control system ECS and sends it to the instruction processing module in real time.

[0203] Step 6: The instruction processing module processes the passing instruction and determines whether there is an instruction prohibiting the horizontal transportation equipment from passing? If there is an instruction prohibiting the horizontal transportation equipment from passing, go to Step 7; otherwise, the display module has no parking license plate number information and the driver can pass.

[0204] Step 7: The instruction processing module processes the instruction to stop the horizontal transportation equipment in the current lane into the license plate number information of the horizontal transportation equipment prohibited from passing and sends it to the display module.

[0205] Step 8: The display module shows the license plate number of the horizontal transport equipment that is prohibited from passing, instructing the driver to stop. Step 9: ECS performs integrated calculations and updates traffic control instructions at intersections in real time. Step 10: The instruction receiving module receives the intersection horizontal transport equipment passage instruction sent by the equipment control system ECS and sends it to the instruction processing module. Step 11: The instruction processing module processes the passage instruction and determines whether there is a horizontal transport equipment ready to pass. If there is an instruction for horizontal transport equipment to prepare for passage, proceed to step 12; otherwise, proceed to step 9.

[0206] Step 12: The display screen flashes the corresponding license plate number to remind the driver that they can prepare to proceed. Step 13: The information acquisition module acquires on-site information in real time and sends it to the information processing module. The information processing module determines whether the horizontal transport equipment has left the intersection. If the horizontal transport equipment has already left the intersection, proceed to step 14; otherwise, proceed to step 12. The display will flash to remind the driver that they are ready to proceed. Step 14: The display screen is updated to remove the display of license plates for departing horizontal transport vehicles.

[0207] In some embodiments, the terminal transportation equipment control system in the mixed traffic scenario may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the terminal transportation equipment control system in the mixed traffic scenario may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Functions for controlling dock transportation equipment in mixed traffic scenarios.

[0208] In this embodiment, the terminal transportation equipment control system in the mixed traffic scenario includes: Cameras are used to collect video data of the monitored area. The cameras are deployed at multiple monitoring nodes and intersections along the road. A display screen is used to show dispatch instructions, which include license plate number, a no-passage status reminder, and the duration of the no-passage restriction; The control terminal is communicatively connected to the camera and the display, and is also communicatively connected to manned and unmanned transportation equipment in the mixed traffic scenario. The control terminal executes a control program, which includes: The data acquisition module is used to acquire the location data of the transportation equipment, which may be manned or unmanned. The data matching module is used to filter out one or more target location data within a pre-stored intersection location range from the location data of multiple transportation devices; The instruction generation module is used to generate dispatch instructions for manned transportation equipment based on target location data, with the principle of normal operation of unmanned transportation equipment as the guideline. The display output module is used to send the dispatching instructions to the corresponding displays within the intersection location range.

[0209] In one specific embodiment, the system is as follows: Figure 3 As shown. The system includes: Equipment Control System (ECS), Traffic Planning Module, Display Screen, Information Acquisition Module, Information Processing Module, Information Output Module, Command Receiving Module, Command Processing Module, Display Module, and other modules.

[0210] The information acquisition module acquires real-time on-site information (such as lane information, abnormal information, personnel information in the lane, and equipment information such as the location and speed of horizontal transport equipment) within a certain range through advanced sensing devices and cameras, and sends it to the information processing module in real time.

[0211] The information processing module processes the on-site information, removes noise from the on-site information through information processing methods such as filters, and sends it to the noise-reduced on-site information output module. At the same time, it sends the display information that needs to be updated (such as canceling the display of license plate numbers) to the display module.

[0212] The information output module sends the noise-reduced field information to the equipment control system ECS.

[0213] The traffic planning module of the equipment control system (ECS) comprehensively processes noise-reduced on-site information, other information, the location, speed, and other information of loading and unloading equipment. Based on the location of the display screen and the vehicle location corresponding to the horizontal transport passage instructions, it sends relevant horizontal transport equipment passage instructions to each display screen at the dock.

[0214] The instruction receiving module receives the horizontal transport equipment passage instruction sent by the equipment control system ECS and sends it to the instruction processing module in real time.

[0215] The instruction processing module processes the horizontal transport passage instruction into the license plate numbers of the horizontal transport equipment that is prohibited from passing and the license plate numbers of the horizontal transport equipment that is ready to pass, and sends them to the display module in real time.

[0216] The display module shows in real time the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are about to be allowed to pass, indicating the vehicles passing through the intersection.

[0217] The display screen needs to show in real time the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are ready to pass, distinguishing between the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are ready to pass by using different colors and whether they are flashing.

[0218] To further understand the technical solution provided in this application, multiple application scenarios are presented. License plate numbers are used in these various application scenarios. In the first application scenario, please refer to... Figure 4 Vehicles with license plate number 1 and 2 were traveling in right-turn lane 1 and straight lane 2, respectively.

[0219] When a vehicle enters the detection range of the intersection display screen, the information acquisition module of the display screen obtains real-time information on the scene through the sensors and cameras installed on the display screen, such as lane information, abnormal information, information of people in the lane, the position and speed of license plate number 1 and license plate number 2, and sends it to the information processing module in real time.

[0220] The information processing module processes the field information, removes noise from the field information through information processing methods such as filters, and sends it to the information output module.

[0221] The information output module sends the noise-reduced field information to the equipment control system ECS.

[0222] The traffic planning module of the equipment control system ECS comprehensively processes the noise-reduced on-site information, other information, the location, speed and other information of the loading and unloading equipment, and concludes that the target locations of license plate number 2 and license plate number 1 are occupied and are not suitable for continued driving. The current optimal arrangement is to stop in the lane. The equipment control system ECS sends a stop instruction to the horizontal transport equipment in the current lane to the display screen.

[0223] The instruction receiving module receives the instruction from the equipment control system ECS to stop the horizontal transport equipment in the current lane and sends it to the instruction processing module in real time.

[0224] The instruction processing module processes the instruction for the horizontal transport equipment in the current lane to stop into license plate number 2 and license plate number 1 of the horizontal transport equipment that is prohibited from passing, and sends it to the display module in real time.

[0225] The display module shows in real time the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are ready to be allowed to pass, such as... Figure 5 As shown, the display screen indicates that license plate number 2 and license plate number 1 are stopped, instructing the driver to stop.

[0226] In the second application scenario, the traffic planning module of the equipment control system (ECS) comprehensively processes the noise-reduced on-site information, other information, the location and speed of the loading and unloading equipment, and other information. The conclusion is that the target location for license plate number 2 is already occupied and not suitable for continued driving. The optimal arrangement is to stop in the lane. The target location for license plate number 1 is about to become available and suitable for continued driving. The current optimal arrangement is to prepare for passage. The equipment control system ECS sends the corresponding instructions to the display screen.

[0227] The instruction receiving module receives the instruction from the equipment control system ECS to stop the horizontal transport equipment in the current lane and sends it to the instruction processing module in real time.

[0228] The instruction processing module processes the instruction for the horizontal transport equipment in the current lane to stop into two parts: license plate number 2 for the horizontal transport equipment that is prohibited from passing and license plate number 1 for the horizontal transport equipment that is about to pass. The instruction is then sent to the display module in real time.

[0229] The display module shows in real time the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are ready to be allowed to pass, such as... Figure 6 As shown, the display screen indicates that license plate number 2 is stopped, and the characters for license plate number 1 change color and flash simultaneously. This instructs the driver of license plate number 2 to stop, and the driver of license plate number 1 can prepare to drive.

[0230] The information acquisition module acquires current information in real time and sends it to the information processing module. When it detects that license plate number 1 has completed a right turn, the information processing module sends the information to the display module, which then cancels the display of license plate number 1's related information on the screen.

[0231] In the third application scenario, such as Figure 7 As shown, a manual vehicle with license plate number 2 is traveling in the straight lane 2, while AGV1 and AGV2 are traveling in the right-turn lane 1 and the straight lane 2, respectively.

[0232] When a vehicle enters the detection range of the intersection display screen, the information acquisition module of the display screen obtains real-time information on the scene through the sensors and cameras installed on the display screen, such as lane information, abnormal information, information of people in the lane, and the location and speed of license plate number 2, and sends it to the information processing module in real time.

[0233] The information processing module processes the field information, removes noise from the field information through information processing methods such as filters, and sends it to the information output module.

[0234] The information output module sends the noise-reduced field information to the equipment control system ECS.

[0235] The traffic planning module of the equipment control system ECS comprehensively processes the noise-reduced on-site information, other information, the location, speed and other information of the loading and unloading equipment, and concludes that license plate number 2 has a risk of colliding with AGV2, and license plate number 2 is not suitable to continue driving. The current optimal arrangement is to stop in the lane. The equipment control system ECS sends a stop instruction to the horizontal transport equipment in the current lane to the display screen.

[0236] The instruction receiving module receives the instruction from the equipment control system ECS to stop the horizontal transport equipment in the current lane and sends it to the instruction processing module in real time.

[0237] The instruction processing module processes the instruction for the horizontal transport equipment in the current lane to stop into the license plate number 2 of the horizontal transport equipment that is prohibited from passing, and sends it to the display module in real time.

[0238] The display module shows in real time the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are ready to be allowed to pass, such as... Figure 8 As shown, the display screen indicates that license plate number 2 is stopped, instructing the driver to stop.

[0239] The traffic planning module of the equipment control system ECS comprehensively processes the noise-reduced on-site information, other information, the location, speed and other information of the loading and unloading equipment, and concludes that license plate number 2 is suitable to continue driving. The current optimal arrangement is to prepare to pass. The equipment control system ECS sends the corresponding instructions to the display screen.

[0240] The instruction receiving module receives the instruction from the equipment control system ECS to stop the horizontal transport equipment in the current lane and sends it to the instruction processing module in real time.

[0241] The instruction processing module processes the instruction for the horizontal transport equipment in the current lane to stop into the license plate number 2 of the horizontal transport equipment that is ready to pass, and sends it to the display module in real time.

[0242] The display module shows in real time the license plate numbers of horizontal transport vehicles that are prohibited from passing and those that are ready to be allowed to pass, such as... Figure 9 As shown, the display screen shows the license plate number 2 changing color and flashing, indicating that the driver of license plate number 2 is ready to drive.

[0243] The information acquisition module acquires current information in real time and sends it to the information processing module. When it detects that license plate number 2 has left the intersection, the information processing module sends the information to the display module, which then cancels the display of license plate number 2's related information on the screen. Figure 10 As shown.

[0244] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A method for controlling dock transportation equipment in a mixed traffic scenario, characterized in that, include: Acquire location data of transportation equipment, which may be manned or unmanned transportation equipment; Filter out one or more target location data within a pre-stored intersection location range from the location data of multiple transportation devices; Based on the principle of ensuring the normal operation of unmanned transportation equipment, dispatch instructions for manned transportation equipment are generated according to the target location data. The dispatch command is sent to the corresponding display within the intersection location range.

2. The method according to claim 1, characterized in that, Obtain the location data of the transportation equipment, including: Obtain the location reported by the transportation equipment, which is generated by the positioning chip of the transportation equipment; Acquire video streams from multiple cameras, extract images containing the transportation equipment from the video streams, and obtain the capture time of the images; The target image with the latest shooting time is selected, and the positioning position of the transportation equipment is corrected according to the latest shooting time and the fixed position of the camera to which the target image belongs.

3. The method according to claim 2, characterized in that, Acquire video streams from multiple cameras, extract images containing the transportation equipment from the video streams, and obtain the capture time of the images, including: The video stream is converted into a sequence of image frames, and each image in the sequence is labeled with its corresponding capture time. The image is preprocessed, including grayscale conversion, noise reduction, and resizing. Using the object detection algorithm in HyperLPR3, the license plate location is identified from the preprocessed image, and the bounding box of the license plate is generated based on the license plate location; The driving lane of the corresponding transportation equipment is determined based on the license plate location. Extract the local image corresponding to the bounding box from the overall image; The local image is segmented into individual characters, and then each character is recognized by a character recognition model. The identified characters are combined in sequence to form a complete license plate number; Save the driving lane and shooting time corresponding to the same license plate number in the same image as key-value pairs, and generate license plate number tags for the key-value pairs.

4. The method according to claim 3, characterized in that, The process involves selecting the target image with the latest capture time and correcting the positioning of the transport equipment based on the latest capture time and the fixed position of the camera to which the target image belongs, including: Save the key-value pairs corresponding to the same license plate number to the same array, and filter out the latest shooting time from the array; Obtain the video stream to which the latest key-value pair belongs, wherein the latest key-value pair is a key-value pair that includes the latest shooting time; The fixed location of the camera to which the video stream belongs is determined based on the source IP of the video stream to which the latest key-value pair belongs; Calculate the speed of the transport vehicle to which the license plate number of the latest key-value pair belongs, based on the video stream to which the latest key-value pair belongs; Obtain the location and timestamp of the transportation equipment, and calculate the time difference between the timestamp and the latest shooting time. The offset distance is obtained by multiplying the time difference by the driving speed. The simulated positioning location is determined by using the fixed position and offset distance of the corresponding camera; The location of the transportation equipment is corrected based on the simulated location.

5. The method according to claim 1, characterized in that, Filter from location data of multiple transportation devices to select one or more target location data within a pre-stored intersection location range, including: Multiple intersection location ranges are pre-stored. The intersection location range is the coordinate range of the intersection monitoring area. The intersection monitoring area is bounded by the farthest shooting distance position of the cameras facing multiple directions at the intersection. The location data of multiple transportation devices are matched with the location range of each intersection to obtain one or more target location data corresponding to the location range of each intersection.

6. The method according to claim 4, characterized in that, Based on the principle of ensuring the normal operation of unmanned transportation equipment, dispatch instructions for manned transportation equipment are generated according to target location data, including: The target location data within the same intersection location range are divided into a first type of location data and a second type of location data. The first type of location data is the location data of unmanned transportation equipment, and the second type of location data is the location data of manned transportation equipment. The occupied lanes and the time of occupation within the location range of the intersection are determined based on the first type of data; The second type of location data corresponding to the occupied lane is determined, and a dispatch instruction is generated for the manned transportation equipment corresponding to the second type of location data. The dispatch instruction includes the license plate number of the manned transportation equipment, a no-passage status reminder, and the duration of the no-passage. The duration of the no-passage is positively correlated with the time of occupation. The scheduling instruction is sent to the display corresponding to the second type of location data.

7. The method according to claim 6, characterized in that, Based on the first type of data, the occupied lanes and the time of occupation within the location range of the intersection are determined, including: The driving lane, driving speed, and driving direction of the unmanned transport equipment are determined based on the first type of location data generated at multiple times. Mark the lane where the unmanned transport equipment is located as an occupied lane; Based on the latest location, speed, and direction of the unmanned transport equipment, calculate the required time for the unmanned transport equipment to leave the intersection location range, and set the required time as the occupied time.

8. The method according to claim 6, characterized in that, Determine the second type of location data corresponding to the occupied lane, and generate a scheduling instruction for the manned transportation equipment corresponding to the second type of location data, including: Based on the second type of location data generated at multiple times, predict the target lane for the corresponding manned transport equipment after it enters the intersection location range. The target lane is the lane where the manned transport equipment passes the center of the intersection and leaves the intersection location range. If the target lane is not an occupied lane, then the manned transport equipment is allowed to pass normally by default. If the target lane is occupied, then determine whether there is a candidate lane parallel to the occupied lane according to the road traffic rules: If not, a dispatch instruction is generated prohibiting the passage of the manned transport equipment; If so, the status of the candidate lane is obtained. If the candidate lane is not occupied, the manned transport equipment is allowed to pass normally by default. If the candidate lane is occupied, a dispatch instruction is generated to prohibit the manned transport equipment from passing.

9. The method according to claim 8, characterized in that, The method further includes: Confirm that the display time of the dispatch instruction for manned transport equipment has reached the occupied time of the occupied lane; Remove the scheduling command from the display's data list.

10. A control system for dock transportation equipment in a mixed traffic scenario, characterized in that, include: Cameras are used to collect video data of the monitored area. The cameras are deployed at multiple monitoring nodes and intersections along the road. A display screen is used to show dispatch instructions, which include license plate number, a no-passage status reminder, and the duration of the no-passage restriction; The control terminal is communicatively connected to the camera and the display, and is also communicatively connected to manned and unmanned transportation equipment in a mixed traffic scenario. The control terminal executes a control program, which includes: The data acquisition module is used to acquire the location data of the transportation equipment, which may be manned or unmanned. The data matching module is used to filter out one or more target location data within a pre-stored intersection location range from the location data of multiple transportation devices; The instruction generation module is used to generate dispatch instructions for manned transportation equipment based on target location data, with the principle of normal operation of unmanned transportation equipment as the guideline. The display output module is used to send the dispatching instructions to the corresponding displays within the intersection location range.

Images