A container and truck alignment information processing method, system and medium

Abstract

The application provides a container and truck alignment information processing method, system and medium, which can be applied to the field of port loading and unloading automation. The application obtains original video streams, environment information and inertial measurement information obtained by a plurality of cameras in a target port area, and performs image enhancement and image stabilization processing to reduce the influence of environment and inertia on the definition of the video stream. Then, the target video stream is subjected to feature point extraction to obtain a plurality of initial key feature points and multi-camera fusion positioning to obtain target key feature points. After analyzing the attitude stability of the container to obtain an attitude stability analysis result, the position deviation of the container and the truck is calculated in combination with the target key feature points to obtain a position deviation calculation result. The loading and unloading prompt information of the target port area is generated according to the position deviation calculation result and a threshold value, so that the parking accuracy of the truck can be effectively improved, and the accuracy and real-time performance of the early warning information can be improved.

Description

Technical Field

[0001] This invention relates to the field of port loading and unloading automation technology, and in particular to a method, system and medium for processing the alignment information of containers and trucks. Background Technology

[0002] In traditional container loading and unloading operations, although intelligent equipment such as cameras and auxiliary systems are used to assist workers in parking and aligning containers, the image quality of cameras is unstable due to environmental factors such as weather, lighting, fog, and vibration. This results in the auxiliary systems being unable to provide workers with stable and effective real-time images. At the same time, the existing auxiliary systems also lack real-time early warning mechanisms, thus failing to provide workers with timely and effective early warning information. Summary of the Invention

[0003] The main objective of this invention is to provide a method, system, and medium for processing the alignment information of containers and trucks, which can effectively improve the parking accuracy of trucks and enhance the accuracy and real-time performance of early warning information.

[0004] To achieve the above objectives, the present invention provides a method for processing alignment information between a container and a truck, the method comprising the following steps:

[0005] Acquire raw video streams and environmental information from multiple cameras in the target port area, as well as inertial measurement information of containers and trucks in the target port area;

[0006] Based on the environmental information and the inertial measurement information, image enhancement and image stabilization processing are performed on the frame images in the original video stream to obtain the target video stream;

[0007] Feature points are extracted from the target video stream to obtain several initial key feature points, including container feature points and truck feature points;

[0008] Multi-camera fusion localization is performed on the initial key feature points to obtain the target key feature points;

[0009] The attitude stability of the container is analyzed based on the target key feature points to obtain the attitude stability analysis results;

[0010] Based on the attitude stability analysis results and the target key feature points, the position deviation of the container and the truck is calculated to obtain the position deviation calculation results.

[0011] Based on the calculated position deviation and the threshold, loading and unloading prompts for the target port area are generated.

[0012] In some embodiments, performing image enhancement and image stabilization processing on frame images in the original video stream based on the environmental information and the inertial measurement information to obtain a target video stream includes:

[0013] Jitter compensation is performed on the frame images in the original video stream based on the inertial measurement information;

[0014] Based on the environmental information, brightness compensation and haze regression processing are performed on the jitter-compensated frame images to obtain the target video stream.

[0015] In some embodiments, the step of performing jitter compensation on frame images in the original video stream based on the inertial measurement information includes:

[0016] Calculate the optical flow between consecutive frames in the original video stream;

[0017] The jitter compensation amount is calculated based on the optical flow and the inertial measurement information;

[0018] Jitter compensation is performed on the frame images in the original video stream according to the jitter compensation amount.

[0019] In some embodiments, the step of extracting feature points from the target video stream to obtain several initial key feature points includes:

[0020] Multi-object parsing is performed on the target video stream based on graph neural networks and attention networks to obtain multi-object parsing results;

[0021] Feature points are extracted based on the multi-object parsing results to obtain several initial key feature points.

[0022] In some embodiments, the step of performing multi-camera fusion localization on the initial key feature points to obtain target key feature points includes:

[0023] Obtain the calibration matrix of the camera device;

[0024] Calculate the image depth corresponding to the initial key feature points;

[0025] The initial key feature points are transformed according to the calibration matrix and the image depth to obtain the target key feature points.

[0026] In some embodiments, the step of analyzing the attitude stability of the container based on the target key feature points to obtain attitude stability analysis results includes:

[0027] Calculate the yaw angle, tilt, and roll of the container based on the target key feature points;

[0028] The attitude stability of the container is analyzed based on the yaw angle, tilt, and roll, and the attitude stability analysis results are obtained.

[0029] In some embodiments, the step of calculating the position deviation between the container and the truck based on the attitude stability analysis results and the target key feature points to obtain the position deviation calculation results includes:

[0030] Calculate the lateral deviation and longitudinal spacing of the container and the truck based on the target key feature points;

[0031] Calculate the angular deviation between the container and the truck based on the attitude stability analysis results;

[0032] The positional deviation calculation results include the lateral deviation, the longitudinal spacing, and the angular deviation.

[0033] Another aspect of the present invention provides a positioning information processing system for containers and trucks, comprising:

[0034] The acquisition module is used to acquire raw video streams and environmental information acquired by multiple camera devices in the target port area, as well as inertial measurement information of containers and trucks in the target port area.

[0035] An image processing module is used to perform image enhancement and image stabilization processing on frame images in the original video stream based on the environmental information and the inertial measurement information to obtain a target video stream;

[0036] The feature extraction module is used to extract feature points from the target video stream to obtain several initial key feature points, including container feature points and truck feature points;

[0037] The positioning module is used to perform multi-camera fusion positioning on the initial key feature points to obtain the target key feature points;

[0038] The analysis module is used to analyze the attitude stability of the container based on the target key feature points and obtain attitude stability analysis results.

[0039] The calculation module is used to calculate the position deviation of the container and the truck based on the attitude stability analysis results and the target key feature points, and to obtain the position deviation calculation results.

[0040] The early warning module is used to generate loading and unloading prompts for the target port area based on the location deviation calculation results and thresholds.

[0041] Another aspect of the present invention provides a positioning information processing system for containers and trucks, comprising:

[0042] At least one processor;

[0043] At least one memory for storing at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0045] Another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0046] The present invention provides the following beneficial effects:

[0047] This application acquires raw video streams, environmental information, and inertial measurement information from multiple cameras in the target port area. Based on the environmental and inertial measurement information, it performs image enhancement and image stabilization on frames in the raw video stream to reduce the impact of environment and inertia on video stream clarity. Then, it extracts feature points from the target video stream to obtain several initial key feature points. These initial key feature points are then used for multi-camera fusion positioning to obtain target key feature points. Next, it analyzes the attitude stability of containers based on the target key feature points to obtain attitude stability analysis results. Finally, it calculates the position deviation of containers and trucks based on the attitude stability analysis results and the target key feature points. Finally, it generates loading and unloading prompts for the target port area based on the position deviation calculation results and a threshold, thereby effectively improving the parking accuracy of trucks and enhancing the accuracy and real-time performance of early warning information. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is a flowchart of a method for processing the alignment information of a container and a truck provided in this application;

[0050] Figure 2 This is a schematic diagram of a positioning information processing system for containers and trucks provided in this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0053] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0054] In related technologies, during traditional container loading and unloading operations, although intelligent equipment such as cameras and auxiliary systems are used to assist workers in aligning parking and containers with their experience, the imaging quality of cameras is unstable due to environmental factors such as weather, lighting, fog, and vibration. This results in the auxiliary systems being unable to provide workers with stable and effective real-time images. At the same time, existing auxiliary systems also lack real-time early warning mechanisms, thus failing to provide workers with timely and effective early warning information.

[0055] Based on this, embodiments of this application provide a method, system, and medium for processing the alignment information of containers and trucks, which can effectively improve the parking accuracy of trucks and enhance the accuracy and real-time performance of early warning information.

[0056] The embodiments of this application will be described in detail below with reference to the accompanying drawings:

[0057] Reference Figure 1 This application provides a method for processing the alignment information between a container and a truck, the method including, but not limited to, the following steps S110 to S170:

[0058] Step S110: Obtain the raw video streams and environmental information acquired by multiple camera devices in the target port area, as well as the inertial measurement information of containers and trucks in the target port area;

[0059] Step S120: Perform image enhancement and image stabilization processing on the frame images in the original video stream based on environmental information and inertial measurement information to obtain the target video stream;

[0060] Step S130: Extract feature points from the target video stream to obtain several initial key feature points, including container feature points and truck feature points;

[0061] Step S140: Perform multi-camera fusion localization on the initial key feature points to obtain the target key feature points;

[0062] Step S150: Analyze the attitude stability of the container based on the key feature points of the target, and obtain the attitude stability analysis results;

[0063] Step S160: Calculate the position deviation of the container and the truck based on the attitude stability analysis results and the key feature points of the target, and obtain the position deviation calculation results;

[0064] Step S170: Generate loading and unloading prompts for the target port area based on the position deviation calculation results and threshold.

[0065] It is understood that the camera device in this embodiment can be a binocular camera or a multi-view camera. The environmental information in this embodiment can be acquired in real time by a pre-set temperature and humidity sensor, and the inertial measurement information in this embodiment can be acquired in real time by an inertial measurement unit (IMU).

[0066] It is understood that, after obtaining the real-time raw video stream, environmental information, and inertial measurement information, this embodiment can perform jitter compensation on the frame images in the raw video stream based on the inertial measurement information, and then perform brightness compensation and haze regression processing on the jitter-compensated frame images based on the environmental information to obtain the target video stream. Specifically, in the jitter compensation process, this embodiment can calculate the jitter compensation amount based on the optical flow between consecutive frame images in the raw video stream and the optical flow and inertial measurement information, and then perform jitter compensation on the frame images in the raw video stream based on the jitter compensation amount. Here, optical flow refers to the phenomenon of image brightness pattern changing over time in consecutive image frames. By calculating the optical flow, the motion speed and direction of each point in the image can be obtained, thereby analyzing the motion state in the scene. The calculation formula for the jitter compensation amount in this embodiment is as follows:

[0067] Jitter compensation = optical flow + inertial measurement information;

[0068] The jitter compensation process is as follows:

[0069] compensated_frame = warp(frame, opticalFlow + IMU_rotation_vector);

[0070] In the formula, compensated_frame represents the result of frame interpolation processing on the frame images in the original video stream; warp represents geometric deformation processing on the image; frame represents the frame image in the original video stream; opticalFlow represents optical flow; and IMU_rotation_vector represents inertial measurement information.

[0071] Specifically, the brightness compensation and haze regression processing in this embodiment can use Retinex and enhancement models to restore nighttime or backlit scenes. After automatically partitioning and enhancing key areas such as guide lines and container corner pieces, brightness compensation or haze removal operations are performed to obtain a stable, dehazed, and brightness-balanced target video stream. Each frame in the target video stream is accompanied by a stabilized pose matrix R_stable.

[0072] It is understood that after obtaining the target video stream, this embodiment can perform multi-object parsing on the target video stream based on graph neural networks and attention networks to obtain multi-object parsing results. Then, based on the multi-object parsing results, feature points are extracted to obtain several initial key feature points. Specifically, the graph neural network and attention network in this embodiment can be a CNN + Edge-feature multi-head network. The multi-object parsing results in this embodiment include, but are not limited to, the truck head, the edge line of the vehicle platform, the corner casting of the container, the chassis lock position (twistlock), the center line of the vehicle body / lane guide line, or the offset auxiliary ruler. The initial key feature points in this embodiment include container feature points and truck feature points. Container feature points ContainerKeypoints = {FL: front left corner casting, FR: front right corner casting, RL: rear left corner casting, RR: rear right corner casting, center: (FL+FR+RL+RR) / 4, top_mid: top center line point}. Truck feature points TruckKeypoints = {head_center, trailer_mid, left_edge_line[], right_edge_line[]}.

[0073] It is understood that, after obtaining the initial key feature points, this embodiment can obtain the target key feature points by acquiring the calibration matrix of the camera device, calculating the image depth corresponding to the initial key feature points, and then performing coordinate transformation on the initial key feature points according to the calibration matrix and image depth. Specifically, the processing in this embodiment unifies the coordinate systems of images acquired by different cameras to the world coordinate system W. Specifically, the calibration matrix of each camera Ci in this embodiment has an intrinsic parameter matrix Ki, an extrinsic parameter matrix [Ri | Ti], and a distortion parameter Di. The image depth acquisition method in this embodiment includes, but is not limited to, a binocular depth acquisition method (standard method), a multi-view cross-referencing (camera array) method, and a "feature plane + size prior" inverse distance calculation method. The binocular depth acquisition method is as follows:

[0074] ;

[0075] In the formula, depth represents image depth; f represents focal length; B represents baseline; and disparity represents parallax.

[0076] The multi-view intersection method utilizes features such as corner pieces / box edges to match feature points in the fields of view of two or more cameras, and then uses triangulation to find the intersection points to obtain the image depth. The inverse distance calculation method uses the fixed dimensions of the container to solve for the image depth.

[0077] In this embodiment, after calculating the calibration matrix and image depth, the pixel coordinates of the initial key feature points in each frame are converted into camera coordinates, and then into world coordinates. Specifically, the process of converting pixel coordinates into camera 3D rays is as follows:

[0078] u = (x - cx) / fx;

[0079] v = (y - cy) / fy;

[0080] ray_camera = normalize([u, v, 1]).

[0081] Then perform 3D point projection: ;

[0082] Then, based on the results of the retroreflection, the process of converting camera coordinates to world coordinates is performed: This allows us to obtain key feature points of the target carrying world coordinates (X, Y, Z) and confidence levels.

[0083] Understandably, this embodiment, after obtaining the target key feature points, calculates the container's yaw angle, trim, and roll based on these points. Then, it analyzes the container's attitude stability based on these parameters to obtain the attitude stability analysis results. Specifically, this embodiment can solve for the positions of the four corner pieces FL, FR, RL, and RR based on the target key feature points. The formula for calculating the container's yaw angle is as follows:

[0084] Yaw = atan2(FR.y - FL.y, FR.x - FL.x);

[0085] The formula for calculating the tilt of a container is as follows:

[0086] Pitch = atan2((FL.z + FR.z) / 2 - (RL.z + RR.z) / 2, box length);

[0087] The formula for calculating the roll displacement of a container is as follows:

[0088] Roll = atan2(FR.z - FL.z, box width);

[0089] This embodiment can calculate the yaw angle, tilt amount, and roll amount to analyze whether the container tilts beyond the limit, and thus obtain the attitude stability analysis results of the container.

[0090] It is understood that, after obtaining the attitude stability analysis results and the target key feature points, this embodiment can calculate the lateral deviation and longitudinal spacing of the container and the truck based on the target key feature points, and calculate the angular deviation of the container and the truck based on the attitude stability analysis results. Specifically, the formula for calculating the lateral deviation is as follows:

[0091] ContainerCenter = (FL + FR + RL + RR) / 4;

[0092] TruckCenterLine = Fits the center line between TruckKeypoints.left_edge_line and right_edge_line;

[0093] Deviation_X = distance(ContainerCenter, TruckCenterLine).

[0094] In the formula, ContainerCenter represents the center position of the container; TruckCenterLine represents the centerline position of the truck; Deviation_X represents the lateral deviation; and distance represents the distance between the two.

[0095] The direction of lateral deviation = {leftward | rightward}; the level of lateral deviation = {green 0~5cm | yellow 5~10cm | red >10cm}.

[0096] The longitudinal spacing calculation process in this embodiment is as follows:

[0097] Distance_Z = |Container.FL.z - Truck.Twistlock_Front.z|;

[0098] In the formula, Distance_Z represents the longitudinal spacing; Container.FL.z represents the front left longitudinal position of the container; Truck.Twistlock_Front.z represents the longitudinal position of the front of the chassis in the truck; || represents taking the absolute value.

[0099] In this embodiment, when Distance_Z < threshold (e.g., 20cm), the system will automatically prompt "You are in position, entering the locked area".

[0100] Specifically, the formula for calculating the angle deviation in this embodiment is as follows:

[0101] AngleDev = |Yaw - theoretical positive angle (=0)|;

[0102] In the formula, AngleDev represents the angle deviation; Yaw represents the attitude stability analysis result.

[0103] This embodiment can analyze whether the container is placed at an angle, whether the vehicle enters the parking lot at an angle, and whether there is a dangerous angle based on the angle deviation.

[0104] It is understood that, in this embodiment, after calculating the lateral deviation, longitudinal spacing, and angular deviation, a state representation = f(Deviation_X, Distance_Z, AngleDev) is formed based on the lateral deviation, longitudinal spacing, and angular deviation. For example, this embodiment can provide loading and unloading prompts for the target port based on this state representation.

[0105] if Deviation_X < 5cm AND AngleDev <1° AND Distance_Z <20cm:

[0106] Status = Perfect Align (Green)

[0107] elif Deviation_X < 10cm:

[0108] Status = Need Adjust (Yellow)

[0109] else:

[0110] State = Danger(Red).

[0111] In some embodiments, the method of this application embodiment can also be executed based on edge processing and cloud control, and the specific processing procedure is shown in Table 1 below:

[0112] Table 1

[0113] Module Function Description Upstream / downstream relationships Imaging and Sensing Layer Images are acquired through binocular or multi-view cameras; information such as attitude and acceleration is sensed by an IMU; and environmental feedback is provided by a temperature and humidity sensor. Provide the raw image and pose data to the edge processing layer. Edge processing layer Performs image enhancement, target recognition (trucks / containers / guide lines / chassis locking), multi-camera fusion positioning and dynamic calibration. Output the target coordinates and state parameters to the control layer. Control and Communication Layer The system performs distance assessment, alignment deviation calculation, and early warning triggering on the identification results; and outputs the results to the VMT, level crossing LED, or central control system via Ethernet or RS485. It connects to the processing layer above and to the linkage equipment below. Environmental adaptation layer It achieves defogging, heating, anti-fouling and anti-vibration control; and forms a closed-loop self-adjustment with the sensor. It forms a feedback loop with the processing layer. Self-inspection and maintenance layer Monitor temperature and humidity, equipment posture, electrical status, and window cleanliness; supports remote reporting and local log storage. It operates independently and reports its status synchronously with the main controller.

[0114] It is understood that, based on the framework of the above table, the method implementation process of this embodiment is shown in Table 2 below:

[0115] Table 2

[0116] step enter Output Core Algorithm / Logic D0 Power supply connection, SPD surge protection and absorption steady-state power supply Voltage monitoring, PoE separation D1 MCU power-on self-test RTC time synchronization, module initialization Module handshake protocol D2 Environmental data (temperature and humidity) Start heating / demisting / air knife control Environmental closed-loop control D3 Image frames + IMU pose data Stabilized video stream jitter compensation algorithm D4 Multi-target recognition (box, vehicle, line, lock position) Target bounding box and feature points CNN+ Edge Detection D5 Binocular or multi-view camera data Spatial coordinates / depth information Triangulation and Calibration Matrix D6 Target coordinates + centerline calibration parameters Deviation angle and distance information Coordinate transformation matrix calculation D7 Evaluation results OSD (Optical Display Screen) overlay guide lines, output to LEDs Real-time deviation display and early warning judgment

[0117] Specifically, when analyzing based on the method of the embodiments of this application, the loading and unloading prompt information may include warning information, and the warning generation process is shown in Table 3 below:

[0118] Table 3

[0119] project Monitoring parameters event Judgment basis action Threshold setting Alignment deviation ±5cm / ±1° Yellow deviation Lateral deviation 5–10 cm Indicator light flashing yellow, OSD display directional arrow Yellow Alert Deviation >10cm —— Red Deviation / Dangerous Angle Deviation > 10 cm or yaw > 5° Audible and visual alarm, stop closing action Red Alert Decreased image clarity MTF < threshold Insufficient clarity MTF decrease, low contrast Automatically triggers air knife, wipers, and heated defrost. Automatic repair Humidity >70% —— High ambient humidity >70% Automatic heating / anti-condensation Environmental anomalies

[0120] As can be seen from the above, the method of this application embodiment can achieve centimeter-level alignment accuracy through multi-source sensor fusion, and can also realize intelligent evaluation and linkage early warning, thereby effectively improving the safety of containers and trucks, and directly outputting alignment guidance images and distance scales, thereby effectively improving the parking accuracy of trucks, as well as improving the accuracy and real-time performance of early warning information.

[0121] Reference Figure 2 This application provides a positioning information processing system for containers and trucks, including but not limited to:

[0122] The acquisition module is used to acquire raw video streams and environmental information from multiple camera devices in the target port area, as well as inertial measurement information of containers and trucks in the target port area.

[0123] The image processing module is used to perform image enhancement and image stabilization processing on the frame images in the original video stream based on environmental information and the inertial measurement information to obtain the target video stream;

[0124] The feature extraction module is used to extract feature points from the target video stream to obtain several initial key feature points, including container feature points and truck feature points.

[0125] The localization module is used to perform multi-camera fusion localization on the initial key feature points to obtain the target key feature points;

[0126] The analysis module is used to analyze the attitude stability of the container based on the target key feature points and obtain the attitude stability analysis results.

[0127] The calculation module is used to calculate the position deviation of the container and the truck based on the attitude stability analysis results and the key feature points of the target, and to obtain the position deviation calculation results.

[0128] The early warning module is used to generate loading and unloading prompts for the target port area based on the position deviation calculation results and thresholds.

[0129] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0130] This embodiment provides a positioning information processing system for containers and trucks, including:

[0131] At least one processor;

[0132] At least one memory for storing at least one program;

[0133] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0134] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0135] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is implemented when executed by a processor. Figure 1 The method shown.

[0136] It is understood that the content of the above method embodiments is applicable to this medium embodiment. The specific functions implemented in this medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0137] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical coding feature maps; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing alignment information between a container and a truck, characterized in that, The method includes the following steps: Acquire raw video streams and environmental information from multiple cameras in the target port area, as well as inertial measurement information of containers and trucks in the target port area; Based on the environmental information and the inertial measurement information, image enhancement and image stabilization processing are performed on the frame images in the original video stream to obtain the target video stream; Feature points are extracted from the target video stream to obtain several initial key feature points, including container feature points and truck feature points; Multi-camera fusion localization is performed on the initial key feature points to obtain the target key feature points; The attitude stability of the container is analyzed based on the target key feature points to obtain the attitude stability analysis results; Based on the attitude stability analysis results and the target key feature points, the position deviation of the container and the truck is calculated to obtain the position deviation calculation results. Based on the calculated position deviation and the threshold, loading and unloading prompts for the target port area are generated.

2. The method according to claim 1, characterized in that, The step of performing image enhancement and image stabilization processing on frame images in the original video stream based on the environmental information and the inertial measurement information to obtain the target video stream includes: Jitter compensation is performed on the frame images in the original video stream based on the inertial measurement information; Based on the environmental information, brightness compensation and haze regression processing are performed on the jitter-compensated frame images to obtain the target video stream.

3. The method according to claim 2, characterized in that, The step of performing jitter compensation on frame images in the original video stream based on the inertial measurement information includes: Calculate the optical flow between consecutive frames in the original video stream; The jitter compensation amount is calculated based on the optical flow and the inertial measurement information; Jitter compensation is performed on the frame images in the original video stream according to the jitter compensation amount.

4. The method according to claim 1, characterized in that, The step of extracting feature points from the target video stream yields several initial key feature points, including: Multi-object parsing is performed on the target video stream based on graph neural networks and attention networks to obtain multi-object parsing results; Feature points are extracted based on the multi-object parsing results to obtain several initial key feature points.

5. The method according to claim 1, characterized in that, The step of performing multi-camera fusion localization on the initial key feature points to obtain target key feature points includes: Obtain the calibration matrix of the camera device; Calculate the image depth corresponding to the initial key feature points; The initial key feature points are transformed according to the calibration matrix and the image depth to obtain the target key feature points.

6. The method according to claim 1, characterized in that, The step of analyzing the attitude stability of the container based on the target key feature points to obtain attitude stability analysis results includes: Calculate the yaw angle, tilt, and roll of the container based on the target key feature points; The attitude stability of the container is analyzed based on the yaw angle, tilt, and roll, and the attitude stability analysis results are obtained.

7. The method according to claim 1, characterized in that, The step of calculating the position deviation of the container and the truck based on the attitude stability analysis results and the target key feature points, and obtaining the position deviation calculation results, includes: Calculate the lateral deviation and longitudinal spacing of the container and the truck based on the target key feature points; Calculate the angular deviation between the container and the truck based on the attitude stability analysis results; The positional deviation calculation results include the lateral deviation, the longitudinal spacing, and the angular deviation.

8. A positioning information processing system for containers and trucks, characterized in that, include: The acquisition module is used to acquire raw video streams and environmental information acquired by multiple camera devices in the target port area, as well as inertial measurement information of containers and trucks in the target port area. An image processing module is used to perform image enhancement and image stabilization processing on frame images in the original video stream based on the environmental information and the inertial measurement information to obtain a target video stream; The feature extraction module is used to extract feature points from the target video stream to obtain several initial key feature points, including container feature points and truck feature points; The positioning module is used to perform multi-camera fusion positioning on the initial key feature points to obtain the target key feature points; The analysis module is used to analyze the attitude stability of the container based on the target key feature points and obtain attitude stability analysis results. The calculation module is used to calculate the position deviation of the container and the truck based on the attitude stability analysis results and the target key feature points, and to obtain the position deviation calculation results. The early warning module is used to generate loading and unloading prompts for the target port area based on the location deviation calculation results and thresholds.

9. A positioning information processing system for containers and trucks, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images