Three-dimensional visualization monitoring system and method for intermodal terminal scene

Abstract

This invention discloses a three-dimensional visualization monitoring system and method for intermodal transport terminal scenarios, specifically relating to the field of three-dimensional visualization monitoring technology. The system includes acquiring terminal images with task indices, extracting the intersection points of container edges using computer vision and pairing them with container corner points, executing a Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals, accepting pose increments when the cost decreases, multiplying the radius by two when the actual decrease reaches the predicted decrease, and keeping the radius unchanged when it does not reach the predicted decrease, rejecting increments and dividing the radius by two when the cost does not decrease, until the increment is zeroed according to the pose register resolution, and outputting the corrected pose; generating container observation paths by correcting the camera pose, identifying container landing tasks that will cut off the last visible line of sight, and completing container verification and priority adjustment before task execution.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional visualization monitoring technology, and more specifically, to a three-dimensional visualization monitoring system and method for intermodal transport station scenarios. Background Technology

[0002] The existing 3D visualization monitoring of intermodal transport terminals mainly uses fixed cameras to collect images of container areas. Through image detection, container identification, target tracking, and multi-view reconstruction, it records the container identity, stacking layer, and container space occupancy. In the centralized container unloading operation in the railway and highway intermodal container area, the yard crane needs to operate continuously according to the loading and unloading plan. It is not possible to stop the machine specifically to verify the container position, nor can it add container unloading operations to obtain supplementary footage. After the top or outer container is placed, the remaining line of sight of the camera to the inner container position will be completely blocked by the container body. Current processing methods mostly update the 3D scene based on the newly added container after the container is dropped, without distinguishing whether the dropping will cut off the last visible line of sight of the inner container. Therefore, if there is an error in the recorded container number, layer, or position of the inner container, the image after dropping can only reflect that the outer stack of containers has been formed, and cannot provide images to verify the status of the inner container. It is not until the inner container is retrieved that the discrepancy between the actual container number and the recorded container position will occur, and there will be no image available to confirm the location of the error during the period of obstruction. The technical problem to be solved by this application is: how to identify the box-dropping action that is about to cut off the last visible line of sight of the box position through image detection, and to complete the three-dimensional scene confirmation before the inner box position is completely blocked without increasing the shutdown and box-turning operations. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a three-dimensional visualization monitoring system and method for intermodal transport terminal scenarios. By correcting the camera pose, a container placement observation path is generated, and container placement tasks that will cut off the last visible line of sight are identified. Container placement verification and order adjustment are completed before task execution, thereby solving the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a three-dimensional visualization monitoring method for intermodal transport station scenarios, comprising: S1. Obtain the station image of the task index, extract the intersection of the box edge lines and pair it with the box corner points through computer vision, and execute the Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals. When the cost decreases, accept the pose increment. When the actual decrease reaches the predicted decrease, the radius is multiplied by two. When it does not reach the predicted decrease, the radius remains unchanged. When the cost does not decrease, reject the increment and divide the radius by two until the increment is zeroed according to the pose register resolution, and output the corrected pose. S2. Generate the box space based on the corrected pose back projection box angle, connect the imaging center and the box position boundary point, assign the rays that do not pass through the box space to the observation hyperedge according to the landing grid number, update the box space according to the task order and delete intersecting rays, and record the task whose cut value changes from non-zero to zero as the line-of-sight closure task through the hypergraph minimum cut algorithm. S3. Before the line-of-sight closure task, correct the unobstructed box surface of the closed box position, segment the characters by inter-class variance, XOR the character skeleton with the ISO template bit by bit and identify the box number in ascending order of the value number, generate the layer position and position grid number in sequence after verification by ISO 6346 rules, splice them to form a verification code, and write the empty box number when the verification fails. S4. When the verification code is inconsistent with the scene record, the station's centralized controller moves the line of sight backwards to close the task, calculates the completion time by dividing the distance by the speed and adding the lift time, deletes the task sequence that is later than the deadline or whose cut value is zero before verification, and reads the first retained task sequence as the target sequence. S5. The station's centralized controller distributes the target sequence in a direct digital control manner. It generates an electrical response mismatch position based on the on / off sequence of the control output circuit and the drive contactor auxiliary circuit. When the electrical response mismatch position is zero, the update scenario is prohibited. When the electrical response mismatch position is zero and the verification code is consistent, the line-of-sight closure task is executed, and the three-dimensional visualization monitoring scenario is updated based on the station image after execution.

[0005] In a preferred embodiment, S1 includes: S11. The station's centralized controller intercepts the control command frame issued by the task number and forms a task image segment from the completion receipt frame. It generates a pixel search side length by projecting the device displacement using computer vision. It matches the sum of the absolute differences of gray levels of the three-by-three pixel blocks in the previous frame within the pixel search side length of the next frame bit by bit. It reads the first and second shifted bits of the sum of the absolute differences of gray levels in ascending order and checks back in reverse. It deletes pixel blocks with inconsistent back-check positions, connects the center pixels of the retained pixel blocks whose Sobel gradient magnitude is not lower than eight neighboring pixels, and outputs the task edge chain. The centralized controller for the site refers to the control device that connects to the site's image acquisition interface, direct digital control interface, and site coordinate table, and captures image frames according to the task number, writes them into coordinate records, and sends control frames. Back-lookup refers to taking the position of the next frame obtained by forward matching of the pixel block of the previous frame as the starting point, recalculating the sum of the absolute differences of gray levels within the search side length of the previous frame pixels, reading the first and second digits of the sum of the absolute differences of gray levels in ascending order and shifting them to obtain the replacement position, and comparing the replacement position with the original pixel position. S12. Read the edge normal vectors in ascending order of the eigenvalues ​​of the pixel covariance matrix of the task edge chain. Find the edge intersection points for edge normal vector pairs with non-zero determinants. Delete the edge intersection points that do not fall into the overlapping area of ​​the circumscribed rectangle of the endpoint. Pair the remaining edge intersection points with the target box corner points in ascending order of polar angle and output the task corner point pairing table. The edge normal vector refers to the unit eigenvector corresponding to the first eigenvalue in ascending order of the two-dimensional covariance matrix formed by subtracting the mean of the coordinates from the pixel coordinates of the task edge chain.

[0006] In a preferred embodiment, S1 further includes: S13. Execute the Lie group Riemann trust region optimization algorithm on the task corner point pairing table. Project the target box corner points according to the current pose and subtract them from the image corner points to generate reprojection residuals. Generate the Jacobian matrix, gradient vector, and quadratic matrix along the six basis vectors of the special Euclidean group. Use the opposite vector of the conjugate residual as the search direction. Generate the step size by dividing the sum of squares of the conjugate residual by the quadratic form of the search direction. When the quadratic form of the search direction is not greater than zero or the step result exceeds the trust region, take the intersection point of the trust region boundary. Continue until the conjugate residual is rounded to zero according to the pose register resolution. Output the candidate pose increment. The process of generating the Jacobian matrix, gradient vector, and quadratic matrix along the six basis vectors of the special Euclidean group is as follows: the current pose is multiplied by the six perturbation poses, which are generated by multiplying the three displacement basis vectors and three rotation basis vectors of the special Euclidean group by a pose register unit and then exponentially mapping them; after each perturbation, the target box corner points are reprojected and the perturbation residual is calculated. The perturbation residual is subtracted from the reprojection residual and divided by the pose register unit to generate the corresponding column of the Jacobian matrix. The gradient vector is generated by multiplying the transpose of the Jacobian matrix by the reprojection residual. The quadratic matrix is ​​generated by multiplying the transpose of the Jacobian matrix by the Jacobian matrix. The pose register resolution refers to the unit change that can be written to the pose field in the field coordinate table. The translation field uses the distance corresponding to the lower limit of the coordinate register, and the rotation field uses the angle corresponding to the lower limit of the angle register. When all fields of pose increment are rounded to zero according to their corresponding units, they are judged to be zero. S14. The candidate pose increment is multiplied by the current pose through a special Euclidean group exponent mapping to generate a candidate pose. The predicted descent is calculated using the gradient vector, the quadratic matrix, and the candidate pose increment. The candidate pose is accepted when the reprojection residual sum of squares is reduced. When the actual descent reaches the predicted descent, the trust region radius is multiplied by two. When the actual descent does not reach the predicted descent, the trust region radius is maintained. When the candidate pose does not reduce the reprojection residual sum of squares, the candidate pose is rejected and the trust region radius is divided by two. This process continues until the candidate pose increment is rounded to zero according to the pose register resolution. The current pose is then written as the corrected pose into the field coordinate table. The process of calculating the predicted descent is as follows: a first-order term is generated by the inner product of the gradient vector and the candidate pose increment; a second-order term is generated by the product of the transpose of the candidate pose increment, the quadratic matrix, and half of the result of the product of the candidate pose increment; and the negative of the sum of the first-order term and the second-order term is taken as the predicted descent.

[0007] In a preferred embodiment, S2 includes: S21. Read the camera imaging center according to the corrected pose, and back-project the image coordinates of the same box angle in different camera images into spatial rays; when the cross product of the direction vectors of the two spatial rays is zero, delete the corresponding ray pair; when the cross product is not zero, find the midpoint of the common perpendicular of the two spatial rays, take the median of the common perpendicular of the same box angle according to the three-dimensional coordinate components to generate the box angle spatial coordinates, and then generate the box space according to the box angle connection order. The direction vector refers to the unit vector pointing from the camera's imaging center to the image coordinate back-projection point in space; a ray pair refers to two spatial rays formed by the same box angle in two different camera images. S22. Read the camera imaging center in the corrected pose and the box position boundary point in the station coordinate table, and connect them to generate the observation ray; calculate the intersection position of the observation ray and the three coordinate intervals of the box space, and classify the observation rays with zero intersection position into the observation hyperedge according to the camera number and the position grid number in the task record, and output the task observation hypermap. The camera imaging center refers to the coordinates of the camera coordinate origin corresponding to the corrected pose in the station coordinate system; the container position boundary point refers to the spatial point stored in the station coordinate table according to the container position grid boundary order; the placement grid number refers to the container position grid number corresponding to the target container position in the task record.

[0008] In a preferred embodiment, S2 further includes: S23. Update the task observation hypergraph according to the task order. When the lifting box space is deleted, restore the observation rays whose intersection position changes from one to zero. When the placement box space is written, delete the observation rays whose intersection position changes from zero to one. Convert the observation hyperedge into a directed edge through Lawler expansion, and use the number of box boundary points contained in the observation hyperedge as the edge capacity. Reuse the residual network of the previous task order to perform preflow advancement and output the task cut value sequence. The task priority refers to the order in which task numbers are written into the Direct Digital Control Task Table according to the order of execution. Lawler expansion refers to generating in-point and out-point for each observation hyperedge, connecting the member vertices in the observation hyperedge to the in-point, connecting the out-point to the member vertices, and writing the edge capacity from the in-point to the out-point as the number of bin boundary points contained in the observation hyperedge; Pre-flow push refers to writing the source node label as the number of vertices and saturating the outgoing edges of the source node, then reading the positive excess vertices in descending order of labels. When there are residual edges with labels that differ by one, push the positive excess amount and the first value of the residual capacity in ascending order. When there are no residual edges with labels that differ by one, add one to the first value of the adjacent vertex labels in ascending order and write it as the current vertex label, until there are no positive excess vertices in the residual network. S24. Read the task number that changes from non-zero to zero in the task cut value sequence, construct the first branch that only writes into the placement box space and the second branch that only deletes into the placement box space, and recalculate the first cut value and the second cut value according to S23. When the first cut value is zero and the second cut value is non-zero, write the task number as the line-of-sight closure task and write it into the box position verification prequence in the direct digital control task table. Otherwise, supplement the station images in the order of the image acquisition time close to the control command time and recalculate the task observation super map until the line-of-sight closure task record is output. The Direct Digital Control Task Table refers to the data table in which the station's centralized controller stores the address of the loading and unloading equipment, the task sequence, the task start time, the preceding task number, and the control frame number according to the task number. The container position verification preceding task refers to the preceding task number written in the line-of-sight closure task record. Only after the verification code corresponding to the preceding task number matches can the station's centralized controller distribute the control frame for the line-of-sight closure task.

[0009] In a preferred embodiment, S3 includes: S31. Read the station image before the time the line-of-sight closure task control command is issued. Calculate the homography matrix based on the four corners of the closed box surface. Transform the unobstructed box surface into a rectangular box surface image. Enumerate the gray levels of the rectangular box surface image and calculate the inter-class variance. Read the first gray level in descending order of the inter-class variance to segment the character region. Perform Zhang-Suen refinement on the character region until the number of pixels deleted in this round is zero. Output the character skeleton string in horizontal order. Among them, the homography matrix refers to the planar projection transformation matrix from the four corners of the closed box surface to the four corners of the rectangular box surface image; The calculation process of inter-class variance is as follows: using the enumerated gray levels as the boundary value, the pixels of the rectangular box image are divided into low gray level group and high gray level group; when the number of pixels in either pixel group is zero, the inter-class variance is written as zero; when the number of pixels in both pixel groups is not zero, the inter-class variance corresponding to the current gray level is obtained by multiplying the number of pixels in the low gray level group, the number of pixels in the high gray level group, and the square of the difference between the gray level mean values ​​of the two pixel groups, and then dividing by the square of the total number of pixels. Zhang-Suen thinning refers to reading adjacent pixels of binary boundary pixels in the character region clockwise according to eight neighbors, deleting boundary pixels that meet the following conditions in two rounds: the number of adjacent black pixels is two to six, the number of zero-to-one transitions is one, and the product of the specified three neighbors is zero, until the number of pixels deleted in the current round is zero. A character skeleton string refers to a binary skeleton block arranged horizontally according to the character region; S32. Scale the character skeleton string bit by bit to the ISO character template grid, and XOR it bit by bit with the same type of ISO character template. Read the template characters in ascending order according to the number of XOR values ​​to form the initial box number. Calculate the check bit of the initial box number according to the ISO 6346 rule. If the calculated check bit is consistent with the image check bit, output the recognized box number. If they are inconsistent, take the next template in the template order and recalculate the check bit. If only one replacement box number passes the check, output the replacement box number. Otherwise, output an empty box number. Among them, the ISO character template grid refers to a fixed-size binary grid corresponding to the eleventh character position of the container number, with the first to fourth positions being letter positions, the fifth to tenth positions being sequence number positions, and the eleventh position being the check digit position; ISO character templates refer to a binary character tile library stored according to an ISO character template grid. Each template corresponds to a letter or number and is used for bitwise XOR with the character skeleton tile. The ISO 6346 standard specifies that the container number consists of four letters, six numbers, and one check digit. To calculate the check digit, the first ten characters are converted into numerical values ​​according to the ISO 6346 character mapping table, and then multiplied by two in ascending order according to the character sequence. The sum is then taken modulo 11 and modulo 10 to obtain the check digit. Image verification bit refers to the recognition result corresponding to the last character of the box number in the rectangular box surface image; S33. Count the horizontal seams from the lower edge of the box surface to the box location base along the vertical centerline of the box surface where the box number is located, and generate a layer position field by adding one to the number of seams; back-project the midpoint of the lower edge of the box surface to the station coordinates after correcting the pose, and generate a location grid number field according to the odd or even number of times the horizontal ray passes through the box location grid boundary; concatenate the box number field, layer position field, and location grid number field in order to form a verification code, and write the verification code into the direct digital control task record corresponding to the line-of-sight closure task; The container grid boundary refers to the plane polygon boundary of the container position stored in the station coordinate table according to the position grid number, which is formed by connecting the container position boundary points corresponding to the same position grid number in sequence. The box number field refers to the field in which the identified box number, replacement box number, or empty box number output by S32 is written in eleven-character length. The empty box number is written as eleven empty characters.

[0010] In a preferred embodiment, S4 includes: S41. The station's centralized controller reads line-of-sight closure tasks with inconsistent verification codes and reads the task numbers from the direct digital control task table according to the execution order to form a task order table. Starting from the next order after the line-of-sight closure task, it reads the insertion position item by item and backtracks along the preceding task number of the task corresponding to the insertion position to the zero number. If the backtracking result contains the line-of-sight closure task number, it stops moving forward. If the backtracking result does not contain the line-of-sight closure task number, it inserts the line-of-sight closure task after the current task, concatenates the task number according to the new order to generate an order code, deletes the order codes that hit the prohibited prefix table, and outputs the candidate order table. The task priority table refers to the sequence of task numbers formed by the central controller of the station reading the task numbers from the direct digital control task table in ascending order according to the execution priority field; the prohibited prefix table refers to the set of task number prefixes that cause the cut value to return to zero before the verification time. Each task number prefix is ​​formed by concatenating the sequence code starting point to the task number that first cut value returns to zero. S42. Recalculate the task time for each sequence code in the candidate sequence table, read the task release time, the completion time of the previous task on the same loading and unloading equipment, and the completion time of the preceding task, and take the first one of the three in descending order as the task start time; generate the task occupation time by dividing the task travel by the equipment running speed and adding the lifting time, and generate the task completion time by adding the task occupation time to the task start time. When the work areas are the same and the time periods overlap, rewrite the start time of the next task to the completion time of the previous task and recalculate in order, delete the sequence code whose task completion time is later than the task end time, and output the time sequence candidate table. The task time refers to the start time and completion time of the task after recalculation according to the sequence code for each task; the release time refers to the starting time when the corresponding task is allowed to enter the distribution queue in the direct digital control task table, which is written by the station central controller after the loading and unloading plan is issued and the target container is confirmed to be in place and the target work area is free.

[0011] In a preferred embodiment, S4 further includes: S43. Update the task observation hypergraph in ascending order of task completion time according to the task candidate table. If the completion times are the same, update in ascending order of task number. When deleting an observation hyperedge, back off the original edge flow. When restoring an observation hyperedge, write the original edge flow to zero, then perform pre-flow advancement and accumulate the cross-edge capacity of the cut partition to obtain the cut value. When the cut value is zero before the verification time, concatenate the sequence code starting point to the current task number into a forbidden prefix and write it into the forbidden prefix table. When all cut values ​​are non-zero before the verification time, write the sequence code into the verification sequence table. Repeat the process until the verification sequence table is not empty. Wherein, the original edge flow refers to the positive flow value of the directed edge stored in the residual network after Lawler expansion; the cut partition cross-edge capacity refers to the sum of the directed edge capacities from the set of vertices reachable by the source to the set of vertices not reachable by the source after the pre-flow push is completed; and the verification time refers to the image acquisition time when the verification code is written into the Direct Digital Control Task Record. S44. Arrange the verification sequence table in ascending order according to the number of shifts after the line-of-sight closure task. If the number of shifts is the same, arrange them in alphabetical order of the sequence code. Read the first sequence code as the target sequence code. The station's centralized controller increments the current direct digital control task table version number by one to generate a write version number. Write the task start time and the previous task number in the standby task table according to the target sequence code, and recursively calculate the cyclic redundancy check value according to the task sequence. If the readback check value is consistent, rewrite the active version pointer to the write version number and distribute the task control frame. If the readback check value is inconsistent, keep the active version pointer and write the write failure record in the task table, and output the target sequence. The lexicographical order of the sequence code refers to comparing the task numbers in the sequence code item by item according to a fixed field length, with the sequence code of the first different task number with the smaller value being placed first; the standby task table refers to a copy of the direct numeric control task table that is not pointed to by the active version pointer.

[0012] In a preferred embodiment, S5 includes: S51. The station's centralized controller performs a bitwise XOR operation between the verification code and the scene record with the same grid number. When the XOR result contains a single value, the line-of-sight closure task is written into the distribution prohibition table. When all XOR results are zero, the loading and unloading equipment address and action code are appended to the task number as the first field, and a cyclic redundancy check value is attached to generate a direct digital control frame. The direct digital control frame is sent at the start time of the corresponding task in the target sequence, and the task control frame number is output. The loading and unloading equipment address and action code refer to the number field on the direct digital control bus used to locate the loading and unloading equipment controller, and the action code refers to the instruction field in the direct digital control frame that represents one of the actions to be performed: picking up the box, moving, dropping the box, or stopping. S52. Read the on / off position of the control output circuit and the on / off position of the drive contactor auxiliary circuit according to the task control frame number. Locate the rising edge and falling edge of the two from the start time of the current task to the start time of the next task. If any on / off position does not form a rising edge and a falling edge, or if the on / off sequence does not conform to the order of control output rising, auxiliary circuit rising, control output falling, auxiliary circuit falling, set the electrical response mismatch position to 1; otherwise, set the electrical response mismatch position to 0 and output the task electrical response record. The on / off position of the control output circuit refers to the sampled status bit output by the central controller of the station to the drive circuit of the loading and unloading equipment. When the output is on, write 1; when the output is off, write 0. The on / off position of the drive contactor auxiliary circuit refers to the sampled status bit fed back by the auxiliary contact of the drive contactor of the loading and unloading equipment. When the contact is closed, write 1; when the contact is open, write 0. The rising edge refers to the sampling position where two adjacent sampling status bits change from zero to one, and the falling edge refers to the sampling position where two adjacent sampling status bits change from one to zero; the electrical response mismatch bit refers to a status word written when the on / off sequence of the control output circuit and the drive contactor auxiliary circuit is inconsistent. S53. When the electrical response mismatch bit in the task electrical response record is one, the station central controller keeps the scene version number unchanged and writes the subsequent task into the distribution prohibition table; when the electrical response mismatch bit is zero, the absolute difference of the station images before and after the line-of-sight closure task is calculated pixel by pixel, the changed area is segmented by inter-class variance, the lower edge endpoint of the first and second changed area in descending order of connected area is read, and the lower edge endpoint is back-projected onto the landing support plane to generate the execution landing grid number; when the execution landing grid number is consistent with the task landing grid number, the landing box space is written into the backup scene table and the readback verification value is calculated. When the readback verification value is consistent, the active scene version pointer is rewritten to the backup scene table version number, and the three-dimensional visualization monitoring scene is output. The scene version number refers to the version number of the 3D scene record currently pointed to by the active scene version pointer. After a valid write to the 3D scene record is completed, the scene version number is incremented by one. The standby scene table refers to the copy of the 3D scene record that is not pointed to by the active scene version pointer.

[0013] A 3D visualization monitoring system for intermodal transport terminal scenarios, comprising a posture correction module, a closure recognition module, a container position verification module, a sequence rearrangement module, and a control write-back module: The pose correction module is used to acquire the station image of the task index, extract the intersection of the box edge lines and pair them with the box corner points through computer vision, and perform the Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals. When the cost decreases, the pose increment is accepted. When the actual decrease reaches the predicted decrease, the radius is multiplied by two. When it does not reach the predicted decrease, the radius remains unchanged. When the cost does not decrease, the increment is rejected and the radius is divided by two until the increment is zeroed according to the pose register resolution, and the corrected pose is output. The closure recognition module is used to generate the box space based on the corrected pose back projection box angle, connect the imaging center and the box position boundary point, classify rays that do not pass through the box space into the observation hyperedge according to the landing grid number, update the box space according to the task order and delete intersecting rays, and record the task whose cut value changes from non-zero to zero as the line closure task through the hypergraph minimum cut algorithm. The container location verification module is used to correct the unobstructed container surfaces of the closed container locations before the line-of-sight closure task. It segments characters by inter-class variance, XORs the character skeleton with the ISO template bit by bit, and identifies the container number in ascending order of a single value. After verification according to the ISO 6346 rule, it generates the layer and location grid numbers in sequence, splices them to form a verification code, and writes an empty container number when the verification fails. The sequence rearrangement module is used to close the task by moving the line of sight backwards one by one when the verification code is inconsistent with the scene record. The completion time is calculated by dividing the distance by the speed and adding the lift time. The task sequence that is later than the deadline or whose cut value is zero before verification is deleted, and the first retained task sequence is read as the target sequence. The control write-back module is used by the station's centralized controller to distribute target sequences in a direct digital control manner. It generates an electrical response mismatch bit based on the on / off sequence of the control output circuit and the drive contactor auxiliary circuit. When the electrical response mismatch bit is one, the update scenario is prohibited. When the electrical response mismatch bit is zero and the verification code is consistent, the line-of-sight closure task is executed, and the three-dimensional visualization monitoring scenario is updated based on the station image after execution.

[0014] The technical effects and advantages of this invention are as follows: 1. This solution constructs an observation superedge for the observation ray and identifies the cut value as zero using the minimum cut. Verification is triggered before the box is placed, which relatively reduces the risk of the inner box position being obscured. 2. The camera pose is optimized and corrected using the Lie group Riemann trust region, so that the box corner back projection corresponds to the station coordinates, thereby reducing the misjudgment of the observation path and box space. 3. Correct the front surface of the sealed box to a rectangular image, and generate a verification code by combining template XOR and ISO check, so that the box number, layer and placement grid number can be verified before obstruction; 4. When the verification codes are inconsistent, the station's centralized controller will shift its line of sight to close the task and recalculate the time, so that the verification waiting is incorporated into the normal task sequence, thereby reducing downtime and box switching. 5. Perform electrical verification monitoring on the direct digital control frame, generate mismatch bit based on the loop continuity order, prohibit scene writing when the device responds abnormally, and relatively suppress false updates. Attached Figure Description

[0015] Figure 1 This is a flowchart outlining the method steps of the present invention; Figure 2 This is a schematic diagram of the system module structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Refer to the instruction manual appendix Figure 1-2 The three-dimensional visualization monitoring method for intermodal transport station scenarios of the present invention includes: S1. Obtain the station image of the task index, extract the intersection of the box edge lines and pair it with the box corner points through computer vision, and execute the Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals. When the cost decreases, accept the pose increment. When the actual decrease reaches the predicted decrease, the radius is multiplied by two. When it does not reach the predicted decrease, the radius remains unchanged. When the cost does not decrease, reject the increment and divide the radius by two until the increment is zeroed according to the pose register resolution, and output the corrected pose. This implementation method is used to convert the box edge line in the task image into the camera correction pose; the site central controller is connected to the site image acquisition interface, direct digital control interface and site coordinate table, and can capture image frames according to the task number, write coordinate records and send control frames; During processing, the station's centralized controller first obtains stable edge lines from the task image, then generates a pairing relationship between the corner points of the container image and the corner points of the target container position. Subsequently, it calculates the pose increment on a special Euclidean group and writes the current pose that meets the accuracy requirements for writing the station coordinate table as the corrected pose. This implementation process includes the following steps: S11 is used to obtain the container edge chain from the task image; the station central controller reads the control command issuance frame and completion receipt frame according to the task number, and extracts the station image between the two frames to form the task image segment; the time difference between adjacent frames is multiplied by the running speed of the loading and unloading equipment to obtain the equipment displacement, the equipment displacement is projected onto the image plane according to the current pose, and the projection length is rounded up to obtain the pixel search side length. For a 3x3 pixel block in the previous frame, shift it bit by bit within the search side length of the pixel in the next frame and calculate the sum of the absolute differences of gray levels in the same position. Read the first shifted bit in ascending order of the sum of the absolute differences of gray levels. When the values ​​are the same, take the shifted bit with the smaller square value of the shift distance first, and then take the first bit according to the field order of horizontal shift amount and vertical shift amount. Starting from the position of the next frame obtained by forward matching of the pixel block in the previous frame, rematch within the search side length of the pixel block in the previous frame and obtain the replacement position. If the replacement position is different from the original pixel position, delete the pixel block. The Sobel gradient magnitude of the retained pixel block is generated by adding the absolute values ​​of the horizontal and vertical gradients. The Sobel gradient magnitude of the center pixel is retained if it is not lower than the eight neighboring pixels. The retained center pixels are connected into task edge chains according to the eight neighboring connectivity relationship. If no center pixel is retained, the station central controller writes the edge missing record and stops the pose update of the current task. S12 is used to generate a task corner point pairing table; for each task edge chain, the mean of pixel coordinates is calculated, and the two-dimensional covariance matrix is ​​obtained by subtracting the mean of pixel coordinates from the pixel coordinates. The unit eigenvector corresponding to the first eigenvalue in ascending order is read as the edge normal vector; the two center pixels corresponding to the first and last eigenvalues ​​in ascending order of the projected coordinates of the task edge chain in the main direction are used as edge endpoints. For two task edge chains, the edge pair is deleted when the determinant of the matrix formed by the normal vectors of the two edges is zero; the intersection point of the edges is obtained when the determinant is not zero; the edge intersection point is deleted when it does not fall into the overlapping area of ​​the bounding rectangle of the endpoints of the two edges. The polar angles of the retained edge line intersections are calculated with the center of the container outline as the origin, and the polar angles of the target container corner points are calculated with the center of the container position as the origin. The polar angles are arranged in ascending counterclockwise order with the positive horizontal direction of the station image as the zero angle direction. If the polar angles are the same, they are arranged in ascending order according to the distance to the center. Then, the retained edge line intersections and the target container corner points are paired one by one, and the task corner point pairing table is output. If the number of retained edge line intersections is different from the number of target container corner points, the station central controller writes a corner point pairing failure record. S13 is used to calculate the candidate pose increment; the station central controller reads the task corner point pairing table, projects the target box corner point onto the image plane according to the current pose, and generates the reprojection residual by subtracting the projection coordinates from the image corner point coordinates; The six basis vectors of the special Euclidean group are read in sequence according to the x-axis translation, y-axis translation, z-axis translation, rotation around the x-axis, rotation around the y-axis, and rotation around the z-axis; the current pose is multiplied by the six perturbation poses, and the perturbation poses are generated by multiplying the corresponding basis vectors by a pose register unit and then exponentially mapping them. After each disturbance, the target box position corner point is reprojected and the disturbance residual is calculated. The Jacobian matrix column is generated by subtracting the reprojection residual from the disturbance residual and dividing by the pose register unit. The gradient vector is obtained by multiplying the reprojection residual by the transpose of the Jacobian matrix. The quadratic matrix is ​​obtained by multiplying the Jacobian matrix by the transpose of the Jacobian matrix. The initial value of the pose increment is the zero vector, the initial value of the conjugate margin is the gradient vector, and the initial value of the search direction is the opposite vector of the conjugate margin. The step size is obtained by dividing the sum of squares of the conjugate margin by the quadratic form of the search direction. If the quadratic form of the search direction is not greater than zero or the step result exceeds the trust region, the intersection point of the search direction and the trust region boundary is taken. Otherwise, the pose increment and conjugate margin are updated, and the search direction is corrected according to the ratio of the sum of squares of the conjugate margins of the two adjacent rounds. When the conjugate margin is rounded to zero according to the resolution of the pose register, the candidate pose increment is output. S14 is used to confirm the candidate pose and write the corrected pose; the candidate pose increment is multiplied by the current pose after being mapped by a special Euclidean exponent to generate the candidate pose; The predicted descent is calculated from first-order and second-order terms. The first-order term is the inner product of the gradient vector and the candidate pose increment, and the second-order term is half of the product of the transpose of the candidate pose increment, the quadratic matrix, and the candidate pose increment. The predicted descent is the negative of the sum of the first-order and second-order terms. The actual descent is the sum of squares of the reprojection residuals of the current pose minus the sum of squares of the reprojection residuals of the candidate pose. The candidate pose is accepted when the sum of squares of the reprojection residuals is reduced. When the actual descent reaches the predicted descent, the trust region radius is multiplied by two. When the actual descent does not reach the predicted descent, the trust region radius is maintained. The candidate pose is rejected and the trust region radius is divided by two when the sum of squares of the reprojection residuals is not reduced. The pose register resolution is the unit change that the pose field in the field coordinate table can write. The translation field uses the distance corresponding to the least significant bit of the coordinate register, and the rotation field uses the angle corresponding to the least significant bit of the angle register. When all fields of the candidate pose increment are rounded to zero according to their corresponding units, the iteration stops, and the current pose is written into the field coordinate table as the correction pose. In this embodiment, the task image segment, task edge chain, task corner point pairing table, candidate pose increment and corrected pose are transmitted according to the task number, and the corrected pose is used for subsequent box space generation and observation ray generation reading. In practical applications: When the yard crane performs the box dropping task, the central controller of the yard captures the task image segment from the frame of control command issuance to the completion receipt frame. The equipment displacement limits the pixel block search range, and erroneous pixel blocks caused by vehicle passage or occlusion by the spreader are checked and deleted. After the task edge chain is generated, the intersection point of the box edge line is paired with the corner point of the target box position and enters the Lie group Riemann trust region optimization. The corrected pose is written into the yard coordinate table. Subsequently, S2 uses the corrected pose to back-project the box angle and calculate the observed hyperedge.

[0018] S2. Generate the box space based on the corrected pose back projection box angle, connect the imaging center and the box position boundary point, assign the rays that do not pass through the box space to the observation hyperedge according to the landing grid number, update the box space according to the task order and delete intersecting rays, and record the task whose cut value changes from non-zero to zero as the line-of-sight closure task through the hypergraph minimum cut algorithm. This implementation method is used to convert the corrected posture into container space and observation relationship, and to identify tasks that will cut off the container observation path during the loading and unloading process. The station's centralized controller first generates the box space based on the corrected pose back projection box angle; then, it establishes a task observation hypermap by forming an observation ray from the camera's imaging center and the box's boundary points; subsequently, it updates the observation hyperedge according to the task order in the direct digital control task table, and obtains the line-of-sight closure task record through the hypermap's minimum cut; this implementation process includes the following steps: S21 is used to generate the box space; the station central controller reads the corrected pose, camera imaging center and task corner point pairing table, takes the image corner points with the same task number and the same box corner order as the same box corner, and back-projects the same box corner in different camera images as a spatial ray; The direction vector is the unit vector pointing from the camera imaging center to the image coordinate back projection space point. The cross product of each coordinate component of the direction vector is calculated after rounding down the least significant bit of the coordinate register of the field coordinate table. When all three components of the cross product are zero, the corresponding ray pair is deleted. When the cross product is not zero, the midpoint of the common perpendicular of the two spatial rays is obtained. The station's centralized controller takes the median of the midpoints of all common perpendiculars of the same box corner according to the x, y, and z coordinates to generate the box corner spatial coordinates. The box corner connection sequence inherits the target box corner point sequence, first connecting the four bottom corners, then connecting the four top corners, and connecting the corresponding upper and lower corner points. The container space is formed by the lower and upper limits of the coordinates of all container corners on the three axes of the station coordinate system; when two spatial rays of the same container corner are missing, the station central controller writes the container corner missing record and retains the container space status corresponding to the previous task number for S22 to read. S22 is used to convert the camera's line of sight into a task observation supermap; the station's centralized controller reads the camera's imaging center in the corrected pose and the box position boundary points in the station's coordinate table. The box position boundary points are spatial points stored in order of box position grid boundaries, and the position grid number is the box position grid number corresponding to the target box position in the task record; the camera's imaging center connects to the box position boundary points to form an observation ray. For each observation ray, the central controller at the site calculates the inlet and outlet parameters according to the x, y, and z coordinate intervals of the enclosure space: When the direction component is not zero, subtract the ray origin component from the lower limit and upper limit of the coordinates respectively, and then divide by the direction component. The first of the two results in ascending order is used as the axis inlet parameter, and the second is used as the axis outlet parameter. When the direction component is zero and the ray origin component is outside the coordinate interval, the intersection bit is written as zero. When the direction component is zero and the ray origin component is inside the coordinate interval, the corresponding coordinate axis is not included in the comparison. When the last ascending digit of the axis entry parameter included in the comparison does not exceed the first ascending digit of the axis exit parameter, the intersection bit is written as one; otherwise, the intersection bit is written as zero. Observation rays with zero intersection points are written into the ray intersection table according to the camera number, the landing grid number, and the box location boundary point number, and are assigned to the observation hyperedge; the observation hyperedge consists of the camera vertex, the landing grid vertex, and the box location boundary point vertex, and the task observation hypergraph is output. S23 is used to observe the super map and calculate the cutting value according to the loading and unloading task progress; the station's centralized controller reads the task sequence from the direct digital control task table, and the task sequence is the order of the task numbers written in the order of execution. When the lifting box space is deleted, the station central controller only recalculates the intersection bits associated with the lifting box space number in the ray intersection table and restores the observation rays whose intersection bits have changed from one to zero; when the placement box space is written, only the intersection bits associated with the placement box space number are recalculated and the observation rays whose intersection bits have changed from zero to one are deleted. The site's centralized controller performs Lawler expansion on the observation hyperedge, generating in-point and out-point for each observation hyperedge. It connects the member vertices in the observation hyperedge to the in-point and the out-point to the member vertices, and writes the edge capacity from the in-point to the out-point as the number of box boundary points contained in the observation hyperedge. The source point is connected to the camera vertex, and the landing grid vertex is connected to the sink point. The edge capacity of the source point and the edge capacity of the sink point are written as the sum of the observation hyperedge capacities plus one. The residual network reuses the calculation results of the previous task order. When deleting an observation superedge, it first backs up the corresponding positive flow. When restoring an observation superedge, it writes the corresponding positive flow to zero. The pre-flow push starts by writing the source vertex label as the number of vertices and saturating the outgoing edges of the source vertex. It reads positive excess vertices in descending order of label. When there are residual edges with labels that differ by one, it pushes the positive excess amount and the first value of the residual capacity in ascending order. When there are no residual edges with labels that differ by one, it adds one to the first value of the adjacent vertex labels in ascending order and writes it as the current vertex label. When there are no positive excess vertices in the residual network, it accumulates the cross-edge capacity of the cut partition and outputs the task cut value sequence. S24 is used to confirm whether the zero cut value was caused by a single container placement task and write it into the line-of-sight closure task record; the station's central controller reads the task number that changed from non-zero to zero in the task cut value sequence, the first branch copies the container space state before the task number is executed, and only writes the placement container space corresponding to the task number before recalculating the first cut value according to S23; the second branch copies the container space state after the task number is executed, and only deletes the placement container space corresponding to the task number before recalculating the second cut value according to S23. When the first cut value is zero and the second cut value is non-zero, the station's centralized controller writes the task number as the line-of-sight closure task and writes the container location verification pre-order into the direct digital control task table; the container location verification pre-order is the previous task number in the line-of-sight closure task record. Only after the verification code corresponding to the previous task number is consistent will the station's centralized controller distribute the control frame for the line-of-sight closure task. When the first cut value is non-zero or the second cut value is zero, the station's centralized controller reads the station images that were not included in the calculation in ascending order of the absolute value of the difference between the image acquisition time and the control command time, regenerates the associated box corner spatial coordinates and updates the associated observation super-edge; when the station images that were not included in the calculation have been read and the cut value relationship is still not satisfied, the direct digital control task table is written to the supplementary acquisition pre-sequence, and the corresponding task number is kept prohibited from execution. In this embodiment, the container space, observation ray, observation super-edge and task cut value sequence are all transmitted according to the task number, and the line of sight closure task record is used for subsequent container position verification and task order rearrangement reading. In practical applications: When the outer container is about to be dropped into the stack, the central controller of the terminal first generates the placement space of the container by back-projecting the corner of the outer container based on the multi-camera images; then it calculates whether the placement space cuts off the observation ray leading to the boundary point of the inner container position; when the cut value of the minimum cut of the supermap changes from non-zero to zero due to this drop, the system does not immediately distribute the drop control frame, but writes the container position verification prequel into the direct digital control task table, so that the inner container position completes the verification before the last observation path is blocked.

[0019] S3. Before the line-of-sight closure task, correct the unobstructed box surface of the closed box position, segment the characters by inter-class variance, XOR the character skeleton with the ISO template bit by bit and identify the box number in ascending order of the value number, generate the layer position and position grid number in sequence after verification by ISO 6346 rules, splice them to form a verification code, and write the empty box number when the verification fails. This implementation method is used to obtain the verification code of the enclosed container position before the line-of-sight closure task is executed; the station's centralized controller first reads the container surface image that is still observable at the enclosed container position, corrects the container surface to a rectangular container surface image, then segments characters from the rectangular container surface image and generates a character skeleton string, subsequently determines the container number using ISO character templates and ISO 6346 rules, and finally combines the layer position and landing grid number to form a verification code and write it into the direct digital control task record; this implementation process includes the following steps: S31 is used to convert the unobstructed surface of the enclosed container into a character skeleton string; the station's centralized controller reads the station image whose acquisition time is earlier than the time when the line-of-sight closure task control command is issued and which covers the enclosed container, reads the container surface that is still reached by the observation ray from the task observation hypermap, and takes the four corners of the enclosed container surface as the homography matrix input; when the four corners of the container surface cannot form a planar projection transformation matrix, a container surface correction failure record is written. The homography matrix is ​​the planar projection transformation matrix from the four corners of the closed box surface to the four corners of the rectangular box surface image. The station's central controller uses the homography matrix to transform the unobstructed box surface into a rectangular box surface image. The gray levels of the rectangular box image are enumerated one by one, and each gray level is used as a boundary value to divide the pixels into low gray level group and high gray level group. When the number of pixels in any pixel group is zero, the inter-class variance is written as zero. When the number of pixels in both pixel groups is not zero, the inter-class variance of the current gray level is obtained by multiplying the number of pixels in the low gray level group, the number of pixels in the high gray level group, and the square of the difference between the gray level mean values ​​of the two pixel groups, and then dividing by the square of the total number of pixels. The character region is then segmented by reading the first gray level in descending order of the inter-class variance. Zhang-Suen refinement reads binary boundary pixels clockwise from eight neighbors. In each round, it first deletes boundary pixels that satisfy the following conditions: the number of adjacent black pixels is two to six, the number of zero-to-one transitions is one, the product of the top pixel and the right pixel and the bottom pixel is zero, and the product of the right pixel and the bottom pixel and the left pixel is zero. Then, it deletes boundary pixels that satisfy the following conditions: the number of adjacent black pixels is two to six, the number of zero-to-one transitions is one, the product of the top pixel and the right pixel and the left pixel is zero, and the product of the top pixel and the bottom pixel and the left pixel is zero. This process continues until the number of pixels deleted in the current round is zero. The refined binary skeleton blocks are arranged horizontally according to the character region, and the character skeleton string is output. S32 is used to generate the container number field from the character skeleton string; the station's central controller reads the ISO character template grid, and the width and height fields of the ISO character template grid are stored in the template library table and correspond to the eleventh character position of the container number. The first to fourth positions are letter positions, the fifth to tenth positions are sequence number positions, and the eleventh position is the check digit position; the binary skeleton blocks in the character skeleton string are scaled bit by bit to the binary grid with the same width and height fields; The first to fourth positions read only letter templates, and the fifth to eleventh positions read only number templates. Each binary skeleton block is XORed bit by bit with the ISO character template at the same position. The template characters are read in ascending order according to the number of XOR values ​​to form the initial box number. If the initial box number does not have eleven characters, an empty box number is output. If the initial box number has eleven characters, the central controller at the site calculates the check digit for the first ten characters according to ISO 6346 rules. The four letters and six numbers are converted into numerical values ​​according to the ISO 6346 character mapping table. The numerical values ​​at each position are multiplied by two powers in ascending order according to the character sequence and then summed. The sum is then modulo eleven and modulo ten to obtain the calculated check digit. If the calculated check digit matches the eleventh image check digit, the box number is identified. When the calculated check bit is inconsistent with the image check bit, only one character position is replaced each time. The replaced character is the next template character in the same position template sequence, and the other characters retain the first template character. The check bit is then recalculated. If only one replacement box number passes the verification, the replacement box number is output. If no replacement box number passes the verification or two or more replacement box numbers pass the verification, an empty box number is output. The box number field is a field that is written with an eleven-character length to identify the box number, replacement box number or empty box number. The empty box number is written as an eleven-character empty box. S33 is used to concatenate the container number field with the layer and location grid number of the enclosed container position to form a verification code; the station's centralized controller uses the vertical centerline of the container surface where the container number is located as the counting line to read the horizontal joint from the lower edge of the container surface to the container position base surface. The base plane of the box location is the bottom bearing plane of the grid number in the station coordinate table. The horizontal joint is the transverse connecting edge line that intersects with the vertical centerline. The position where the first gray level difference is zero is not included in the horizontal joint. When the transverse connecting edge line crosses the vertical centerline, the count is incremented by one. The number of joints is incremented by one to generate the layer field. The central controller of the station projects the midpoint of the lower edge of the box surface to the box base surface after correcting the posture, and obtains the station coordinate landing point; the box grid boundary is the box planar polygon boundary stored in the station coordinate table according to the landing grid number, which is formed by connecting the box boundary points corresponding to the same landing grid number in sequence; A horizontal ray is drawn from the station coordinate point in the positive horizontal direction. When the number of times the horizontal ray crosses the grid boundary of the box is odd, the point is assigned to the corresponding box number. When the horizontal ray crosses the vertex of the box grid boundary, only the boundary edge intersecting with the horizontal ray on both sides is counted once. When the station coordinate point is located on the box grid boundary, the box number with the smaller number on both sides of the boundary is read. When the back projection ray does not intersect the box base surface, an abnormal box number record is written. The verification code is formed by concatenating the container number field, the layer field, and the placement grid number field in sequence. The container number field is eleven characters, the layer field is written as four binary bits, and the placement grid number field is written according to the binary bit width corresponding to the total number of container positions in the station. If there are insufficient bits, zeros are added on the left. The verification code is written into the direct digital control task record corresponding to the line-of-sight closure task, so that S4 can compare it bit by bit with the scene record. In this embodiment, the rectangular box surface image, character skeleton string, box number field, layer field and location grid number field are all from the site image and site coordinate table before the line-of-sight closure task is executed. The verification code is written into the direct digital control task record and used as the basis for determining whether the line-of-sight closure task is moved backward. In practical applications: When the outer container is about to cover the inner container, the central controller of the station reads the station image of the previous frame where the inner container surface can be seen. First, it completes the homography correction using the four corners of the container surface, and then obtains the container number skeleton through inter-class variance and Zhang-Suen refinement. If the container number is damaged and the first template combination fails the ISO 6346 verification, the system only performs single character replacement verification. If a unique pass result cannot be obtained, an empty container number is written, so that the subsequent S4 will postpone the line-of-sight closing task instead of directly releasing the obstruction action.

[0020] S4. When the verification code is inconsistent with the scene record, the station's centralized controller moves the line of sight backwards to close the task, calculates the completion time by dividing the distance by the speed and adding the lift time, deletes the task sequence that is later than the deadline or whose cut value is zero before verification, and reads the first retained task sequence as the target sequence. This implementation method is used to adjust the execution priority of the line-of-sight closure task when the verification code is inconsistent with the scene record, so that the observation path is still maintained before the closure position is completed. The station's centralized controller reads the task number, the preceding task number, the task release time, the task deadline time, and the loading and unloading equipment information from the direct digital control task table, first generates a candidate priority, then recalculates the task time, then verifies the cut value using the task observation hypermap, and finally writes it into the target task sequence. This implementation process includes the following steps: S41 is used to generate candidate order for line-of-sight closure tasks; the station's centralized controller reads line-of-sight closure tasks with inconsistent verification codes and reads the task numbers from the Direct Digital Control task table in ascending order of execution order field to form a task order table; the prefix table is prohibited from being empty during the first execution, and the prefix table is prohibited from reading the task number prefix written by S43 during subsequent executions; The station's centralized controller reads the insertion position sequentially from the next position after the line-of-sight closure task, and then searches back level by level along the preceding task number of the corresponding task to the zero number. The zero number is the termination number written when the preceding task number is empty, and does not correspond to the loading and unloading task. If the lookup results contain the line-of-sight closure task number, stop moving forward; if the lookup results do not contain the line-of-sight closure task number, insert the line-of-sight closure task after the current task, and concatenate the task numbers in the new order to generate a sequence code. If the sequence code prefix hits the prohibited prefix list, the sequence code is deleted; if it does not hit, it is written into the candidate sequence code list. If no sequence code is written into the candidate sequence code list, the station's centralized controller keeps the line of sight closed and prohibits the execution of the task. S42 is used to recalculate the task start time and task completion time for the candidate priority list; the station central controller reads the priority code item by item according to the candidate priority list, and reads the task release time, the completion time of the previous task on the same loading and unloading equipment and the completion time of the preceding task according to the priority code. The task release time is the starting time when the task is allowed to enter the distribution queue in the Direct Digital Control Task Table. It is generated after the three records of loading and unloading plan issuance, target container arrival, and target work area vacancy are all written. When there is no previous task on the same loading and unloading equipment, the completion time of the previous task is written as the task release time. When the previous task number is zero, the completion time of the previous task is written as the task release time. The three times are arranged in descending order, with the first one being the task start time; the task duration is generated by dividing the task distance by the equipment operating speed and adding the lifting time; the task completion time is generated by adding the task start time to the task duration. If two tasks have the same work area number, and the start time of one task is earlier than the completion time of the other task, and the start time of the other task is earlier than the completion time of one task, it is determined that the time period overlaps. The station's centralized controller rewrites the start time of the latter task to the completion time of the former task, and continues to recalculate the subsequent tasks according to the sequence code until no task start time is rewritten in one round of traversal. If the completion time of any task is later than the deadline of the task, the sequence number is deleted, and the remaining sequence numbers, along with the task times, are written into the time sequence candidate table. S43 is used to check whether the candidate order of the task observation hypermap is cut off before the verification; the station central controller updates the task observation hypermap in ascending order of task completion time in the time sequence candidate table, and updates it in ascending order of task number when the completion times are the same. When deleting a watch edge, read the positive flow value of the directed edge stored in the residual network after Lawler expansion as the original edge flow, and back the flow value of the same value along the reverse edge. When restoring the watch edge, write the corresponding positive flow to zero. The station's centralized controller then executes pre-flow propulsion. After completion, it adds the directed edge capacities of the set of vertices reachable from the source point to the set of vertices not reachable from the source point to obtain the cut partition's cross-edge capacity, and writes the cut partition's cross-edge capacity as the current cut value. The verification time is the image acquisition time when the verification code is written into the direct digital control task record; when the task completion time is earlier than the verification time and the cut value is zero, the station central controller will concatenate the sequence code from the start point to the current task number into a prohibited prefix and write it into the prohibited prefix table; when the task completion time is equal to the verification time, no prohibited prefix will be written; when all cut values ​​before the verification time are non-zero, the sequence code will be written into the verification sequence table. When the candidate priority list is empty and the verification priority list is still empty, the station's centralized controller writes the supplementary acquisition sequence into the direct digital control task list and keeps the line of sight closed to prevent the task from being executed. S44 is used to write the verification priority table into a copy of the direct digital control task table; the station central controller sorts the verification priority table in ascending order according to the number of shifts after the line of sight closed tasks. If the number of shifts is the same, it is sorted in alphabetical order of priority codes. The alphabetical order of priority codes is to compare the task numbers in the priority codes item by item according to a fixed field length, and the priority code with the smaller value of the first different task number is placed first; the first priority code is read as the target priority code; The standby task table is a copy of the Direct Digital Control task table that is not pointed to by the active version pointer. The active version pointer is the pointer field of the station central controller pointing to the currently effective version of the Direct Digital Control task table. The station's centralized controller increments the current Direct Digital Control task table version number by one to generate a written version number. In the standby task table, it writes the task start time and the previous task number item by item according to the target sequence code, and recursively calculates the cyclic redundancy check value according to the check polynomial, initial value and output XOR value saved in the Direct Digital Control task table. After the backup task list is written, the station's central controller rereads and recalculates the cyclic redundancy check value with the same parameters. When the readback check value is consistent, the active version pointer is rewritten to the written version number, and the pending distribution flag is written, and the target task sequence is output. When the readback check value is inconsistent, the active version pointer remains unchanged and the task list write failure record is written. The direct digital control frame is generated and sent by S5. In this embodiment, the candidate priority list, the time-series candidate list, the verification priority list and the target task sequence are passed according to the line-of-sight closure task number, and the task cut value verification result determines whether the line-of-sight closure task can be moved to after the verification is completed. In practical applications: When the inner container location verification code is inconsistent with the scene record, the site's centralized controller will shift the container dropping task that is about to block the inner container location to the next position, and check whether the shift violates the previous task, equipment occupation, work area occupation, and task deadline. The sequence code checked by the time will then enter the task observation super map for verification. If a candidate sequence has already had its cut value reduced to zero before the verification image is collected, the task number prefix of the candidate sequence will be written into the prohibited prefix table, and subsequent sequence generation will not repeat the same error path.

[0021] S5. The station's centralized controller distributes the target sequence in a direct digital control manner. It generates an electrical response mismatch position based on the on / off sequence of the control output circuit and the drive contactor auxiliary circuit. When the electrical response mismatch position is zero, the update scenario is prohibited. When the electrical response mismatch position is zero and the verification code is consistent, the line-of-sight closure task is executed, and the three-dimensional visualization monitoring scenario is updated based on the station image after execution. This implementation method is used to complete the control of loading and unloading equipment, execution status verification, and 3D scene write-back after the target task sequence is written. The station's centralized controller first compares the verification code with the scene record, and generates a direct digital control frame after the verification is passed. Then, it reads the sampling status of the control output circuit and the drive contactor auxiliary circuit to determine whether the loading and unloading equipment drive link responds in sequence. Finally, it confirms the placement result by combining the station images before and after the line-of-sight closure task execution, and writes it into the 3D scene record according to the scene version number. This implementation process includes the following steps: S51 is used to generate and send direct digital control frames after verification is passed; the scene record is the box number field, layer field, position grid number field and box space record stored in the three-dimensional scene record currently pointed to by the active scene version pointer, according to the position grid number index. The station's centralized controller concatenates the box number field, floor field, and landing grid number field of the same landing grid number in the scene record according to the verification code field, and then XORs them bit by bit with the verification code. The box number field is written according to the character encoding, and the floor field and landing grid number field are written according to the binary encoding. If the field length is insufficient, zeros are added to the left. When the XOR result contains a single value, the station's central controller will write the line-of-sight blocking task number, the reason for prohibition, and the writing time into the distribution prohibition table; When all XOR results are zero, the station's centralized controller reads the task number, task start time, loading / unloading equipment address, and action code from the target task sequence output by S44. The loading / unloading equipment address is the number field of the controller for locating the loading / unloading equipment on the direct digital control bus, and the action code is the instruction field indicating picking up the box, moving, dropping the box, or stopping. The station's centralized controller generates a direct digital control frame with the task number as the first field, followed by the loading / unloading equipment address and action code, and then appended with a cyclic redundancy check value. When the start time of the task corresponding to the target task sequence is reached, the station's centralized controller first queries the distribution prohibition table. If the task number is not matched, a direct digital control frame is sent and the task control frame number is output. If the task number is matched, no direct digital control frame is sent. S52 is used to confirm whether a direct digital control frame has formed a drive response through electrical verification monitoring; the station central controller reads the on / off position of the control output circuit and the on / off position of the drive contactor auxiliary circuit according to the task control frame number. The on / off position of the control output circuit is the sampled status bit output by the station central controller to the drive circuit of the loading and unloading equipment. When the output is on, write 1; when the output is off, write 0. The on / off position of the auxiliary circuit of the drive contactor is the sampling status bit fed back by the auxiliary contact of the drive contactor of the loading and unloading equipment. When the contact is closed, write 1; when the contact is open, write 0. The sampling period is provided by the sampling period field of the direct digital control interface. The sampling segment starts from the previous sampling bit at the start time of the current task and ends from the previous sampling bit at the start time of the next task. If there is no next task, the sampling segment ends from the next sampling bit at the completion time of the current task. The rising edge indicates the sampling position where two adjacent sampling status bits change from zero to one, and the falling edge indicates the sampling position where two adjacent sampling status bits change from one to zero. When the two edges fall at the same sampling position, the reading sequence of the control output circuit precedes that of the drive contactor auxiliary circuit. If any on / off position fails to generate one rising edge and one falling edge, or if the edge sequence is not control output rising, auxiliary circuit rising, control output falling, auxiliary circuit falling, the station central controller will set the electrical response mismatch to one; if the edge sequence conforms to the above sequence, the station central controller will set the electrical response mismatch to zero and output the task electrical response record. S53 is used to update the 3D visualization monitoring scene based on the electrical response results and the positioning image; when the electrical response mismatch bit in the task electrical response record is one, the station central controller keeps the scene version number unchanged and writes the successor task whose previous task number in the direct digital control task table is equal to the current task number into the distribution prohibition table. When the electrical response mismatch is zero, the station's centralized controller reads the station images before and after the line-of-sight closure task is executed, calculates the absolute difference of the gray values ​​at the same pixel position pixel by pixel, and segments the changed area according to the inter-class variance calculation method of S31. The central controller of the station performs eight-neighbor connectivity marking on the changed area, reads the first changed area in descending order of connected area, and reads the first changed area in ascending order of the horizontal coordinate of the lower edge midpoint when the connected areas are the same, and takes the lower edge midpoint of the changed area; the landing support plane is the bearing plane below the layer corresponding to the task landing grid number, which is generated by superimposing the layer height on the box base surface in the station coordinate table; After the midpoint of the lower edge is back-projected to the positioning support plane after the corrected pose, the execution positioning grid number is generated according to the odd or even number of times the horizontal ray passes through the boundary of the box grid. When the execution placement grid number is inconsistent with the task placement grid number, the site central controller keeps the scene version number unchanged, writes the placement anomaly record, and writes the subsequent task into the distribution prohibition table; when the execution placement grid number is consistent with the task placement grid number, the site central controller writes the placement box space into the backup scene table according to the execution placement grid number, and writes the box number field and layer field in the verification code, and then calculates the readback verification value. When the readback verification values ​​are consistent, the site central controller rewrites the active scene version pointer to the backup scene table version number, increments the scene version number by one, and outputs the 3D visualization monitoring scene; when the readback verification values ​​are inconsistent, the site central controller keeps the active scene version pointer unchanged and writes a write failure record to the scene table. In this embodiment, the target task sequence, direct digital control frame, task electrical response record and three-dimensional scene record are linked according to the task number, so that the three-dimensional scene is only written after the verification code is consistent, the drive loop response sequence is correct and the placement image is confirmed. In practical applications: When the yard crane is preparing to perform a box-dropping task that will obstruct the inner box position, the yard central controller first confirms that the verification code is consistent with the scene record of the same dropping grid number, and then sends a direct digital control frame; if the drive contactor auxiliary circuit does not rise before the control output circuit descends, the electrical response mismatch bit is written as 1, and the 3D scene is not updated; if the electrical response mismatch bit is zero, the yard central controller then uses the images before and after box dropping to extract the midpoint of the lower edge of the changed area, and back-projects it to obtain the execution dropping grid number. Only when the execution dropping grid number is consistent with the task dropping grid number will the dropping box space be written into the standby scene table.

[0022] Furthermore, the present invention also includes a three-dimensional visualization monitoring system for intermodal transport terminal scenarios, the system comprising a pose correction module, a closure recognition module, a container position verification module, a sequence rearrangement module, and a control write-back module: The pose correction module is used to acquire the station image of the task index, extract the intersection of the box edge lines and pair them with the box corner points through computer vision, and perform the Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals. When the cost decreases, the pose increment is accepted. When the actual decrease reaches the predicted decrease, the radius is multiplied by two. When it does not reach the predicted decrease, the radius remains unchanged. When the cost does not decrease, the increment is rejected and the radius is divided by two until the increment is zeroed according to the pose register resolution, and the corrected pose is output. The closure recognition module is used to generate the box space based on the corrected pose back projection box angle, connect the imaging center and the box position boundary point, classify rays that do not pass through the box space into the observation hyperedge according to the landing grid number, update the box space according to the task order and delete intersecting rays, and record the task whose cut value changes from non-zero to zero as the line closure task through the hypergraph minimum cut algorithm. The container location verification module is used to correct the unobstructed container surfaces of the closed container locations before the line-of-sight closure task. It segments characters by inter-class variance, XORs the character skeleton with the ISO template bit by bit, and identifies the container number in ascending order of a single value. After verification according to the ISO 6346 rule, it generates the layer and location grid numbers in sequence, splices them to form a verification code, and writes an empty container number when the verification fails. The sequence rearrangement module is used to close the task by moving the line of sight backwards one by one when the verification code is inconsistent with the scene record. The completion time is calculated by dividing the distance by the speed and adding the lift time. The task sequence that is later than the deadline or whose cut value is zero before verification is deleted, and the first retained task sequence is read as the target sequence. The control write-back module is used by the station's centralized controller to distribute target sequences in a direct digital control manner. It generates an electrical response mismatch bit based on the on / off sequence of the control output circuit and the drive contactor auxiliary circuit. When the electrical response mismatch bit is one, the update scenario is prohibited. When the electrical response mismatch bit is zero and the verification code is consistent, the line-of-sight closure task is executed, and the three-dimensional visualization monitoring scenario is updated based on the station image after execution.

[0023] Working principle: This solution first captures images of the site according to the loading and unloading tasks, extracts the intersection points of the container edges using computer vision, and corrects the camera pose using the Lie group Riemann trust region optimization algorithm. Then, based on the corrected pose, it generates the container space and container position observation rays, classifies the rays that can still observe the container position boundary into the observation hyperedge, and simulates the lifting and lowering process according to the task order. It finds the line-of-sight closure task that will make the container position observation path zero by using the hypergraph minimum cut. Before the line-of-sight closure task is executed, the system identifies the container number, layer, and placement grid number of the closed container position and generates a verification code. If the verification code is inconsistent with the scene record, the site central controller moves the line-of-sight closure task backward and rearranges the task order. After the verification is passed, the task is issued through direct digital control. At the same time, the control output circuit and drive contactor auxiliary circuit are used to check the equipment response. Finally, the 3D visualization monitoring scene is updated by combining the site images after execution. For example, a yard crane is preparing to place a container on the outside of a stack, but this placement will block the inner container position. The system first determines whether this placement will cut off the last observation path of the inner container position based on the camera footage and the spatial relationship of the stack. Before placement, it reads the visible image of the inner container surface to identify the container number, layer number, and container position. If the identification result is inconsistent with the scene record, the system does not immediately perform the occlusion action, but adjusts the order in the tasks that do not affect the deadline, so that the inner container position remains visible until the verification is completed. After the verification is consistent, the central controller of the yard issues the container placement command, and uses electrical circuit feedback and the image after placement to confirm that the task has indeed been completed, and then writes the placement result into the 3D scene.

[0024] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional visualization monitoring method for intermodal transport station scenarios, characterized in that, include: S1. Obtain the station image of the task index, extract the intersection of the box edge lines and pair it with the box corner points through computer vision, and execute the Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals. When the cost decreases, accept the pose increment. When the actual decrease reaches the predicted decrease, the radius is multiplied by two. When it does not reach the predicted decrease, the radius remains unchanged. When the cost does not decrease, reject the increment and divide the radius by two until the increment is zeroed according to the pose register resolution, and output the corrected pose. S2. Generate the box space based on the corrected pose back projection box angle, connect the imaging center and the box position boundary point, assign the rays that do not pass through the box space to the observation hyperedge according to the landing grid number, update the box space according to the task order and delete intersecting rays, and record the task whose cut value changes from non-zero to zero as the line-of-sight closure task through the hypergraph minimum cut algorithm. S3. Before the line-of-sight closure task, correct the unobstructed box surface of the closed box position, segment the characters by inter-class variance, XOR the character skeleton with the ISO template bit by bit and identify the box number in ascending order of the value number, generate the layer position and position grid number in sequence after verification by ISO 6346 rules, splice them to form a verification code, and write the empty box number when the verification fails. S4. When the verification code is inconsistent with the scene record, the station's centralized controller moves the line of sight backwards to close the task, calculates the completion time by dividing the distance by the speed and adding the lift time, deletes the task sequence that is later than the deadline or whose cut value is zero before verification, and reads the first retained task sequence as the target sequence. S5. The station's centralized controller distributes the target sequence in a direct digital control manner. It generates an electrical response mismatch position based on the on / off sequence of the control output circuit and the drive contactor auxiliary circuit. When the electrical response mismatch position is zero, the update scenario is prohibited. When the electrical response mismatch position is zero and the verification code is consistent, the line-of-sight closure task is executed, and the three-dimensional visualization monitoring scenario is updated based on the station image after execution.

2. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 1, characterized in that: S1 includes: S11. The station's centralized controller intercepts the control command frame issued by the task number and forms a task image segment from the completion receipt frame. It generates a pixel search side length by projecting the device displacement using computer vision. It matches the sum of the absolute differences of gray levels of the three-by-three pixel blocks in the previous frame within the pixel search side length of the next frame bit by bit. It reads the first and second shifted bits of the sum of the absolute differences of gray levels in ascending order and checks back in reverse. It deletes pixel blocks with inconsistent back-check positions, connects the center pixels of the retained pixel blocks whose Sobel gradient magnitude is not lower than eight neighboring pixels, and outputs the task edge chain. S12. Read the edge normal vectors in ascending order of the eigenvalues ​​of the pixel covariance matrix of the task edge chain. Find the edge intersection points for edge normal vector pairs with non-zero determinants. Delete the edge intersection points that do not fall into the overlapping area of ​​the circumscribed rectangle of the endpoint. Pair the remaining edge intersection points with the target box corner points in ascending order of polar angle and output the task corner point pairing table.

3. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 2, characterized in that: S1 also includes: S13. Execute the Lie group Riemann trust region optimization algorithm on the task corner point pairing table. Project the target box corner points according to the current pose and subtract them from the image corner points to generate reprojection residuals. Generate the Jacobian matrix, gradient vector, and quadratic matrix along the six basis vectors of the special Euclidean group. Use the opposite vector of the conjugate residual as the search direction. Generate the step size by dividing the sum of squares of the conjugate residual by the quadratic form of the search direction. When the quadratic form of the search direction is not greater than zero or the step result exceeds the trust region, take the intersection point of the trust region boundary. Continue until the conjugate residual is rounded to zero according to the pose register resolution. Output the candidate pose increment. S14. The candidate pose increment is multiplied by the current pose using a special Euclidean group exponent mapping to generate a candidate pose. The predicted descent is calculated using the gradient vector, the quadratic matrix, and the candidate pose increment. The candidate pose is accepted when the sum of squared reprojection residuals is reduced. When the actual descent reaches the predicted descent, the trust region radius is multiplied by two. When the actual descent does not reach the predicted descent, the trust region radius is maintained. The candidate pose is rejected when the sum of squared reprojection residuals is not reduced, and the trust region radius is divided by two. This process continues until the candidate pose increment is rounded to zero according to the pose register resolution. The current pose is then written into the field coordinate table as the corrected pose.

4. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 3, characterized in that: S2 includes: S21. Read the camera imaging center according to the corrected pose, and back-project the image coordinates of the same box angle in different camera images into spatial rays; when the cross product of the direction vectors of the two spatial rays is zero, delete the corresponding ray pair; when the cross product is not zero, find the midpoint of the common perpendicular of the two spatial rays, take the median of the common perpendicular of the same box angle according to the three-dimensional coordinate components to generate the box angle spatial coordinates, and then generate the box space according to the box angle connection order. S22. Read the camera imaging center in the corrected pose and the box position boundary point in the station coordinate table, and connect them to generate the observation ray; calculate the intersection position of the observation ray and the three coordinate intervals of the box space, and classify the observation rays with zero intersection position into the observation hyperedge according to the camera number and the position grid number in the task record, and output the task observation hypermap.

5. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 4, characterized in that: S2 also includes: S23. Update the task observation hypergraph according to the task order. When the lifting box space is deleted, restore the observation rays whose intersection position changes from one to zero. When the placement box space is written, delete the observation rays whose intersection position changes from zero to one. Convert the observation hyperedge into a directed edge through Lawler expansion, and use the number of box boundary points contained in the observation hyperedge as the edge capacity. Reuse the residual network of the previous task order to perform preflow advancement and output the task cut value sequence. S24. Read the task number that changes from non-zero to zero in the task cut value sequence, construct the first branch that only writes into the placement box space and the second branch that only deletes into the placement box space, and recalculate the first cut value and the second cut value according to S23. When the first cut value is zero and the second cut value is non-zero, write the task number as the line-of-sight closure task and write it into the box position verification prequence in the direct digital control task table. Otherwise, supplement the station images in the order of the image acquisition time close to the control command time and recalculate the task observation super map until the line-of-sight closure task record is output.

6. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 5, characterized in that: S3 includes: S31. Read the station image before the time the line-of-sight closure task control command is issued. Calculate the homography matrix based on the four corners of the closed box surface. Transform the unobstructed box surface into a rectangular box surface image. Enumerate the gray levels of the rectangular box surface image and calculate the inter-class variance. Read the first gray level in descending order of the inter-class variance to segment the character region. Perform Zhang-Suen refinement on the character region until the number of pixels deleted in this round is zero. Output the character skeleton string in horizontal order. S32. Scale the character skeleton string bit by bit to the ISO character template grid, and XOR it bit by bit with the same type of ISO character template. Read the template characters in ascending order according to the number of XOR values ​​to form the initial box number. Calculate the check bit of the initial box number according to the ISO 6346 rule. If the calculated check bit is consistent with the image check bit, output the recognized box number. If they are inconsistent, take the next template in the template order and recalculate the check bit. If only one replacement box number passes the check, output the replacement box number. Otherwise, output an empty box number. S33. Count the horizontal seams from the lower edge of the box surface to the box location base along the vertical centerline of the box surface where the box number is located, and generate a layer position field by adding one to the number of seams; back-project the midpoint of the lower edge of the box surface to the station coordinates after correcting the pose, and generate a location grid number field according to the odd or even number of times the horizontal ray passes through the box location grid boundary; concatenate the box number field, layer position field, and location grid number field in order to form a verification code, and write the verification code into the direct digital control task record corresponding to the line-of-sight closure task.

7. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 6, characterized in that: S4 includes: S41. The station's centralized controller reads line-of-sight closure tasks with inconsistent verification codes and reads the task numbers from the direct digital control task table according to the execution order to form a task order table. Starting from the next order after the line-of-sight closure task, it reads the insertion position item by item and backtracks along the preceding task number of the task corresponding to the insertion position to the zero number. If the backtracking result contains the line-of-sight closure task number, it stops moving forward. If the backtracking result does not contain the line-of-sight closure task number, it inserts the line-of-sight closure task after the current task, concatenates the task number according to the new order to generate an order code, deletes the order codes that hit the prohibited prefix table, and outputs the candidate order table. S42. Recalculate the task time for each sequence code in the candidate sequence table, read the task release time, the completion time of the previous task on the same loading / unloading equipment, and the completion time of the preceding task, and take the first one of the three in descending order as the task start time; generate the task occupancy time by dividing the task travel by the equipment running speed and adding the lifting time, and generate the task completion time by adding the task start time to the task occupancy time. When the work areas are the same and the time periods overlap, rewrite the start time of the next task to the completion time of the previous task and recalculate in order, delete the sequence code whose task completion time is later than the task end time, and output the time sequence candidate table.

8. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 7, characterized in that: S4 also includes: S43. Update the task observation hypergraph in ascending order of task completion time according to the task candidate table. If the completion times are the same, update in ascending order of task number. When deleting an observation hyperedge, back off the original edge flow. When restoring an observation hyperedge, write the original edge flow to zero, then perform pre-flow advancement and accumulate the cross-edge capacity of the cut partition to obtain the cut value. When the cut value is zero before the verification time, concatenate the sequence code starting point to the current task number into a forbidden prefix and write it into the forbidden prefix table. When all cut values ​​are non-zero before the verification time, write the sequence code into the verification sequence table. Repeat the process until the verification sequence table is not empty. S44. Arrange the verification sequence table in ascending order according to the number of shifts after the line-of-sight closure task. If the number of shifts is the same, arrange them in alphabetical order of the sequence code. Read the first sequence code as the target sequence code. The station's centralized controller increments the current direct digital control task table version number by one to generate a write version number. Write the task start time and the previous task number in the standby task table according to the target sequence code, and recursively calculate the cyclic redundancy check value according to the task sequence. If the readback check value is consistent, rewrite the active version pointer to the write version number and distribute the task control frame. If the readback check value is inconsistent, keep the active version pointer and write the write failure record in the task table, and output the target sequence.

9. The three-dimensional visualization monitoring method for intermodal transport station scenarios according to claim 8, characterized in that: S5 includes: S51. The station's centralized controller performs a bitwise XOR operation between the verification code and the scene record with the same grid number. When the XOR result contains a single value, the line-of-sight closure task is written into the distribution prohibition table. When all XOR results are zero, the loading and unloading equipment address and action code are appended to the task number as the first field, and a cyclic redundancy check value is attached to generate a direct digital control frame. The direct digital control frame is sent at the start time of the corresponding task in the target sequence, and the task control frame number is output. S52. Read the on / off position of the control output circuit and the on / off position of the drive contactor auxiliary circuit according to the task control frame number. Locate the rising edge and falling edge of the two from the start time of the current task to the start time of the next task. If any on / off position does not form a rising edge and a falling edge, or if the on / off sequence does not conform to the order of control output rising, auxiliary circuit rising, control output falling, auxiliary circuit falling, set the electrical response mismatch position to 1; otherwise, set the electrical response mismatch position to 0 and output the task electrical response record. S53. When the electrical response mismatch bit in the task electrical response record is one, the station centralized controller keeps the scene version number unchanged and writes the subsequent task into the distribution prohibition table; when the electrical response mismatch bit is zero, the absolute difference of the station image before and after the line-of-sight closure task is calculated pixel by pixel, the changed area is segmented by inter-class variance, the lower edge endpoint of the first and second changed area in descending order of connected area is read, and the lower edge endpoint is back-projected onto the landing support plane to generate the execution landing grid number; when the execution landing grid number is consistent with the task landing grid number, the landing box space is written into the backup scene table and the readback verification value is calculated. When the readback verification value is consistent, the active scene version pointer is rewritten to the backup scene table version number, and the three-dimensional visualization monitoring scene is output.

10. A three-dimensional visualization monitoring system for intermodal transport terminal scenes, used to implement the three-dimensional visualization monitoring method for intermodal transport terminal scenes as described in any one of claims 1-9, the system comprising a pose correction module, a closure recognition module, a container position verification module, a sequence rearrangement module, and a control write-back module, characterized in that: The pose correction module is used to acquire the station image of the task index, extract the intersection of the box edge lines and pair them with the box corner points through computer vision, and perform the Lie group Riemann trust region optimization algorithm at the cost of the sum of squared reprojection residuals. When the cost decreases, the pose increment is accepted. When the actual decrease reaches the predicted decrease, the radius is multiplied by two. When it does not reach the predicted decrease, the radius remains unchanged. When the cost does not decrease, the increment is rejected and the radius is divided by two until the increment is zeroed according to the pose register resolution, and the corrected pose is output. The closure recognition module is used to generate the box space based on the corrected pose back projection box angle, connect the imaging center and the box position boundary point, classify rays that do not pass through the box space into the observation hyperedge according to the landing grid number, update the box space according to the task order and delete intersecting rays, and record the task whose cut value changes from non-zero to zero as the line closure task through the hypergraph minimum cut algorithm. The container location verification module is used to correct the unobstructed container surfaces of the closed container locations before the line-of-sight closure task. It segments characters by inter-class variance, XORs the character skeleton with the ISO template bit by bit, and identifies the container number in ascending order of a single value. After verification according to the ISO 6346 rule, it generates the layer and location grid numbers in sequence, splices them to form a verification code, and writes an empty container number when the verification fails. The sequence rearrangement module is used to close the task by moving the line of sight backwards one by one when the verification code is inconsistent with the scene record. The completion time is calculated by dividing the distance by the speed and adding the lift time. The task sequence that is later than the deadline or whose cut value is zero before verification is deleted, and the first retained task sequence is read as the target sequence. The control write-back module is used by the station's centralized controller to distribute target sequences in a direct digital control manner. It generates an electrical response mismatch bit based on the on / off sequence of the control output circuit and the drive contactor auxiliary circuit. When the electrical response mismatch bit is one, the update scenario is prohibited. When the electrical response mismatch bit is zero and the verification code is consistent, the line-of-sight closure task is executed, and the three-dimensional visualization monitoring scenario is updated based on the station image after execution.

Images