Intelligent positioning and safety control methods, devices, equipment and media for stacker cranes

Abstract

This invention relates to the field of intelligent equipment control technology, and discloses a method, device, equipment, and medium for intelligent positioning and safety control of a stacker crane. The method includes: acquiring images of the aisle environment using a depth camera of the target stacker crane; identifying the basic visual features and initial positioning of the aisle environment images; performing structural correlation on the basic visual features to obtain the target visual structural features; updating the initial positioning; generating the motion trajectory of the target stacker crane using the updated positioning; performing drift detection and global correction on the motion trajectory to obtain a corrected trajectory; monitoring operational deviations on the corrected trajectory to obtain deviation monitoring records; and performing collision prevention control on the target stacker crane to obtain the target control result. This invention can achieve stable and reliable high-precision positioning in narrow, low-texture, and structurally repetitive warehouse environments, effectively suppressing feature matching errors and cumulative drift, and significantly improving the repositioning success rate and operational safety of the stacker crane.

Description

Technical Field

[0001] This invention relates to the field of intelligent equipment control technology, and in particular to a method, device, equipment and medium for intelligent positioning and safety control of stacker cranes. Background Technology

[0002] Automated storage and retrieval systems (AS / RS) are crucial infrastructure in modern logistics systems. Their operational efficiency and safety directly depend on the high-precision positioning and stable operation of stacker cranes within the aisles. When stacker cranes travel at high speeds through narrow aisles and perform goods storage and retrieval operations, they must rely on real-time positioning systems for precise attitude control to prevent deviations, fork misalignment, or collisions. However, current methods for stacker crane positioning and safety control still face numerous technical bottlenecks.

[0003] First, automated storage and retrieval systems (AS / RS) typically feature elongated, low-texture, and highly repetitive environments, with rack uprights and beams forming numerous regular and similar geometric structures. This environment makes it difficult for vision-based SLAM systems to extract stable and discriminative feature points, leading to fuzzy or incorrect feature associations and insufficient positioning accuracy. Second, due to the long movement paths of stacker cranes, purely visual positioning is prone to cumulative errors, meaning positioning drift continuously accumulates. When occlusion, changes in lighting, or a sudden lack of available feature points occur, the system may experience repositioning failures, affecting the stacker crane's continuous operation capability between racks. Third, the confined operating space of stacker cranes in high-density AS / RS environments means that posture deviations can directly lead to inaccurate picking and placing of goods, and even pose a risk of collisions between forks, racks, or the equipment itself, severely impacting the safety and operational efficiency of the warehousing system.

[0004] In summary, existing technologies urgently need a method for intelligent positioning and safety control of stacker cranes that integrates multi-source positioning perception, enhances the ability to correlate environmental features, and has real-time early warning of abnormal deviations. This would enable precise position perception, suppression of operational errors, and safe and reliable operation control, thereby improving the overall stability and intelligence level of automated warehouses.

[0005] Therefore, existing stacker cranes are prone to feature matching errors and cumulative drift when positioned in narrow, low-texture, and repetitive warehousing environments, leading to repositioning failures and increased operational safety risks. Summary of the Invention

[0006] This invention provides a method, device, equipment, and medium for intelligent positioning and safety control of stacker cranes. Its main purpose is to solve the problem that existing stacker cranes are prone to feature matching errors and cumulative drift when positioning in narrow, low-texture, and structurally repetitive storage environments, leading to repositioning failures and increased operational safety risks.

[0007] Firstly, to achieve the above objectives, the present invention provides a method for intelligent positioning and safety control of a stacker crane, comprising:

[0008] The depth camera of the target stacker crane is used to acquire images of the tunnel environment, and the basic visual features and initial positioning of the tunnel environment images are identified.

[0009] By performing structural association on the basic visual features, the target visual structural features are obtained;

[0010] The initial positioning is updated based on the target visual structural features, and the updated positioning is used to generate the motion trajectory of the target stacker crane.

[0011] The motion trajectory is drift detected and globally corrected to obtain the corrected trajectory;

[0012] The work action deviation is monitored on the correction trajectory to obtain a deviation monitoring record;

[0013] Collision prevention control is performed on the target stacker based on the deviation monitoring records to obtain the target control result.

[0014] Secondly, the present invention also provides an intelligent positioning and safety control device for a stacker crane, comprising:

[0015] The initial positioning and recognition module is used to acquire tunnel environment images using the depth camera of the target stacker crane, and to identify the basic visual features and initial positioning of the tunnel environment images;

[0016] The feature structure association module is used to perform structural association on the basic visual features to obtain the target visual structure features;

[0017] The motion trajectory generation module is used to update the initial positioning based on the target visual structural features, and to generate the motion trajectory of the target stacker crane using the updated positioning;

[0018] The trajectory drift correction module is used to perform drift detection and global correction on the motion trajectory to obtain a corrected trajectory;

[0019] The motion deviation monitoring module is used to monitor the work motion deviation of the correction trajectory and obtain a deviation monitoring record;

[0020] The collision prevention control module is used to perform collision prevention control on the target stacker crane based on the deviation monitoring records, and obtain the target control result.

[0021] Thirdly, the present invention also provides an electronic device, the electronic device comprising:

[0022] At least one processor; and,

[0023] A memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the aforementioned intelligent positioning and safety control method for a stacker crane.

[0025] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described intelligent positioning and safety control method for a stacker crane.

[0026] This invention utilizes the depth camera of the target stacker crane to acquire images of the aisle environment, extracts basic visual features, and combines these with prior structural information about the aisle space to obtain stable and reliable target visual structural features. This updated initial positioning and generated a motion trajectory. Subsequently, through trajectory drift detection and global correction, a precise corrected trajectory is obtained. Based on this, operational deviation monitoring and collision prevention control are implemented, forming a closed-loop control mechanism. This effectively suppresses feature matching errors and cumulative drift caused by narrow aisles, low texture, and repetitive shelf structures, improving positioning accuracy and robustness, achieving reliable repositioning, and ensuring stacker crane operation safety through movement deviation monitoring and collision prevention control. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating an intelligent positioning and safety control method for a stacker crane according to an embodiment of the present invention.

[0029] Figure 2 This is a functional block diagram of a stacker crane intelligent positioning and safety control device provided in an embodiment of the present invention.

[0030] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0033] This application provides a method for intelligent positioning and safety control of a stacker crane. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the device provided in this application: a server, a terminal, etc. In other words, the method for intelligent positioning and safety control of a stacker crane can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0034] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent positioning and safety control method for a stacker crane according to an embodiment of the present invention. In this embodiment, the intelligent positioning and safety control method for a stacker crane includes:

[0035] S1. Use the depth camera of the target stacker crane to acquire images of the tunnel environment, and identify the basic visual features and initial positioning of the tunnel environment images.

[0036] In this embodiment of the invention, the target stacker crane is a stacking device that performs automated storage and retrieval tasks in an automated warehouse. A depth camera is a 3D imaging sensor capable of simultaneously acquiring scene image information and distance information. The aisle environment image is visual image data reflecting the rack structure and spatial layout within the warehouse aisles. Basic visual features are geometric structural features extracted from the aisle environment image for positioning and identification. Initial positioning is the positioning result based on visual features, estimating the current position and attitude of the stacker crane.

[0037] In detail, the depth camera at the front end of the target stacker crane is used to collect depth and grayscale data of the warehouse aisle in real time. The acquired aisle environment images are processed for denoising and parallax correction. Based on edge detection, point cloud clustering and geometric structure extraction algorithms, basic visual features such as shelf uprights, shelf boundaries and aisle centerlines are identified. On this basis, a local three-dimensional feature point set is constructed by combining depth information, and feature matching is performed with a preset warehouse environment map or structural prior. The pose estimation algorithm is used to solve the translation vector and rotation attitude of the stacker crane, thereby completing the initial positioning of the current running position.

[0038] This invention utilizes the depth camera of the target stacker crane to acquire images of the tunnel environment and identify the basic visual features and initial positioning. It can acquire spatial constraint information between the stacker crane and the storage structure in real time without the need for additional positioning base stations or high-cost sensors, thereby achieving rapid perception of the operating position and attitude estimation.

[0039] S2. Perform structural association on the basic visual features to obtain the target visual structural features.

[0040] In this embodiment of the invention, the target visual structural features are a set of visual features that can fully reflect the spatial structure of the alleyway after structural association processing.

[0041] In detail, the step of performing structural association on the basic visual features to obtain target visual structural features includes:

[0042] The prior layout relationship of the aisle space where the target stacker crane is located is generated based on the pre-acquired warehouse racking layout parameters and aisle geometry prior information;

[0043] The basic visual features are associated with the prior layout relationship to obtain visual structure association features;

[0044] Geometric consistency screening is performed on the visual structure association features to obtain the target visual structure features.

[0045] In this embodiment of the invention, based on the collected warehouse racking layout parameters, including structural information such as racking spacing, shelf height, and column position, as well as aisle geometric prior information such as aisle width, track centerline and relative relationship with racks, an a priori layout model describing the aisle spatial topology is constructed; then these a priori geometric constraints are associated with the spatial coordinate system of the stacker crane area to form the three-dimensional structural layout relationship of the aisle where the target stacker crane is currently located.

[0046] Furthermore, based on the extracted basic visual features, including local visual information such as corners, edges, and planes identified in the aisle environment image, as well as the prior layout relationship of the aisle where the target stacker crane is located, including geometric information such as rack spacing, shelf height, column position, aisle width, and the relative relationship between the track centerline and the rack, the basic visual features are matched with these prior geometric constraints to establish the correlation between visual features and aisle structure, thereby forming visual structure correlation features that can reflect visual information and spatial topological relationships.

[0047] In this embodiment of the invention, geometric consistency analysis is performed on the constructed visual structure association features, including matching verification of position, orientation, scale and topology. Feature points that do not conform to the geometric constraints of the lane or may have matching errors are eliminated, and reliable feature points that both conform to the visual features and satisfy the prior geometric constraints of the lane are retained, and finally the filtered target visual structure features are obtained.

[0048] This invention establishes a correlation between basic visual features and the prior geometric layout of the alleyway space, which not only enhances the stability and discriminativeness of the features, but also effectively suppresses feature matching errors and cumulative drift caused by the narrow alleyways, low texture, and repetitive shelf structures.

[0049] S3. Update the initial positioning based on the target visual structural features, and use the updated positioning to generate the motion trajectory of the target stacker crane.

[0050] In this embodiment of the invention, positioning update is a process of correcting and optimizing the initial positioning using the target's visual structural features to improve positioning accuracy. The motion trajectory is the spatial position path of the target stacker crane within the aisle, changing over time.

[0051] Specifically, the step of updating the initial positioning based on the target visual structural features and generating the motion trajectory of the target stacker crane using the updated positioning includes:

[0052] Extract the acceleration and angular velocity of the target's visual structural features;

[0053] Using the acceleration and the acceleration, a motion prior analysis is performed on the initial positioning to obtain the target stacker crane's positioning to be updated;

[0054] The target visual structural features are mapped to the location to be updated, and error analysis is performed on the location to be updated based on the mapping results to obtain the reprojection error;

[0055] The structural prior constraints of the alleyway space are generated based on the prior layout relationship;

[0056] The reprojection error and the structural prior constraints are used to jointly optimize the localization to be updated, resulting in an updated localization.

[0057] Based on the initial positioning and the updated positioning, the target stacker crane is subjected to trajectory analysis to obtain the motion trajectory.

[0058] In this embodiment of the invention, based on the selected target visual structural features, the displacement changes of each feature point of the stacker crane in the aisle over time are analyzed using the visual information of consecutive frames. The instantaneous acceleration and angular velocity information of the stacker crane along the aisle are further extracted, reflecting the motion state and attitude changes of the stacker crane.

[0059] Furthermore, the extracted acceleration and angular velocity are combined with the initial positioning of the stacker crane to perform kinematic deduction, analyze the possible motion trajectory and position changes of the stacker crane in the aisle, and obtain the positioning to be updated after considering motion constraints, laying the foundation for improving positioning accuracy.

[0060] In this embodiment of the invention, based on the location to be updated, the visual structural features of the target are mapped in three-dimensional space to a two-dimensional image plane or a corresponding coordinate system. The deviation between the predicted position and the actual observed position, i.e., the reprojection error, is analyzed to quantify the consistency between the visual features and the positioning results.

[0061] In this embodiment of the invention, the structural prior constraints of the aisle space are constructed by utilizing the previously acquired warehouse racking layout parameters and aisle geometric prior information, including racking spacing, shelf height, column position, aisle width, and the relative relationship between the track centerline and the racking, providing geometric and topological constraints for positioning optimization.

[0062] Furthermore, by combining visual reprojection errors with prior constraints on the roadway structure, a joint optimization method is employed to adjust and correct the updated positioning, eliminating abnormal or inconsistent features to obtain a more accurate and stable updated positioning, thereby improving the positioning accuracy and robustness of the stacker crane. Moreover, by utilizing the temporal relationship between the initial positioning and the updated positioning, the spatial position of the stacker crane in the roadway over time is analyzed to generate a complete stacker crane motion trajectory.

[0063] This invention improves the positioning accuracy and stability of stacker cranes in complex warehousing environments by combining reliable visual structural features to correct initial position information. At the same time, by generating precise motion trajectories, it enables real-time monitoring and analysis of the stacker crane's motion status, providing a safe and reliable trajectory reference for automated storage and retrieval operations, reducing collision risks and improving operational efficiency.

[0064] S4. Perform drift detection and global correction on the motion trajectory to obtain the corrected trajectory.

[0065] In this embodiment of the invention, drift detection is the process of identifying and analyzing positional deviations in the motion trajectory caused by sensor errors or inaccurate positioning. Global correction is the process of correcting the detected trajectory drift using prior environmental information or global constraints, restoring the trajectory to a more accurate spatial position. The corrected trajectory is a more accurate and reliable stacker crane motion path obtained after drift detection and global correction processing.

[0066] In detail, the drift detection and global correction of the motion trajectory to obtain the corrected trajectory includes:

[0067] The positioning difference between two adjacent frames in the motion trajectory is obtained one by one;

[0068] When the positioning difference is greater than a preset difference threshold, the motion trajectory is taken as a potential drift trajectory;

[0069] Based on the historical operating trajectory of the target stacker crane and the lane structure layout of the lane space obtained in advance, identify the loop candidate region of the movement trajectory;

[0070] Using the target visual structural features, semantic rematching is performed on the alleyway environment image and the loop candidate region of the potential drift trajectory to generate semantic matching degree and geometric consistency constraint information.

[0071] A global constraint system is constructed based on the semantic matching degree and the geometric consistency constraint information;

[0072] The potential drift trajectory is corrected according to the global constraint system to obtain the corrected trajectory.

[0073] In this embodiment of the invention, the position information of the stacker crane in two consecutive frames of the motion trajectory is compared to obtain the position change or difference value between the two frames, so as to quantify the motion offset of the stacker crane in a continuous time step. Furthermore, when the positioning difference exceeds the difference threshold, it is considered that there may be cumulative drift or instantaneous positioning error in the motion trajectory, and the motion trajectory is marked as a potential drift trajectory.

[0074] In this embodiment of the invention, by combining historical operating data of the target stacker crane with prior layout information of the aisle space, including rack layout, aisle width, and track centerline, the regions where the current motion trajectory may form loops are analyzed to determine candidate loop regions for global correction. The candidate loop regions are spatial areas in the stacker crane's motion trajectory that are inferred from historical operating trajectories and aisle space structure layout, suggesting possible loops or repeated path traversal. These regions are used for feature matching and trajectory optimization during global correction and drift correction.

[0075] In this embodiment of the invention, the step of using the target visual structural features to perform semantic re-matching on the alleyway environment image and the loop candidate region of the potential drift trajectory, and generating semantic matching degree and geometric consistency constraint information, includes:

[0076] Semantic segmentation is performed on the alleyway environment image of the potential drift trajectory, and discriminative structural feature points are identified based on the segmentation results;

[0077] The discriminative structural feature points and the target visual structural features are associated to obtain a semantic feature set;

[0078] The semantic feature set and the loop closure candidate region are matched point by point to obtain a number of matching feature points;

[0079] Based on the positional correspondence of the matching feature points, the semantic matching degree between the potential drift trajectory and the loop closure candidate region is calculated;

[0080] Based on the semantic matching degree, a deviation analysis is performed on the matching feature points to obtain feature point deviation information;

[0081] Geometric consistency constraint information is generated based on the feature point deviation information.

[0082] In this embodiment of the invention, the alleyway environment image corresponding to the potential drift trajectory is divided into different regions according to object categories or structural features, such as shelves, passageways, and pillars, using a semantic segmentation method. Then, key feature points that can effectively distinguish different spatial structures are extracted from the segmentation results of the alleyway environment image. These distinguishable structural feature points reflect important geometric information of the alleyway environment.

[0083] Furthermore, the extracted discriminative structural feature points are matched and associated with the previously obtained target visual structural features to form a feature set that can simultaneously reflect visual features and semantic information, called the semantic feature set, which provides a highly reliable feature basis for accurate trajectory correction.

[0084] In this embodiment of the invention, each feature point in the semantic feature set is mapped to the spatial location of the loop candidate region. By using a point-to-point correspondence and matching algorithm, feature points with similar positions and semantic attributes in the loop region are found to obtain a set of matching feature points, which provides a basis for the alignment of the trajectory with the loop region.

[0085] Furthermore, based on the positional differences of the matching feature points in the potential drift trajectory and the loop closure candidate region, the spatial consistency between the two is quantified, and the semantic matching degree is calculated to reflect the degree of semantic and spatial matching between the trajectory and the loop closure region. The calculation formula is as follows:

[0086]

[0087] in, Indicates semantic matching degree. This represents the total number of matching feature points. This represents the i-th matching feature point in the potential drift trajectory. This represents the i-th matching feature point in the i-th loop closure candidate region. This represents the scale parameter that controls the matching tolerance.

[0088] In this embodiment of the invention, the obtained semantic matching degree is used to analyze the deviations of the matched feature points in terms of position, orientation, and scale, generating deviation information for each feature point, reflecting the local error distribution of the potential drift trajectory relative to the loop area. Furthermore, the deviation information of each feature point is integrated to construct geometric constraints that reflect the consistency between the trajectory and the tunnel structure, including position, orientation, and spatial relationship constraints, for subsequent global optimization and trajectory correction, ensuring that the stacker crane's movement trajectory conforms to the physical geometry of the tunnel environment.

[0089] In this embodiment of the invention, the obtained semantic matching degree and geometric consistency information are integrated to establish a global constraint system that reflects the overall consistency of the trajectory and the spatial constraint relationship. Furthermore, the global constraint system is used to optimize and adjust potential drift trajectories. By correcting positional deviations and cumulative drift, the trajectory is made consistent with the tunnel structure and visual features, ultimately obtaining an accurate and reliable corrected trajectory for the safe operation and motion analysis of the stacker crane.

[0090] This invention significantly improves positioning accuracy and trajectory reliability by identifying and correcting the cumulative drift caused by sensor errors or low-texture, repetitive structures in the environment during the movement of a stacker crane in aisles. The corrected trajectory not only ensures the stable operation of the stacker crane in complex storage environments, but also provides an accurate motion basis for subsequent operation monitoring and collision prevention control.

[0091] S5. Perform operation action deviation monitoring on the correction trajectory to obtain deviation monitoring records.

[0092] In this embodiment of the invention, operational deviation monitoring is the process of detecting and analyzing the deviation between the actual movement trajectory and the expected operational trajectory of the stacker crane during storage and retrieval operations. The deviation monitoring record is a data set that records the operational deviation information detected during the stacker crane's operation.

[0093] In detail, the step of monitoring the deviation of the operation on the correction trajectory to obtain a deviation monitoring record includes:

[0094] The actual execution parameter set of the current fork extension, lifting position and lateral movement position of the target stacker crane is extracted based on the correction trajectory.

[0095] Obtain the planned position parameter set corresponding to the target stacker crane, and subtract the actual execution parameter set from the planned position parameter set to obtain the operation action deviation.

[0096] When the deviation of the operation is less than or equal to a preset first threshold, the normal safety zone is taken as the current operation state of the target stacker crane;

[0097] When the deviation of the operation action is greater than a preset first threshold and less than a preset second threshold, the warning buffer is taken as the current operation state of the target stacker.

[0098] When the deviation of the operation action is greater than or equal to a preset second threshold, the dangerous collision zone is taken as the current operation state of the target stacker crane;

[0099] The target stacker crane is monitored based on the deviation of the operation action and the current operation status, and a deviation monitoring record is generated.

[0100] In this embodiment of the invention, the specific motion information of the stacker crane at the current time point is obtained from the correction trajectory, including the extension length of the forks, the lifting height, and the position of the lateral movement. These parameters constitute the actual execution parameter set, reflecting the actual action state of the stacker crane in the aisle operation.

[0101] Furthermore, the pre-set set of planned position parameters of the stacker crane in the planned operation task is obtained, including the fork extension amount, lifting position and lateral movement position. The set of planned position parameters is compared with the set of actual execution parameters item by item to obtain the deviation of the operation action, which is used to quantify the deviation between the actual execution action and the planned action of the stacker crane.

[0102] In this embodiment of the invention, when the deviation of the operational action does not exceed a preset first threshold, the current operation of the stacker crane is considered to be within the normal range and highly safe, and is marked as a normal safety zone, indicating that the stacker crane's operating status is safe and reliable. When the deviation of the operational action exceeds the first threshold but is still less than the second threshold, it indicates that the stacker crane's operation has a certain deviation, but has not yet reached a dangerous level, and is marked as a warning buffer zone to alert operators or monitoring personnel to potential risks. When the deviation of the operational action reaches or exceeds the second threshold, the stacker crane's operation deviates severely and may cause a collision or operational accident, and is marked as a dangerous collision zone as a high-risk status warning to facilitate immediate safety measures.

[0103] Furthermore, by combining the deviation of the operation actions with the current operating status of the stacker crane, the stacker crane operation process is monitored in real time. Deviation data, status information, and abnormal warnings are recorded in the deviation monitoring record, providing reliable data support for operation safety assessment, anomaly analysis, and subsequent optimization.

[0104] This invention analyzes in real time the deviations between the actual and planned actions of the stacker crane during storage and retrieval operations, such as the fork extension, lifting position, and lateral movement position. This allows for timely identification of operational anomalies or potential risks. The deviation monitoring records not only provide quantitative data on the stacker crane's action accuracy and safety status, but also provide a basis for operation optimization, fault warning, and safety management.

[0105] S6. Based on the deviation monitoring records, collision prevention control is performed on the target stacker crane to obtain the target control result.

[0106] In this embodiment of the invention, collision prevention control is a process that uses deviation monitoring records to adjust and intervene in the movement trajectory and operation of the stacker crane in real time to avoid collisions with racks or obstacles. The target control result is the safe action or adjusted operation result performed by the stacker crane after collision prevention control, reflecting the effectiveness of the control measures.

[0107] In detail, the step of performing collision prevention control on the target stacker crane based on the deviation monitoring records to obtain the target control result includes:

[0108] The deviation trend of the target stacker crane is analyzed based on the deviation monitoring records to obtain the deviation trend characteristics;

[0109] Obtain the minimum safe distance between the target stacker crane and the rack obstacle, and generate the current distance between the target stacker crane and the rack obstacle based on the deviation trend characteristics and the correction trajectory;

[0110] When the current distance is less than the minimum safe distance, the preset safety control strategy library is invoked, and the speed suppression strategy and path fine-tuning strategy in the safety control strategy library are extracted;

[0111] Based on the speed suppression strategy and the path fine-tuning strategy, a preventive control command is generated;

[0112] The target stacker crane is collide-prevention is performed using the aforementioned prevention and control command to obtain the target control result.

[0113] In this embodiment of the invention, the data on the change of stacker crane movement deviation over time in the deviation monitoring record are used to analyze the development trend of the stacker crane movement deviation during operation, extract feature information reflecting the rate, direction and law of deviation change, and form deviation trend features.

[0114] Furthermore, by pre-setting a safe distance between the stacker crane and the rack or other obstacles, and combining deviation trend characteristics and correction trajectory information, the actual distance between the current position of the stacker crane and the rack obstacle is analyzed, reflecting the spatial relationship between the stacker crane and the obstacle in real time.

[0115] In this embodiment of the invention, when the actual distance is lower than the minimum safe distance, a potential collision risk is determined, and the safety control strategy library is automatically invoked to select a speed suppression strategy and a path fine-tuning strategy suitable for the current scenario to ensure the safe operation of the stacker crane.

[0116] In this embodiment of the invention, by combining the extracted speed suppression and path fine-tuning strategies, preventive control commands are generated that can control the speed of the stacker crane and fine-tune its movement path, for real-time intervention in the stacker crane's actions. Furthermore, the generated preventive control commands are sent to the stacker crane for execution, causing it to slow down, adjust its path, or stop during operation, thereby avoiding collisions with racks or obstacles. Ultimately, a safe and controllable target control result is obtained, ensuring the safety and stability of automated warehousing operations.

[0117] This invention analyzes the deviations and trends of stacker crane movements in real time, enabling early detection of potential collision risks. Based on safety control strategies, it adjusts the speed and path of the stacker crane in a timely manner, achieving proactive intervention in dangerous actions. This not only effectively prevents the stacker crane from colliding with shelves or other obstacles, but also improves the safety, reliability, and continuous operation efficiency of automated warehouses.

[0118] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0119] like Figure 2The diagram shown is a functional block diagram of a stacker crane intelligent positioning and safety control device provided in an embodiment of the present invention.

[0120] This disclosure provides a stacker crane intelligent positioning and safety control device, which corresponds one-to-one with the stacker crane intelligent positioning and safety control method described in the above embodiments. Figure 2 As shown, the intelligent positioning and safety control device 100 for a stacker crane can be installed in an electronic device. According to its functions, the intelligent positioning and safety control device 100 for a stacker crane includes an initial positioning identification module 101, a feature structure association module 102, a motion trajectory generation module 103, a trajectory drift correction module 104, a motion deviation monitoring module 105, and a collision prevention control module 106. Detailed descriptions of each functional module are as follows:

[0121] The initial positioning and recognition module 101 is used to acquire tunnel environment images using the depth camera of the target stacker crane, and to identify the basic visual features and initial positioning of the tunnel environment images;

[0122] The feature structure association module 102 is used to perform structural association on the basic visual features to obtain target visual structure features;

[0123] The motion trajectory generation module 103 is used to update the initial positioning based on the target visual structural features, and generate the motion trajectory of the target stacker crane using the updated positioning;

[0124] The trajectory drift correction module 104 is used to perform drift detection and global correction on the motion trajectory to obtain a corrected trajectory.

[0125] The motion deviation monitoring module 105 is used to monitor the work motion deviation of the correction trajectory and obtain a deviation monitoring record.

[0126] The collision prevention control module 106 is used to perform collision prevention control on the target stacker based on the deviation monitoring record, and obtain the target control result.

[0127] In one embodiment, the feature structure association module 102 performs structural association on the basic visual features to obtain target visual structural features, for the following purposes:

[0128] The prior layout relationship of the aisle space where the target stacker crane is located is generated based on the pre-acquired warehouse racking layout parameters and aisle geometry prior information;

[0129] The basic visual features are associated with the prior layout relationship to obtain visual structure association features;

[0130] Geometric consistency screening is performed on the visual structure association features to obtain the target visual structure features.

[0131] In one embodiment, the motion trajectory generation module 103 performs a positioning update based on the target visual structural features, and generates the motion trajectory of the target stacker crane using the updated positioning, for the following purposes:

[0132] Extract the acceleration and angular velocity of the target's visual structural features;

[0133] Using the acceleration and the acceleration, a motion prior analysis is performed on the initial positioning to obtain the target stacker crane's positioning to be updated;

[0134] The target visual structural features are mapped to the location to be updated, and error analysis is performed on the location to be updated based on the mapping results to obtain the reprojection error;

[0135] The structural prior constraints of the alleyway space are generated based on the prior layout relationship;

[0136] The reprojection error and the structural prior constraints are used to jointly optimize the localization to be updated, resulting in an updated localization.

[0137] Based on the initial positioning and the updated positioning, the target stacker crane is subjected to trajectory analysis to obtain the motion trajectory.

[0138] In one embodiment, the trajectory drift correction module 104 performs drift detection and global correction on the motion trajectory to obtain a corrected trajectory, for the following purposes:

[0139] The positioning difference between two adjacent frames in the motion trajectory is obtained one by one;

[0140] When the positioning difference is greater than a preset difference threshold, the motion trajectory is taken as a potential drift trajectory;

[0141] Based on the historical operating trajectory of the target stacker crane and the lane structure layout of the lane space obtained in advance, identify the loop candidate region of the movement trajectory;

[0142] Using the target visual structural features, semantic rematching is performed on the alleyway environment image and the loop candidate region of the potential drift trajectory to generate semantic matching degree and geometric consistency constraint information.

[0143] A global constraint system is constructed based on the semantic matching degree and the geometric consistency constraint information;

[0144] The potential drift trajectory is corrected according to the global constraint system to obtain the corrected trajectory.

[0145] In one embodiment, the trajectory drift correction module 104 performs semantic re-matching of the alleyway environment image and the loop candidate region of the potential drift trajectory using the target visual structural features, generating semantic matching degree and geometric consistency constraint information, for:

[0146] Semantic segmentation is performed on the alleyway environment image of the potential drift trajectory, and discriminative structural feature points are identified based on the segmentation results;

[0147] The discriminative structural feature points and the target visual structural features are associated to obtain a semantic feature set;

[0148] The semantic feature set and the loop closure candidate region are matched point by point to obtain a number of matching feature points;

[0149] Based on the positional correspondence of the matching feature points, the semantic matching degree between the potential drift trajectory and the loop closure candidate region is calculated;

[0150] Based on the semantic matching degree, a deviation analysis is performed on the matching feature points to obtain feature point deviation information;

[0151] Geometric consistency constraint information is generated based on the feature point deviation information.

[0152] In one embodiment, the motion deviation monitoring module 105 performs motion deviation monitoring on the correction trajectory and obtains a deviation monitoring record, which is used for:

[0153] The actual execution parameter set of the current fork extension, lifting position and lateral movement position of the target stacker crane is extracted based on the correction trajectory.

[0154] Obtain the planned position parameter set corresponding to the target stacker crane, and subtract the actual execution parameter set from the planned position parameter set to obtain the operation action deviation.

[0155] When the deviation of the operation is less than or equal to a preset first threshold, the normal safety zone is taken as the current operation state of the target stacker crane;

[0156] When the deviation of the operation action is greater than a preset first threshold and less than a preset second threshold, the warning buffer is taken as the current operation state of the target stacker.

[0157] When the deviation of the operation action is greater than or equal to a preset second threshold, the dangerous collision zone is taken as the current operation state of the target stacker crane;

[0158] The target stacker crane is monitored based on the deviation of the operation action and the current operation status, and a deviation monitoring record is generated.

[0159] In one embodiment, the collision prevention control module 106 performs collision prevention control on the target stacker crane based on the deviation monitoring records to obtain a target control result, for the following purposes:

[0160] The deviation trend of the target stacker crane is analyzed based on the deviation monitoring records to obtain the deviation trend characteristics;

[0161] Obtain the minimum safe distance between the target stacker crane and the rack obstacle, and generate the current distance between the target stacker crane and the rack obstacle based on the deviation trend characteristics and the correction trajectory;

[0162] When the current distance is less than the minimum safe distance, the preset safety control strategy library is invoked, and the speed suppression strategy and path fine-tuning strategy in the safety control strategy library are extracted;

[0163] Based on the speed suppression strategy and the path fine-tuning strategy, a preventive control command is generated;

[0164] The target stacker crane is collide-prevention is performed using the aforementioned prevention and control command to obtain the target control result.

[0165] In this invention, the specific limitations of the intelligent positioning and safety control device for a stacker crane can be found in the above-described limitations of the intelligent positioning and safety control method for a stacker crane, and will not be repeated here. Each module in the aforementioned intelligent positioning and safety control device for a stacker crane can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0166] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0167] The depth camera of the target stacker crane is used to acquire images of the tunnel environment, and the basic visual features and initial positioning of the tunnel environment images are identified.

[0168] By performing structural association on the basic visual features, the target visual structural features are obtained;

[0169] The initial positioning is updated based on the target visual structural features, and the updated positioning is used to generate the motion trajectory of the target stacker crane.

[0170] The motion trajectory is drift detected and globally corrected to obtain the corrected trajectory;

[0171] The work action deviation is monitored on the correction trajectory to obtain a deviation monitoring record;

[0172] Collision prevention control is performed on the target stacker based on the deviation monitoring records to obtain the target control result.

[0173] In the several embodiments provided by this invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0174] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0175] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0176] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0177] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.

[0178] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0179] The depth camera of the target stacker crane is used to acquire images of the tunnel environment, and the basic visual features and initial positioning of the tunnel environment images are identified.

[0180] By performing structural association on the basic visual features, the target visual structural features are obtained;

[0181] The initial positioning is updated based on the target visual structural features, and the updated positioning is used to generate the motion trajectory of the target stacker crane.

[0182] The motion trajectory is drift detected and globally corrected to obtain the corrected trajectory;

[0183] The work action deviation is monitored on the correction trajectory to obtain a deviation monitoring record;

[0184] Collision prevention control is performed on the target stacker based on the deviation monitoring records to obtain the target control result.

[0185] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0186] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0187] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0188] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0189] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0190] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0192] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0193] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for intelligent positioning and safety control of a stacker crane, characterized in that, The method includes: The depth camera of the target stacker crane is used to acquire images of the tunnel environment, and the basic visual features and initial positioning of the tunnel environment images are identified. Structural associations are performed on basic visual features to obtain the target visual structural features; Extract the acceleration and angular velocity from the stacker crane's motion data, perform motion prior analysis, and obtain the positioning to be updated; Calculate the reprojection error of the target visual structural features to the localization to be updated; Obtain structural prior constraints generated based on the prior layout of the alleyways; By integrating reprojection error and structural prior constraints, the localization to be updated is jointly optimized to obtain the updated localization. Based on the initial and updated positioning, trajectory analysis is performed on the target stacker crane to obtain its motion trajectory; The motion trajectory is drift detected and globally corrected to obtain the corrected trajectory; The work action deviation is monitored on the correction trajectory to obtain a deviation monitoring record; Collision prevention control is performed on the target stacker based on the deviation monitoring records to obtain the target control result.

2. The intelligent positioning and safety control method for stacker cranes as described in claim 1, characterized in that, The process of structurally associating basic visual features to obtain target visual structural features includes: Based on the warehouse racking layout parameters and aisle geometry prior information, the prior layout relationship of the aisle space where the target stacker crane is located is generated; By associating basic visual features with prior layout relationships, visual structure association features are obtained. Geometric consistency screening is performed on the visual structure association features to obtain the target visual structure features.

3. The intelligent positioning and safety control method for stacker cranes as described in claim 2, characterized in that, The motion trajectory is drift detected and globally corrected to obtain a corrected trajectory, including: Compare the positioning difference between adjacent frames in the motion trajectory; if it is greater than a threshold, it is determined to be a potential drift trajectory. Based on historical trajectories and alleyway layouts, identify candidate loop areas; By utilizing the visual structural features of the target, semantic re-matching is performed on potential drift trajectories and loop candidate regions to generate matching degree and geometric constraints; Global constraints are constructed based on the matching degree and geometric constraints to correct potential drift trajectories and obtain corrected trajectories.

4. The intelligent positioning and safety control method for stacker cranes as described in claim 3, characterized in that, The step of utilizing the visual structural features of the target to perform semantic re-matching of potential drift trajectories and loop closure candidate regions, generating matching scores and geometric constraints, includes: Semantic segmentation is performed on environmental images of potential drift trajectories to identify discriminative structural feature points; By associating the aforementioned feature points with the visual structural features of the target, a semantic feature set is constructed; The semantic feature set is matched with the loop closure candidate region to obtain the matching feature points; Calculate the semantic matching degree of the matching points, analyze the feature point deviation based on the matching degree, and generate geometric consistency constraint information.

5. The intelligent positioning and safety control method for stacker cranes as described in claim 1, characterized in that, The work action deviation is monitored on the correction trajectory to obtain a deviation monitoring record, including: The actual execution position parameters of the stacker crane forks and platform are obtained based on the calibration trajectory. Calculate the deviation between the actual execution location parameters and the planned location parameters; The current operation status is determined based on the comparison result between the deviation and the preset threshold. Based on the deviation and the operation status, a deviation monitoring record is generated.

6. The intelligent positioning and safety control method for stacker cranes as described in claim 1, characterized in that, The step of performing collision prevention control on the target stacker crane based on the deviation monitoring records to obtain the target control result includes: Analyze the deviation trends of work actions based on deviation monitoring records; Obtain the minimum safe distance, and combine the deviation trend with the correction trajectory to calculate the current distance between the stacker crane and the obstacle; If the current distance is less than the minimum safe distance, then the speed suppression and path fine-tuning strategy is invoked from the safety control strategy library; The strategy is used to generate and execute preventive control commands to obtain collision avoidance control results.

7. A stacker crane intelligent positioning and safety control device, characterized in that, The device includes: The initial positioning and recognition module is used to acquire images of the tunnel environment using the depth camera of the target stacker crane, and to identify the basic visual features and initial positioning of the tunnel environment images. The feature structure association module is used to perform structural association on basic visual features to obtain target visual structural features; The motion trajectory generation module is used to update the initial positioning based on the visual structural features of the target, and to generate the motion trajectory of the target stacker crane using the updated positioning. The trajectory drift correction module is used to perform drift detection and global correction on the motion trajectory to obtain a corrected trajectory; The motion deviation monitoring module is used to monitor the work motion deviation of the correction trajectory and obtain a deviation monitoring record; The collision prevention control module is used to perform collision prevention control on the target stacker crane based on the deviation monitoring records, and obtain the target control result.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a stacker crane intelligent positioning and safety control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a stacker crane intelligent positioning and safety control method as described in any one of claims 1 to 7.

Images