Linkage cooperative monitoring method and system and intelligent inspection robot

Abstract

The invention belongs to the field of retired lithium batteries, and discloses a linkage cooperative monitoring method and system and an intelligent inspection robot, and the method comprises the steps that a mobile terminal constructs a three-dimensional map through a synchronous positioning and map construction technology, and synchronously collects a monitoring picture of a static terminal; identifying common-view feature points in the three-dimensional map and the monitoring picture, and calculating a conversion matrix of coordinate systems of the static end and the mobile end through a matrix conversion algorithm; establishing a calibration error compensation model, and iteratively optimizing calibration parameters by using multi-view data of the mobile terminal; and the mobile terminal and / or the static terminal detects an abnormal event and autonomously triggers an alarm so as to realize monitoring linkage response. According to the invention, the problem of information isolated island in a traditional monitoring mode is thoroughly solved, the monitoring blind area is effectively eliminated, and the omnibearing coverage of the monitoring range is realized.

Description

Technical Field

[0001] The present invention relates to the field of retired lithium batteries, and particularly to a linkage collaborative monitoring method, system and intelligent inspection robot. Background Art

[0002] At present, the inspection work in retired lithium battery warehouses mainly relies on manual operation, which has many drawbacks. Manual inspection is inefficient, and the missed inspection rate is high. Moreover, retired lithium batteries themselves have potential risks such as flammability, smoke generation and toxic gas release, which pose a high safety threat to manual inspection. Therefore, it is particularly urgent to develop a robot dedicated to inspecting retired lithium battery warehouses to replace manual inspection. The existing warehouse safety monitoring systems mainly adopt the technical scheme of independent operation of "fixed warehouse monitoring cameras + inspection robots", but this scheme has the following significant deficiencies: 1. Information island problem: Data cannot be interchanged between the fixed warehouse monitoring cameras and the inspection robots. After the camera detects an abnormality, it is necessary to manually dispatch the robot, resulting in a long response delay and seriously affecting the emergency handling efficiency.

[0003] 2. Monitoring coverage blind area: Fixed cameras have monitoring dead angles, and the inspection paths of robots are fixed, so there are also blind areas during the inspection process, and they cannot dynamically respond to key areas, resulting in ineffective monitoring of some key areas.

[0004] 3. High false alarm rate: The warehouse area is large, and the monitoring distance of fixed cameras is far, so it is easy to cause false alarms due to factors such as environmental interference, increasing the unnecessary alarm handling cost.

[0005] 4. One-way trigger mechanism: Only a single terminal of the robot or the camera is supported to initiate an alarm, lacking a collaborative mechanism and unable to achieve multi-terminal linkage alarm, reducing the overall reliability of the system.

[0006] Therefore, how to provide a linkage collaborative monitoring method, system and intelligent inspection robot is an urgent problem to be solved at present. Summary of the Invention

[0007] Embodiments of the present invention provide a linkage collaborative monitoring method to solve the above technical problems in the prior art.

[0008] To provide a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary part is not a general review, nor is it intended to identify key / important constituent elements or delineate the protection scope of these embodiments. Its sole purpose is to present some concepts in a simple form as a preamble to the subsequent detailed description.

[0009] According to the first aspect of the embodiments of the present invention, a linkage collaborative monitoring method is provided.

[0010] In one embodiment, the linkage and collaborative monitoring method includes: The mobile device constructs a three-dimensional map through simultaneous localization and mapping technology and synchronously collects the monitoring images of the stationary device; Identify the co-visible feature points in the three-dimensional map and the monitoring images, and calculate the transformation matrix between the coordinate systems of the stationary device and the mobile device through the matrix transformation algorithm; Establish a calibration error compensation model and use the multi-view data of the mobile device to iteratively optimize the calibration parameters; When an abnormal event is detected by the mobile device and / or the stationary device, an alarm is triggered autonomously to achieve a monitoring linkage response.

[0011] In one embodiment, the mobile device constructs a three-dimensional map through simultaneous localization and mapping technology, including: Obtain the pose matrix of the mobile device in the world coordinate system; Extract the physical feature points in the three-dimensional map and record the world coordinates of the mobile device.

[0012] In one embodiment, after synchronously collecting the monitoring images of the stationary device, it further includes: Extract two-dimensional feature points through the feature detection algorithm; Identify fixed markers by combining geometric constraints or semantic segmentation.

[0013] In one embodiment, the step of identifying the co-visible feature points in the three-dimensional map and the monitoring images, and calculating the transformation matrix between the coordinate systems of the stationary device and the mobile device through the matrix transformation algorithm includes: Based on the pose matrix and the initial external parameters, estimate the external parameter matrix of the mobile device, and project the three-dimensional feature points in the world coordinates of the mobile device onto the image plane; Construct a nearest neighbor matching model between the projected points and the image features, and screen the coordinate matching point pairs with the reprojection error lower than the preset threshold; Based on the coordinate matching point pairs of the three-dimensional feature points and the two-dimensional feature points, optimize the matrix transformation algorithm; Use the optimized matrix transformation algorithm to solve the rotation vector and the translation vector, and convert them into an external parameter matrix of a preset specification to obtain the transformation function of the transformation matrix.

[0014] In one embodiment, after the step of identifying the co-visible feature points in the three-dimensional map and the monitoring images, and calculating the transformation matrix between the coordinate systems of the stationary device and the mobile device through the matrix transformation algorithm, it further includes: Iteratively optimize the random sample consensus model and verify the inliers; Jointly solve using the matching data under different pose matrices; Evaluate the confidence of the external parameter estimation through the covariance matrix.

[0015] In one embodiment, the iterative optimization of the random sampling point set solving model and the verification of inliers further include: An external parameter estimation model for the stationary end established by a matrix transformation algorithm, and verifying whether it satisfies the current estimation of the random sampling point set solving model, and coordinate matching point pairs with a reprojection error less than the threshold.

[0016] In one embodiment, the establishment of a calibration error compensation model and the iterative optimization of calibration parameters using multi-view data of the mobile terminal include: Constructing a calibration error compensation model based on multi-source residuals, simultaneously defining a set of calibration parameters to be optimized, and performing staged non-linear optimization, Performing iterative optimization through the random sampling point set solving model, and performing convergence determination and dynamic recalibration.

[0017] In one embodiment, when the mobile terminal and / or the stationary end detect an abnormal event, an alarm is triggered autonomously to achieve monitoring linkage response, including: If the stationary end detects an abnormal event, the control end receives the coordinate signal and sends it to the mobile terminal; the mobile terminal automatically plans a path and navigates to the vicinity of the abnormal point found by the stationary end for investigation; the mobile terminal uploads the monitoring screen to the cloud platform to form a linkage monitoring; If the mobile terminal detects an abnormal event, the alarm mechanism of the mobile terminal itself is triggered; the control end receives the coordinate signal and sends it to the stationary end, and the stationary end automatically pops up a window and triggers an alarm; the alarm mechanisms of the mobile terminal and the stationary end work together to form a multi-view linkage monitoring.

[0018] According to the second aspect of the embodiments of the present invention, a linkage collaborative monitoring system is provided.

[0019] In one embodiment, the linkage collaborative monitoring system includes: A map construction module for the mobile terminal to construct a three-dimensional map through simultaneous localization and mapping technology and synchronously collect the monitoring screen of the stationary end; A matrix transformation module for identifying the co-visible feature points in the three-dimensional map and the monitoring screen, and calculating the transformation matrix of the coordinate systems of the stationary end and the mobile terminal through a matrix transformation algorithm; A parameter calibration module for establishing a calibration error compensation model and iteratively optimizing calibration parameters using multi-view data of the mobile terminal; An alarm trigger module for the mobile terminal and / or the stationary end to detect an abnormal event and autonomously trigger an alarm to achieve monitoring linkage response.

[0020] In some embodiments, the intelligent inspection robot includes: A wheeled base; A housing disposed at the top of the wheeled base; An alarm and a gas sensor are symmetrically arranged on one side of the top of the housing; A lidar is arranged on the other side of the top of the housing; A thermal imaging dual-spectrum pan-tilt is arranged on the top of the housing and is located between the lidar, the alarm and the gas sensor; An automatic charging current collecting device is arranged at one end of the wheeled base; A depth camera is arranged at the end of the housing.

[0021] The technical solution provided by the embodiment of the present invention may include the following beneficial effects: 1) Improve inspection efficiency and safety guarantee: The mobile terminal can realize round-the-clock uninterrupted inspection operations, and accurately detect various potential hazards with high-frequency and high-precision detection standards. More importantly, it can replace manual labor to enter high-risk areas, effectively avoiding the risk of harm to personnel caused by harsh environments such as high temperature, smoke, and toxic gases, and building a comprehensive safety defense line in all directions.

[0022] 2) Break the information barrier and eliminate monitoring blind spots: By means of the intelligent linkage mechanism between the stationary end and the mobile end of the warehouse, seamless sharing of data between the two parties is realized, completely breaking the information island. This linkage mode significantly reduces the monitoring blind spots, ensuring that the monitoring scope is free of dead spots and fully covered, providing all-round and three-dimensional data support for the safety management of the warehouse.

[0023] 3) Achieve cost optimization: Replace traditional manual inspections with mobile terminals, significantly reducing the investment in labor costs. At the same time, through the coordinated cooperation between the mobile end and the stationary end, the deployment requirements for high-density stationary ends can be reduced, further optimizing the equipment cost, and achieving a double improvement in economic benefits and safety management.

[0024] 4) The present invention not only completely solves the information island problem in the traditional monitoring mode, but also effectively eliminates the monitoring blind spots, achieving full coverage of the monitoring scope. Through the coordinated cooperation between the mobile end and the stationary end, a comprehensive and three-dimensional intelligent inspection system is constructed, providing a solid and reliable guarantee for the safety management of retired lithium battery warehouses, and comprehensively protecting the safe and stable operation of the warehouses.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0027] Figure 1 is a flowchart of a linkage and collaborative monitoring method shown according to an exemplary embodiment; Figure 2 It is a structural block diagram of an interlocking collaborative monitoring system shown according to an exemplary embodiment; Figure 3 It is a right view of an intelligent inspection robot shown according to an exemplary embodiment; Figure 4 It is a rear view of an intelligent inspection robot shown according to an exemplary embodiment; Figure 5 It is a left view of an intelligent inspection robot shown according to an exemplary embodiment; Figure 6 It is a front view of an intelligent inspection robot shown according to an exemplary embodiment; Figure 7 It is a top view of an intelligent inspection robot shown according to an exemplary embodiment; Figure 8 It is a bottom view of an intelligent inspection robot shown according to an exemplary embodiment. Detailed implementation manners

[0028] The following description and drawings fully illustrate the specific implementation manners herein, enabling those skilled in the art to practice them. Parts and features of some embodiments may be included in or substituted for parts and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents of the claims. In this document, terms such as "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a structure, device or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such structure, device or equipment. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the structure, device or equipment including the said element. The various embodiments herein are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0029] In this document, the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing this document and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In the description of this document, unless otherwise specified and defined, the terms "mounted", "connected", and "coupled" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the communication inside two components. It can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0030] In this document, unless otherwise stated, the term "plurality" means two or more.

[0031] In this document, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0032] In this document, the term "and / or" is an associative relationship describing an object, indicating that there can be three relationships. For example, A and / or B means: A or B, or, the three relationships of A and B.

[0033] It should be understood that although each step in the flowchart is displayed sequentially according to the indication of the arrow, these steps are not necessarily executed sequentially in the order indicated by the arrow. Unless there is a clear description in this document, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0034] Each module in the device or system of this application can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0035] Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0036] Figure 1An embodiment of a linkage collaborative monitoring method of the present invention is shown.

[0037] In this alternative embodiment, the linkage collaborative monitoring method includes: Step S101, the mobile terminal constructs a three-dimensional map through synchronous positioning and map construction technology, and synchronously collects the monitoring images of the stationary terminal; Specifically, a robot (i.e., the mobile terminal) constructs a three-dimensional map of a warehouse through laser SLAM (i.e., synchronous positioning and map construction technology), and synchronously collects the images of a fixed camera (i.e., the stationary terminal); Step S102, identify the co-visible feature points in the three-dimensional map and the monitoring images, and calculate the transformation matrix of the coordinate systems of the stationary terminal and the mobile terminal through a matrix transformation algorithm; Specifically, identify the co-visible feature points (such as shelf corners and fire signs) in the field of view of the fixed camera, and calculate the transformation matrix between the camera and the robot coordinate systems through the PnP algorithm (i.e., the matrix transformation algorithm); Step S103, establish a calibration error compensation model, and use the multi-view data of the mobile terminal to iteratively optimize the calibration parameters; Step S104, when an abnormal event is detected by the mobile terminal and / or the stationary terminal, an alarm is triggered autonomously to achieve a monitoring linkage response.

[0038] In one embodiment, the construction of the three-dimensional map by the mobile terminal through synchronous positioning and map construction technology includes: Obtain the pose matrix of the mobile terminal in the world coordinate system; Extract the physical feature points in the three-dimensional map and record the world coordinates of the mobile terminal.

[0039] Specifically, obtain in real time the pose matrix T world→robot =[R∣t] of the robot output by laser SLAM in the world coordinate system, where R represents the rotation matrix and t represents the translation vector; Extract the significant physical feature points (such as shelf corners and columns) in the current laser scan, and record their world coordinates P world ={p i ∈R 3 ∣i = 1,…,n}, where p i represents the three-dimensional coordinates of the i-th significant physical feature point extracted by laser SLAM from the environment, and n represents the total number of feature points extracted in a single-frame scan of laser SLAM; In one embodiment, after synchronously collecting the monitoring images of the stationary terminal, it further includes: Extract two-dimensional feature points through a feature detection algorithm; Identify fixed markers in combination with geometric constraints or semantic segmentation.

[0040] Specifically, synchronously collect image frames, and extract 2D feature (i.e., two-dimensional feature) points P through a feature detection algorithm (ORB / SIFT). image ={q j ∈R 2 ∣j = 1,…,m}, where q j represents the pixel coordinates of the j-th 2D feature point extracted by the camera from the image, and m represents the total number of 2D feature points extracted from a single-frame image; Identify fixed markers by combining geometric constraints or semantic segmentation to enhance feature reliability; In one embodiment, identifying the co-visible feature points in the 3D map and the monitoring screen, and calculating the transformation matrix between the static end and the mobile end coordinate systems through a matrix transformation algorithm includes: Estimate the external parameter matrix of the mobile end based on the pose matrix and the initial external parameters, and project the 3D feature points in the world coordinates of the mobile end onto the image plane; Construct a nearest neighbor matching model between the projected points and the image features, and screen the coordinate matching point pairs with a reprojection error lower than a preset threshold; Optimize the matrix transformation algorithm based on the coordinate matching point pairs of the 3D feature points and the 2D feature points; Use the optimized matrix transformation algorithm to solve the rotation vector and the translation vector, and convert them into an external parameter matrix of a preset specification to obtain the transformation function of the transformation matrix.

[0041] Specifically, based on the robot pose matrix T world→robot and the initial external parameter estimate T cam→world , project the laser feature point P world onto the image plane: ; where K is the camera internal parameter matrix, is the projection function with distortion correction, projected_points represents the set of 2D pixel coordinates obtained by projecting the 3D feature points (in the world coordinate system) detected by the lidar onto the camera image plane, R init represents the initial rotation matrix, and t init represents the initial translation vector; Construct a nearest neighbor matching model between the projected points and the image features (accelerated by KD-Tree), and screen the matching pairs with a reprojection error lower than the threshold δ: ; where matched_pairs represents the set of corresponding relationships between the 3D environmental feature points (p i ) detected by laser SLAM and the 2D image feature points (q j ) extracted by the camera, and is used to establish cross-sensor coordinate association; Using the matched 3D-2D point pairs (P world , P image ), construct the PnP (i.e., matrix transformation algorithm) optimization: ; Among them, k represents the camera internal parameter matrix (3×3), which projects the 3D points in the camera coordinate system onto the image plane, represents the square of the Euclidean distance, and calculates the error between the projected point and the actual image point; Adopt the EPnP algorithm (i.e., the optimized matrix transformation algorithm) to solve the rotation vector and the translation vector t, and convert them into a 4×4 external parameter matrix: ; Among them, is the conversion function from the rotation vector to the matrix, R rod represents the Rodrigues rotation matrix, and t represents the position of the camera origin in the world coordinate system; In one embodiment, after identifying the co-visible feature points in the three-dimensional map and the monitoring screen, and calculating the conversion matrix between the stationary end and the mobile end coordinate systems through the matrix transformation algorithm, it further includes: Iteratively optimize the random sampling point set solution model and verify the inliers; Adopt the matching data under different pose matrices for joint solution; Evaluate the confidence of the external parameter estimation through the covariance matrix.

[0042] Specifically, adopt RANSAC (Random Sample Consensus) to iteratively optimize, the random sampling point set solution model and verify the inliers: Collect the matching data under K different poses for joint solution to improve the calibration accuracy; Evaluate the confidence of the external parameter estimation through the covariance matrix (J is the Jacobian matrix).

[0043] In one embodiment, the iteratively optimizing the random sampling point set solution model and verifying the inliers further includes: Establish an external parameter estimation model of the stationary end through the matrix transformation algorithm, and verify whether it satisfies the current estimated random sampling point set solution model and the coordinate matching point pairs with the reprojection error less than the threshold.

[0044] Specifically, the model refers to the camera external parameter estimation model established through the PnP algorithm, that is, the projection transformation relationship from the 3D laser feature points to the 2D image features; Inliers: The matching point pairs that satisfy the current estimation model and have a reprojection error less than the threshold .

[0045] In one embodiment, establishing the calibration error compensation model and using the multi-view data of the mobile device to iteratively optimize the calibration parameters includes: Constructing a calibration error compensation model based on multi-source residuals, defining a set of calibration parameters to be optimized, and performing staged non-linear optimization, Iteratively optimizing by solving the model with a randomly sampled point set, and performing convergence determination and dynamic recalibration.

[0046] Specifically, 1) Multi-source residual modeling and parameter definition; Residual construction: Define the pose-related reprojection residual: ; Wherein, p ij represents the j-th laser 3D feature point collected under the i-th robot pose, q ij represents the image 2D feature point matched with p ij ; Define the time synchronization residual:

[0047] Wherein, T interp represents the continuous pose estimation function generated by spline interpolation; Set of parameters to be optimized: X = { T cam→world , δT 1,…, δT N , Δ t}; T cam→world : 6-DOF camera extrinsic parameters; Pose correction amount; Δ t : Sensor delay parameter; 2) Staged non-linear optimization Adopt a progressive optimization strategy to ensure convergence stability: a. The first stage (initial adjustment of extrinsic parameters): Fix δT i = 0, Δ t = 0; Single-objective optimization of camera extrinsic parameters T cam→world ; b. The second stage (joint pose optimization): Release pose correction amount δT i ; Optimization objective:

[0048] Among them, represents the reprojection residual; c. Third stage (full parameter optimization): Introduce the time delay parameter Δ t； Use spline interpolation to achieve continuous pose estimation: 3). Implementation of the robust optimization engine Core solver: Construct an optimization problem based on Ceres Solver; Use sparse Cholesky decomposition to accelerate the solution; Enhancement of key technologies: The Huber kernel function suppresses the influence of outliers; Adaptive regularization (dynamic adjustment of the coefficient λ); Covariance weighting: Set Σij according to the confidence of feature point detection; Among them, Σij represents the covariance matrix of observation uncertainty; 4). Convergence determination and dynamic recalibration; Convergence conditions (both satisfied): Average reprojection error < 0.5 pixels; Parameter change amount

[0049] Number of iterations ≥ 100.

[0050] In one embodiment, when the mobile terminal and / or the stationary terminal detects an abnormal event, it autonomously triggers an alarm to implement the monitoring linkage response, including: If the stationary terminal detects an abnormal event, the control terminal receives the coordinate signal and sends it to the mobile terminal; the mobile terminal automatically plans a path and navigates to the vicinity of the abnormal point found by the stationary terminal for investigation; the mobile terminal uploads the monitoring screen to the cloud platform to form a linkage monitoring; If the mobile terminal detects an abnormal event, it triggers its own alarm mechanism; the control terminal receives the coordinate signal and sends it to the stationary terminal, and the stationary terminal automatically pops up a window and triggers an alarm; the alarm mechanisms of the mobile terminal and the stationary terminal work together to form a multi-perspective linkage monitoring.

[0051] Specifically, the event-triggered linkage mechanism includes: 1) Trigger condition: The fixed camera detects an abnormal event (such as smoke, temperature rise, open fire, etc.); Response process: Step 1: The fixed camera detects an anomaly; Step 2: The central control module receives the coordinates and sends them to the patrol robot; Step 3: The patrol robot automatically constructs a path through navigation and goes to the vicinity of the anomaly point detected by the fixed camera for investigation; Step 4: The robot uploads the video footage to the cloud platform to form a linkage monitoring; 2) Trigger condition: The patrol robot pan-tilt head detects an abnormal event (such as smoke, temperature rise, open fire, etc.); Response process: 1. Anomaly detection and alarm: After the patrol robot pan-tilt head detects an abnormal event, it immediately triggers the local alarm mechanism; 2. Information transmission and scheduling: After the central control module receives the alarm signal and its precise coordinates from the patrol robot, it quickly sends the relevant information to the fixed camera monitoring room; 3. Linkage response: The fixed camera near the patrol robot automatically pops up and triggers an alarm, notifying the relevant personnel in the monitoring room in a timely manner; 4. Linkage monitoring: The alarm mechanisms of the patrol robot and the fixed camera monitoring room work together to form a multi-perspective linkage monitoring combining near and far, providing comprehensive on-site information support for emergency response.

[0052] Figure 2 An embodiment of a linkage collaborative monitoring system of the present invention is shown.

[0053] In one embodiment, the linkage collaborative monitoring system includes: A map construction module 201, configured to construct a three-dimensional map through simultaneous localization and mapping technology on the mobile end and synchronously collect the monitoring images of the static end; A matrix conversion module 202, configured to identify the co-visible feature points in the three-dimensional map and the monitoring images, and calculate the conversion matrix between the coordinate systems of the static end and the mobile end through a matrix conversion algorithm; A parameter calibration module 203, configured to establish a calibration error compensation model and iteratively optimize the calibration parameters by using the multi-perspective data of the mobile end; A trigger alarm module 204, configured to detect an abnormal event on the mobile end and / or the static end and independently trigger an alarm to achieve a monitoring linkage response.

[0054] Figures 3 - 8 An embodiment of an intelligent patrol robot of the present invention is shown.

[0055] In one embodiment, the intelligent patrol robot includes: A wheeled base 1; A housing 2, disposed at the top of the wheeled base 1; An alarm 3 and a gas sensor 4 are symmetrically arranged on one side of the top of the housing 2; A lidar 5 is arranged on the other side of the top of the housing 2; A thermal imaging dual-spectrum pan-tilt 6 is arranged on the top of the housing 2 and is located between the lidar 5, the alarm 3 and the gas sensor 4; An automatic charging current collection device 7 is arranged at one end of the wheeled base 1; A depth camera 8 is arranged at the end of the housing 2.

[0056] Specifically, the intelligent inspection robot of the present invention is specifically aimed at retired lithium batteries: 1. Retired lithium batteries have the following characteristics in terms of safety: a. Thermal runaway risk: The internal structure of the retired battery is aged, and its thermal stability becomes poor. It is easy to cause thermal runaway under overcharge, over-discharge or high-temperature environments, resulting in battery fire and explosion; b. Short-circuit risk: The electrical connection of the retired battery may have problems such as breakage, corrosion, and looseness, which are likely to cause short circuits and lead to safety accidents; c. Poor consistency: After long-term use of the retired battery, the consistency of each battery in the battery pack becomes poor, and some batteries may be overcharged or over-discharged, increasing the safety risk; d. Electrolyte leakage: The electrolyte of the retired battery contains harmful substances such as fluorides. If it leaks, it will cause harm to the environment and human health; The detection methods for potential risks of lithium batteries include: a. Abnormal temperature rise: When the battery temperature exceeds the normal operating range (generally 60°C) and continues to rise, there may be a safety risk; b. Temperature change rate: If the battery temperature rises rapidly within a short period of time, such as the temperature rise rate ≥ 1°C / s and lasts for a certain period of time (such as 2 seconds), it indicates that a violent chemical reaction may be occurring inside the battery, and the risk of thermal runaway is extremely high; c. Combustible gas detection: During the use or testing of the battery, if flammable gases mainly composed of hydrogen, methane, ethylene, carbon monoxide, etc. are detected, this is usually a signal that the battery thermal runaway is about to occur or has already occurred.

[0057] Based on the characteristics of the above retired lithium batteries, specific gas sensors and thermal imaging dual-spectrum pan-tilts are accurately selected, and thresholds are set for key parameters to achieve accurate and timely detection of potential risks of retired lithium batteries, providing a reliable guarantee for safety monitoring.

[0058] 2. Heterogeneous camera automatic calibration technology: Through the dynamic matching of the SLAM map and visual feature points, online calibration of heterogeneous cameras at the minute level is achieved.

[0059] 3. Interlocking monitoring mechanism: Establish an interlocking monitoring mechanism for fixed cameras in the warehouse and inspection robots to break data silos, achieve data sharing, and effectively eliminate monitoring blind spots.

[0060] 4. Two-way dynamic trigger mechanism: Break the traditional master-slave architecture and adopt a two-way dynamic trigger mechanism. Both the robot and the camera can be used as trigger sources, and the dynamic allocation of tasks is realized through the central control platform, improving the system response speed and collaborative efficiency.

[0061] The interlocking and collaborative monitoring system equipped with this robot realizes the real-time interaction and collaborative response of monitoring data by deeply linking the thermal imaging dual-spectrum pan-tilt on the robot with the fixed cameras in the warehouse. When the robot first discovers a danger during the inspection process, it can not only trigger an alarm independently but also immediately link with the warehouse cameras, causing the warning information to automatically pop up on the operation interface of the monitoring room and quickly focus on the danger area. Conversely, if the warehouse camera first captures an abnormal situation, it will immediately activate the interlocking mechanism and quickly notify the inspection robot to rush to the vicinity of the danger site at the fastest speed to conduct on-site investigation and detailed assessment.

[0062] With the help of this efficient interlocking mechanism, not only the information silo problem in the traditional monitoring mode is completely solved, but also the monitoring blind spots are effectively eliminated, achieving full coverage of the monitoring scope. Through the coordinated cooperation of the robot and the camera, a comprehensive and three-dimensional intelligent inspection system is built, providing a solid and reliable guarantee for the safety management of the retired lithium battery warehouse and comprehensively protecting the safe and stable operation of the warehouse.

[0063] The present invention is not limited to the structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A linkage and collaborative monitoring method, characterized in that, Including: The mobile terminal constructs a three-dimensional map through simultaneous localization and mapping technology and synchronously collects the monitoring images of the stationary terminal; Identify the co-visible feature points in the three-dimensional map and the monitoring images, and calculate the transformation matrix between the coordinate systems of the stationary terminal and the mobile terminal through the matrix transformation algorithm; Establish a calibration error compensation model and use the multi-view data of the mobile terminal to iteratively optimize the calibration parameters; When the mobile terminal and / or the stationary terminal detects an abnormal event, an alarm is triggered automatically to achieve monitoring linkage response.

2. The linkage and collaborative monitoring method according to claim 1, wherein The construction of the three-dimensional map by the mobile terminal through simultaneous localization and mapping technology includes: Obtain the pose matrix of the mobile terminal in the world coordinate system; Extract the physical feature points in the three-dimensional map and record the world coordinates of the mobile terminal.

3. The linkage and collaborative monitoring method according to claim 1, characterized in that, After synchronously collecting the monitoring images of the stationary terminal, it further includes: Extract two-dimensional feature points through the feature detection algorithm; Identify the fixed markers by combining geometric constraints or semantic segmentation.

4. The linkage and collaboration monitoring method according to claim 1, wherein The identification of the co-visible feature points in the three-dimensional map and the monitoring images, and the calculation of the transformation matrix between the coordinate systems of the stationary terminal and the mobile terminal through the matrix transformation algorithm include: Based on the pose matrix and the initial external parameters, estimate the external parameter matrix of the mobile terminal, and project the three-dimensional feature points in the world coordinates of the mobile terminal onto the image plane; Construct the nearest neighbor matching model between the projection points and the image features, and screen the coordinate matching point pairs with the reprojection error lower than the preset threshold; Based on the coordinate matching point pairs of the three-dimensional feature points and the two-dimensional feature points, optimize the matrix transformation algorithm; Use the optimized matrix transformation algorithm to solve the rotation vector and the translation vector, and convert them into the external parameter matrix of the preset specification to obtain the transformation function of the transformation matrix.

5. The linkage and collaborative monitoring method according to claim 1, wherein After the identification of the co-visible feature points in the three-dimensional map and the monitoring images, and the calculation of the transformation matrix between the coordinate systems of the stationary terminal and the mobile terminal through the matrix transformation algorithm, it further includes: Iteratively optimize the random sample consensus solution model and verify the inliers; Jointly solve using the matching data under different pose matrices; Evaluate the confidence of the external parameter estimation through the covariance matrix.

6. The linkage and collaboration monitoring method according to claim 5, wherein The iterative optimization of the random sample consensus solution model and the verification of the inliers further include: Establish the external parameter estimation model of the stationary terminal through the matrix transformation algorithm, and verify whether it satisfies the current estimated random sample consensus solution model and the coordinate matching point pairs with the reprojection error less than the threshold.

7. The linkage and collaborative monitoring method according to claim 1, wherein The establishment of the calibration error compensation model and the use of the multi-view data of the mobile terminal to iteratively optimize the calibration parameters include: Construct the calibration error compensation model based on the multi-source residuals, and at the same time define the set of calibration parameters to be optimized and perform staged non-linear optimization; Perform iterative optimization through the random sample consensus solution model, and perform convergence determination and dynamic recalibration.

8. The linkage and collaborative monitoring method according to claim 1, wherein, The detection of an abnormal event by the mobile terminal and / or the stationary terminal, and the automatic triggering of an alarm to achieve monitoring linkage response includes: If the stationary terminal detects an abnormal event, the control terminal receives the coordinate signal and sends it to the mobile terminal; the mobile terminal automatically plans the path and navigates to the vicinity of the abnormal point found by the stationary terminal for investigation; the mobile terminal uploads the monitoring image to the cloud platform to form a linkage monitoring; If an abnormal event is detected on the mobile device, trigger the alarm mechanism of the mobile device itself; the control end receives the coordinate signal and sends it to the stationary end, and the stationary end automatically pops up a window and triggers an alarm; the alarm mechanisms of the mobile device and the stationary end work together to form a multi-perspective linkage monitoring.

9. A linkage and collaborative monitoring system, characterized in that, This linkage collaborative monitoring system includes: A map construction module, which is used for the mobile device to construct a three-dimensional map through simultaneous localization and mapping technology and synchronously collect the monitoring images of the stationary end; A matrix conversion module, which is used to identify the co-visible feature points in the three-dimensional map and the monitoring images, and calculate the conversion matrix of the coordinate systems of the stationary end and the mobile device through the matrix conversion algorithm; A parameter calibration module, which is used to establish a calibration error compensation model and use the multi-perspective data of the mobile device to iteratively optimize the calibration parameters; A trigger alarm module, which is used for the mobile device and / or the stationary end to detect an abnormal event and autonomously trigger an alarm to achieve a monitoring linkage response.

10. An intelligent inspection robot for implementing the linkage and collaborative monitoring operation of the linkage and collaborative monitoring method described in any one of claims 1-8, characterized in that, Including: A wheeled base; A housing, which is arranged at the top of the wheeled base; An alarm and a gas sensor, which are symmetrically arranged on one side of the top of the housing; A lidar, which is arranged on the other side of the top of the housing; A thermal imaging dual-spectrum pan-tilt, which is arranged on the top of the housing and is located between the lidar, the alarm and the gas sensor; An automatic charging power collection device, which is arranged at one end of the wheeled base; A depth camera, which is arranged at the end of the housing.

Images