Method for detecting integrity of frame line of logistics park based on quadruped robot vision

Abstract

This invention relates to the technical field of data analysis, specifically to a method for visually inspecting the integrity of frame lines in a logistics park using a quadruped robot. The method includes: acquiring multiple frame line detection samples from the quadruped robot during a target monitoring period, along with feature information for each frame line detection sample; distinguishing between reference samples and samples to be corrected among the multiple frame line detection samples; based on the pose of the frame line detection samples in the world coordinate system and the detection time, associating and matching the multiple reference samples with the multiple samples to be corrected to determine multiple sample matching pairs from the multiple frame line detection samples; in the multiple sample matching pairs, correcting the pose of the corresponding sample to be corrected based on the pose of the reference sample in each sample matching pair to obtain multiple frame line target samples; and performing frame line integrity detection based on the multiple frame line target samples to obtain the detection result. This invention can improve the accuracy and reliability of frame line detection results.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and specifically to a method for visually inspecting the integrity of logistics park outlines using a quadruped robot. Background Technology

[0002] With the widespread adoption of automated logistics technology, the integrity of road markings (such as AGV guide lines and warehouse location boundaries) in logistics parks directly impacts the operational efficiency and safety of warehousing systems. Currently, using mobile robots equipped with vision sensors for automated inspection has become a mainstream trend. Compared to wheeled robots, quadruped robots, with their superior obstacle-crossing capabilities, can adapt to discontinuous road surfaces such as speed bumps and cable trenches that may exist in logistics parks, thus demonstrating unique advantages in complex inspection scenarios.

[0003] However, when a quadruped robot performs a typical trot gait, its body inevitably generates periodic pitch and vertical vibrations with a frequency of approximately 2Hz to 3Hz. This high-frequency attitude change causes real-time fluctuations in the angle of the rigidly connected camera optical axis relative to the ground. In low-cost inspection scenarios lacking high-precision external absolute positioning systems (such as RTK-GNSS or high-beam LiDAR SLAM), when relying solely on vision or odometry for position estimation, the aforementioned body pitch causes significant back-and-forth drift in the projected coordinates of the same ground physical point across multiple consecutive frames. This projection error is non-Gaussian and periodic, making it difficult for traditional mean filtering algorithms based on multi-frame overlay to effectively align the observation data. This leads to divergence in line width measurement data, making it difficult to accurately distinguish between the actual physical wear of the marking lines and the observation noise caused by body sway. Summary of the Invention

[0004] The purpose of this invention is to provide a method for visual inspection of the integrity of logistics park outlines based on quadruped robots, which solves the technical problem of poor detection performance of quadruped robots in low-cost inspection scenarios.

[0005] In a first aspect, one embodiment of the present invention provides a method for visually inspecting the integrity of logistics park outlines using a quadruped robot, the method comprising: Multiple frame detection samples of a quadruped robot during a target monitoring period and feature information of each frame detection sample are obtained. The target monitoring period is a time period that traces back a set duration from the current moment. The set duration is longer than the duration of the quadruped robot's gait cycle. The feature information includes the pose of the corresponding frame detection sample in the world coordinate system and its corresponding confidence risk value. The confidence risk value is used to indicate the degree of body sway of the quadruped robot at the corresponding detection moment. Among the multiple frame detection samples, reference samples and samples to be corrected are distinguished. The reference samples are frame detection samples with a confidence risk value less than a risk threshold, and the samples to be corrected are frame detection samples with a confidence risk value greater than or equal to the risk threshold. Based on the pose and detection time of the frame detection sample in the world coordinate system, multiple reference samples and multiple samples to be corrected are associated and matched to determine multiple sample matching pairs from the multiple frame detection samples. In the plurality of sample matching pairs, the pose of the corresponding sample to be corrected is corrected based on the pose of the reference sample of each sample matching pair to obtain a plurality of frame target samples. The integrity of the frame lines is detected based on the multiple frame line target samples, and the detection results are obtained.

[0006] In some embodiments, the confidence risk value of the frame detection sample is determined based on the pitch angular velocity and vertical acceleration of the quadruped robot at the corresponding detection time, wherein the pitch angular velocity and the corresponding confidence risk value are positively correlated, and the vertical acceleration and the corresponding confidence risk value are positively correlated.

[0007] In some embodiments, the risk threshold is the product of the median confidence risk value of the plurality of frame detection samples and a preset adjustment coefficient, wherein the preset adjustment coefficient is greater than 1.

[0008] In some embodiments, the step of associating and matching multiple reference samples with multiple samples to be corrected based on the pose and detection time of the frame detection samples in the world coordinate system to determine multiple sample matching pairs from the multiple frame detection samples includes: A difference matrix is ​​constructed based on multiple reference samples and multiple samples to be corrected. Each element in the difference matrix corresponds to a candidate sample pair, which includes one reference sample and one sample to be corrected. The element values ​​of the matrix elements are used to indicate the matching cost between the corresponding reference sample and the corresponding sample to be corrected. The matching cost is determined based on the detection time deviation and pose deviation between the corresponding reference sample and the corresponding sample to be corrected. The difference matrix is ​​subjected to bipartite graph optimal matching processing to obtain matching results, and the multiple candidate sample pairs included in the matching results are determined as multiple sample matching pairs.

[0009] In some embodiments, the pose includes position and width, and the pose deviation includes position offset distance and width deviation.

[0010] In some embodiments, the matching cost is the average of the normalized value of the position offset distance, the normalized value of the width deviation, and the normalized value of the detection time deviation.

[0011] In some embodiments, the pose includes a position, the position includes a horizontal axis coordinate and a vertical axis coordinate, the vertical axis corresponding to the vertical axis coordinate indicates the direction of travel of the quadruped robot, and the horizontal axis corresponding to the horizontal axis coordinate indicates the direction perpendicular to the direction of travel of the quadruped robot. The step of correcting the pose of the corresponding sample to be corrected based on the pose of the reference sample in each sample matching pair to obtain multiple bounding box target samples includes: In the plurality of sample matching pairs, the horizontal axis coordinate of the corresponding sample to be corrected is replaced based on the horizontal axis coordinate of the reference sample of each sample matching pair, and the confidence risk value of the corresponding sample to be corrected is replaced based on the confidence risk value of the reference sample of each sample matching pair, so as to obtain a plurality of corrected samples. Multiple corrected samples and multiple reference samples were all identified as target samples for the frame line.

[0012] In some embodiments, the pose includes position and width; The steps for performing frame integrity detection based on the multiple frame target samples and obtaining the detection results include: Among the multiple grid regions included in the frame detection area, the grid region containing at least one frame target sample is determined as a valid grid region; The widths of at least one frame target sample included in each valid grid are fused to determine the fused width of each valid grid. The integrity of the frame lines is checked based on the fusion width of each valid grid, and the detection results are obtained.

[0013] In some embodiments, the step of fusing the widths of at least one frameline target sample included in each valid grid to determine the fused width of each valid grid includes: The width of at least one frame target sample included in each effective grid is weighted and calculated to obtain the fusion width of each effective grid. The calculation weight of the frame target sample width is negatively correlated with its corresponding confidence risk value.

[0014] In some embodiments, the step of performing frame line integrity detection based on the fusion width of each valid grid to obtain the detection result includes: Analyze the width dispersion of at least one frame target sample within each effective grid to determine the observation fluctuation index for each effective grid; Among multiple valid grids, the valid grids with observation fluctuation index greater than or equal to the observation fluctuation threshold are filtered out to obtain multiple target grids; The frame line integrity is detected based on the multiple target grids, and the detection results are obtained.

[0015] The present invention has the following beneficial effects: After acquiring multiple frame detection samples along with their poses and confidence risk values, the samples are first differentiated using risk thresholds to identify reference samples corresponding to stable robot states and samples to be corrected corresponding to swaying robot states. Then, based on the pose and detection time of each frame detection sample, the reference samples and samples to be corrected are correlated and matched to determine the reference samples and samples to be corrected that indicate the same real-world region. Multiple sample matching pairs are then constructed. The pose of the corresponding sample to be corrected is then corrected based on the pose of the reference sample. Finally, frame integrity detection is performed to minimize data interference caused by the swaying of the quadruped robot and ensure the accuracy and reliability of the final output detection results. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for visually inspecting the integrity of logistics park outlines using a quadruped robot, as provided in an embodiment of the present invention. Detailed Implementation

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0018] The following describes in detail, with reference to the accompanying drawings, a specific scheme of the method for visually inspecting the integrity of logistics park frame lines based on quadruped robots provided by the present invention.

[0019] It should be understood that the present invention is specifically applied to low-cost inspection scenarios for quadruped robots (where there is a lack of high-precision external absolute positioning systems (such as RTK-GNSS or high-beam laser radar SLAM)).

[0020] In one embodiment, the present invention provides a method for visually inspecting the integrity of logistics park outlines using a quadruped robot, such as... Figure 1 As shown, the method includes: Step S1: Obtain multiple frame detection samples from the quadruped robot during the target monitoring period, as well as the feature information of each frame detection sample.

[0021] The target monitoring period is a time period that traces back a set duration (which can be set to 0.5 seconds based on experience) from the current moment. The set duration is longer than the gait cycle of the quadruped robot. The feature information includes the pose of the corresponding frame detection sample in the world coordinate system and its corresponding confidence risk value. The confidence risk value is used to indicate the degree of body sway of the quadruped robot at the corresponding detection moment.

[0022] In this invention, the quadruped robot is equipped with a vision sensor (such as a monocular camera or a depth camera) and an inertial measurement unit (IMU).

[0023] Since the robot collects data while in motion, even a small time difference between the image acquisition moment and the IMU data acquisition moment can lead to a huge pose calculation error. Therefore, this invention employs a hardware pulse triggering method for clock synchronization. Specifically, the vision sensor is configured to send a high-level pulse signal to the onboard processor and IMU via a hardware interface at the instant each exposure ends. The processor captures the rising edge of this signal and marks it as the reference timestamp for the corresponding frame image (assuming it's the k-th frame). At any moment The processor performs the following data acquisition actions: reads data from the IMU buffer and obtains the time using linear interpolation. fuselage three-axis angular velocity and triaxial acceleration And it mainly extracts the pitch rate of the fuselage. and vertical acceleration .

[0024] The specific process for extracting samples for frame detection is as follows: For the first Image detection is performed on the frame image to extract the frame region indicating the park boundary. Then, the frame region is skeletonized to obtain several consecutive skeleton points. Then, for each skeleton point, the camera intrinsic and extrinsic parameters are used to project it onto the standard horizontal plane of the world coordinate system. The positions of each skeleton point in the world coordinate system are calculated (using instantaneous projected coordinates). (represented) and width (which can be determined by the maximum inscribed circle diameter or vertical span of the skeleton point within the frame area along the normal direction).

[0025] Specifically, the confidence risk value of the frame detection sample is determined based on the pitch angular velocity and vertical acceleration of the quadruped robot at the corresponding detection time, wherein the pitch angular velocity and the corresponding confidence risk value are positively correlated, and the vertical acceleration and the corresponding confidence risk value are positively correlated.

[0026] Both the pitch angular velocity and vertical acceleration mentioned above can be obtained by reading data from the IMU buffer.

[0027] The pitch angular velocity mentioned above is used to reflect the body nodding amplitude of the quadruped robot at the corresponding detection moment. The greater the body nodding amplitude, the greater the pitch angular velocity, the more significant the body shaking, and the more severe the interference of body shaking on the frame detection sample obtained at the corresponding detection moment, and the lower the data reliability.

[0028] Vertical acceleration is used to indicate the degree of swaying of the quadruped robot in the vertical direction. The greater the swaying amplitude, the greater the vertical acceleration and the more significant the shaking of the robot body. In this case, the frame detection sample obtained at the corresponding detection time will be more severely affected by the shaking of the robot body, and the data reliability will be lower.

[0029] For example, confidence risk value It can be represented as: in, This represents the absolute value of the pitch angular velocity of the sample detected within the corresponding frame. The reference constant for angular velocity (which can be the average of the absolute values ​​of multiple pitch angular velocities determined by the IMU while the quadruped robot is standing still; in this embodiment, it is set to...) This is used to perform numerical normalization on the aforementioned pitch angular velocity. This represents the vertical acceleration and gravitational acceleration constant of the sample detected within the corresponding frame. (Pick The absolute value of the difference, This represents the acceleration reference constant (which can detect the average of multiple vertical acceleration peaks when the feet of a quadruped robot impact the ground; in this embodiment, it is set to ). This is used to numerically normalize vertical acceleration. This is a balance coefficient (based on experience, it is set to 0.5), used to balance the contribution weights of angular velocity and vertical acceleration to the confidence risk value.

[0030] Step S2: Distinguish between reference samples and samples to be corrected among the multiple frame detection samples. The reference samples are frame detection samples with a confidence risk value less than a risk threshold, and the samples to be corrected are frame detection samples with a confidence risk value greater than or equal to the risk threshold.

[0031] The reference sample can be understood as the frame line detection sample obtained under stable conditions, while the sample to be corrected can be understood as the frame line detection sample obtained under swaying conditions.

[0032] In one example, the risk threshold is the product of the median confidence risk value of the plurality of frame detection samples and a preset adjustment coefficient, wherein the preset adjustment coefficient is greater than 1.

[0033] In this example, based on the above settings, the risk threshold can be dynamically adapted to multiple bounding box detection samples to ensure the reliability of the risk threshold value, thereby making the reference sample and the sample to be corrected based on the risk threshold more accurate.

[0034] The preset adjustment coefficient can be set to a range of 1.2 to 1.5 based on experience.

[0035] In another example, the risk threshold can be preset to a certain experimental value (such as 0.3) to reduce the computational overhead during the implementation of the plan.

[0036] Step S3: Based on the pose and detection time of the frame detection sample in the world coordinate system, perform association matching on multiple reference samples and multiple samples to be corrected to determine multiple sample matching pairs from the multiple frame detection samples.

[0037] Analysis revealed that although different frame detection samples may originate from different image frames, the continuity of image capture means that different frame detection samples may indicate the same real-world frame portion. In this case, once the reference sample and the sample to be corrected indicating the same real-world frame portion are identified, the pose of the reference sample can be used as confidence data to adaptively correct the position of the corresponding sample to be corrected. This suppresses data interference introduced by camera shake and provides a solid data foundation for subsequent frame integrity analysis.

[0038] Therefore, the step of associating and matching multiple reference samples with multiple samples to be corrected based on the pose and detection time of the frame detection samples in the world coordinate system to determine multiple sample matching pairs from the multiple frame detection samples specifically includes: A difference matrix is ​​constructed based on multiple reference samples and multiple samples to be corrected. Each element in the difference matrix corresponds to a candidate sample pair, which includes one reference sample and one sample to be corrected. The element values ​​of the matrix elements are used to indicate the matching cost between the corresponding reference sample and the corresponding sample to be corrected. The matching cost is determined based on the detection time deviation and pose deviation between the corresponding reference sample and the corresponding sample to be corrected. The difference matrix is ​​subjected to bipartite graph optimal matching processing to obtain matching results, and the multiple candidate sample pairs included in the matching results are determined as multiple sample matching pairs.

[0039] In the above setup, the construction of the difference matrix is ​​used to abstract the problem of association matching between the reference sample and the sample to be corrected. Then, the bipartite graph optimal matching algorithm is used to process the difference matrix to determine the optimal matching result, that is, to dynamically determine the association matching relationship between the reference sample and the sample to be corrected.

[0040] In this invention, multiple reference samples and multiple sets of samples to be corrected are specifically used as the left and right vertices of the bipartite graph. The element values ​​of the matrix elements in the difference matrix are used as the edge weights connecting the vertices. The Hungarian Algorithm or the KM algorithm is used to perform the bipartite graph optimal matching operation on the difference matrix.

[0041] It should be understood that the greater the detection time deviation, the greater the distance between the detection time of the corresponding reference sample and the detection time of the corresponding sample to be corrected, and the lower the probability that the two indicate the same real frame line part; and the greater the pose deviation, the lower the probability that the two indicate the same real frame line part.

[0042] Furthermore, the pose includes position and width, and the pose deviation includes position offset distance and width deviation.

[0043] Furthermore, the matching cost is the average of the normalized value of the position offset distance, the normalized value of the width deviation, and the normalized value of the detection time deviation.

[0044] In the above settings, the position offset distance, width deviation, and detection time deviation are numerically normalized to eliminate differences in different numerical units and ensure the usability of the calculated matching cost.

[0045] For example, the above matching cost It can be represented as: in, Represents the distance normalization factor, defined , The preset maximum allowable drift distance (set based on experience) ), Indicates the first The first reference sample and the first The positional offset distance between the samples to be corrected (i.e., the Euclidean distance between their positions). Define the width normalization factor. , The standard width of the markings (based on experience) m), Indicates the first The first reference sample and the first The width deviation between the samples to be corrected (i.e., the absolute difference in their widths). Represents the time normalization factor, defined , Specifically, it involves setting the duration. Indicates the first The first reference sample and the first The detection time deviation between the two samples to be corrected (the time difference between their detection times).

[0046] Step S4: In the plurality of sample matching pairs, the pose of the corresponding sample to be corrected is corrected based on the pose of the reference sample of each sample matching pair to obtain a plurality of frame target samples.

[0047] The position includes horizontal axis coordinates and vertical axis coordinates. The vertical axis coordinates indicate the direction of travel of the quadruped robot, and the horizontal axis coordinates indicate the direction perpendicular to the direction of travel of the quadruped robot (which can be approximately understood as the direction of the frame width).

[0048] Analysis revealed that the periodic pitching behavior during the robot's swaying primarily interferes with frame width detection, causing continuous jumps in frame width. However, the position of the quadruped robot's travel direction is almost unaffected. Therefore, the step of correcting the pose of the corresponding sample to be corrected based on the pose of the reference sample in each of the multiple sample matching pairs to obtain multiple frame target samples specifically includes: In the plurality of sample matching pairs, the horizontal axis coordinate of the corresponding sample to be corrected is replaced based on the horizontal axis coordinate of the reference sample of each sample matching pair, and the confidence risk value of the corresponding sample to be corrected is replaced based on the confidence risk value of the reference sample of each sample matching pair, so as to obtain a plurality of corrected samples. Multiple corrected samples and multiple reference samples were all identified as target samples for the frame line.

[0049] Based on the above settings, the data correction of the frame line detection samples obtained under the condition of the machine body shaking can be completed easily, providing a solid data foundation for the subsequent frame line integrity detection.

[0050] Step S5: Perform frame integrity detection based on the multiple frame target samples to obtain the detection results.

[0051] The step of performing frame integrity detection based on the multiple frame target samples to obtain the detection result includes: Among the multiple grid regions included in the frame detection area, the grid region containing at least one frame target sample is determined as a valid grid region; The widths of at least one frame target sample included in each valid grid are fused to determine the fused width of each valid grid. The integrity of the frame lines is checked based on the fusion width of each valid grid, and the detection results are obtained.

[0052] Based on the above settings, regional gridding is used to simplify and filter the data of multiple target samples with frame lines, thereby reducing computational overhead and noise interference while ensuring the accuracy of subsequent processing results.

[0053] In one example, the frame detection area can be divided into multiple grid areas, with the size of each grid area set to 1cm × 1cm.

[0054] Specifically, the step of fusing the widths of at least one frameline target sample included in each valid grid to determine the fused width of each valid grid includes: The width of at least one frame target sample included in each effective grid is weighted and calculated to obtain the fusion width of each effective grid. The calculation weight of the frame target sample width is negatively correlated with its corresponding confidence risk value.

[0055] For example, when an effective grid includes two or more target samples, the cumulative value of the reciprocals of the confidence risk values ​​of the two or more target samples included in the effective grid can be calculated first to obtain the baseline value of the two or more target samples included in the effective grid. Then, in the two or more target samples included in the effective grid, the ratio of the reciprocal of the confidence risk value of each target sample to its corresponding baseline value is determined as the calculation weight corresponding to its width. In addition, for a certain effective grid, if it includes two or more target samples and there is at least one target sample with a confidence risk value of 0, the fusion width of the effective grid is the average value of the widths of one or more target samples with a confidence risk value of 0.

[0056] The steps for performing frame integrity checks based on the fusion width of each valid grid to obtain the detection results include: Analyze the width dispersion of at least one frame target sample within each effective grid to determine the observation fluctuation index for each effective grid; Among multiple valid grids, the valid grids with observation fluctuation index greater than or equal to the observation fluctuation threshold are filtered out to obtain multiple target grids; The frame line integrity is detected based on the multiple target grids, and the detection results are obtained.

[0057] In this invention, the range of the width of at least one frame target sample within each effective grid is specifically used as the observation fluctuation index for each effective grid.

[0058] The aforementioned observed fluctuation index can be set to 5 mm based on experience.

[0059] In this invention, the above-mentioned frame line integrity detection operation specifically includes: In multiple target meshes, the absolute difference between the blending width of each target mesh and the reference width of the frame line is calculated, and the ratio of this absolute difference to the reference width of the frame line is calculated to determine the frame line defect ratio of each target mesh. Then, among multiple target grids, the target grids with a frame line defect rate less than or equal to the rate threshold (e.g., 10%) are identified as normal grids, while the target grids with a frame line defect rate greater than the rate threshold (e.g., 10%) are identified as abnormal grids. If the number of abnormal grids is greater than or equal to the number threshold (e.g., 30), output the detection result indicating the abnormality of the frame integrity detection (to remind maintenance personnel to repair the missing parts of the frame). If the number of abnormal grids is less than the threshold, the output indicates that the integrity of the frame line is normal.

[0060] The aforementioned frame reference width is the standard width of the frame in its undamaged state. This value needs to be set based on the actual working conditions, and this invention does not limit it.

[0061] In summary, after acquiring multiple frame detection samples along with their poses and confidence risk values, the samples are first differentiated using risk thresholds to identify reference samples corresponding to stable robot states and samples to be corrected corresponding to swaying robot states. Then, based on the pose and detection time of each frame detection sample, the reference samples and samples to be corrected are correlated and matched to determine the reference samples and samples to be corrected that indicate the same real-world region. Based on this, multiple sample matching pairs are constructed. Then, the pose of the corresponding sample to be corrected is corrected based on the pose of the reference sample. Finally, frame integrity detection is performed to minimize data interference caused by the swaying of the quadruped robot and ensure the accuracy and reliability of the final output detection results.

[0062] It should be noted that the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some implementations, multitasking and parallel processing are possible or may be advantageous.

Claims

1. A method for detecting the integrity of a logistics yard frame line based on vision of a quadruped robot, characterized in that, The method includes: Multiple frame detection samples of a quadruped robot during a target monitoring period and feature information of each frame detection sample are obtained. The target monitoring period is a time period that traces back a set duration from the current moment. The set duration is longer than the duration of the quadruped robot's gait cycle. The feature information includes the pose of the corresponding frame detection sample in the world coordinate system and its corresponding confidence risk value. The confidence risk value is used to indicate the degree of body sway of the quadruped robot at the corresponding detection moment. Among the multiple frame detection samples, reference samples and samples to be corrected are distinguished. The reference samples are frame detection samples with a confidence risk value less than a risk threshold, and the samples to be corrected are frame detection samples with a confidence risk value greater than or equal to the risk threshold. Based on the pose and detection time of the frame detection sample in the world coordinate system, multiple reference samples and multiple samples to be corrected are associated and matched to determine multiple sample matching pairs from the multiple frame detection samples. In the plurality of sample matching pairs, the pose of the corresponding sample to be corrected is corrected based on the pose of the reference sample of each sample matching pair to obtain a plurality of frame target samples. The integrity of the frame lines is detected based on the multiple frame line target samples, and the detection results are obtained. 2.The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 1, wherein, The confidence risk value of the frame detection sample is determined based on the pitch angular velocity and vertical acceleration of the quadruped robot at the corresponding detection time. The pitch angular velocity and the corresponding confidence risk value are positively correlated, and the vertical acceleration and the corresponding confidence risk value are positively correlated. 3.The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 1, wherein, The risk threshold is the product of the median confidence risk value of the plurality of frame detection samples and a preset adjustment coefficient, wherein the preset adjustment coefficient is greater than 1.

4. The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 1, wherein, The step of associating and matching multiple reference samples with multiple samples to be corrected based on the pose and detection time of the frame detection samples in the world coordinate system to determine multiple sample matching pairs from the multiple frame detection samples includes: A difference matrix is ​​constructed based on multiple reference samples and multiple samples to be corrected. Each element in the difference matrix corresponds to a candidate sample pair, which includes one reference sample and one sample to be corrected. The element values ​​of the matrix elements are used to indicate the matching cost between the corresponding reference sample and the corresponding sample to be corrected. The matching cost is determined based on the detection time deviation and pose deviation between the corresponding reference sample and the corresponding sample to be corrected. The difference matrix is ​​subjected to bipartite graph optimal matching processing to obtain matching results, and the multiple candidate sample pairs included in the matching results are determined as multiple sample matching pairs.

5. The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 4, characterized in that, The pose includes position and width, and the pose deviation includes position offset distance and width deviation. 6.The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 5, wherein, The matching cost is the average of the normalized value of the position offset distance, the normalized value of the width deviation, and the normalized value of the detection time deviation.

7. The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 1, wherein, The pose includes a position, which includes a horizontal axis coordinate and a vertical axis coordinate. The vertical axis coordinate indicates the direction of travel of the quadruped robot, and the horizontal axis coordinate indicates the direction perpendicular to the direction of travel of the quadruped robot. The step of correcting the pose of the corresponding sample to be corrected based on the pose of the reference sample in each sample matching pair to obtain multiple bounding box target samples includes: In the plurality of sample matching pairs, the horizontal axis coordinate of the corresponding sample to be corrected is replaced based on the horizontal axis coordinate of the reference sample of each sample matching pair, and the confidence risk value of the corresponding sample to be corrected is replaced based on the confidence risk value of the reference sample of each sample matching pair, so as to obtain a plurality of corrected samples. Multiple corrected samples and multiple reference samples were all identified as target samples for the frame line. 8.The method for detecting the integrity of the frame line of the logistics yard based on the quadruped robot vision according to claim 7, wherein, The pose includes position and width; The steps for performing frame integrity detection based on the multiple frame target samples and obtaining the detection results include: Among the multiple grid regions included in the frame detection area, the grid region containing at least one frame target sample is determined as a valid grid region; The widths of at least one frame target sample included in each valid grid are fused to determine the fused width of each valid grid. The integrity of the frame lines is checked based on the fusion width of each valid grid, and the detection results are obtained.

9. The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 8, characterized in that, The step of fusing the widths of at least one frameline target sample included in each valid grid to determine the fused width of each valid grid includes: The width of at least one frame target sample included in each effective grid is weighted and calculated to obtain the fusion width of each effective grid. The calculation weight of the frame target sample width is negatively correlated with its corresponding confidence risk value.

10. The method for detecting the integrity of the frame line of the logistics yard based on the vision of the quadruped robot according to claim 8, wherein, The steps for performing frame line integrity checks based on the fusion width of each effective grid to obtain the check results include: Analyze the width dispersion of at least one frame target sample within each effective grid to determine the observation fluctuation index for each effective grid; Among multiple valid grids, the valid grids with observation fluctuation index greater than or equal to the observation fluctuation threshold are filtered out to obtain multiple target grids; The frame line integrity is detected based on the multiple target grids, and the detection results are obtained.

Images