Method, device, equipment and medium for detecting material handling equipment

Abstract

The application discloses a material processing equipment detection method, device, equipment and medium, and the method comprises the following steps: determining a detection parameter and a current pose state of the material processing equipment; the detection parameter comprises an expected pose state and an initial scanning path; determining whether the material processing equipment meets a pose matching condition according to the current pose state and the expected pose state; if not, calculating a pose deviation matrix of the material processing equipment, and correcting the initial scanning path according to the pose deviation matrix to obtain a target scanning path; sending the target scanning path to a scanning device, so that the scanning device scans the material processing equipment according to the target scanning path to obtain point cloud data of the material processing equipment; receiving the point cloud data sent by the scanning device, and detecting the material processing equipment based on the point cloud data to obtain a detection result of the material processing equipment. Through the technical scheme of the embodiment of the application, the detection accuracy of the material processing equipment can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of detection technology, and in particular to a detection method, apparatus, equipment and medium for material handling equipment. Background Technology

[0002] In the tobacco production process, material handling equipment (such as tobacco packaging equipment) serves as the core tooling for conveying, organizing, and packaging tobacco products such as cigarette packs and cigarette sticks. Its quality directly determines the pass rate of finished tobacco packaging products and the continuous and stable operation capability of the production line. Therefore, quality inspection of this equipment is particularly important.

[0003] Currently, the main method for quality inspection of material handling equipment is to preset a fixed scanning path, control the scanning equipment to perform scanning operations and collect point cloud data, and then conduct quality inspection based on the collected point cloud data. However, this method is prone to incomplete point cloud data collection, which reduces the accuracy of the inspection. If the scanning path is manually recalibrated, it is not only cumbersome and inefficient, but also highly dependent on the operator's experience, making it difficult to meet the actual needs of online rapid inspection.

[0004] Therefore, there is an urgent need to propose a new method to solve the above problems. Summary of the Invention

[0005] This invention provides a detection method, apparatus, equipment, and medium for material handling equipment, which can effectively improve the detection accuracy of material handling equipment.

[0006] In a first aspect, embodiments of the present invention provide a detection method for a material handling device, the method comprising: Determine the detection parameters and current pose state of the material handling equipment; the detection parameters include the desired pose state and the initial scan path. Determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. If the conditions are not met, the pose deviation matrix of the material handling equipment is calculated, and the initial scanning path is corrected according to the pose deviation matrix to obtain the target scanning path. The target scanning path is sent to the scanning device so that the scanning device scans the material handling device according to the target scanning path to obtain point cloud data of the material handling device; The device receives the point cloud data sent by the scanning device and performs detection on the material handling device based on the point cloud data to obtain the detection result of the material handling device.

[0007] The technical solution of this invention first determines the detection parameters and current pose state of the material handling equipment. This not only provides a data foundation for subsequent pose comparison but also enables adaptive matching of detection parameters for different equipment, effectively ensuring the accuracy and reliability of subsequent whole-machine inspection. Next, it determines whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. If it does, the initial scanning path is sent to the scanning device, allowing the scanning device to scan the material handling equipment according to the initial scanning path, obtaining point cloud data of the material handling equipment. This improves inspection efficiency while ensuring scanning accuracy. If it does not meet the conditions, the pose deviation matrix of the material handling equipment is calculated, and the initial scanning path is corrected based on the pose deviation matrix to obtain the target scanning path. This ensures that the generated target scanning path conforms to the actual placement pose of the equipment, effectively avoiding scanning blind spots, trajectory misalignment, and partial missed scans caused by equipment pose deviations, improving the adaptability of the scanning path to actual working conditions. Simultaneously, no manual intervention is required to adjust the scanning path, significantly reducing manual intervention and human error, effectively improving the automation level and inspection efficiency of the overall inspection operation. Subsequently, the target scanning path is sent to the scanning device, enabling the device to scan the material handling equipment according to the path and obtain point cloud data. This effectively avoids scanning blind spots and missed scans caused by equipment pose deviations, ensuring high integrity, effectiveness, and reliability of the collected point cloud data. This provides high-quality raw data support for subsequent material handling equipment testing, thereby improving the accuracy and stability of the overall equipment testing results. Finally, the point cloud data sent by the scanning device is received, and the material handling equipment is tested based on this data to obtain the testing results. This effectively improves the intelligence, accuracy, and reliability of equipment testing. Therefore, the technical solution of this invention can solve the problem in the prior art where using a fixed preset scanning path easily leads to incomplete point cloud data acquisition, thus reducing the accuracy of equipment testing.

[0008] Secondly, embodiments of the present invention also provide a detection device for a material handling equipment, the device comprising: The determination module is used to determine the detection parameters and current pose state of the material handling equipment; the detection parameters include the desired pose state and the initial scanning path. The judgment module is used to determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. The correction module is used to calculate the pose deviation matrix of the material handling equipment if the conditions are not met, and to correct the initial scanning path according to the pose deviation matrix to obtain the target scanning path. The sending module is used to send the target scanning path to the scanning device, so that the scanning device scans the material handling device according to the target scanning path to obtain point cloud data of the material handling device; The detection module is used to receive the point cloud data sent by the scanning device, and to detect the material handling equipment based on the point cloud data to obtain the detection result of the material handling equipment.

[0009] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the detection method of the material handling equipment according to any embodiment of the present invention.

[0010] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, characterized in that the computer-executable instructions, when executed by a computer processor, implement the detection method of the material handling equipment described in any embodiment of the present invention.

[0011] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the detection device of the material handling equipment, or it may be packaged separately from the processor of the detection device of the material handling equipment; this application does not impose any limitations on this.

[0012] The descriptions of the second, third, and fourth aspects in this application can be referenced to the detailed description of the first aspect; and the beneficial effects described in the second, third, and fourth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0013] In this application, the name of the detection device in the aforementioned material handling equipment does not limit the equipment or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0014] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic flowchart of a detection method for a material handling equipment provided in an embodiment of the present invention; Figure 2 A schematic flowchart of a detection method for another material handling equipment provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a detection device for a material handling equipment provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0018] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0019] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0020] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0021] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, without conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0022] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0024] Figure 1 This is a flowchart illustrating a detection method for a material handling equipment according to an embodiment of the present invention. This embodiment is applicable to situations requiring quality inspection of material handling equipment. The method is executed by a detection device for the material handling equipment, which can be implemented using software and / or hardware. For example, the device can be integrated into an electronic device. The detection method for the material handling equipment in this embodiment specifically includes the following steps: Step 110: Determine the detection parameters and current position status of the material handling equipment.

[0025] Specifically, the detection parameters are pre-set benchmark parameters based on actual working conditions and detection requirements, used for quality inspection of material handling equipment. The detection parameters include the desired pose state and the initial scanning path. The desired pose state is the pre-calibrated and established standard benchmark pose of the equipment (desired position information and desired orientation angle) in the benchmark coordinate system of the detection station. The current pose state is the actual pose of the material handling equipment to be inspected in the benchmark coordinate system of the detection station (such as current position information and current orientation angle), used for comparison with the desired pose. The benchmark coordinate system of the detection station is a coordinate system constructed with a fixed positioning point (such as a positioning pin, limit block, or camera installation position) within the detection station of the material handling equipment as the origin. It is used to uniformly calibrate the desired pose, characterize the current actual pose of the equipment, and realize the benchmark mapping transformation between pixel coordinates and physical actual coordinates. The detection station is a dedicated and limited working area for completing the quality inspection of the material handling equipment. The initial scanning path is a pre-planned and set standard scanning motion trajectory of the material handling equipment in the desired pose state, used for the scanning equipment to move along this trajectory and collect point cloud data of the material handling equipment. Material handling equipment refers to specialized tobacco packaging process equipment used in the tobacco production chain to complete the conveying, transfer, positioning, clamping, and straightening of finished and semi-finished tobacco products such as cigarette packs, cartons, and boxes, and to connect with packaging stations. Specifically, it can include various types of tobacco packaging equipment, such as tobacco packaging machine main units, cigarette pack conveying devices, and cigarette box palletizing and transferring equipment.

[0026] In practice, the detection parameters (such as desired pose, initial scan path, and ideal feature point set) of the material handling equipment can be obtained by querying a pre-defined attribute information and detection parameter correspondence table based on the equipment's attribute information (such as model and serial number). The attribute information and detection parameter correspondence table is a pre-established association mapping data table. This table uses the inherent attribute information of the material handling equipment as an index to form a one-to-one matching and binding relationship with the specific detection parameters. For example, assuming the attribute information is the equipment model, the detection parameters include the desired pose, initial scan path, and ideal feature point set; if the equipment model is A1, then the desired pose is (500.0, 200.0, 0 degrees), the initial scan path is (480, 180) → (520, 180) → (520, 220) → (480, 220), and the ideal feature point set is {(450, 180), (550, 180), (500, 220)}.

[0027] Secondly, image acquisition devices (such as cameras) are used to acquire real-time images of preset feature points on the top of the material handling equipment (such as right-angle corners of the top frame of the equipment, vertices of the top tooling reference bosses, etc.), obtaining real-shot images containing these feature points. Then, using feature detection algorithms (such as Harris corner detection algorithm, Shi-Tomasi high-quality corner detection algorithm, etc.), the actual feature point set corresponding to the equipment under test is extracted from the real-shot images, and the pixel coordinates corresponding to each feature point are calculated.

[0028] Next, a pre-calibrated camera transformation matrix is ​​invoked to establish a coordinate mapping relationship between the camera pixel plane and the physical XY plane of the detection station (i.e., the detection station's reference coordinate system). Then, relying on the singular value decomposition algorithm, the ideal feature point set and the actual feature point set are matched and solved to find the optimal rigid transformation between the two sets of feature point sets, thereby obtaining the translational and rotational deviations of the equipment in the detection station's reference coordinate system. Finally, by combining the planar translation and rotational attitude quantities, the actual position and attitude information of the material handling equipment in the detection station's reference coordinate system are calculated, thus obtaining the current pose state of the material handling equipment.

[0029] In this embodiment, the above steps not only provide a data foundation for subsequent pose comparison, but also achieve adaptive matching of detection parameters of different devices, effectively ensuring the accuracy and reliability of subsequent whole-machine testing.

[0030] Step 120: Determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state.

[0031] If satisfied, proceed to step 130; otherwise, proceed to step 140.

[0032] Specifically, the pose matching condition is a preset pose deviation judgment threshold rule, which is used to determine whether the actual pose of the device is within the allowable error range.

[0033] In practice, the material handling equipment can be determined to meet the pose matching conditions based on the current pose state and the desired pose state. For example, the positional deviation and rotational deviation between the current pose state and the desired pose state can be calculated. If both the positional deviation and rotational deviation are within the preset allowable error threshold range, the pose matching conditions are met. If the positional deviation and / or rotational deviation exceed the preset allowable error threshold range, the pose matching conditions are not met.

[0034] If the pose matching condition is met, it means that the actual placement and posture of the material handling equipment are close to the standard ideal state, with no obvious pose deviation. In this case, step 130 can be executed directly. If the pose matching condition is not met, it means that the material handling equipment has a position deviation or posture deflection, and the actual pose deviates from the ideal standard pose. Scanning path correction is required. In this case, step 140 can be executed.

[0035] In this embodiment, the above steps provide a reliable basis for determining whether to select the original scanning path or correct the scanning path, effectively avoiding the waste of resources and invalid point cloud data collection caused by blind scanning, while ensuring the integrity of the scan, thereby improving the accuracy and reliability of the overall detection of the material handling equipment.

[0036] Step 130: Send the initial scan path to the scanning device so that the scanning device can scan the material handling equipment according to the initial scan path and obtain the point cloud data of the material handling equipment.

[0037] Specifically, the scanning equipment is a detection instrument used to acquire three-dimensional spatial information of the surface of material handling equipment, such as a 3D laser scanner, structured light camera, or lidar. It can move or adjust its viewing angle according to the received scanning path to complete a 3D scan of the material handling equipment and collect corresponding point cloud data. Point cloud data is the set of three-dimensional spatial coordinate points collected by the scanning equipment after performing a 3D scan of the material handling equipment surface.

[0038] In practice, after determining that the material handling equipment meets the pose matching conditions, the initial scanning path can be sent to the scanning equipment directly through the data interaction interface with the scanning equipment (such as the Ethernet interface), so that the scanning equipment can scan the material handling equipment according to the initial scanning path and obtain the point cloud data of the material handling equipment.

[0039] In this embodiment, the above steps can improve the efficiency of the detection operation while ensuring scanning accuracy.

[0040] Step 140: Calculate the pose deviation matrix of the material handling equipment, and correct the initial scanning path according to the pose deviation matrix to obtain the target scanning path.

[0041] Specifically, the pose deviation matrix is ​​a mathematical matrix used to describe the rigid transformation relationship between the current actual pose of the material handling equipment and its desired pose. The target scanning path is the scanning trajectory obtained after applying the pose deviation matrix to the initial scanning path for deviation compensation and trajectory correction.

[0042] In the specific implementation, firstly, the current position coordinates and rotation angle are parsed from the current pose state, and the standard position coordinates and standard rotation angle are parsed from the desired pose state. Secondly, the position deviation (including lateral and longitudinal deviations) and attitude rotation deviation between the current pose and the desired pose are calculated. These two types of deviations are modeled into a matrix according to the two-dimensional planar rigid body transformation rules, and the pose deviation matrix is ​​constructed and solved through matrix operations. Then, the original coordinates, preset travel direction, and standard scanning angle of all scanning points on the initial scanning path are extracted. Using the aforementioned pose deviation matrix as the core correction basis, each scanning point on the initial scanning path is corrected one by one. Specifically, this includes: correcting the original coordinates of each point through matrix multiplication (e.g., corrected coordinates = pose deviation matrix × original coordinates); and simultaneously, based on the rotation component in the pose deviation matrix, the travel direction angle and scanning angle of each point are rotated and corrected (e.g., target travel direction angle = initial travel direction angle + rotation deviation), ensuring that each scanning point matches the actual pose of the device.

[0043] Finally, after completing translation compensation and rotational transformation correction for all points on the initial scanning path, all corrected scanning points are integrated to generate a new scanning trajectory that adapts to the actual pose of the material handling equipment. This trajectory is the target scanning path.

[0044] In this embodiment, the above steps ensure that the generated target scanning path conforms to the actual placement posture of the equipment, effectively avoiding scanning blind spots, trajectory misalignment, and partial missed scans caused by equipment posture deviations, thus improving the adaptability of the scanning path to actual working conditions. Simultaneously, no manual intervention is required to adjust the scanning path, significantly reducing manual intervention and human error, and effectively improving the overall automation level and efficiency of the inspection operation.

[0045] Step 150: Send the target scanning path to the scanning device so that the scanning device can scan the material handling equipment according to the target scanning path and obtain the point cloud data of the material handling equipment.

[0046] In practice, after obtaining the target scanning path, the target scanning path can be directly sent to the scanning device through the data interaction interface (such as Ethernet interface) with the scanning device, so that the scanning device can scan the material handling equipment according to the target scanning path and obtain the point cloud data of the material handling equipment.

[0047] In this embodiment, the above steps can effectively avoid scanning blind spots and missed scans caused by device pose offset, ensuring that the collected point cloud data has high integrity, strong effectiveness, and excellent reliability, providing high-quality raw data support for the subsequent detection of material handling equipment, thereby improving the accuracy and stability of the overall detection results of the equipment.

[0048] Step 160: Receive point cloud data sent by the scanning device, and perform detection on the material handling equipment based on the point cloud data to obtain the detection results of the material handling equipment.

[0049] Specifically, the test results are conclusions about the quality of material handling equipment obtained after processing and analyzing point cloud data. For example, the test results may include dimensional errors, surface defects, assembly position deviations, structural deformation, and whether the overall equipment meets the assembly requirements of the production process.

[0050] In practice, the system receives point cloud data from the scanning device and inputs it into a pre-trained quality inspection model to obtain the inspection results of the material handling equipment. The quality inspection model is a detection model obtained by training a deep learning model with samples and iteratively optimizing parameters using historical point cloud data from different types of material handling equipment and their corresponding standard inspection results.

[0051] In this embodiment, the above steps effectively improve the intelligence level, accuracy, and reliability of equipment detection.

[0052] The detection method for material handling equipment provided in this invention first determines the detection parameters and current pose state of the material handling equipment. This not only provides a data foundation for subsequent pose comparison but also enables adaptive matching of detection parameters for different equipment, effectively ensuring the accuracy and reliability of subsequent whole-machine detection. Next, it determines whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. If it does, the initial scanning path is sent to the scanning device, allowing the scanning device to scan the material handling equipment according to the initial scanning path, obtaining point cloud data of the material handling equipment. This improves detection efficiency while ensuring scanning accuracy. If it does not meet the conditions, the pose deviation matrix of the material handling equipment is calculated, and the initial scanning path is corrected based on the pose deviation matrix to obtain the target scanning path. This ensures that the generated target scanning path conforms to the actual placement pose of the equipment, effectively avoiding scanning blind spots, trajectory misalignment, and partial missed scans caused by equipment pose deviations, improving the adaptability of the scanning path to actual working conditions. Simultaneously, no manual intervention is required to adjust the scanning path, significantly reducing manual intervention and human error, effectively improving the automation level and detection efficiency of the overall detection operation. Subsequently, the target scanning path is sent to the scanning device, enabling the device to scan the material handling equipment according to the path and obtain point cloud data. This effectively avoids scanning blind spots and missed scans caused by equipment pose deviations, ensuring high integrity, effectiveness, and reliability of the collected point cloud data. This provides high-quality raw data support for subsequent material handling equipment testing, thereby improving the accuracy and stability of the overall equipment testing results. Finally, the point cloud data sent by the scanning device is received, and the material handling equipment is tested based on this data to obtain the testing results. This effectively improves the intelligence, accuracy, and reliability of equipment testing. Therefore, the technical solution of this invention can solve the problem in the prior art where using a fixed preset scanning path easily leads to incomplete point cloud data acquisition, thus reducing the accuracy of equipment testing.

[0053] Figure 2 This is a schematic flowchart illustrating another detection method for a material handling device provided in an embodiment of the present invention. This embodiment is a specific implementation based on the above embodiments. In this embodiment, the method may further include: Step 210: Obtain the current appearance image of the material handling equipment.

[0054] Specifically, the current appearance image is a two-dimensional digital image that reflects the visible external shape and structural outline of the material handling equipment, obtained by real-time shooting of the material handling equipment through image acquisition devices (such as industrial cameras and smart cameras).

[0055] In practice, after receiving the arrival trigger signal from the automated guided vehicle (AGV), an image acquisition device installed at the inspection station captures the current appearance image of the material handling equipment. The arrival trigger signal is a status trigger command signal issued after the AGV has traveled and stopped at the designated position on the inspection station, completing precise positioning. It indicates that the material handling equipment has arrived at the inspection station and is ready, triggering the initiation of subsequent inspection processes. The AGV is an intelligent material handling device with autonomous navigation, path planning, and automatic driving functions.

[0056] It should be noted that the material handling equipment to be inspected is transported to the designated inspection station by an automated guided vehicle (AGV). Upon receiving a dispatch instruction, the AGV automatically transports the equipment to the inspection station. Once the AGV reaches the inspection station and precisely stops at the designated location, it sends a positioning trigger signal. The dispatch instruction is a control command issued to the AGV, specifying the task assignment, departure time, and target station information to control the AGV's initiation and execution of the equipment transfer task.

[0057] In this embodiment, the above steps provide a data foundation for subsequently determining the detection parameters and current pose status of the material handling equipment.

[0058] Step 211: Determine the type of material handling equipment based on the current appearance image to obtain the target equipment type.

[0059] Specifically, the target equipment type refers to the specific equipment type (such as equipment type name or type code) of the material handling equipment to be inspected, determined after image recognition or classification processing of the current appearance image. The type of material handling equipment is a category identifier formed by classifying the material handling equipment according to its function, structure, or model.

[0060] In practice, after obtaining the current appearance image, it can be preprocessed (such as image enhancement, noise reduction, size normalization, etc.). Then, the preprocessed image is input into a pre-trained type determination model to obtain the target equipment type. The type determination model is a classification model obtained by training and iteratively optimizing a deep learning model (such as MobileNet-V3, YOLO series, etc.) based on appearance image samples of different types of material handling equipment and their corresponding equipment type labels.

[0061] In this embodiment, the above steps enable adaptive detection and adaptation for different types of devices, improve the automation and efficiency of device classification and identification, and provide accurate preliminary classification basis for subsequent detection, ensuring the continuity and adaptability of the overall detection process.

[0062] Step 212: Determine the detection parameters based on the target equipment type and the preset detection configuration library.

[0063] Specifically, the preset detection configuration library is a database or configuration file pre-built according to the actual application scenario. The library stores various device types and their corresponding detection parameters. The detection parameters include the desired pose state and the initial scan path.

[0064] In practice, after obtaining the target device type, a search can be performed in the preset detection configuration library based on the target device type to match and obtain the corresponding detection parameters. For example, the expected pose corresponding to device type A2 is (200.0, 200.0, 0 degrees), the initial scanning path is (120, 85) → (360, 85) → (120, 420) → (360, 420), and the ideal feature point set is {(450, 160), (540, 180), (500, 210)}.

[0065] In this embodiment, the above steps enable the adaptive retrieval of specific detection parameters according to the device type, reducing manual intervention and thereby improving the automation level of the detection process and the overall detection accuracy.

[0066] Furthermore, before step 212, the method includes: obtaining the current detection task; determining whether the device type and the target device type in the current detection task match; if they match, triggering the execution of determining the detection parameters based on the target device type and the preset detection configuration library.

[0067] Specifically, the current testing task is the single complete testing operation to be executed, which includes basic information about the material handling equipment to be tested, such as equipment type, equipment number, testing requirements, and other task-related configuration information.

[0068] In practice, the current inspection task can be obtained first through the upper-level system (such as the workshop manufacturing execution system) or user input. Then, the equipment type in the current inspection task is matched and verified with the target equipment type obtained above. If the two match, it means that the material handling equipment to be inspected is consistent with the equipment category of this inspection task, that is, the equipment identity is compliant and correct. At this time, the step of determining the inspection parameters according to the target equipment type and the preset inspection configuration library is triggered. If the two do not match, it means that the actual equipment to be inspected on site does not match the equipment category reserved for the current inspection task, that is, the equipment is wrongly delivered or the task matching is abnormal. At this time, an alarm message of type mismatch can be sent to the terminal of the staff and the subsequent inspection parameter configuration process is terminated.

[0069] In this embodiment, the above steps can quickly identify problems such as misdelivery of equipment and abnormal tasks, reduce ineffective operations, save workstation resources, and improve overall detection and control capabilities and operational efficiency.

[0070] Optionally, the detection parameters may also include multiple device positioning markers and theoretical reference features.

[0071] Step 213: Based on multiple equipment positioning markers and theoretical reference features, perform feature extraction and matching calculation on the current appearance image to obtain the current pose state of the material handling equipment.

[0072] Specifically, equipment positioning markers are highly recognizable feature points pre-marked on the equipment's external structure, such as right-angle corners of the frame, vertices of tooling reference bosses, centers of reflective dots, and corners of QR codes. Each marker corresponds to a desired coordinate (including desired physical coordinates and desired two-dimensional coordinates) and a region of interest (ROI). Theoretical reference features are ideal standard feature information such as standard contour features, structural edge features, and geometric dimension features pre-defined for each equipment type, serving as the theoretical benchmark for actual image feature comparison.

[0073] In the specific implementation, based on the pre-set region parameters of each equipment positioning marker, the current appearance image is partially cropped to obtain the current region image corresponding to each equipment positioning marker. Then, feature extraction is performed on these region images to obtain the current image features of each equipment positioning marker. Next, the current image features of each equipment positioning marker are matched with the corresponding theoretical reference features to obtain the matching results for each equipment positioning marker. Then, the successfully matched equipment positioning markers are identified as target markers. When the number of target markers reaches or exceeds a preset threshold, pose calculation is performed using these target markers to obtain the current pose state of the material handling equipment.

[0074] In this embodiment, the above steps can effectively avoid background interference and improve the accuracy and stability of pose detection.

[0075] Further, step 213 may specifically include: performing region cropping processing on the current appearance image based on multiple device positioning markers to obtain the current region image of multiple device positioning markers; extracting features from the current region images of multiple device positioning markers to obtain the current image features of multiple device positioning markers; performing feature matching on the current image features of multiple device positioning markers using theoretical reference features to obtain the matching results of multiple device positioning markers; determining the device positioning markers with successful matching results as target markers; and performing pose calculation processing on multiple target markers when the number of target markers is not less than a preset number to obtain the current pose state of the material handling equipment.

[0076] Specifically, the current region image is a sub-image obtained by cropping the current appearance image based on the device positioning marker, containing only the local area surrounding the marker. The current image features are image feature data extracted from the current region image corresponding to the device positioning marker. The target marker is the device positioning marker that has successfully matched the features. The preset quantity is a minimum threshold number of target markers required for reliable pose calculation, pre-set according to actual conditions or needs.

[0077] In the specific implementation, for each device positioning marker, according to its preset ROI parameters, the corresponding local sub-image is cropped from the current appearance image to obtain the current area image of the device positioning marker. Next, feature extraction algorithms (such as scale-invariant feature transform and corner detection algorithms) are used to extract features from the current region image of the device's positioning marker, obtaining the current image features of the device's positioning marker. Then, the extracted current image features are compared with preset theoretical reference features to obtain the matching result of the device's positioning marker. If the matching result is successful, the device's positioning marker is determined as the target marker; if the matching fails, the marker is discarded and not included in subsequent calculations.

[0078] When the number of target markers obtained through screening is not less than a preset number (e.g., 3), the obtained target markers are used for pose calculation to determine the current pose state of the material handling equipment. The specific process is as follows: First, a one-to-one mapping relationship is established between the actual pixel coordinates of the target markers and the expected physical coordinates of the corresponding equipment positioning markers. Second, a pre-calibrated camera intrinsic parameter matrix (including focal length and principal point parameters) and lens distortion coefficients are introduced, and the image coordinates are distorted and associated with the coordinate system by combining the coordinate transformation relationship between the camera coordinate system and the detection station reference coordinate system. Then, a perspective multi-point pose calculation algorithm is used to calculate the pose parameters of the camera relative to the detection station reference coordinate system. Finally, the calculated pose parameters are projected onto the horizontal plane of the detection station to obtain the planar position and deflection angle of the material handling equipment under the detection station, i.e., the current pose state of the equipment. If the number of target markers obtained after screening is less than the preset number, it is determined that there are insufficient effective feature points and the pose calculation conditions are not met. At this time, an error message can be triggered, and a prompt message indicating that the number of feature points is insufficient can be sent so that staff can handle it in time and ensure that the subsequent pose detection process can be carried out normally.

[0079] In this embodiment, the efficiency and accuracy of pose detection are improved through the above steps.

[0080] Step 214: Determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state.

[0081] If satisfied, proceed to step 215; otherwise, proceed to step 216.

[0082] Step 215: Send the initial scan path to the scanning device so that the scanning device can scan the material handling equipment according to the initial scan path and obtain the point cloud data of the material handling equipment.

[0083] Step 216: Based on the current two-dimensional position information of multiple target markers and the expected two-dimensional position information of multiple device positioning markers, construct the covariance matrix of rigid body transformation.

[0084] Specifically, the current two-dimensional position information is the two-dimensional pixel coordinates of the target marker point in the camera pixel coordinate system. The desired two-dimensional position information is the theoretical standard two-dimensional coordinates of the equipment positioning marker point in the camera pixel coordinate system under standard reference conditions, serving as a reference for pose comparison. The camera pixel coordinate system is a two-dimensional Cartesian coordinate system defined on the digital image plane. It can take the upper left (or lower left) corner of the camera image as the origin, with pixels as the basic unit. The horizontal axis represents the column number (horizontal direction) of the pixel, and the vertical axis represents the row number (vertical direction). This coordinate system directly describes the pixel position of each feature point in the image. Rigid body transformation is a mathematical mapping in a two-dimensional plane that transforms one set of points (actual position) to another set of points (desired position) through rotation and translation. It is assumed that the material handling equipment is a rigid object with no structural deformation, and only a spatial position transformation model involving overall translation and rotation exists. The covariance matrix is ​​a matrix used to characterize the interaction between the two sets of points (target marker point and equipment positioning marker point).

[0085] In the specific implementation, the centroid of the point set composed of multiple target markers is first calculated to obtain the first centroid c1, and its calculation formula is as follows: Where N is the total number of valid marker points, i.e., the number of one-to-one matching pairs between target marker points and device positioning marker points, (u i v i ( ) represents the current two-dimensional position information of the i-th target marker point. Simultaneously, the centroid of the set of multiple device positioning marker points is calculated to obtain the second centroid c2, whose calculation formula is: , where (x i y i Let p be the expected two-dimensional position information of the i-th device positioning marker. Then, subtract the first centroid from the coordinates of each target marker to obtain the centroid-free position information p of the target marker. i Simultaneously, subtract the second centroid from the coordinates of each device positioning marker to obtain the centroid-free position information q of the device positioning marker. iThen, based on the decentroided position information of multiple target markers and multiple device positioning markers, the covariance matrix H of the rigid body transformation is constructed, and the calculation formula is as follows: .

[0086] In this embodiment, the above steps provide a data foundation for the subsequent calculation of the pose deviation matrix.

[0087] Step 217: Perform singular value decomposition on the covariance matrix to obtain the pose deviation matrix of the material handling equipment.

[0088] In the specific implementation, the covariance matrix H is first decomposed into the following form: H = UAV T Where U and V are both 2×2 orthogonal matrices (called the left singular vector matrix and the right singular vector matrix, respectively); A is a 2×2 diagonal matrix. Then, the optimal rotation matrix R is calculated using the formula: R = VU T Next, calculate the optimal translation vector t, using the formula: t = c2 - R·c1. Finally, construct the pose deviation matrix ΔT using the optimal rotation matrix and the optimal translation vector. ;in, , , , These are the components of the optimal rotation matrix R; , The horizontal and vertical components of the optimal translation vector t.

[0089] In this embodiment, the accuracy of the obtained pose deviation matrix is ​​improved through the above steps.

[0090] Step 218: Correct the initial scanning path according to the pose deviation matrix to obtain the target scanning path.

[0091] Step 219: Send the target scanning path to the scanning device so that the scanning device can scan the material handling equipment according to the target scanning path and obtain the point cloud data of the material handling equipment.

[0092] It should be noted that, in order to improve the accuracy, anti-interference ability and 3D reconstruction quality of point cloud data acquisition, the point cloud data acquired in this embodiment can preferably be blue light point cloud data. This data can be acquired by a blue light structured light 3D scanning device. Compared with traditional laser point clouds, its imaging effect on reflective, dark or complex curved workpieces is more stable, which can effectively reduce ambient light interference, improve the integrity and accuracy of point cloud data, and provide a more reliable data foundation for subsequent 3D model construction, registration and comparison and deviation analysis.

[0093] Step 220: Receive the point cloud data sent by the scanning device, and perform detection on the material handling equipment based on the point cloud data to obtain the detection result of the material handling equipment.

[0094] Optionally, the detection parameters may also include the desired 3D model.

[0095] Furthermore, the material handling equipment is inspected based on point cloud data to obtain the inspection results, including: constructing a current three-dimensional model of the material handling equipment based on point cloud data; performing registration comparison and deviation analysis on the current three-dimensional model and the expected three-dimensional model to obtain the inspection results of the material handling equipment.

[0096] Specifically, the desired 3D model is a 3D digital model of the material handling equipment under ideal standard conditions. It typically originates from computer-aided design drawings of the equipment, standard models obtained through reverse engineering, or high-precision reference models established by scanning standard equipment. This model includes the complete geometry, dimensions, and structural features of the equipment, serving as a theoretical benchmark for subsequent comparison and evaluation. The current 3D model is a real-time 3D digital model of the material handling equipment constructed from point cloud data collected on-site, after point cloud preprocessing (such as denoising, registration, and fusion) and 3D reconstruction, reflecting the equipment's current actual structural form and positional status.

[0097] In the specific implementation, the obtained point cloud data is first preprocessed, including outlier removal, noise filtering, point cloud simplification, and hole filling. Invalid and stray points caused by environmental interference are eliminated, while retaining the effective 3D point cloud information of the equipment itself. Then, 3D reconstruction algorithms (such as Poisson reconstruction, Delaunay triangulation, and moving least squares) are used to reconstruct the 3D data from the preprocessed point cloud data, generating the current 3D model of the material handling equipment. Next, using the key structural features and reference positioning locations of the equipment as constraints, point cloud registration algorithms (such as random sampling consistency registration algorithm and fast point feature histogram registration algorithm) are used to perform global coarse registration and fine registration between the current 3D model and the desired 3D model. This completes the spatial alignment and attitude matching of the two models in the same reference coordinate system, eliminating comparison errors caused by position offset and angle deflection.

[0098] Finally, based on the accurate registration of the two models, deviation analysis was carried out. The corresponding contours of the models, the installation positions of key components, the external dimensions and shape features were checked one by one to calculate parameters such as spatial position deviation, attitude deflection deviation and geometric dimension deviation. The various deviation indicators were comprehensively analyzed to obtain the detection results of the material handling equipment's shape, posture deviation and dimensional compliance.

[0099] For example, after completing the registration and alignment of the current 3D model and the desired 3D model, the specific detection steps can be as follows: (1) Basic data preparation: The relevant areas of the key detection parts (such as bolt holes and assembly surfaces) preset by the equipment are taken as the ROI areas. (2) Missing parts detection: Missing parts detection is carried out using the surface deviation method. Specifically, the ideal point set Pc corresponding to the surface (such as the bolt hole surface) of the key detection parts (such as bolts) in the desired 3D model is extracted, as well as the actual collected point cloud data Ps, and the points o that the actual point cloud data falls into the ROI of the key detection parts are calculated. i The root mean square deviation Erms from the ideal point set Pc is calculated using the following formula: ,in, For point arrive The nearest Euclidean distance is then determined. Next, it is determined whether Erms is greater than the preset fit tolerance threshold. If it is, it indicates that the critical inspection part (bolt hole) is missing; otherwise, it is determined that the critical inspection part is installed and is considered present.

[0100] (3) Misassembly detection: After determining that the part exists, the points within the ROI of the critical inspection parts (such as bolts) are checked. Perform a least-squares fit on the cylinder, with the goal of minimizing the sum of squared distances Ec from all points to the cylinder surface. ,in, The norm of a vector; is the center point of the cylinder; v is the fitted axial vector of the cylinder; r is the fitted radius of the cylinder; This refers to the number of point clouds within the ROI of the critical detection part, i.e., the total number of effective 3D points participating in cylinder fitting. Let Ec be a single 3D point within the ROI. Then, by minimizing Ec, the center point, axial vector, and fitted radius of the cylinder are obtained. Next, specification determination is performed: if |r-rc|≤Ttol, the critical inspection part (e.g., bolt) is considered to be of correct specification, where rc is the standard radius of the correct specification critical inspection part (e.g., bolt), obtained from the desired 3D model; Ttol is a preset radius tolerance threshold, i.e., the maximum allowable deviation between the actual radius and the standard radius of the critical inspection part (e.g., bolt). If |r-re|≤Ttol, the incorrect specification critical inspection part (e.g., bolt) is considered to be assembled, where re is the standard radius of the incorrect specification critical inspection part (e.g., bolt), i.e., the standard radius value of other models that do not conform to the target specification.

[0101] (4) Tilt Detection: Obtain the standard axial vector vc of the key inspection part (such as a bolt) from the desired 3D model, and calculate the tilt angle α between the fitted axial vector v and the standard axial vector vc. The specific calculation formula is: α = arccos(v·vc / (||v||||vc||)). If the tilt angle is greater than the preset angle threshold, the key inspection part (such as a bolt) is determined to be tilted; otherwise, it is determined not to be tilted. Finally, summarize the inspection data of missing parts, incorrect parts, and tilted parts to obtain the overall inspection results of the material handling equipment.

[0102] In this embodiment, the accuracy and reliability of the detection results are improved through the above steps.

[0103] Step 221: Send the test results to the staff's terminal.

[0104] In practice, after obtaining the test results, the results can be sent to the staff's terminals (such as mobile phones, computers, etc.) so that the staff can view the equipment assembly status in real time, quickly grasp the location of defects and deviation data, and carry out on-site verification, fault rectification and operation and maintenance scheduling in a timely manner.

[0105] Furthermore, based on the obtained detection results, deviation heatmaps and deviation comparison tables can be generated for easy viewing and analysis by staff. The deviation heatmap uses color differences to visually present the overall deviation distribution between the current 3D model and the desired 3D model, allowing for quick identification of abnormal deviation areas. The deviation comparison table records the quantified deviation values ​​and judgment results at each key detection location in a structured manner, facilitating data retention, traceability verification, and manual assistance in analysis.

[0106] The detection method for material handling equipment provided in this invention first acquires the current appearance image of the material handling equipment, providing a data foundation for subsequently determining the detection parameters and current pose state of the equipment. Based on the current appearance image, the type of material handling equipment is determined, resulting in the target equipment type. This achieves adaptive detection adaptation for different equipment types, improving the automation and efficiency of equipment classification and recognition, and providing accurate pre-classification basis for subsequent detection, ensuring the continuity and adaptability of the overall detection process. Detection parameters are determined according to the target equipment type and a preset detection configuration library. This allows for adaptive retrieval of specific detection parameters according to the equipment type, reducing manual intervention and thus improving the automation and overall accuracy of the detection process. Feature extraction and matching calculations are performed on the current appearance image based on multiple equipment positioning markers and theoretical reference features to obtain the current pose state of the material handling equipment. This effectively avoids background interference and improves the accuracy and stability of pose detection. Finally, the current pose state and the desired pose state are used to determine whether the material handling equipment meets the pose matching conditions. If the conditions are met, the initial scan path is sent to the scanning device, allowing it to scan the material handling equipment and obtain point cloud data. This improves inspection efficiency while maintaining scanning accuracy. If the conditions are not met, a covariance matrix of rigid body transformation is constructed based on the current 2D position information of multiple target markers and the expected 2D position information of multiple equipment positioning markers. This provides a data foundation for subsequent calculation of the pose deviation matrix. Singular value decomposition is performed on the covariance matrix to obtain the pose deviation matrix of the material handling equipment, improving its accuracy. The initial scan path is then corrected based on the pose deviation matrix to obtain the target scan path. This ensures the generated target scan path closely matches the actual placement pose of the equipment, effectively avoiding scanning blind spots, trajectory misalignment, and partial missed scans caused by equipment pose deviations, thus improving the adaptability of the scan path to the actual working conditions. Furthermore, no manual intervention is required to adjust the scan path, significantly reducing human intervention and operational errors, effectively improving the overall automation level and inspection efficiency. Subsequently, the target scanning path is sent to the scanning device, enabling it to scan the material handling equipment according to the target scanning path and obtain point cloud data of the equipment. This effectively avoids scanning blind spots and missed scans caused by equipment pose offset, ensuring high integrity, effectiveness, and reliability of the collected point cloud data. This provides high-quality raw data support for subsequent inspection of the material handling equipment, thereby improving the accuracy and stability of the overall equipment inspection results. The system receives the point cloud data sent by the scanning device and performs inspections on the material handling equipment based on this data, obtaining the inspection results. This effectively improves the intelligence, accuracy, and reliability of equipment inspection.The test results are sent to the staff's terminals so that they can view the equipment assembly status in real time, quickly grasp the location of defects and deviation data, and promptly carry out on-site verification, fault rectification, and operation and maintenance scheduling. Therefore, the technical solution of this invention can solve the problem in the prior art that the use of a fixed preset scanning path easily leads to incomplete point cloud data acquisition, thereby reducing the accuracy of equipment testing.

[0107] Figure 3 This is a schematic diagram of the structure of a detection device for a material handling equipment provided in an embodiment of the present invention. This device belongs to the same inventive concept as the detection methods for the material handling equipment in the above embodiments. For details not described in detail in the embodiments of the detection device for the material handling equipment, please refer to the embodiments of the detection methods for the material handling equipment described above.

[0108] like Figure 3 As shown, the device includes: The determination module 310 is used to determine the detection parameters and current pose state of the material handling equipment; the detection parameters include the desired pose state and the initial scanning path. The judgment module 320 is used to determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. The correction module 330 is used to calculate the pose deviation matrix of the material handling equipment if the conditions are not met, and to correct the initial scanning path according to the pose deviation matrix to obtain the target scanning path. The sending module 340 is used to send the target scanning path to the scanning device, so that the scanning device scans the material handling device according to the target scanning path to obtain point cloud data of the material handling device; The detection module 350 is used to receive the point cloud data sent by the scanning device, and to detect the material handling device based on the point cloud data to obtain the detection result of the material handling device.

[0109] Based on the above embodiments, the determining module 310 determines the detection parameters and current pose state of the material handling equipment, including: Acquire the current appearance image of the material handling equipment; determine the type of the material handling equipment based on the current appearance image to obtain the target equipment type; determine the detection parameters according to the target equipment type and a preset detection configuration library; the detection parameters also include multiple equipment positioning markers and theoretical reference features; perform feature extraction and matching calculation on the current appearance image according to the multiple equipment positioning markers and the theoretical reference features to obtain the current pose state of the material handling equipment.

[0110] Based on the above embodiments, the determining module 310 performs feature extraction and matching calculation on the current appearance image according to the plurality of equipment positioning markers and the theoretical reference features to obtain the current pose state of the material handling equipment, including: Based on the multiple device positioning markers, the current appearance image is processed by region cropping to obtain the current region image of the multiple device positioning markers; features are extracted from the current region images of the multiple device positioning markers to obtain the current image features of the multiple device positioning markers; feature matching is performed on the current image features of the multiple device positioning markers using the theoretical reference features to obtain the matching results of the multiple device positioning markers; the device positioning markers with successful matching results are determined as target markers; if the number of target markers is not less than a preset number, pose calculation is performed on the multiple target markers to obtain the current pose state of the material handling equipment.

[0111] Based on the above embodiments, the device further includes: The type discrimination module is used to obtain the current detection task before determining the detection parameters based on the target device type and the preset detection configuration library; determine whether the device type in the current detection task matches the target device type; if they match, trigger the execution of determining the detection parameters based on the target device type and the preset detection configuration library.

[0112] Based on the above embodiments, the correction module 330 calculates the pose deviation matrix of the material handling equipment, including: Based on the current two-dimensional position information of multiple target markers and the expected two-dimensional position information of multiple device positioning markers, a covariance matrix of rigid body transformation is constructed; singular value decomposition is performed on the covariance matrix to obtain the pose deviation matrix of the material handling equipment.

[0113] Based on the above embodiments, the detection parameters further include a desired three-dimensional model. The detection module 350 detects the material handling equipment based on the point cloud data to obtain the detection result of the material handling equipment, including: A current 3D model of the material handling equipment is constructed based on the point cloud data; the current 3D model and the desired 3D model are registered, compared, and subjected to deviation analysis to obtain the detection results of the material handling equipment.

[0114] Based on the above embodiments, the device further includes: The display module is used to send the test results to the staff's terminal after obtaining the test results from the material handling equipment.

[0115] The detection device for the material handling equipment provided in the embodiments of the present invention can execute the detection method for the material handling equipment provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0116] It is worth noting that in the embodiments of the detection device of the above-mentioned material handling equipment, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0117] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0118] like Figure 4 As shown, electronic device 4 is represented in the form of a general-purpose computing electronic device. The components of electronic device 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0119] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0120] Electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 4, including volatile and non-volatile media, removable and non-removable media.

[0121] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0122] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0123] Electronic device 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 4, and / or with any device that enables electronic device 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of electronic device 4 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be combined with electronic device 4 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0124] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, such as implementing the detection method of the material handling equipment provided in the embodiments of the present invention.

[0125] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the detection method of the material handling equipment provided in any embodiment of the present invention.

[0126] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements, for example, the detection method of the material handling equipment provided in this invention.

[0127] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0128] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0129] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0130] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0132] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with relevant laws and regulations.

[0133] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for detecting material handling equipment, characterized in that, The method includes: Determine the detection parameters and current pose state of the material handling equipment; the detection parameters include the desired pose state and the initial scan path. Determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. If the conditions are not met, the pose deviation matrix of the material handling equipment is calculated, and the initial scanning path is corrected according to the pose deviation matrix to obtain the target scanning path. The target scanning path is sent to the scanning device so that the scanning device scans the material handling device according to the target scanning path to obtain point cloud data of the material handling device; The device receives the point cloud data sent by the scanning device and performs detection on the material handling device based on the point cloud data to obtain the detection result of the material handling device.

2. The method according to claim 1, characterized in that, Determine the detection parameters and current position and orientation of the material handling equipment, including: Acquire a current external image of the material handling equipment; Based on the current appearance image, the type of material handling equipment is determined to obtain the target equipment type; The detection parameters are determined based on the target device type and a preset detection configuration library; the detection parameters also include multiple device positioning markers and theoretical reference features. Based on the multiple device positioning markers and the theoretical reference features, feature extraction and matching calculations are performed on the current appearance image to obtain the current pose state of the material handling equipment.

3. The method according to claim 2, characterized in that, Based on the multiple device positioning markers and the theoretical reference features, feature extraction and matching calculations are performed on the current appearance image to obtain the current pose state of the material handling equipment, including: Based on the multiple device positioning markers, the current appearance image is processed by region cropping to obtain the current region image of the multiple device positioning markers; Feature extraction is performed on the current region images of multiple device positioning markers to obtain the current image features of multiple device positioning markers; Using the theoretical reference features, feature matching is performed on the current image features of multiple device positioning markers to obtain matching results for multiple device positioning markers. The device location markers that are successfully matched are designated as target markers. If the number of target markers is not less than a preset number, pose calculation is performed on multiple target markers to obtain the current pose state of the material handling equipment.

4. The method according to claim 2, characterized in that, Before determining the detection parameters based on the target device type and the preset detection configuration library, the method further includes: Get the current detection task; Determine whether the device type in the current detection task matches the target device type; If a match is found, execution is triggered to determine the detection parameters based on the target device type and the preset detection configuration library.

5. The method according to claim 3, characterized in that, Calculate the pose deviation matrix of the material handling equipment, including: Based on the current two-dimensional position information of multiple target markers and the expected two-dimensional position information of multiple device positioning markers, a covariance matrix of rigid body transformation is constructed. The covariance matrix is ​​subjected to singular value decomposition to obtain the pose deviation matrix of the material handling equipment.

6. The method according to claim 1, characterized in that, The detection parameters also include a desired 3D model. Based on the point cloud data, the material handling equipment is detected to obtain the detection results of the material handling equipment, including: Construct a current 3D model of the material handling equipment based on the point cloud data; The current 3D model and the desired 3D model are registered, compared, and subjected to deviation analysis to obtain the detection results of the material handling equipment.

7. The method according to claim 1, characterized in that, After obtaining the test results from the material handling equipment, the following is also included: The test results are sent to the staff's terminal.

8. A detection device for material handling equipment, characterized in that, The device includes: The determination module is used to determine the detection parameters and current pose state of the material handling equipment; the detection parameters include the desired pose state and the initial scanning path. The judgment module is used to determine whether the material handling equipment meets the pose matching conditions based on the current pose state and the desired pose state. The correction module is used to calculate the pose deviation matrix of the material handling equipment if the conditions are not met, and to correct the initial scanning path according to the pose deviation matrix to obtain the target scanning path. The sending module is used to send the target scanning path to the scanning device, so that the scanning device scans the material handling device according to the target scanning path to obtain point cloud data of the material handling device; The detection module is used to receive the point cloud data sent by the scanning device, and to detect the material handling equipment based on the point cloud data to obtain the detection result of the material handling equipment.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the detection method of the material handling equipment according to any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the detection method of any of the material handling equipment described in claims 1-7.

Images