Structural damage detection method and system

Abstract

The present application discloses a method and system for detecting structural damage, which includes: when the component to be inspected is in motion, sequentially collecting image data of the component to be inspected through multiple image acquisition devices, and generating corresponding first point cloud data based on each of the image data; based on the camera extrinsics of each of the image acquisition devices, converting the multiple first point cloud data into multiple second point cloud data in the same coordinate system; constructing a three-dimensional model corresponding to the component to be inspected based on the multiple second point cloud data; and determining the degree of structural damage to the component to be inspected based on the size information of the three-dimensional model. This method can accurately detect the three dimensions of the component to be inspected in a non-contact manner, and then accurately determine the degree of structural damage to the component to be inspected in a moving state. When the degree of structural damage to the component to be inspected is relatively serious, it prompts the user to replace the component to be inspected, thereby avoiding structural damage to the component to be inspected and causing production accidents.

Description

Technical Field

[0001] The present application relates to the technical field of non-destructive testing, and in particular to a structural damage detection method and system. Background Art

[0002] Scraper conveyors are essential tools for daily operations in coal mines. Under prolonged loads like coal, the scraper chain can deform and eventually break, impacting conveying operations. Replacing a deformed chain is relatively easy, typically taking only about 30 minutes. However, disassembly, reassembly, and cleaning the entire conveyor chain after a chain break is more challenging, typically requiring approximately 20 hours to complete. This time-consuming process can severely impact mine operations and reduce production capacity. Detecting the extent of chain damage before it breaks and replacing it promptly when severe damage occurs can help ensure safe and efficient production in coal mines.

[0003] Structural damage detection for such moving parts is usually performed through manual visual inspection or using handheld devices. However, inspectors need to enter narrow tunnels to inspect the chains, which not only faces high safety risks, but also has low inspection efficiency, long detection time, and low accuracy. Summary of the Invention

[0004] This application provides a structural damage detection method and system. The technical solutions adopted in the embodiments of this application are as follows:

[0005] On one hand, the present application provides a structural damage detection method, comprising:

[0006] When the component to be inspected is in motion, multiple image acquisition devices are used to sequentially acquire image data of the component to be inspected, and corresponding first point cloud data are generated based on each of the image data; wherein the multiple image acquisition devices have different camera angles, and the first point cloud data is point cloud data in the camera coordinate system of the image acquisition device;

[0007] Based on camera extrinsic parameters of each of the image acquisition devices, converting the plurality of first point cloud data into a plurality of second point cloud data in the same coordinate system; wherein the camera extrinsic parameters are calibrated based on a first timing parameter and a first motion parameter, the first timing parameter is used to identify a sampling time interval and a sampling order of the plurality of image acquisition devices, and the first motion parameter is used to characterize a motion state of the component to be inspected;

[0008] constructing a three-dimensional model corresponding to the component to be inspected based on the plurality of second point cloud data;

[0009] The degree of structural damage of the component to be inspected is determined based on the dimensional information of the three-dimensional model.

[0010] In some embodiments, the plurality of image acquisition devices are sequentially arranged along a first direction, where the first direction is a moving direction of the component to be inspected; sequentially acquiring image data of the component to be inspected by the plurality of image acquisition devices includes:

[0011] Image data of the component to be inspected is collected in sequence along a second direction by a plurality of image collection devices, wherein the second direction is opposite to the moving direction of the component to be inspected.

[0012] In some embodiments, the plurality of image acquisition devices include more than three image acquisition devices; and sequentially acquiring image data of the component to be inspected by the plurality of image acquisition devices includes:

[0013] Collecting image data of the component to be inspected at one end in the moving direction by an image acquisition device;

[0014] Collecting image data of the side of the component to be inspected in the moving direction by at least one of the image acquisition devices;

[0015] The image data of the component to be inspected at the other end in the moving direction is collected by another image collection device.

[0016] In some embodiments, the component to be inspected is a chain; determining the degree of structural damage of the component to be inspected based on the dimensional information of the three-dimensional model includes:

[0017] determining the length of the chain links of the chain in the length direction of the chain;

[0018] If the length is greater than a first threshold, determining that the link needs to be replaced;

[0019] When the length is less than the first threshold, it is determined that the chain link does not need to be replaced.

[0020] In some embodiments, the method further comprises:

[0021] Acquiring the first timing parameter and the first motion parameter;

[0022] Determining a parallel vector in the moving direction of the component to be inspected based on the first timing parameter and the first motion parameter; wherein the parallel vector is used to compensate for the sampling time intervals of multiple image acquisition devices and the displacement of the component to be inspected in the moving direction;

[0023] The camera extrinsic parameters of each of the image acquisition devices are calibrated based on the parallel vectors.

[0024] In some embodiments, the sampling time interval is configured to shorten as the moving speed of the component to be inspected increases, and to lengthen as the moving speed of the component to be inspected decreases.

[0025] Another aspect of the present application provides a structural damage detection system, comprising:

[0026] Multiple image acquisition devices, configured to sequentially acquire image data of the component to be inspected while the component to be inspected is in motion, and to generate corresponding first point cloud data based on each image data; wherein the multiple image acquisition devices have different camera angles, and the first point cloud data is point cloud data in the camera coordinate system of the image acquisition device;

[0027] A processing device is configured to convert a plurality of the first point cloud data into a plurality of second point cloud data in the same coordinate system based on the camera extrinsic parameters of each of the image acquisition devices; construct a three-dimensional model corresponding to the component to be inspected based on the plurality of the second point cloud data; and determine the degree of structural damage of the component to be inspected based on the size information of the three-dimensional model; wherein the camera extrinsic parameter is calibrated based on a first timing parameter and a first motion parameter, the first timing parameter is used to identify the sampling time interval and sampling order of the plurality of the image acquisition devices, and the first motion parameter is used to characterize the motion state of the component to be inspected.

[0028] In some embodiments, the plurality of image acquisition devices are sequentially arranged along a first direction, where the first direction is the moving direction of the component to be inspected; the plurality of image acquisition devices are specifically configured as follows:

[0029] The image data of the component to be inspected is collected sequentially along a second direction; wherein the second direction is opposite to the moving direction of the component to be inspected.

[0030] In some embodiments, the plurality of image acquisition devices include more than three image acquisition devices, wherein:

[0031] One of the image acquisition devices is configured to: acquire image data of the component to be inspected at one end in the moving direction;

[0032] At least one of the image acquisition devices is configured to: acquire image data of the side of the component to be inspected in the moving direction;

[0033] The other image acquisition device is configured to acquire image data of the component to be inspected at the other end in the moving direction.

[0034] In some embodiments, the component to be inspected is a chain; the processing device is specifically configured as follows:

[0035] determining the length of the chain links of the chain in the length direction of the chain;

[0036] If the length is greater than a first threshold, determining that the link needs to be replaced;

[0037] When the length is less than the first threshold, it is determined that the chain link does not need to be replaced.

[0038] The structural damage detection method of the embodiment of the present application calibrates the camera extrinsics in advance based on the sampling time intervals and sampling orders of multiple image acquisition devices, as well as the first motion parameters that can characterize the motion state of the component to be inspected; when the component to be inspected is in motion, image data of the component to be inspected is sequentially acquired by multiple image acquisition devices to avoid mutual interference among the multiple image acquisition devices, and corresponding first point cloud data are generated based on each image data; based on the pre-calibrated camera extrinsics, the multiple first point cloud data are converted into multiple second point cloud data in the same coordinate system; based on the multiple second point cloud data, a three-dimensional model of the component to be inspected can be accurately constructed, and by detecting the dimensional information of the three-dimensional model, the purpose of non-contact and accurate detection of the dimensional information of the component to be inspected is achieved, thereby accurately judging the degree of structural damage of the component to be inspected. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a structural damage detection method according to an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a scenario of a structural damage detection method according to an embodiment of the present application;

[0041] Figure 3 A schematic diagram of image data collected by an image acquisition device;

[0042] Figure 4 is a schematic diagram of the second point cloud data in the world coordinate system;

[0043] Figure 5 is a schematic diagram of the three-dimensional model;

[0044] Figure 6 Schematic diagram of a method for detecting the size information of a three-dimensional model;

[0045] Figures 7 to 9 Schematic diagram of the image acquisition process during chain movement;

[0046] Figure 10 Schematic diagram of the principle of compensating the spatial position of the image acquisition device for parallel vectors;

[0047] Figure 11 This is a structural block diagram of a structural damage detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0049] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0050] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0051] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0052] It should also be understood that although the present application has been described with reference to certain specific examples, those skilled in the art will be able to implement many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0053] The above and other aspects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0054] Specific embodiments of the present application will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments described are merely examples of the present application and may be implemented in a variety of ways. Familiar and / or repetitive functions and structures are not described in detail to avoid obscuring the present application with unnecessary or redundant details. Therefore, the specific structural and functional details described herein are not intended to be limiting, but rather serve merely as a basis and representative basis for the claims to teach those skilled in the art to variously utilize the present application with substantially any suitable detailed structure.

[0055] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may all refer to one or more of the same or different embodiments according to the present application.

[0056] The present application provides a structural damage detection method that accurately detects the dimensions of a component under inspection in a non-contact manner, thereby accurately determining the extent of structural damage to the component under inspection while in motion. If the extent of structural damage to the component under inspection is severe, the user is prompted to replace the component under inspection, thereby preventing structural damage to the component under inspection from causing production accidents. This method is applicable to detecting the extent of structural damage to various types of components under inspection while in motion, such as chains or other types of components under inspection, and the specific type of component under inspection is not limited herein.

[0057] Figure 1 This is a flow chart of the structural damage detection method according to an embodiment of the present application. Figure 1 As shown, the structural damage detection method of the embodiment of the present application may specifically include the following steps.

[0058] S101 , when a component to be inspected is in motion, image data of the component to be inspected is sequentially collected by a plurality of image acquisition devices, and corresponding first point cloud data is generated based on each image data.

[0059] Optionally, the image acquisition device may include a depth camera or other three-dimensional camera. Depth cameras include, but are not limited to, structured light depth cameras, time-of-flight depth cameras, and binocular stereo vision depth cameras. Optionally, the image acquisition device may employ, for example, a Gray code stripe area array camera. Gray code stripe area array cameras offer high detection accuracy and facilitate accurate detection of the three dimensions of the inspected component.

[0060] Optionally, multiple image acquisition devices can be set at positions adjacent to the motion path of the component to be inspected, and can be set in sequence along the motion path of the component to be inspected. Relative to the motion path of the component to be inspected, the camera angles of each image acquisition device may be different, so that the image of the component to be inspected can be comprehensively captured from multiple angles.

[0061] Taking the part to be inspected as a chain, and the chain moving in a straight line as an example, cameras A, B and C can be set in sequence along the moving direction of the chain. Cameras A, B and C can all be Gray code fringe array cameras. Cameras A, B and C can be configured to have different camera angles, such as Figure 2 shown.

[0062] Of course, in specific implementation, the position and camera angle of the image acquisition device can be set according to factors such as the shape and movement state of the inspected component, so as to clearly capture the external contour image of the inspected component, and is not limited to the above settings.

[0063] Optionally, in addition to presetting the positions and camera angles of the multiple image acquisition devices, the sampling time intervals and sampling order of the multiple image acquisition devices may also be pre-set. For example, if the multiple image acquisition devices are sequentially arranged along the motion path of the component to be inspected, the multiple image acquisition devices may be configured to sequentially capture images along the direction of motion of the component to be inspected or in a direction opposite to the motion direction. Of course, the multiple image acquisition devices may also sequentially capture images of the component to be inspected in other orders. Optionally, the multiple image acquisition devices may be configured to sequentially sample at equal time intervals, and the sampling time intervals of image acquisition devices in adjacent sampling orders may also vary. This configuration may be specifically based on the positions of the image acquisition devices, the camera angles, and the motion state of the component to be inspected.

[0064] Optionally, during the inspection process, multiple image acquisition devices can be controlled to sequentially acquire image data of the components to be inspected based on a pre-configured sampling order and sampling time interval. Figure 3 The images shown in a, b, c and d may be captured by different image capture devices.

[0065] Based on the acquired image data of the component to be inspected, the acquired image data can be processed based on the camera intrinsic parameters of each image acquisition device to acquire first point cloud data corresponding to each image data. The first point cloud data is point cloud data in the camera coordinate system of the image acquisition device, and the first point cloud data includes both the plane coordinates of the pixel points and the depth coordinates of the pixel points.

[0066] Optionally, based on the acquired image data, the image processing chip of each depth camera can directly process the image data into the first point cloud data based on its own camera intrinsic parameters and other parameters, or the image processor can separately obtain the image data collected by each image acquisition device, as well as the camera intrinsic parameters and other parameters of each image acquisition device, and then obtain the corresponding first point cloud data based on the image data.

[0067] S102: Based on the camera extrinsic parameters of each of the image acquisition devices, convert the plurality of first point cloud data into a plurality of second point cloud data in the same coordinate system.

[0068] Optionally, based on setting the sampling time intervals and sampling order of multiple image acquisition devices, first timing parameters including the sampling time intervals and sampling order of the multiple image acquisition devices can be obtained. The motion state of the component to be inspected can be pre-determined, and first motion parameters capable of characterizing the motion state of the component to be inspected can be obtained. For example, taking the example of the component to be inspected moving in a straight line, the first motion parameters can include the motion direction and speed of the component to be inspected. Based on the obtained first timing parameters and first motion parameters, the camera extrinsic parameters can be pre-calibrated based on the first timing parameters and first motion parameters.

[0069] Optionally, based on the first point cloud data corresponding to each image data, the corresponding first point cloud data can be converted to the same coordinate system based on the camera extrinsic parameters of each image acquisition device to form multiple second point cloud data. For example, the first point cloud data in the camera coordinate system can be converted to the world coordinate system based on the rotation matrix and the translation vector to form the second point cloud data in the world coordinate system, such as Figure 4 shown.

[0070] It can be understood that the same coordinate system can be a world coordinate system, a camera coordinate system of a certain image acquisition device, or other coordinate systems. As long as the first point cloud data in the camera coordinate system of each image acquisition device can be converted to the same coordinate system based on the calibrated camera extrinsics to form multiple second point cloud data, the type of the same coordinate system is not limited here.

[0071] S103: Construct a three-dimensional model corresponding to the component to be inspected based on the plurality of second point cloud data.

[0072] Optionally, multiple second point cloud data located in the same coordinate system can be integrated into target point cloud data, which can describe the overall external contour of the component to be inspected. Of course, the overall external contour described here does not necessarily include the external contours of all surfaces of the component to be inspected, but should be understood as a relatively complete external contour that meets the detection requirements. For example, if it is necessary to detect the length dimension of the component to be inspected, then the target point cloud data can accurately describe the external contour of the component to be inspected in the length direction; if it is necessary to detect the width dimension of the component to be inspected, then the target point cloud data can accurately describe the external contour of the component to be inspected in the width direction, and does not necessarily include, for example, the external contour of the bottom surface or other surfaces.

[0073] Optionally, once target point cloud data is acquired, noise reduction processing can be performed on the target point cloud data to remove clutter and noise points, obtaining valid target point cloud data and avoiding interference from clutter and noise points. This not only improves the accuracy of the constructed 3D model but also reduces the amount of data processing required for dimensional modeling. Optionally, feature point clouds within the valid target point cloud data can be identified, and boundary points extracted from them. A 3D model of the component to be inspected can be constructed based on the feature point clouds and boundary points.

[0074] Still taking the chain as an example, the constructed three-dimensional model can be as follows: Figure 5As shown, it can be seen that the outer contour of the three-dimensional model is consistent or basically consistent with the outer contour of the chain. By detecting the size information of the three-dimensional model, the purpose of detecting the size information of the chain can be indirectly achieved without the need for contact with the chain and without affecting the operation of the chain.

[0075] S104: Determine the degree of structural damage of the component to be inspected based on the size information of the three-dimensional model.

[0076] Optionally, one or more target dimensions may be determined in advance based on the motion state, force direction, and historical damage data of the component to be inspected. For example, if the component to be inspected primarily moves along its length, the force applied to the component to be inspected is primarily tension in the lengthwise direction, and historical damage data indicates that structural damage to the component to be inspected is primarily due to deformation of the component along the lengthwise direction caused by tension, then the lengthwise direction of the component to be inspected may be determined as the target dimension.

[0077] Optionally, reference data may be set in advance based on the dimensional information of the component to be inspected in the target dimension. The reference data may include the correspondence between the dimensional information in the target dimension and the degree of structural damage. Optionally, the initial dimensional information in the target dimension of the component to be inspected when no structural damage occurs when it leaves the factory may be determined based on the design parameters of the component to be inspected. The critical dimensional information in the target dimension when structural damage occurs to the component to be inspected may also be determined based on historical damage data. Multiple dimensional information ranges may be set based on the initial dimensional information and the critical dimensional information, and the degree of structural damage corresponding to each dimensional information range may be determined. For example, the degree of structural damage may be divided into three levels: mild damage, moderate damage, and severe damage. Correspondingly, three dimensional information ranges may be set based on the initial dimensional information and the critical dimensional information, corresponding to mild damage, moderate damage, and severe damage, respectively.

[0078] Optionally, after constructing a three-dimensional model of the component to be inspected, the dimensional information of the three-dimensional model in the target dimension can be detected. For example, the dimensional information of the inspected component in the length direction, width direction, axial direction, circumferential direction or other specific directions can be detected. After obtaining the dimensional information of the three-dimensional model in the target dimension, it can be compared with the reference data to determine the structural damage process of the component to be inspected. When relatively verified structural damage occurs, prompt operations or early warning operations can also be performed. For example, a prompt message can be sent to the management personnel, or while ensuring safety, the operation of the component to be inspected can be suspended, and the user can be prompted that the component to be inspected has relatively serious structural damage, and the user is asked to perform maintenance to avoid structural damage to the component to be inspected and cause an accident.

[0079] Optionally, the initial size information of the component to be inspected in the target dimension can also be obtained based on image acquisition. For example, in the initial state of the component to be inspected, image data of the component to be inspected can be acquired using an image acquisition device, first point cloud data and second point cloud data can be acquired, and a three-dimensional model can be constructed. The size information of the three-dimensional model in the target dimension can be used as the initial size information of the component to be inspected in the target dimension.

[0080] The structural damage detection method of the embodiment of the present application calibrates the camera extrinsics in advance based on the sampling time intervals and sampling orders of multiple image acquisition devices, as well as the first motion parameters that can characterize the motion state of the component to be inspected; when the component to be inspected is in a motion state, the image data of the component to be inspected is sequentially acquired by the multiple image acquisition devices to avoid interference between the multiple image acquisition devices, and corresponding first point cloud data are generated based on each image data; based on the pre-calibrated camera extrinsics, the multiple first point cloud data are converted into multiple second point cloud data in the same coordinate system; based on the multiple second point cloud data, a three-dimensional model of the component to be inspected can be accurately constructed, and by detecting the dimensional information of the three-dimensional model, the purpose of non-contact and accurate detection of the dimensional information of the component to be inspected is achieved, thereby accurately judging the degree of structural damage of the component to be inspected.

[0081] In some embodiments, the plurality of image acquisition devices are sequentially arranged along a first direction, where the first direction is a moving direction of the component to be inspected; sequentially acquiring image data of the component to be inspected by the plurality of image acquisition devices includes:

[0082] Image data of the component to be inspected is collected in sequence along a second direction by a plurality of image collection devices, wherein the second direction is opposite to the moving direction of the component to be inspected.

[0083] That is, multiple image acquisition devices can be set in sequence along the moving direction of the part to be inspected, and multiple image acquisition devices can sample in sequence along the opposite direction of the moving direction of the part to be inspected. In this way, a single image acquisition device can capture a larger range of external contour images of the part to be inspected, and multiple image acquisition devices can work together to capture a relatively complete external contour image of the part to be inspected, which is beneficial to improving the completeness of the three-dimensional model relative to the part to be inspected.

[0084] Cooperate Figure 2As shown, the component to be inspected may be a chain, which may be, for example, a chain of a scraper conveyor, and the chain may move in a straight line along its length. Optionally, multiple image acquisition devices such as camera A, camera B, and camera C may be arranged in sequence at positions adjacent to the chain along the moving direction of the chain. Camera A, camera B, and camera C may have different camera angles, and the chain may move in a straight line from camera A to camera C. During image acquisition, when a link in the chain enters the image acquisition range, camera C may first acquire image data of the one or more links, and then camera B may acquire image data of the one or more links, and then camera A may acquire image data of the link, as shown in FIG. Figures 7 to 9 Optionally, camera C, camera B, and camera A may sequentially capture image data of the component to be inspected at the same preset time interval. For example, camera C, camera B, and camera A may cyclically capture image data of the component to be inspected at 20-second intervals.

[0085] In some embodiments, the plurality of image acquisition devices include more than three image acquisition devices; and sequentially acquiring image data of the component to be inspected by the plurality of image acquisition devices includes:

[0086] Collecting image data of the component to be inspected at one end in the moving direction by an image acquisition device;

[0087] Collecting image data of the side of the component to be inspected in the moving direction by at least one of the image acquisition devices;

[0088] The image data of the component to be inspected at the other end in the moving direction is collected by another image collection device.

[0089] Optionally, multiple image acquisition devices may be sequentially arranged along the movement path of the component to be inspected, with the imaging ranges of the multiple image acquisition devices collectively constituting a target acquisition range. The movement path of the component to be inspected within the target acquisition range may be defined as a target path segment, which may have opposing first and second ends. When the component to be inspected moves from the first end to the second end and enters the target path segment from the first end, an image acquisition device located near the second end may first perform image acquisition to capture image data at one end of the component to be inspected in the movement direction. When the component to be inspected moves to the middle of the target path segment, one or more image acquisition devices located near the middle of the target path segment may sequentially capture image data of the lateral portions of the component to be inspected in the movement direction. When the component to be inspected moves to a position near the second end of the target path segment, an image acquisition device located near the first end of the target path segment may capture image data at the other end of the component to be inspected in the movement direction.

[0090] For example, Figure 2Taking the specific example shown as an example, the imaging ranges of cameras A, B and C jointly define the target path. The length of the target path segment, the moving speed of the chain, the sampling time intervals of the three cameras and the imaging ranges of the three cameras can be configured as follows: when a link in the chain enters the target path segment from the first end, the camera C close to the second end collects image data at one end of the link; when the link moves to the middle of the target path segment, the camera B collects image data of the side of the link; when the link moves to the second end of the target path segment, the camera C collects image data at the other end of the link, in order to obtain a relatively complete external contour image of the link, so as to construct a relatively complete three-dimensional model of the link to accurately detect the degree of structural damage of the link.

[0091] In some embodiments, the component to be inspected is a chain; determining the degree of structural damage of the component to be inspected based on the dimensional information of the three-dimensional model includes:

[0092] determining the length of the chain links of the chain in the length direction of the chain;

[0093] If the length is greater than a first threshold, determining that the link needs to be replaced;

[0094] When the length is less than the first threshold, it is determined that the chain link does not need to be replaced.

[0095] Alternatively, historical damage data indicates that chains typically damage by deformation of chain links along their length under tension. Excessive deformation can lead to breakage. Therefore, reference data can be pre-set based on information about the initial length of the chain links and their critical length. This reference data can include correspondences between multiple length ranges and the degree of structural damage to the chain links. The lower limit of the length range corresponding to severe damage to the chain links can be set as the first threshold.

[0096] In specific implementation, multiple cameras can be set in sequence along the moving direction of the chain, such as Figure 2 As shown. Multiple cameras can collect image data of the chain in a moving state in sequence based on a pre-set sampling order and sampling time interval, as shown in FIG. Figure 3 After acquiring the image data, the first point cloud data can be acquired based on the camera intrinsic parameters of the camera, and then the first point cloud data in the camera coordinate system of each camera can be converted to the world coordinate system based on the external parameters of each camera to form multiple second point cloud data, such as Figure 4 Then, a three-dimensional model of the chain can be constructed based on the plurality of second point cloud data, as shown in FIG. Figure 5 shown.

[0097] When the three-dimensional model of the chain is completed, the length of the chain link in the longitudinal direction of the chain can be detected, such as Figure 6 This length is then compared with a first threshold value, as shown in FIG. If the length is greater than the first threshold value, the corresponding chain link is determined to be severely damaged, and a prompt message may be sent to the user to prompt the user to replace the chain link to avoid production delays due to chain link damage. If the length is less than the first threshold value, the chain link is not severely damaged and is unlikely to suffer structural damage, so testing can continue.

[0098] In some embodiments, the method may further include:

[0099] Acquiring the first timing parameter and the first motion parameter;

[0100] Determining a parallel vector in the moving direction of the component to be inspected based on the first timing parameter and the first motion parameter; wherein the parallel vector is used to compensate for the sampling time intervals of multiple image acquisition devices and the displacement of the component to be inspected in the moving direction;

[0101] The camera extrinsic parameters of each of the image acquisition devices are calibrated based on the parallel vectors.

[0102] Since the component to be inspected is in motion and multiple image acquisition devices do not capture image data of the component to be inspected at the same time, it is not possible to calibrate the camera extrinsic parameters based on the fixed position relationship of the multiple image acquisition devices. It is necessary to compensate for the sampling time interval of the image acquisition device and the displacement of the component to be inspected in the moving direction.

[0103] by Figure 2 Taking the application scenario shown in the figure as an example, when the chain link X enters the target path segment from the first end, the camera C first collects the image data of the chain link X. At this time, the relative position relationship between the chain link X and the camera C is as follows: Figure 7 When link X moves to the middle of the target path segment, camera B collects image data of link X. At this time, the relative position relationship between link X and camera B is as follows: Figure 8 When the chain link X moves to the second end of the target path segment, the camera A collects the image data of the chain link X. At this time, the relative position relationship between the chain link X and the camera A is as follows: Figure 9 If the position of chain link X when camera B captures the image is taken as the reference, then cameras A and C should be located at Figure 10 Therefore, the positions of cameras A and C need to be compensated using translation vectors Y and Z.

[0104] Optionally, the motion state of the component to be inspected can be detected in advance to obtain a first motion parameter that can characterize the motion state of the component to be inspected. Taking the component to be inspected as a chain, and the chain moving in a straight line along its own length as an example, the first motion parameter may include the moving direction and moving speed of the chain. Optionally, when the image acquisition device is installed and the sampling time interval and sampling order are configured, the first timing parameters including the sampling time interval and sampling order can be configured to the data processor. Based on the acquisition of the first timing parameters and the first motion parameters, the translation vector of each image acquisition device can be determined, the spatial position of the image acquisition device can be corrected based on the translation vector, and the camera extrinsic parameters of each image acquisition device can be determined based on the corrected spatial position to compensate for the sampling time interval of the image acquisition device and the displacement of the component to be inspected in the moving direction.

[0105] In some embodiments, the sampling time interval is configured to shorten as the moving speed of the component to be inspected increases, and to lengthen as the moving speed of the component to be inspected decreases.

[0106] In practice, the moving speed of the part to be inspected may not be fixed but may change dynamically. In this case, the moving speed of the part to be inspected can be detected and the sampling interval can be configured based on the moving speed of the part to be inspected. In this way, even if the moving speed of the part to be inspected changes dynamically, multiple image acquisition devices can still capture image data that meets the requirements.

[0107] Still Figure 2 Taking the application scenario shown above as an example, once multiple cameras are installed, their positions and camera angles are typically fixed, and the target path segments defined by these cameras are also fixed. Based on this, if the chain's speed changes, the sampling interval can be dynamically configured based on the chain's speed to ensure that the translation vectors of each camera in the chain's direction of movement remain unchanged, thereby ensuring that the camera extrinsics of each camera remain unchanged. This eliminates the need for users to recalibrate camera extrinsics and allows point cloud data conversion based on pre-set camera extrinsics.

[0108] See also Figure 11 As shown, the embodiment of the present application also provides a structural damage detection system, including:

[0109] Multiple image acquisition devices 201 are configured to sequentially acquire image data of the component to be inspected while the component to be inspected is in motion, and to generate corresponding first point cloud data based on each image data; wherein the multiple image acquisition devices 201 have different camera angles, and the first point cloud data is point cloud data in the camera coordinate system of the image acquisition device 201;

[0110] The processing device 202 is configured to convert the plurality of first point cloud data into a plurality of second point cloud data in the same coordinate system based on the camera extrinsic parameters of each of the image acquisition devices 201; construct a three-dimensional model corresponding to the component to be inspected based on the plurality of second point cloud data; and determine the degree of structural damage of the component to be inspected based on the dimensional information of the three-dimensional model; wherein the camera extrinsic parameters are calibrated based on a first timing parameter and a first motion parameter, the first timing parameter is used to identify the sampling time interval and sampling order of the plurality of image acquisition devices 201, and the first motion parameter is used to characterize the motion state of the component to be inspected.

[0111] In some embodiments, the plurality of image acquisition devices 201 are sequentially arranged along a first direction, where the first direction is the moving direction of the component to be inspected; the plurality of image acquisition devices 201 are specifically configured as follows:

[0112] The image data of the component to be inspected is collected sequentially along a second direction; wherein the second direction is opposite to the moving direction of the component to be inspected.

[0113] In some embodiments, the plurality of image acquisition devices 201 include more than three image acquisition devices 201, wherein:

[0114] One of the image acquisition devices 201 is configured to: acquire image data of the component to be inspected at one end in the moving direction;

[0115] At least one of the image acquisition devices 201 is configured to: acquire image data of the side of the component to be inspected in the moving direction;

[0116] The other image acquisition device 201 is configured to acquire image data of the component to be inspected at the other end in the moving direction.

[0117] In some embodiments, the component to be inspected is a chain; the processing device 202 is specifically configured as follows:

[0118] determining the length of the chain links of the chain in the length direction of the chain;

[0119] If the length is greater than a first threshold, determining that the link needs to be replaced;

[0120] When the length is less than the first threshold, it is determined that the chain link does not need to be replaced.

[0121] In some embodiments, the processing device 202 is further configured to:

[0122] Acquiring the first timing parameter and the first motion parameter;

[0123] Determine a parallel vector in the moving direction of the component to be inspected based on the first timing parameter and the first motion parameter; wherein the parallel vector is used to compensate for the sampling time intervals of the plurality of image acquisition devices 201 and the displacement of the component to be inspected in the moving direction;

[0124] The camera extrinsic parameters of each of the image acquisition devices 201 are calibrated based on the parallel vectors.

[0125] In some embodiments, the sampling time interval is configured to shorten as the moving speed of the component to be inspected increases, and to lengthen as the moving speed of the component to be inspected decreases.

[0126] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A structural damage detection method comprising: When the component to be inspected is in motion, multiple image acquisition devices are used to sequentially acquire image data of the component to be inspected, and corresponding first point cloud data are generated based on each of the image data; wherein the multiple image acquisition devices have different camera angles, and the first point cloud data is point cloud data in the camera coordinate system of the image acquisition device; Based on camera extrinsic parameters of each of the image acquisition devices, converting the plurality of first point cloud data into a plurality of second point cloud data in the same coordinate system; wherein the camera extrinsic parameters are calibrated based on a first timing parameter and a first motion parameter, the first timing parameter is used to identify a sampling time interval and a sampling order of the plurality of image acquisition devices, and the first motion parameter is used to characterize a motion state of the component to be inspected, and the first motion parameter includes a motion direction and a motion speed of the component to be inspected; constructing a three-dimensional model corresponding to the component to be inspected based on the plurality of second point cloud data; The degree of structural damage of the component to be inspected is determined based on the dimensional information of the three-dimensional model.

2. The method according to claim 1, wherein The plurality of image acquisition devices are sequentially arranged along a first direction, where the first direction is a moving direction of the component to be inspected; The sequentially collecting image data of the component to be inspected by multiple image acquisition devices includes: Image data of the component to be inspected is collected in sequence along a second direction by a plurality of image collection devices, wherein the second direction is opposite to the moving direction of the component to be inspected.

3. The method according to claim 1, wherein The plurality of image acquisition devices include more than three image acquisition devices; The sequentially collecting image data of the component to be inspected by multiple image acquisition devices includes: Collecting image data of the component to be inspected at one end in the moving direction by an image acquisition device; Collecting image data of the side of the component to be inspected in the moving direction by at least one of the image acquisition devices; The image data of the component to be inspected at the other end in the moving direction is collected by another image collection device.

4. The method according to claim 1, wherein The component to be inspected is a chain; and determining the degree of structural damage of the component to be inspected based on the dimensional information of the three-dimensional model includes: determining the length of the chain links of the chain in the length direction of the chain; If the length is greater than a first threshold, determining that the link needs to be replaced; When the length is less than the first threshold, it is determined that the chain link does not need to be replaced.

5. The method according to claim 1, wherein The method further comprises: Acquiring the first timing parameter and the first motion parameter; Determining a parallel vector in the moving direction of the component to be inspected based on the first timing parameter and the first motion parameter; wherein the parallel vector is used to compensate for the sampling time intervals of multiple image acquisition devices and the displacement of the component to be inspected in the moving direction; The camera extrinsic parameters of each of the image acquisition devices are calibrated based on the parallel vectors.

6. The method according to claim 1, wherein The sampling time interval is configured to shorten as the moving speed of the component to be inspected increases, and to lengthen as the moving speed of the component to be inspected decreases.

7. A structural damage detection system comprising: Multiple image acquisition devices, configured to sequentially acquire image data of the component to be inspected while the component to be inspected is in motion, and to generate corresponding first point cloud data based on each image data; wherein the multiple image acquisition devices have different camera angles, and the first point cloud data is point cloud data in the camera coordinate system of the image acquisition device; A processing device is configured to convert a plurality of the first point cloud data into a plurality of the second point cloud data in the same coordinate system based on the camera extrinsic parameters of each of the image acquisition devices; construct a three-dimensional model corresponding to the component to be inspected based on the plurality of the second point cloud data; and determine the degree of structural damage of the component to be inspected based on the size information of the three-dimensional model; wherein the camera extrinsic parameter is calibrated based on a first timing parameter and a first motion parameter, the first timing parameter is used to identify the sampling time interval and sampling order of the plurality of the image acquisition devices, the first motion parameter is used to characterize the motion state of the component to be inspected, and the first motion parameter includes the motion direction and motion speed of the component to be inspected.

8. The system according to claim 7, wherein: The plurality of image acquisition devices are sequentially arranged along a first direction, where the first direction is the moving direction of the component to be inspected; the plurality of image acquisition devices are specifically configured as follows: The image data of the component to be inspected is collected sequentially along a second direction; wherein the second direction is opposite to the moving direction of the component to be inspected.

9. The system according to claim 7, wherein: The plurality of image acquisition devices include more than three image acquisition devices, wherein: One of the image acquisition devices is configured to: acquire image data of the component to be inspected at one end in the moving direction; At least one of the image acquisition devices is configured to: acquire image data of the side of the component to be inspected in the moving direction; The other image acquisition device is configured to acquire image data of the component to be inspected at the other end in the moving direction.

10. The system according to claim 7, wherein: The component to be inspected is a chain; the processing device is specifically configured as follows: determining the length of the chain links of the chain in the length direction of the chain; If the length is greater than a first threshold, determining that the link needs to be replaced; When the length is less than the first threshold, it is determined that the chain link does not need to be replaced.

Images