Train part anomaly detection method and device, electronic equipment and medium

Abstract

The application discloses a train part abnormality detection method and device, electronic equipment and medium, and the method comprises the following steps: obtaining 3D detection data and vehicle type information of a train to be detected, wherein the detection data is image data or point cloud data; the 3D detection data is divided into a plurality of part regions according to the vehicle type information, and each part region contains more than one part; the abnormal state of each part in each part region is detected respectively to determine the abnormal region in each part region; and the abnormal type is determined according to the overlapping relationship between the abnormal region and each part in the corresponding part. The application has the advantages of simple implementation method, low cost, high detection efficiency and precision, low repeated alarm rate and false alarm rate.

Description

Technical Field

[0001] This invention relates to the field of train fault detection technology, and in particular to a method, device, electronic equipment, and medium for detecting abnormalities in train components. Background Technology

[0002] To ensure train operation safety, effective train fault detection is essential. With the continuous development of machine vision technology, train fault detection methods are gradually shifting from manual inspection to automation and intelligence. Intelligent detection methods can significantly improve the intelligence and efficiency of inspections while reducing costs. However, current machine vision-based train fault detection typically involves acquiring 2D images of the component under test, performing foreground segmentation to obtain the component's image, and then separating the parts from non-part areas within the component to determine abnormal states. This method requires precise segmentation of each part within the train system for abnormal state assessment. Since trains contain numerous components, this necessitates building numerous detection models and performing extensive part segmentation and other processing. This process is not only complex and time-consuming but also lacks high accuracy and is prone to duplicate alarms and false alarms. For example, when a component within the train malfunctions, other parts within that component may be affected and unable to function properly. If the above detection method is used, both the component and the affected parts will be detected simultaneously, triggering alarms when the parts may actually be functioning correctly, leading to false alarms. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, device, electronic equipment, and medium for detecting abnormalities in train parts that is simple to implement, low in cost, high in detection efficiency and accuracy, and low in repetitive alarm rate and false alarm rate.

[0004] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0005] A method for detecting abnormalities in train components, comprising the following steps:

[0006] Acquire 3D detection data and vehicle type information of the train to be inspected, wherein the detection data is image data or point cloud data;

[0007] The 3D detection data is divided into multiple component regions based on the vehicle type information, and each component region contains more than one component.

[0008] Abnormal state detection is performed on each component in each of the aforementioned component regions to identify the abnormal regions in each component region;

[0009] Determine the abnormal type according to the overlapping relationship between the abnormal area and each part in the corresponding component.

[0010] Further, the step of segmenting the 3D detection data into a plurality of component areas according to the vehicle type information comprises:

[0011] Determine segmentation parameters according to the vehicle type information, wherein the segmentation parameters comprise position information of each component area;

[0012] According to the line number relationship between the 3D detection data of the to-be-detected train and the 3D detection data of the corresponding template train and the segmentation parameters, calculate the offset line number and the area length of each component area on the 3D detection data;

[0013] Segment according to the calculated offset line number and area length of each component area on the 3D detection data to obtain a plurality of component areas.

[0014] Further, the position information of each component area comprises any one or more of the offset line number of the starting position of each component area from the position of the train head, the position of each component area in the train car, and the offset line number of the starting position of each component area from the specified reference position of the current train car.

[0015] Further, the step of detecting the abnormal state of each component in each of the component areas to determine the abnormal area in each component area comprises:

[0016] Obtain the current component area and the template data corresponding to the current component area;

[0017] Iterate through each component in the current component area to detect the abnormal state;

[0018] When detecting the abnormal state of each component, extract a to-be-detected component area from the current component area, register the extracted to-be-detected component area with the corresponding template data, and determine the abnormal area according to the registration result.

[0019] Further, the step of extracting a to-be-detected component area from the current component area and registering the extracted to-be-detected component area with the corresponding template data comprises:

[0020] Obtain the position information of the to-be-detected component in the corresponding template data, and extract the current required template data from the corresponding template data of the to-be-detected component according to the obtained position information;

[0021] Spatially expand the position information of the to-be-detected component in the corresponding template data to obtain expanded position information;

[0022] According to the extended position information, a part region to be detected is intercepted from the current part region;

[0023] The intercepted part region to be detected is registered with the intercepted current required template data.

[0024] Further, before the registration of the intercepted part region to be detected with the corresponding template data, a data preprocessing of down-sampling of the part region to be detected is further included, and after the registration of the intercepted part region to be detected with the corresponding template data, a data preprocessing of down-sampling and de-noising of the part region to be detected is further included.

[0025] Further, according to the registration result, if the fitting degree between the part region to be detected and the corresponding template data is less than a preset threshold, the whole part region to be detected is determined as an abnormal region, otherwise, by finding difference points between the part region to be detected and the corresponding template data, the abnormal region is formed by the found difference points.

[0026] Further, the finding of the difference points between the part region to be detected and the corresponding template data includes: if there is a target point on the part region to be detected satisfying a preset first condition, or there is a target point on the corresponding template data of the part region to be detected satisfying a preset second condition, the target point is determined as a difference point; the preset first condition is that there is no point matching the target point in a spherical space with a preset overlap region distance threshold as a radius in the corresponding template data, and the preset second condition is that there is no point matching the target point in a spherical space with a preset overlap region distance threshold as a radius in the part region to be detected.

[0027] Further, after the difference points are found, effective abnormal points are intercepted from the range of the registered template data, and the abnormal region is formed by the effective difference points in a clustering manner.

[0028] Further, when the abnormal region in each part region is determined, a geometric feature judgment of the abnormal region is further included, and the abnormal region whose geometric feature does not satisfy a preset condition is removed.

[0029] Further, the determination of the abnormal type according to the overlap relationship between the abnormal region and each part in the corresponding part includes:

[0030] An overlap region between the abnormal region and all part regions in the corresponding template data is obtained;

[0031] A proportion of the overlap region in the abnormal region is determined;

[0032] If the proportion of the overlapping region in the abnormal region exceeds a preset first threshold, it is determined that the part is abnormal, otherwise, whether the part is abnormal is determined according to the proportion of the overlapping region in the part region and / or the geometric characteristics of the abnormal region.

[0033] Further, the determination of whether the part is abnormal according to the proportion of the overlapping region in the part region and / or the geometric characteristics of the abnormal region comprises: if the proportion of the overlapping region in the part region exceeds a preset second threshold, or the proportion of the overlapping region in the part region is less than the preset second threshold and the abnormal region satisfies a preset geometric characteristic, it is determined that the part is abnormal.

[0034] Further, the determination of the abnormal type according to the overlapping relationship between the abnormal region and each part in the corresponding part further comprises: if there are multiple target abnormal regions that overlap with the same part region and the overlapping region exceeds a preset threshold, the target abnormal regions are merged to determine the abnormal type according to the overlapping relationship between the merged abnormal region and each part in the corresponding part.

[0035] A train part abnormality detection device comprises:

[0036] A to-be-detected data acquisition module is configured to acquire 3D detection data and vehicle type information of a to-be-detected train, wherein the detection data is image data or point cloud data.

[0037] A region segmentation module is configured to segment the 3D detection data into multiple part regions according to the vehicle type information, wherein each part region contains one or more parts.

[0038] An abnormal region detection module is configured to detect the abnormal state of each part in each part region and determine an abnormal region in each part region.

[0039] An abnormal type determination module is configured to determine the abnormal type according to the overlapping relationship between the abnormal region and each part in the corresponding part.

[0040] Further, the abnormal region detection module comprises:

[0041] A data acquisition unit is configured to acquire a current part region and template data corresponding to the current part region.

[0042] A traversal unit is configured to traverse each part in the current part region to detect the abnormal state.

[0043] When the traversal unit detects the abnormal state of each part, the to-be-detected part region is intercepted from the current part region, the intercepted to-be-detected part region is registered with the corresponding template data, and the abnormal region is determined according to the registration result.

[0044] Further, the abnormal type judging module judging the abnormal type according to the overlapping relationship between the abnormal area and each part in the corresponding component comprises: the abnormal type judging module judging the abnormal type according to the overlapping relationship between the abnormal area and each part in the corresponding component comprises: obtaining the overlapping area between the abnormal area and all part areas in the corresponding template data; judging the proportion of the overlapping area in the abnormal area; if the proportion of the overlapping area in the abnormal area exceeds a preset first threshold, determining that it is a part abnormality, otherwise, judging whether it is a component abnormality according to the proportion of the overlapping area in the part area and the geometric feature of the abnormal area; if the proportion of the overlapping area in the part area exceeds a preset second threshold, or the proportion of the overlapping area in the part area is less than the preset second threshold and the abnormal area satisfies a preset geometric feature, determining that it is a component abnormality.

[0045] An electronic device comprising a processor and a memory, the memory being configured to store a computer program, and the processor being configured to execute the computer program to perform the method as described above.

[0046] A computer readable storage medium storing a computer program, the computer program being configured to perform the method as described above when executed.

[0047] Compared with the prior art, the advantages of the present application are that: the present application detects structural abnormalities by obtaining 3D image / point cloud data of the train, effectively reduces the system complexity and hardware cost under the premise of ensuring the abnormal detection rate of parts, and considers the correlation between component abnormalities and part abnormalities, detects abnormalities in each component area by first segmenting a plurality of large component areas, then detects abnormalities in each component area, i.e., detects the whole single component first, determines the abnormal area of each component area, and then confirms the abnormal type according to the overlapping relationship between the abnormal area and the part, without the need to construct a large number of single part detection models and perform a large number of part segmentation processes, which not only effectively reduces the implementation complexity, but also accurately detects the abnormalities in the parts under the premise of avoiding false detection of non-part areas of the component, while avoiding repeated alarms and false alarms, effectively reducing the workload in the development process. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is the implementation flowchart of the train part abnormality detection method of the embodiment.

[0049] Figure 2 is the implementation flowchart of the abnormality detection of the component area in the specific application embodiment of the present application.

[0050] Figure 3is a schematic diagram of the structure principle of the train part abnormality detection device in the specific application embodiment of the present application. DETAILED DESCRIPTION

[0051] The present application is further described below in conjunction with the drawings and specific preferred embodiments, but the protection scope of the present application is not limited thereby.

[0052] As shown in Figure 1 , the steps of the train part abnormality detection method of the present embodiment include:

[0053] S01. Data acquisition: acquire 3D detection data and vehicle type information of the train to be detected, the detection data being image data or point cloud data;

[0054] S02. Region segmentation: segment the 3D detection data into a plurality of part regions according to the vehicle type information, each part region containing one or more parts;

[0055] S03. Abnormal region detection: respectively detect the abnormal state of each part in each part region to determine the abnormal region in each part region;

[0056] S04. Abnormal type judgment: judge the abnormal type according to the overlapping relationship between the abnormal region and each part in the corresponding part.

[0057] Considering that when the train part has a fault, it is highly likely to be accompanied by a certain degree of structural abnormality in space, such as part loss, loosening, deformation, and attachment of foreign matter, etc. Such structural abnormalities will present obvious geometric differences in 3D images / point clouds, and the influence of water stains, stains, and day-night light differences on the surface of the vehicle body is small. The present embodiment fully utilizes the characteristics of 3D image / point cloud data, detects structural abnormalities by acquiring 3D image / point cloud data of the train, and effectively reduces the system complexity and hardware cost under the premise of ensuring the part abnormality detection rate. At the same time, considering the correlation between part abnormalities and part abnormalities, the 3D image / point cloud data of the train to be detected is first segmented into a plurality of part regions to be detected, each region containing one or more parts of the same kind, and then each part region is detected for abnormalities, i.e. detecting the whole single part first to determine the abnormal region of each part region, and then confirming the abnormal type (part abnormality or part abnormality) according to the overlapping relationship between the abnormal region and the part. Without constructing a large number of single part detection models and without performing a large number of part segmentation processing, it not only effectively reduces the implementation complexity, but also accurately detects the abnormalities of the parts in which the parts are located under the premise of avoiding false detection of part non-part regions, while avoiding repeated alarms and false alarms, effectively reducing the workload in the development process.

[0058] In the embodiment, step S01 is specifically implemented by relative movement (which can be configured to tend to uniform motion) between the data acquisition device (library inspection robot and trackside acquisition device, etc.) and the train to be inspected, and the 3D image / point cloud of the train is collected in a line scanning manner. The detection data can be directly obtained from the data acquisition device, or can be obtained by storing and then obtaining. When the same train is data collected, the overall line number deviation of two data collection can be configured to be not more than 0.5%, and the collection accuracy is 2pixel / mm (after the image / point cloud data plane of a flat box body plane is flattened, that is, the depth information is uniform to the same value, whether it is on the longitudinal straight line of the train or on the transverse straight line, the distance between adjacent two pixel points is about 0.5mm. In order to achieve the effect in the above scheme, the collection accuracy needs to be not less than 2pixel / mm). Taking a 20m car as an example, the collected 3D image / point cloud line number is 40000±200, and the car type information can be analyzed from the gray information of the 3D image / point cloud data. Then, the car type is obtained by combining prior knowledge. Of course, it can also be provided by other modules, or even in the form of a specified car type.

[0059] Since the parts in the corresponding position of the same type of train have no obvious difference in structure, that is, the position structure of the parts in the same type of train is fixed, after the train type is determined, the prior information can be used to accurately segment each part region. In the embodiment, the vehicle type information is obtained to determine the segmentation parameters of the segmentation of each part region, and the 3D detection data is segmented into a plurality of part regions by combining the segmentation parameters. The specific steps include:

[0060] S201. Determine the segmentation parameters according to the vehicle type information, and the segmentation parameters include the position information of each part region;

[0061] S202. According to the line number relationship between the 3D detection data of the train to be inspected and the 3D detection data of the corresponding template train and the segmentation parameters, the offset line number and the region length of each part region in the 3D detection data are calculated;

[0062] S203. Segment according to the offset line number and the region length of each part region in the 3D detection data calculated, to obtain a plurality of part regions.

[0063] In a specific application embodiment, the segmentation parameter can be a set of one-dimensional data, including the offset line number of the starting position of each component region from the head position of the vehicle, and the difference between two adjacent data represents the line number of the previous component region; then, according to the line number ratio between the 3D detection data of the to-be-detected train and the corresponding template train 3D detection data and the segmentation parameter, the offset line number and the region length of each to-be-detected component region in the to-be-detected detection data (image / point cloud data) are calculated, and each to-be-detected component region is segmented. The segmentation parameter can also be a set of two-dimensional data, the first dimension represents the reference region (such as the first car) where each component region is located; the second dimension represents the offset line number of the starting position of each component region from a certain reference position (such as the car boundary or the axle position) of the current car, and the difference between two adjacent data represents the line number of the previous component region, and then the above-mentioned reference position is calculated by using the traditional boundary positioning algorithm, and then according to the line number ratio between the current reference region in the to-be-detected train 3D image / point cloud and the template train 3D detection data and the segmentation parameter, the offset line number and the region length of each to-be-detected component region in the to-be-detected data are calculated, and each to-be-detected component region is segmented. It can be understood that, in addition to the above-mentioned parameters, other types of parameters can also be used according to actual needs to achieve more accurate or faster segmentation.

[0064] Preferably, after segmentation, the following constraint can be configured: the offset value of a certain component of the first vehicle from the starting position of the to-be-detected component region to which it belongs is denoted as y1, the offset value of the corresponding component of the second vehicle from the starting position of the to-be-detected component region to which it belongs is denoted as y2, and |y1-y2| is not more than 60 lines.

[0065] The step S03 of the embodiment detects the abnormal state of each component in each component region, and the specific steps for determining the abnormal region in each component region include:

[0066] S301. Obtain the current component region and the template data corresponding to the current component region;

[0067] S302. Traverse each component in the current component region to detect the abnormal state;

[0068] S303. When detecting the abnormal state of each component, the to-be-detected component region is intercepted from the current component region, the intercepted to-be-detected component region is registered with the corresponding template data, and the abnormal region is determined according to the registration result.

[0069] The embodiment obtains the template data corresponding to the to-be-inspected train according to the vehicle type information, and then performs abnormality detection on each to-be-inspected component large area. When detecting a single to-be-inspected component large area, after obtaining the to-be-inspected component large area and the corresponding template data, each component in the current to-be-inspected component large area is traversed, and then abnormality detection is performed on a single component to obtain the abnormal area in the component. The component area can be quickly subjected to overall abnormality detection, and the abnormal area in each component area can be determined.

[0070] In a specific application embodiment, the to-be-inspected component large area and the corresponding template data can be obtained in sequence according to the physical arrangement order of the to-be-inspected component large area, and then each component in the current to-be-inspected component large area is traversed, that is, a single component is subjected to abnormality detection to obtain the abnormal area in the component. Through the above manner, the system can analyze the previously collected data when the data collection device collects data, and the effect of collecting and analyzing simultaneously can be achieved. The to-be-inspected component large areas can also be classified according to the component types to which they belong, the to-be-inspected component large areas of a certain type of component are analyzed first, and then the to-be-inspected component large areas of other types of components are analyzed. The to-be-inspected component large areas of the same type of component can be analyzed in sequence according to the collection order or in other specified order. Through this type of manner, when the same type of component is analyzed, the previously obtained template image / point cloud can be directly analyzed, the throughput of obtaining template data is effectively reduced, and the analysis efficiency can be faster in a scenario where the timeliness requirement for analysis is not very high. Of course, other types of collection and analysis manners can also be used according to actual needs.

[0071] In a specific application embodiment, the template data (image / point cloud file) is obtained from 3D image / point cloud data collected when all components of the train are normal, and further includes a component position information parameter file. The component position information parameter file corresponds to the template image / point cloud file one by one, and is used to record the number, name and position information of each component in the corresponding template image / point cloud. Specifically, an xyz space coordinate system is established for the 3D image / point cloud. The position information of each component in the component position information parameter file includes six dimensions: [x0, y0, z0, Δx, Δy, Δz]. x0 is the starting position in the x direction. Taking single-line data collected by the data collection device as an example, the direction from the starting point to the ending point is defined as the positive direction of the x direction. y0 is the starting position in the y direction. Taking a component large area as an example, the direction from the first line data to the last line data is defined as the positive direction of the y direction. z0 is the starting position in the z direction. The z direction represents the depth information between the data collection device and the train. Specifically, the direction from the data collection device to the train can be taken as the positive direction of the z direction. Δx is the span in the x direction, Δy is the span in the y direction, and Δz is the span in the z direction. The span unit is uniformly mm.

[0072] In step S303 of the embodiment, the part region to be detected is cut from the current part region, and the cut part region to be detected is matched with the corresponding template data. The specific steps include:

[0073] S331. The position information of the part to be detected in the corresponding template data is obtained, and the current required template data is cut from the corresponding template data of the part to be detected according to the obtained position information.

[0074] S332. The position information of the part to be detected in the corresponding template data is expanded in space to obtain expanded position information.

[0075] S333. The part region to be detected is cut from the current part region according to the expanded position information.

[0076] S334. The cut part region to be detected is matched with the cut current required template data.

[0077] In order to avoid safety hazards, it is usually difficult to collect a sufficient number of real failure samples for key parts. In the embodiment, the logic similar to the sliding window matching is adopted to match the template part to the part region to be detected (the size is greater than that of the template part), instead of matching the part region to be detected to the template part, which can greatly reduce the workload of data labeling. If the data collection accuracy is within the expected range, the part region to be detected is cut according to the physical size, and there is almost no missed detection problem in the target detection process. In addition, matching a small region to a large region can also effectively improve the matching adaptation degree and further ensure the detection accuracy.

[0078] Before the part region to be detected is cut from the large part region to be detected according to the expanded position information in the embodiment, the spatial expansion parameter is determined, that is, the position information of the part in the template data is expanded in space before being cut, which can avoid the influence caused by the relative deviation between the collection equipment and the train in space during the two data collections. The above-mentioned relative deviation may be caused by the vibration of the collection equipment, the shaking of the train, or the installation error of the part, which can further improve the detection accuracy.

[0079] In the embodiment, if the adaptation degree between the part region to be detected and the corresponding template data is less than a preset threshold in the abnormal region determined according to the matching result, it is determined that the entire part region to be detected is an abnormal region, otherwise, the difference points between the part region to be detected and the corresponding template data are found, and the abnormal region is formed by the found difference points.

[0080] In a specific application embodiment, when the component region to be detected is registered with the template data, a template component is first obtained: the position information of a component is obtained from the template data, and the template component is cut out from the template image / point cloud file according to the position information; if the current 6-dimensional position information is x0, y0, z0, Δx, Δy, Δz, the template component is cut out with (x0, y0, z0) as the starting point and (x0+Δx, y0+Δy, z0+Δz) as the ending point; a small region of the component to be detected is obtained: the position information is expanded in space, and a small region of the component to be detected is cut out from a large region of the component to be detected according to the expanded position information, so that the region contains only a single component; then the cut-out template component is registered with the small region of the component to be detected, and if the fitting degree is lower than the fitting threshold, the component is directly recorded as abnormal and the region is the entire small region of the component to be detected; otherwise, subsequent steps of finding difference points and forming abnormal regions are performed.

[0081] Taking the detection of a bottom component of a train as an example, the expansion parameters in space are as follows:

[0082] The expansion parameter in the x direction is ±20, so as to avoid the influence of the relative deviation in the x direction, which can be caused by the shaking of the collection device or the snake-like movement of the train or the installation error of the component;

[0083] The expansion parameter in the y direction is ±30, so as to avoid the influence of the relative deviation in the y direction, which can be caused by the deviation of the data segmentation algorithm to be detected, or the unstable collection frequency of the collection device, or the deviation of the train speed measuring device, or the installation error of the component;

[0084] The expansion parameters in the z direction are -100 and 20 respectively, so as to avoid the influence of the relative deviation in the z direction, which can be caused by the unevenness of the plane carried by the collection device during movement, or the shaking of the train itself, or the installation error of the component.

[0085] Finally, the small region of the component to be detected is cut out with (x0-20, y0-30, z0-100) as the starting point and (x0+Δx+20, y0+Δy+30, z0+Δz+20) as the ending point.

[0086] Preferably, the preset threshold of the fitting degree can be set to 40%, and the registration algorithm can specifically adopt point-to-plane icp registration and be divided into 3 levels of coarse-to-fine registration. It can be understood that the preset threshold of the fitting degree, the registration algorithm, the registration radius and the number of iterations can all be selected and configured according to actual needs.

[0087] Further, the embodiment further comprises data preprocessing of the to-be-detected data before registration of the intercepted to-be-detected component region and the corresponding template data, and data preprocessing of the to-be-detected data after registration of the intercepted to-be-detected component region and the corresponding template data. That is, the small region image / point cloud data of the template component and the to-be-detected component are preprocessed before and after registration. The preprocessing before registration is referred to as first preprocessing, and the preprocessing after registration is referred to as second preprocessing. The first preprocessing only performs downsampling, and the second preprocessing performs downsampling and denoising at the same time after downsampling. The two downsamplings can each be voxel downsampling. The size of the first sampling is larger than that of the second sampling. Downsampling with a large size parameter before registration can reduce the registration time while ensuring the registration accuracy. After registration, downsampling with a small size parameter is performed before subsequent processing, which can ensure the data accuracy of subsequent processing and avoid false detection and missed detection of abnormal parts. Preferably, the denoising can be performed by spherical radius filtering, which can effectively reduce the interference of noise on subsequent processing.

[0088] In the embodiment, finding the difference points between the to-be-detected component region and the corresponding template data specifically comprises: if a target point on the to-be-detected component region satisfies a preset first condition or a target point on the template data corresponding to the to-be-detected component region satisfies a preset second condition, the target point is determined as a difference point; the preset first condition is that there is no point matching the target point in a spherical space with a preset overlap region distance threshold as a radius in the corresponding template data, and the preset second condition is that there is no point matching the target point in a spherical space with a preset overlap region distance threshold as a radius in the to-be-detected component region. That is, if a point on the template image / point cloud cannot find a point on the to-be-detected image / point cloud in a spherical space with the overlap region distance threshold as a radius, the point is determined as a difference point. Similarly, if a point on the to-be-detected image / point cloud cannot find a point on the template image / point cloud in a spherical space with the overlap region distance threshold as a radius, the point is also determined as a difference point. In this way, the difference points between the to-be-detected component region and the template data can be quickly and effectively found based on the overlap region distance threshold.

[0089] The overlap region distance threshold can be configured according to different component types. For example, the overlap region distance threshold of small parts such as M8 bolts, small rubber plugs, and hinges can be set to 5; the overlap region distance threshold of components such as air outlets with a mesh diameter greater than 20 mm and joints can be set to 9; and the overlap region distance threshold of other components can be set to 7.

[0090] It can be understood that when the overlap region between each abnormal region and all part regions is considered, the overlap can be spatial overlap or point overlap. The judgment of whether the points overlap can be achieved by using the same principle as the abnormal point finding. The overlap region distance threshold can be set to 2.

[0091] After identifying the discrepancies, valid outliers are extracted from the registered template data. These valid outliers are then clustered to form outlier regions. In a specific application embodiment, the template component range does not include the direction of depth information; that is, the direction of depth information is not limited. The clustering algorithm can be the dbscan density-based clustering algorithm, or other types of clustering algorithms can be used depending on the actual needs.

[0092] Furthermore, in this embodiment, when identifying abnormal regions in each component area, the geometric features of the abnormal regions are also judged, and abnormal regions whose geometric features do not meet the preset conditions are eliminated. The aforementioned geometric features can specifically be size and area. By pre-setting size thresholds (e.g., 7 for components containing small parts, 9 for other components) and area thresholds (e.g., 0 for components containing parts, 10*10 for components without parts), if the size of the abnormal region in both the x and y directions is not lower than the size threshold, it is retained; otherwise, the abnormal region is eliminated. If the projected area of ​​the abnormal region in the z-plane is not lower than the area threshold, it is retained; otherwise, the abnormal region is eliminated. The aforementioned projected area can be calculated as follows: the z-value of the points within the abnormal region is assigned to 0, and then voxel downsampling is performed with a sampling size of 1, thereby calculating the number of points after downsampling. Geometric features can also be thin-walled features, wire bundle features, etc. Thin-walled features can be specifically defined as follows: in the 3D image / point cloud data of the abnormal region, the normal vectors of points exceeding a specified proportion are parallel to the z-plane; or the data is divided into several segments in the x-direction with a step size of Δx = 10, and the Δy of each segment is < 2. Wire bundle features can be defined as follows: project the 3D image / point cloud data of the abnormal region onto the z-plane, then perform voxel downsampling with a sampling size of 1, retaining the points with the largest connected components, and then confirming whether these points satisfy the latter description method of thin-walled features. The Δx and Δy parameters need to be adjusted according to the actual wire bundle; for example, Δx can be set to 100 and Δy to 10 for a certain wire bundle. It is understood that the specific type of geometric feature and the threshold settings can be selected and configured according to actual needs.

[0093] In specific application embodiments, such as Figure 2 As shown, the detailed steps for anomaly detection in a single component area are as follows:

[0094] Get template component: Obtain the position information of a component from the template data, and extract the template component from the template image / point cloud file based on the position information;

[0095] Obtain a small area of ​​the component to be inspected: Spatially expand the above location information, and extract a small area of ​​the component to be inspected from the large area of ​​the component to be inspected based on the expanded location information, so that the area contains only a single component;

[0096] Registration: register the template part with the small area of the part to be detected, if the degree of fitting is lower than the fitting threshold, directly record as part abnormality, the area is the entire small area of the part to be detected; otherwise, find the difference points to form the abnormal area;

[0097] Abnormal area formation: find the difference points between the detected area and the template data by overlapping the area distance threshold, intercept the difference points in the range of the template part after registration as effective abnormal points, and cluster the effective difference points to form the abnormal area;

[0098] Screening of abnormal area: judge the geometric features of the abnormal area, and eliminate the abnormal area that does not meet certain geometric features.

[0099] In this embodiment, the abnormal type is judged according to the overlapping relationship between the abnormal area and each part in the corresponding part:

[0100] Obtain the overlapping area between the abnormal area and all part areas in the corresponding template data;

[0101] Judge the proportion of the overlapping area in the abnormal area;

[0102] If the proportion of the overlapping area in the abnormal area exceeds a preset first threshold, it is determined that the part is abnormal, otherwise, whether it is a part abnormality is judged according to the proportion of the overlapping area in the part area and / or the geometric features of the abnormal area. If the proportion of the overlapping area between the abnormal area of the part and the part area on the abnormal area is large (greater than the preset first threshold), it indicates that the abnormal area is a part-dominant abnormal area, and the abnormal area is probably caused by part abnormality, which can be determined as part abnormality. On the contrary, if the proportion of the overlapping area between the abnormal area of the part and the part area on the abnormal area is small (less than the preset first threshold), it may be part abnormality or no abnormality, and it cannot be directly determined whether it is part abnormality, so further according to the proportion of the overlapping area in the part area and the geometric features of the abnormal area, whether it is part abnormality is judged.

[0103] In the embodiment, determining whether the component is abnormal according to the proportion of the overlapping region in the part region and / or the geometric feature of the abnormal region comprises: if the proportion of the overlapping region in the part region exceeds the preset second threshold, or the proportion of the overlapping region in the part region is less than the preset second threshold, and the abnormal region satisfies the preset geometric feature, it is determined that the component is abnormal. If the proportion of the overlapping region on the part region is large (greater than the preset second threshold), it indicates that the abnormal region is likely to be an abnormal region dominated by a non-part, and the abnormal region is probably caused by a fault of the component itself, and the part itself has no abnormality, and the component abnormality can be directly determined. If the proportion of the overlapping region on the part region is small (less than the preset second threshold), but the geometric feature is satisfied, it indicates that the abnormal region is not affected by the part, and whether the component is abnormal can be determined according to the geometric feature of the abnormal region. For the case that the proportion of the overlapping region on the part region is less than the preset second threshold but the geometric feature is satisfied, an abnormally strict condition can be set, and the component is determined to be abnormal only when the specified type of geometric feature is satisfied to ensure the accuracy, and if the geometric feature is not satisfied, the abnormal region is directly removed.

[0104] The preset first threshold and the preset second threshold can be flexibly configured according to actual needs.

[0105] The embodiment can fully utilize the overlapping relationship between the abnormal region and the part to determine the type of the abnormality, and can determine the fault and position of the part without constructing a detection module of the part and segmenting the part region, which can greatly improve the detection accuracy and efficiency, and can effectively avoid repeated alarm or false alarm between the component and the part abnormality.

[0106] In the embodiment, determining the type of the abnormality according to the overlapping relationship between the abnormal region and each part in the corresponding component further comprises: if there are multiple target abnormal regions overlapping with the same part region and the overlapping region exceeds the preset threshold, it indicates that the multiple target abnormal regions are related to the same part, and each target abnormal region is merged to determine the type of the abnormality according to the overlapping relationship between the merged abnormal region and each part in the corresponding component. The merged abnormal region can be determined to be a part abnormality or a component abnormality according to the preset first threshold and the preset second threshold.

[0107] There can be a large span of parts (such as a lying screw) in the train, or there can be a large area of attachments on the parts (corresponding to part abnormalities). For such cases, after the image is divided according to the parts, the large span of parts or the large area of attachments can be divided into multiple different areas. If the above steps are directly used for abnormality judgment, multiple abnormal points can exist, resulting in repeated alarms. In this embodiment, whether each abnormal area has a large overlap with the same part area can be used to identify the case where the attachments on the same part or component are divided into multiple areas. After the identified abnormal areas are merged and processed, abnormality judgment can be performed, which can effectively avoid repeated alarms.

[0108] Taking a lying screw as an example, multiple abnormal areas have a large overlap with the same part area according to the above steps. Then, the detected abnormal areas are merged. When the abnormality of the merged abnormal area is judged, if the overlap area between the abnormal area of the part and the part area accounts for a large proportion (greater than a preset first threshold) on the abnormal area, it indicates that the abnormal area is a part-dominant abnormal area, and it is determined as a part abnormality. On the contrary, if the overlap area between the abnormal area of the part and the part area accounts for a small proportion (less than a preset first threshold) on the abnormal area, it is further determined whether the overlap area accounts for a large proportion (greater than a preset second threshold) on the part area. If the overlap area accounts for a large proportion (greater than a preset second threshold) on the part area, it indicates that the abnormal area is likely to be a non-part-dominant abnormal area, and it can be directly determined as a part abnormality. If the overlap area accounts for a small proportion (less than a preset second threshold) on the part area, but satisfies the geometric characteristics, it indicates that the abnormal area is not affected by the part. Whether the geometric characteristics of the abnormal area meet the final determination of the part abnormality is determined.

[0109] The above steps of this embodiment use geometric characteristics to determine the merging of each abnormal area and the elimination of invalid abnormal areas, so that the detection result is more reasonable. When it is determined as a part abnormality, the final part abnormality type can be further determined according to the part type. When it is determined as a part abnormality, the final part abnormality type can be further determined according to the part type.

[0110] This embodiment specifically determines the overlap relationship between the abnormal area and each part in the current part through a core processing module, and classifies the abnormal area, specifically including:

[0111] If the abnormal area is mainly composed of a part, that is, a part-dominant abnormal area, it is recorded as a first case;

[0112] If the abnormal area contains part / whole area of a part, but is not a part-dominant abnormal area, that is, a part-affected abnormal area, it is recorded as a second case;

[0113] If multiple abnormal regions are with the same small part as II case, record as III case;

[0114] If abnormal region does not belong to any of the above cases, record as IV case.

[0115] Then further process the abnormal region combined with the classification result, record the final effective abnormal region, which specifically includes:

[0116] For multiple abnormal regions of III case, perform merging processing, and then judge the overlapping relationship with each part again;

[0117] For abnormal region of I case, directly record, and define the abnormality as part abnormality, and then analyze the abnormal type combined with different part types;

[0118] For abnormal region of II case, directly record, and define the abnormality as part abnormality; If needed, the abnormality of the corresponding part can also be further recorded; At the same time, the abnormal type can be analyzed combined with different part / part types;

[0119] For abnormal region of IV case, perform geometric feature judgment again, only record the abnormal region meeting certain geometric features, define as part abnormality, and then analyze the abnormal type combined with different part types.

[0120] The following takes the judgment of the relationship between the abnormal region and each part in the current part in the specific application embodiment as an example, and the specific steps include:

[0121] Step 1: Abnormal region classification includes:

[0122] ① The main component of the abnormal region is a certain part, specifically, the proportion of a certain part region in the abnormal region is more than 50%, it is considered that the abnormal region is dominated by the part, and it is recorded as I case, and the subsequent processing is not performed;

[0123] ② The abnormal region contains part / all of the region of a certain part, specifically, the proportion of the abnormal region in a certain part region is more than 15%, it is considered that the abnormal region is affected by the part, and it is recorded as II case;

[0124] ③ Multiple abnormal regions are with the same small part as II case, record as III case;

[0125] ④ Abnormal region does not belong to any of the above cases, record as IV case.

[0126] Step 2: Further process the abnormal region combined with the classification result, which specifically includes:

[0127] ① For multiple abnormal regions of the III case, perform merging processing, and after merging, perform step 1 again to determine the case to which the merged abnormal region belongs;

[0128] ② For the abnormal region of the I case, directly record and define the abnormality as a part abnormality, and then analyze the abnormality type in combination with different part types, such as the abnormality types of bolts, such as loss and loosening, the abnormality types of joints, such as falling off and loosening, and the abnormality types of rubber plugs, such as loss and opening;

[0129] ③ For the abnormal region of the II case, directly record and define the abnormality as a part abnormality, and then analyze the abnormality type in combination with different part types, such as the abnormality types of boxes, such as deformation (including breakage) and foreign matter attachment; in this embodiment, when a large foreign matter is attached to the box and blocks one or more bolts, only the abnormality of part foreign matter attachment is recorded;

[0130] ④ For the abnormal region of the IV case, perform geometric feature judgment again, and only record the abnormal region that meets certain geometric features, define it as a part abnormality, and then analyze the abnormality type in combination with different part types, such as the abnormality types of boxes, such as deformation (including breakage) and foreign matter attachment.

[0131] The geometric feature threshold value used in this step can be set more leniently, that is, the part abnormality not affected by the part can use a larger threshold value, for example, the size threshold value can be uniformly set to 10, and the size of the abnormal region in the x and y directions is not less than the value, and the abnormal region is retained; otherwise, the abnormal region is rejected; the area threshold value of the uneven multi-angled part can be set to 50*50, and the others are set to 0, the projection area of the abnormal region in the z plane is not less than the value, and the abnormal region is retained; otherwise, the abnormal region is rejected.

[0132] ⑤ Record the abnormality result, including but not limited to part number, name, position information and abnormality type.

[0133] When the structural abnormality of the train part is detected in this embodiment, first, according to prior knowledge, a large region of the part to be detected is segmented from the 3D image / point cloud data of the train to be detected, and the region contains one or more parts of the same kind. Then, according to the corresponding template data of the train to be detected, each part in the large region of the part to be detected is traversed, and the single part is detected for abnormality to obtain the abnormal region in the part, and then the abnormal type is determined in combination with the part type and the geometric feature of the abnormal region. The detection of the structural abnormality of the train part can be effectively realized, when the single part is detected for abnormality, the abnormal type of the part is determined by judging the overlapping relationship between the abnormal region and each part in the current part, which can greatly improve the detection accuracy and efficiency, and effectively avoid repeated alarm or false alarm between part and part abnormality.

[0134] The train part abnormality detection device of the embodiment comprises:

[0135] A to-be-detected data acquisition module is configured to acquire 3D detection data of a to-be-detected train and vehicle type information, and the detection data is image data or point cloud data.

[0136] A region segmentation module is configured to segment the 3D detection data into a plurality of part regions according to the vehicle type information, and each part region contains one or more parts.

[0137] An abnormal region detection module is configured to detect the abnormal state of each part in each part region respectively, and determine an abnormal region in each part region.

[0138] An abnormal type judgment module is configured to judge the abnormal type according to the relationship between the abnormal region and each part in the corresponding part.

[0139] The abnormal region detection module in the embodiment specifically comprises:

[0140] A data acquisition unit is configured to acquire a current part region and template data corresponding to the current part region.

[0141] A traversal unit is configured to traverse each part in the current part region to detect the abnormal state.

[0142] When the traversal unit detects the abnormal state of each part, a to-be-detected part region is intercepted from the current part region, the intercepted to-be-detected part region is registered with the corresponding template data, and an abnormal region is determined according to the registration result.

[0143] In the embodiment, the abnormal type judgment module judges the abnormal type according to the overlapping relationship between the abnormal region and each part in the corresponding part, which comprises: acquiring an overlapping region between the abnormal region and all part regions in the corresponding template data; judging the proportion of the overlapping region in the abnormal region; if the proportion of the overlapping region in the abnormal region exceeds a preset first threshold, it is determined that the part is abnormal, otherwise, whether the part is abnormal is determined according to the proportion of the overlapping region in the part region and the geometric characteristics of the abnormal region; if the proportion of the overlapping region in the part region exceeds a preset second threshold, or the proportion of the overlapping region in the part region is less than the preset second threshold and the abnormal region satisfies a preset geometric characteristic, it is determined that the part is abnormal.

[0144] In the embodiment, a data acquisition device is further provided to acquire the to-be-detected data, the to-be-detected data acquisition module is further provided to acquire 3D image / point cloud data acquired by the data acquisition device as the to-be-detected data, the template data acquisition module is further provided to acquire template data corresponding to the to-be-detected data, and a display module is further provided to present the detection result.

[0145] In a specific application embodiment, as shown in Figure 3 The region segmentation module, the abnormal region detection module, and the abnormal type judgment module are implemented by one detection module, i.e., the region segmentation, the abnormal region detection, and the abnormal type judgment functions are implemented by one detection module. First, the relative movement (tending to uniform motion) between the data acquisition device and the train occurs to collect the 3D image / point cloud of the train in a line scanning manner. For the same train, the overall line number deviation of two data acquisition configurations does not exceed a certain range. After the data acquisition device collects the 3D image / point cloud data, the detection data acquisition module obtains the detection data, obtains the template data corresponding to the detection data through the template data, and divides the 3D detection data into multiple component regions according to the vehicle type information. Each component region contains one or more components. The detection module detects the abnormal state of each component in each component region, determines the abnormal region in each component region, and judges the abnormal type according to the overlapping relationship between the abnormal region and each part in the corresponding component. The detection module specifically judges the abnormal type according to the overlapping area ratio between the abnormal region and all part regions in the corresponding template data. If the overlapping area ratio exceeds a preset first threshold, it is determined that the part is abnormal. Then, the size of the overlapping area ratio on the part region is further judged. If the overlapping area ratio on the part region is large (greater than a preset second threshold), it indicates that the abnormal region is likely to be a non-part dominated abnormal region, and the component abnormality can be directly determined. If the overlapping area ratio on the part region is small (less than the preset second threshold), but the geometric characteristics are met, it indicates that the abnormal region is not affected by the part. Whether the component is abnormal is finally determined according to whether the geometric characteristics of the abnormal region meet the requirements. If there are multiple target abnormal regions overlapping with the same part region and the overlapping area exceeds the preset threshold, the target abnormal regions are merged and then re-judged. Finally, the detection result is displayed by the detection module, and the abnormal parts are marked and highlighted on the image.

[0146] The train component abnormality detection device of the embodiment corresponds to the train component abnormality detection method described above, and will not be described again.

[0147] The embodiment also provides an electronic device including a processor and a memory. The memory is used to store a computer program, and the processor is used to execute the computer program to perform the method described above.

[0148] The embodiment also provides a computer readable storage medium storing a computer program. The computer program is executed to implement the method described above.

[0149] The above are only the preferred embodiments of the present application, and do not limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments without departing from the technical solutions of the present application, and according to the technical essence of the present application, should fall within the scope of protection of the technical solutions of the present application.

Claims

1. A train component abnormality detection method characterized by comprising the steps of The method comprises the following steps: acquiring 3D detection data and vehicle type information of a train to be detected, wherein the detection data is image data or point cloud data; segmenting the 3D detection data into a plurality of component regions according to the vehicle type information, wherein each component region contains one or more components; performing abnormal state detection on each component in each component region to determine an abnormal region in each component region; judging an abnormal type according to an overlapping relationship between the abnormal region and each part in the corresponding component; the step of judging an abnormal type according to an overlapping relationship between the abnormal region and each part in the corresponding component comprises the following steps: acquiring an overlapping region between the abnormal region and all part regions in the corresponding template data; judging a proportion of the overlapping region in the abnormal region; if the proportion of the overlapping region in the abnormal region exceeds a preset first threshold, determining that the part is abnormal, otherwise, judging whether the component is abnormal according to the proportion of the overlapping region in the part region and the geometric characteristics of the abnormal region.

2. The train component anomaly detection method of claim 1, wherein, the step of segmenting the 3D detection data into a plurality of component regions according to the vehicle type information comprises the following steps: determining segmentation parameters according to the vehicle type information, wherein the segmentation parameters include position information of each component region; calculating offset line numbers and region lengths of each component region on the 3D detection data according to a line number relationship between the 3D detection data of the train to be detected and the 3D detection data of the corresponding template train and the segmentation parameters; segmenting according to the calculated offset line numbers and region lengths of each component region on the 3D detection data to obtain a plurality of component regions.

3. The train component anomaly detection method of claim 2, wherein, the position information of each component region includes any one or more of an offset line number of a starting position of each component region from a position of a train head, a position of each component region in a train car, and an offset line number of a starting position of each component region from a specified reference position of a current train car.

4. The train component anomaly detection method of claim 1, wherein the step of performing abnormal state detection on each component in each component region to determine an abnormal region in each component region comprises the following steps: acquiring a current component region and template data corresponding to the current component region; performing abnormal state detection on each component in the current component region; when performing abnormal state detection on each component, cutting out a detection component region from the current component region, registering the cut-out detection component region with the corresponding template data, and determining an abnormal region according to the registration result.

5. The train component anomaly detection method of claim 4, wherein, the step of cutting out a detection component region from the current component region and registering the cut-out detection component region with the corresponding template data comprises the following steps: acquiring position information of the detection component in the corresponding template data, and cutting out current required template data from the corresponding template data of the detection component according to the acquired position information; spatially expanding the position information of the detection component in the corresponding template data to obtain expanded position information; cutting out a detection component region from the current component region according to the expanded position information; registering the cut-out detection component region with the cut-out current required template data.

6. The train component anomaly detection method of claim 4, wherein, The data preprocessing before the registration of the intercepted component region to be detected and the corresponding template data comprises downsampling of the to-be-detected data, and the data preprocessing after the registration of the intercepted component region to be detected and the corresponding template data comprises downsampling and denoising of the to-be-detected data.

7. The train component anomaly detection method of claim 4, wherein, The abnormal region is determined according to the registration result, and if the fitting degree between the component region to be detected and the corresponding template data is less than a preset threshold, the entire component region to be detected is determined as the abnormal region, otherwise, the difference points between the component region to be detected and the corresponding template data are found, and the abnormal region is formed by the found difference points.

8. The train component anomaly detection method of claim 7, wherein, The difference points between the component region to be detected and the corresponding template data are found by: if a target point on the component region to be detected satisfies a preset first condition or a target point on the corresponding template data of the component region to be detected satisfies a preset second condition, the target point is determined as a difference point; the preset first condition is that there is no point matching the target point in a spherical space with a preset overlap region distance threshold as a radius in the corresponding template data, and the preset second condition is that there is no point matching the target point in a spherical space with a preset overlap region distance threshold as a radius in the component region to be detected.

9. The train component anomaly detection method of claim 7, wherein, After the difference points are found, effective abnormal points are intercepted from the range of the template data after registration, and the abnormal region is formed by clustering of the effective difference points.

10. The train component abnormality detection method according to any one of claims 1 to 9, characterized by When the abnormal region in each component region is determined, geometric feature judgment of the abnormal region is further performed, and the abnormal region whose geometric feature does not satisfy a preset condition is removed.

11. The train component anomaly detection method of claim 1, wherein, The component abnormality is determined according to the proportion of the overlap region in the part region, and the geometric feature judgment of the abnormal region, which comprises: if the proportion of the overlap region in the part region exceeds a preset second threshold, or the proportion of the overlap region in the part region is less than the preset second threshold and the abnormal region satisfies a preset geometric feature, the component abnormality is determined.

12. The train component anomaly detection method of claim 1, wherein, The abnormal type is determined according to the overlap relationship between the abnormal region and each part in the corresponding component, which further comprises: if multiple target abnormal regions overlap with the same part region and the overlap region exceeds a preset threshold, the target abnormal regions are merged to determine the abnormal type according to the overlap relationship between the merged abnormal region and each part in the corresponding component.

13. A train component abnormality detection device characterized by comprising: The method comprises: a to-be-detected data acquisition module configured to acquire 3D detection data of a to-be-detected train and vehicle type information, the detection data being image data or point cloud data; a region segmentation module configured to segment the 3D detection data into multiple component regions according to the vehicle type information, each component region containing one or more components; an abnormal region detection module configured to detect an abnormal state of each component in each component region to determine an abnormal region in each component region; an abnormal type determination module configured to determine an abnormal type according to an overlap relationship between the abnormal region and each part in the corresponding component; and an abnormal type determination module configured to determine an abnormal type according to an overlap relationship between the abnormal region and each part in the corresponding component. The abnormal type judging module judges the abnormal type according to the overlapping relationship between the abnormal area and each part in the corresponding component, and includes: obtaining an overlapping area between the abnormal area and all part areas in the corresponding template data; judging a proportion of the overlapping area in the abnormal area; if the proportion of the overlapping area in the abnormal area exceeds a preset first threshold, determining that the part is abnormal, otherwise, judging whether the component is abnormal according to the proportion of the overlapping area in the part area and the geometric feature of the abnormal area; if the proportion of the overlapping area in the part area exceeds a preset second threshold, or the proportion of the overlapping area in the part area is less than the preset second threshold and the abnormal area satisfies a preset geometric feature, determining that the component is abnormal.

14. The train component anomaly detection apparatus of claim 13, wherein The abnormal area detection module includes: a data acquisition unit configured to acquire a current component area and template data corresponding to the current component area; a traversal unit configured to traverse each component in the current component area to detect an abnormal state; when the traversal unit detects the abnormal state of each component, the traversal unit intercepts a to-be-detected component area from the current component area, registers the intercepted to-be-detected component area with the corresponding template data, and determines an abnormal area according to a registration result.

15. An electronic device comprising a processor and a memory for storing a computer program, characterized in that The processor is configured to execute the computer program to perform the method of any one of claims 1-12.

16. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed, implements the method of any one of claims 1-12.

Images