A method and system for identifying abnormality of square hole locks of railway vehicles

Abstract

The present application discloses a method and system for identifying abnormalities of square hole locks of rail vehicles, and the method comprises: collecting a first image of a vehicle to be inspected; locating first square hole lock position information and through-passage position information of the square hole lock and the through-passage on the first image by using a target detection algorithm; performing image stitching on the first image according to the through-passage position information and the first square hole lock position information to obtain a panoramic image of the carriage and second square hole lock position information of the square hole lock; cropping the panoramic image of the carriage according to the second square hole lock position information to obtain a square hole lock area image, extracting a lock core triangle mark from the square hole lock area image by using a semantic segmentation algorithm to obtain a lock core triangle mark; calculating a bottom edge midline angle of the lock core triangle mark, and judging whether the square hole lock has an abnormal condition according to the bottom edge midline angle.

Description

Technical Field

[0001] The present application relates to the technical field of rail vehicle maintenance, and in particular to a method and system for identifying abnormalities of square hole locks of rail vehicles. Background Art

[0002] In the operation and maintenance of rail vehicles, the skirts need to be opened frequently for equipment maintenance and inspection. As the locking parts of the skirts, square hole locks are used very frequently. There are many devices inside the skirts and they are concentrated. Once the square hole locks have problems, the normal operation of the vehicle will be affected.

[0003] At present, manual inspection is still the conventional method for fastener inspection. Manual inspection has the following risks: operators are prone to fatigue, the quality of inspection is overly dependent on the responsibility of the maintenance personnel, and there is a risk of missed inspection; manual inspection efficiency is low and the inspection time is long; lack of information support, inspection records and common problem information are inconvenient to store and retrieve, which is not conducive to statistics and traceability, and cannot be retrieved remotely, which is not convenient for problem investigation and supervision and management. Therefore, in the process of rail vehicle fastener inspection, how to improve inspection efficiency and realize information traceability has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a method and system for identifying abnormalities in square hole locks of rail vehicles, so as to at least solve the problems of low efficiency in manual inspection of rail vehicle locks, lack of information support, inconvenience in troubleshooting abnormal conditions of lock parts, inconvenience in supervision and management, and extreme imbalance in the number of positive and negative samples in the abnormal situation detection process based on deep learning.

[0005] The present invention provides an abnormality identification method, comprising:

[0006] Image acquisition step: acquiring a first image of the vehicle to be inspected;

[0007] The square hole lock and the through-passage positioning step: positioning the square hole lock and the through-passage position information on the first image by using a target detection algorithm;

[0008] Image stitching step: stitching the first image according to the through-tunnel position information and the first square hole lock position information to obtain a panoramic view of the carriage and second square hole lock position information of the square hole lock;

[0009] The lock core triangle mark extraction step: according to the second square hole lock position information, the car panoramic image is cropped to obtain a square hole lock area image, and the lock core triangle mark is extracted from the square hole lock area image by a semantic segmentation algorithm to obtain the lock core triangle mark;

[0010] Square hole lock abnormality judgment step: calculate the bottom edge centerline angle of the lock core triangle mark, and judge whether the square hole lock has an abnormal condition according to the bottom edge centerline angle.

[0011] In the above-mentioned abnormality identification method, the square hole lock and through-channel positioning step includes:

[0012] Performing data cleaning on the first image to obtain the second image containing the square hole lock and the through passage;

[0013] The second image is annotated to obtain a first image data set, and the first image data set is enhanced to obtain a second image data set.

[0014] In the above-mentioned abnormality identification method, the square hole lock and through-channel positioning step further comprises:

[0015] The second image data set is trained by the target detection algorithm to obtain a square hole lock and through-channel positioning model;

[0016] The square hole lock and through-passage positioning model detects the first image to obtain the first square hole lock position information of the square hole lock in the first image and the through-passage position information of the through-passage in the first image.

[0017] In the above-mentioned abnormality identification method, the image stitching step includes:

[0018] The first images are stitched to obtain the panoramic image of the carriage and the second square hole lock position information of the square hole lock on each carriage of the vehicle to be inspected.

[0019] In the above-mentioned abnormality identification method, the lock core triangle mark extraction step includes:

[0020] A third image data set containing the square hole lock is selected from the first image data set.

[0021] In the above-mentioned abnormality identification method, the lock core triangle mark extraction step further includes:

[0022] The square hole lock dataset is obtained by cropping according to the annotation box information in the third image dataset.

[0023] In the above-mentioned abnormality identification method, the lock core triangle mark extraction step further includes:

[0024] Performing data annotation on the square hole lock data set to obtain a fourth image data set, and performing data enhancement on the fourth image data set to obtain a fifth image data set;

[0025] The fifth image data set is trained to obtain a lock core triangle mark extraction model, and the lock core triangle mark extraction model is used to extract the lock core triangle mark from the square hole lock area image to obtain the lock core triangle mark.

[0026] In the above-mentioned abnormality identification method, the square hole lock abnormality identification step includes:

[0027] The vertex coordinates of the lock core triangle mark are obtained, the side length data of the lock core triangle mark is calculated according to the vertex coordinates, and the bottom edge midline angle is calculated according to the side length data.

[0028] In the above-mentioned abnormality identification method, the square hole lock abnormality identification step further comprises:

[0029] Preset bottom edge midline angle range;

[0030] When the bottom edge centerline angle is within the preset bottom edge centerline angle range, the square hole lock determines that the abnormal condition exists.

[0031] The present invention also provides an abnormality identification system, wherein the abnormality identification method described above is applicable, and the abnormality identification system comprises:

[0032] Image acquisition unit: acquires a first image of the vehicle to be inspected through a color array camera;

[0033] A square hole lock and through-passage positioning unit: a first square hole lock position information and a through-passage position information of the square hole lock and the through-passage on the first image are positioned by a target detection algorithm;

[0034] Image stitching unit: stitching the second image according to the through-passage position information and the square hole lock position information to obtain a panoramic view of the carriage and the second square hole lock position information of the square hole lock;

[0035] A lock core triangle mark extraction unit: according to the second square hole lock position information, the car panoramic image is cropped to obtain a square hole lock area image, and the lock core triangle mark is extracted from the square hole lock area image by a semantic segmentation algorithm to obtain the lock core triangle mark;

[0036] The square hole lock abnormality judgment unit calculates the bottom edge center line angle of the lock core triangle mark, and judges whether the square hole lock has an abnormal condition according to the bottom edge center line angle.

[0037] Compared with the related art, the present invention proposes a method and system for identifying abnormalities of square hole locks of rail vehicles. The method uses a high-frequency color area array camera to collect the side image of the vehicle to be inspected, obtains two adjacent frames of images with effective overlap, and improves the clarity of the features of small parts such as square hole locks; the compartment where the square hole lock is located is determined by identifying the segmentation of the through-channel, and the square hole lock is accurately located by using image stitching, so that when an abnormal condition occurs in the square hole lock, the abnormality or fault location of the square hole lock is reported, the variation troubleshooting and supervision and management are carried out, and the efficiency of abnormality detection is improved; in the abnormality detection method based on the deep learning network of the present invention, there is no need to use the image of the square hole lock with abnormalities as a negative sample, and pre-label and train it, which solves the problem that the number of positive and negative samples is extremely unbalanced and negative samples are difficult to obtain; after extracting the triangular mark of the square hole lock core, the present invention combines geometric knowledge to judge whether the square hole lock has an abnormal condition; the present invention replaces manual detection or inspection to a large extent, improves the detection efficiency of parts such as rail vehicle fasteners, and realizes information traceability.

[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 This is a flowchart of a method for identifying anomalies of square hole locks of rail vehicles according to an embodiment of the present application;

[0041] Figure 2 is a flow chart of a method for identifying anomalies of square hole locks of rail vehicles according to an embodiment of the present application;

[0042] Figure 3 It is a flow chart of constructing a square hole lock and through-channel positioning model according to an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of a square hole locking result according to an embodiment of the present application;

[0044] Figure 5 is a flowchart of image stitching steps according to an embodiment of the present application;

[0045] Figure 6 This is a flow chart of constructing a lock core triangle mark extraction model according to an embodiment of the present application;

[0046] Figure 7 is a schematic diagram of the extraction result of the square hole lock triangle mark according to an embodiment of the present application;

[0047] Figure 8 is a schematic diagram of an isosceles triangle according to an embodiment of the present application;

[0048] Fig. 9 It is a structural schematic diagram of the rail vehicle square hole lock abnormality identification system of the present invention.

[0049] Wherein, the accompanying drawings are marked as follows:

[0050] Image acquisition unit: 51;

[0051] Square hole lock and through-channel positioning unit: 52;

[0052] Image stitching unit: 53;

[0053] Lock cylinder triangle mark extraction unit: 54. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0055] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the disclosure of the present application, some changes in design, manufacturing or production based on the technical content disclosed in the present application are just conventional technical means, and should not be understood as the disclosure of the present application is insufficient.

[0056] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0057] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0058] The present invention provides a method and system for identifying abnormalities in square hole locks of rail vehicles, which can replace manual inspections to a large extent, improve detection efficiency, and realize information-based traceability. In addition, existing anomaly detection methods based on deep learning usually require positive and negative samples for annotation and training. However, the probability of rail vehicle failure is very low, and negative samples are difficult to collect and the number is extremely small. The present invention does not require abnormal square hole lock images to be annotated and trained as negative samples in advance, which solves the problem that the number of positive and negative samples is extremely unbalanced and negative samples are difficult to obtain.

[0059] The present invention is described below in conjunction with specific embodiments.

[0060] Embodiment 1

[0061] This embodiment also provides a method for identifying abnormalities in square hole locks of rail vehicles. Figures 1 to 8 , Figure 1 This is a flowchart of a method for identifying anomalies of square hole locks of rail vehicles according to an embodiment of the present application; Figure 2 is a flow chart of a method for identifying anomalies of square hole locks of rail vehicles according to an embodiment of the present application; Figure 3 It is a flow chart of constructing a square hole lock and through-channel positioning model according to an embodiment of the present application; Figure 4 is a schematic diagram of a square hole locking result according to an embodiment of the present application; Figure 5 is a flowchart of image stitching steps according to an embodiment of the present application; Figure 6 This is a flow chart of constructing a lock core triangle mark extraction model according to an embodiment of the present application; Figure 7 is a schematic diagram of the extraction result of the square hole lock triangle mark according to an embodiment of the present application; Figure 8 Schematic diagram of an isosceles triangle according to an embodiment of the present application. Figures 1 to 8 As shown, the anomaly identification method includes:

[0062] Image acquisition step S1: acquiring a first image of the vehicle to be inspected;

[0063] Square hole lock and through passage positioning step S2: locating first square hole lock position information and through passage position information of the square hole lock and the through passage on the first image by using a target detection algorithm;

[0064] Image stitching step S3: stitching the first image according to the through-passage position information and the first square hole lock position information to obtain a panoramic view of the carriage and second square hole lock position information of the square hole lock;

[0065] Lock core triangle mark extraction step S4: according to the second square hole lock position information, the car panoramic image is cropped to obtain a square hole lock area image, and the lock core triangle mark is extracted from the square hole lock area image by a semantic segmentation algorithm to obtain the lock core triangle mark;

[0066] Square hole lock abnormality judgment step S5: Calculate the bottom edge centerline angle of the lock core triangle mark, and judge whether the square hole lock has an abnormal condition according to the bottom edge centerline angle.

[0067] In an embodiment, the image acquisition step S1 comprises:

[0068] A first image of the vehicle to be inspected is acquired.

[0069] In a specific implementation, the present invention uses a high-frequency color area array camera acquisition device to accumulate and acquire the passing data of vehicles to be inspected in the past three months, including two-dimensional images of the vehicle side of vehicles of various models and numbers at different passing times, i.e., the first image. The speed measuring device controls the external triggering of the camera to effectively overlap the two adjacent frames of images collected, wherein the overlapping area cannot be too large, so as to reduce the workload and running time of the later processing, and the overlapping area cannot be too small, so as to ensure that there are enough pairs of feature points successfully matched between the two adjacent frames of images in the later image stitching process; in the field of rail vehicle maintenance, the acquisition of vehicle side images and fault detection are mostly based on line scan cameras. In order to adapt to the higher passing speed, the camera line frequency cannot be very high, generally 2K, and the image resolution is not high; and the line scan camera has a large field of view, and the pixel proportion of components such as square hole locks in the image is very small, and features such as corners and edges are not prominent; however, in the present invention, since the two adjacent frames of images collected by the high-frequency color area array camera overlap effectively, the features of small components such as square hole locks in the collected two-dimensional image of the vehicle side of the vehicle to be inspected are very clear.

[0070] In an embodiment, the square hole lock and through-channel positioning step S2 includes:

[0071] Performing data cleaning on the first image to obtain a second image containing the square hole lock and the through-channel;

[0072] Annotating the first image to obtain a first image dataset, and performing image enhancement on the first image dataset to obtain a second image dataset;

[0073] The second image data set is trained by a target detection algorithm to obtain a square hole lock and through-channel positioning model;

[0074] The square hole lock and through-passage positioning model detects the first image to obtain first square hole lock position information of the square hole lock in the first image and through-passage position information of the through-passage in the first image.

[0075] In the specific implementation, data processing is first performed, specifically, images containing square hole locks and through-channels are sorted and screened out, namely, the second images, which are used for training data sets and test data sets; data annotation is performed, specifically, the annotation tool LabelImg is used to generate an XML annotation file in the PascalVOC format, namely, the first image data set; then data enhancement is performed, specifically, data enhancement is performed on the annotated image data set by image mirroring and image contrast adjustment operations to obtain an enhanced image data set, namely, the second image data set; finally, the model is constructed, specifically, based on Yolo The v5 network, i.e., the target detection algorithm, trains the enhanced training data set and the test data set, i.e., the second image data set, to obtain a square hole lock positioning model and a through-passage positioning model, wherein the square hole lock and through-passage positioning model is used to detect the first image, and obtain the first square hole lock position information of the square hole lock in the first image and the through-passage position information of the through-passage in the first image; the square hole lock and through-passage positioning model is evaluated to obtain the position of the square hole lock in the two-dimensional image of the vehicle side to be repaired, i.e., the first square hole lock position information and the position of the through-passage in the two-dimensional image of the vehicle side to be repaired, i.e., the first image, i.e., the through-passage position information; wherein, the target detection algorithm realizes accurate positioning of the square hole lock and the through-passage based on the Yolo v5 network; wherein, the main evaluation indicators include average accuracy, loss function curve, accuracy and recall rate, etc. If the evaluation indicators are not ideal, the parameter training is reset, i.e., the model is tuned.

[0076] In an embodiment, the image stitching step S3 includes:

[0077] The first images are stitched to obtain a panoramic view of the carriage and second square hole lock position information of the square hole locks on each carriage of the vehicle to be inspected.

[0078] In a specific implementation, a 6-carriage vehicle has 5 gangways, and a train is divided into 6 carriages. Each carriage has multiple square hole locks. According to the position information of the gangway on the two-dimensional image of the vehicle side obtained by the square hole locks and the gangway positioning step S2, the two-dimensional images between the two gangways are spliced ​​to obtain a panoramic view of the carriage; specifically, first, feature point extraction is performed. There are many definitions of feature points in the CV field, such as scale-invariant feature feature point detectors SIFT, SURF, BRISK, ORB, etc. Among them, ORB and BRISK are usually more invariant to rotation and affine transformation. Although they are the most effective algorithms for detecting a large number of features, the matching time of such a large number of features will prolong the total image matching time. On the contrary, ORB and BRISK performs the fastest image matching, but its accuracy is compromised. Therefore, for all types of geometric transformations, SIFT and BRISK have the highest overall accuracy. The present invention adopts the detector of BELID, and the descriptor uses the BoostedSCC algorithm to select K feature points. The BoostedSCC algorithm is an improvement on AdaBoost, which uses an enhancement algorithm to select a set of features discriminatively and combine them to produce a strong description. This method improves the calculation speed, the execution time is close to ORB, the accuracy is similar to SIFT, and the best features can be selected for the task. Then the feature points are matched, specifically, the feature descriptors of the two images are traversed, wherein the feature points containing the feature point information are selected. The feature vector is then calculated, and the quality of the match, that is, the distance between the vectors, is calculated. The feature points are sorted according to the distance, and the matching results of the first N features are displayed under a certain confidence level. Among them, the present invention selects the FLANNMatch method; secondly, image matching, specifically, after obtaining the matching point set of the two images, the homography transformation matrix is ​​solved based on the RANSAC method; secondly, projection transformation, specifically, the two images are converted to the same plane, and this plane is called the composite panoramic plane; secondly, splicing calculation, specifically, the deformed image coordinate range of each input image is first calculated to obtain the output image size, and the output image size can be easily determined by mapping the four corners of each source image and calculating the minimum and maximum values ​​of the coordinates (x, y) image size; then, it is necessary to calculate the offsets xoffset and yoffset of the origin of the specified reference image relative to the output panoramic image; finally, using the inverse deformation described above, the pixels of each input image are mapped to the plane defined by the reference image, and the forward deformation and reverse deformation of the points are performed respectively; secondly, image fusion is performed, specifically, the pixel colors are fused in the overlapping area to avoid seams, wherein the present invention uses the simplest feathering method, which uses the weighted average color value to fuse the overlapping pixels; secondly, a panoramic image of the carriage is generated, specifically, the above operation is repeated for multiple two-dimensional images between the two through channels to generate a panoramic image of the carriage; finally, the square hole lock coordinates are transformed to the panoramic image of the carriage to obtain a small image of the square hole lock area.

[0079] In an embodiment, the lock core triangle mark extraction step S4 includes:

[0080] Selecting a third image data set containing the square hole lock from the first image data set;

[0081] The square hole lock dataset is obtained by cropping according to the annotation box information in the third image dataset;

[0082] Performing data annotation on the square hole lock dataset to obtain a fourth image dataset, and performing data enhancement on the fourth image dataset to obtain a fifth image dataset;

[0083] The fifth image data set is trained to obtain a lock core triangle mark extraction model, and the lock core triangle mark extraction model is used to extract the lock core triangle mark from the square hole lock area image to obtain the lock core triangle mark.

[0084] In the specific implementation, firstly, the square hole lock area image is obtained, specifically, the image containing the square hole lock and the annotation file corresponding to the image, i.e., the third data set, are separately screened out from the first image data set. The third image data set includes the square hole lock image and its annotation file. The annotation file records the position of the square hole lock on the two-dimensional image. The square hole lock data set is obtained by cutting according to this position information and used for training the semantic segmentation model. Then, data processing is performed, specifically, the annotation tool Labelme is used to annotate the triangular contour of the lock core in the square hole lock data set, and after generating the json annotation file, i.e., the fourth image data set and the label image, the fourth image data set is digitized. According to the enhancement, specifically, the annotated square hole lock dataset, i.e., the fourth image dataset, is subjected to image rotation and image contrast adjustment operations to obtain an enhanced image dataset, i.e., the fifth image dataset; finally, the model is constructed, and the enhanced training dataset and test dataset are trained by the semantic segmentation algorithm to obtain a lock core triangle logo extraction model. After the lock core triangle logo extraction model is evaluated, the lock core triangle logo is extracted from the square hole lock area image by the lock core triangle logo extraction model to obtain the lock core triangle logo; wherein, the semantic segmentation algorithm is implemented based on the Unet network; the main evaluation indicators include average intersection-over-union ratio, class pixel accuracy, loss function curve, etc. If the evaluation indicators are not ideal, the parameter training is reset, i.e., model tuning.

[0085] In an embodiment, the square hole lock abnormality identification step S5 includes:

[0086] Obtain the vertex coordinates of the lock core triangle mark, calculate the side length data of the lock core triangle mark according to the vertex coordinates, and calculate the bottom edge midline angle according to the side length data;

[0087] Preset bottom edge midline angle range;

[0088] When the bottom edge center line angle is within the preset bottom edge center line angle range, the square hole lock determines that an abnormal condition exists.

[0089] In a specific implementation, the properties of an isosceles triangle are: its two sides are equal in length, the bisector of its vertex angle, the midline on the base, and the height on the base coincide, which is abbreviated as "the three lines of an isosceles triangle merge into one". Since the lock core triangle of the square hole lock is marked as an isosceles triangle, based on the above properties of the isosceles triangle, it can be determined whether the square hole lock has rotated, that is, whether there is an abnormal condition, based on the angle of the midline of the base.

[0090] Specifically, first determine the coordinates of the three vertices of the lock core triangle mark of the square hole lock, calculate the three side lengths of the lock core triangle mark, and count them as a, b, and c respectively, and let a=b, where the vertices corresponding to the three sides are A(X a ,Y a )、B(X b ,Y b )、C(X c ,Y c ), then a=|BC|=|AC|=b, C is the vertex angle; then calculate the angle of the midline of the base, where the midpoint of the base is D(X d ,Y d ),but

[0091]

[0092]

[0093] The angle of the straight line where the center line CD is located, that is, the angle of the midline of the base is:

[0094]

[0095] When the angle of the bottom midline of the triangle mark on the lock core of the square hole lock is within the angle range of angle∈(-100°,80°), that is, when the top angle of the triangle mark on the lock core does not point to the word "off", it is determined that there is an abnormality in the square hole lock.

[0096] Embodiment 2

[0097] This embodiment provides specific implementation steps of a method for identifying anomalies of square hole locks of rail vehicles. The specific implementation steps of the anomaly identification method are as follows:

[0098] Step 1: Image acquisition step S1, using a high-frequency color area array camera to acquire a two-dimensional image of the side of the vehicle to be inspected;

[0099] Step 2: Positioning the square hole lock and the gangway Step S2, positioning the position information of the square hole lock and the gangway on the two-dimensional image of the vehicle side by using a target detection algorithm;

[0100] Step 3: Image stitching step S3, according to the position information of the gangway on the two-dimensional image of the vehicle side, the two-dimensional images between the gangways are stitched to obtain the position information of the square hole lock on each car and the panoramic view of the car;

[0101] Step 4: Extracting the lock core triangle mark Step S4, according to the position information of the square hole lock on each carriage, after cropping the carriage panorama to obtain the square hole lock area image, the lock core triangle mark is extracted from the square hole lock area image by using a semantic segmentation algorithm to obtain the lock core triangle mark of the square hole lock;

[0102] Step 5: Square hole lock abnormality judgment step S5, after calculating the side length data of the lock core triangle mark according to the vertex coordinates of the lock core triangle mark, calculate the bottom edge centerline angle according to the side length data. If the bottom edge centerline angle of the lock core triangle mark is within the preset bottom edge centerline angle range, the square hole lock is judged to have an abnormal condition, that is, when the top angle of the lock core triangle mark does not point to the word "off", it is judged that the square hole lock has an abnormal condition.

[0103] Embodiment 3

[0104] This embodiment also provides a rail vehicle square hole lock abnormality identification system. Fig. 9 FIG. 1 is a schematic diagram of the structure of the rail vehicle square hole lock abnormality identification system of the present invention. Fig. 9 As shown, the anomaly recognition system of the invention is applicable to the following anomaly recognition method, and the anomaly recognition system includes:

[0105] Image acquisition unit: acquires a first image of the vehicle to be inspected through a color array camera;

[0106] Square hole lock and through-passage positioning unit: locates the first square hole lock position information and through-passage position information of the square hole lock and the through-passage on the first image through a target detection algorithm;

[0107] Image stitching unit: stitching the first image according to the through-channel position information and the square hole lock position information to obtain a panoramic view of the carriage and second square hole lock position information of the square hole lock;

[0108] The lock core triangle mark extraction unit: according to the second square hole lock position information, the car panoramic image is cropped to obtain the square hole lock area image, and the lock core triangle mark is extracted from the square hole lock area image by using the semantic segmentation algorithm to obtain the lock core triangle mark;

[0109] Square hole lock abnormality judgment unit: calculates the bottom edge midline angle of the lock core triangle mark, and judges whether the square hole lock has an abnormal condition according to the bottom edge midline angle.

[0110] In summary, the present invention provides a method and system for identifying abnormal square hole locks of rail vehicles, which effectively overlaps two adjacent frames of images collected by a high-frequency color area array camera, thereby improving the feature clarity of small parts such as square hole locks. In addition, the method proposed by the present invention does not require abnormal square hole lock images to be pre-labeled and trained as negative samples, thus solving the problem of extreme imbalance in the number of positive and negative samples.

[0111] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the protection scope of the attached claims.

Claims

1. A method for identifying abnormalities of square hole locks of railway vehicles, characterized in that: The abnormality identification method comprises: Image acquisition step: acquiring a first image of the vehicle to be inspected; The square hole lock and the through-passage positioning step: positioning the square hole lock and the through-passage position information on the first image by using a target detection algorithm; Image stitching step: stitching the first image according to the through-tunnel position information and the first square hole lock position information to obtain a panoramic view of the carriage and second square hole lock position information of the square hole lock; The lock core triangle mark extraction step: according to the second square hole lock position information, the car panoramic image is cropped to obtain a square hole lock area image, and the lock core triangle mark is extracted from the square hole lock area image by a semantic segmentation algorithm to obtain the lock core triangle mark; Square hole lock abnormality judgment step: calculating the bottom edge centerline angle of the lock core triangle mark, and judging whether the square hole lock has an abnormality according to the bottom edge centerline angle; The image stitching step includes: stitching the first image to obtain the panoramic view of the carriage and the second square hole lock position information of the square hole lock on each carriage of the vehicle to be inspected; The square hole lock abnormality judgment step also includes: Preset bottom edge midline angle range; When the bottom edge centerline angle is within the preset bottom edge centerline angle range, the square hole lock determines that the abnormal condition exists.

2. The abnormality identification method according to claim 1, characterized in that: The square hole lock and through-channel positioning steps include: Performing data cleaning on the first image to obtain a second image containing the square hole lock and the through passage; The second image is annotated to obtain a first image data set, and the first image data set is enhanced to obtain a second image data set.

3. The abnormality identification method according to claim 2, characterized in that: The square hole lock and through-channel positioning step also includes: The second image data set is trained by the target detection algorithm to obtain a square hole lock and through-channel positioning model; The square hole lock and through-passage positioning model detects the first image to obtain the first square hole lock position information of the square hole lock in the first image and the through-passage position information of the through-passage in the first image.

4. The abnormality identification method according to claim 2, characterized in that: The lock core triangle mark extraction step comprises: A third image data set containing the square hole lock is selected from the first image data set.

5. The abnormality identification method according to claim 4, characterized in that: The lock core triangle mark extraction step also includes: The square hole lock dataset is obtained by cropping according to the annotation box information in the third image dataset.

6. The abnormality identification method according to claim 5, characterized in that: The lock core triangle mark extraction step also includes: Performing data annotation on the square hole lock data set to obtain a fourth image data set, and performing data enhancement on the fourth image data set to obtain a fifth image data set; The fifth image data set is trained to obtain a lock core triangle mark extraction model, and the lock core triangle mark is extracted from the square hole lock area image by the lock core triangle mark extraction model to obtain the lock core triangle mark.

7. The abnormality identification method according to claim 1, characterized in that: The square hole lock abnormality judgment step comprises: The vertex coordinates of the lock core triangle mark are obtained, the side length data of the lock core triangle mark is calculated according to the vertex coordinates, and the bottom edge midline angle is calculated according to the side length data.

8. An abnormality identification system, characterized in that: The abnormality identification system comprises: Image acquisition unit: acquires a first image of the vehicle to be inspected through a color array camera; A square hole lock and through-passage positioning unit: a first square hole lock position information and a through-passage position information of the square hole lock and the through-passage on the first image are positioned by a target detection algorithm; Image stitching unit: stitching the first image according to the through-passage position information and the first square hole lock position information to obtain a panoramic view of the carriage and second square hole lock position information of the square hole lock; A lock core triangle mark extraction unit: according to the second square hole lock position information, the car panoramic image is cropped to obtain a square hole lock area image, and the lock core triangle mark is extracted from the square hole lock area image by a semantic segmentation algorithm to obtain the lock core triangle mark; Square hole lock abnormality judgment unit: calculates the bottom edge midline angle of the lock core triangle mark, and judges whether the square hole lock has an abnormal condition according to the bottom edge midline angle; Wherein, the image stitching unit comprises: stitching the first image to obtain the panoramic image of the carriage and the second square hole lock position information of the square hole lock on each carriage of the vehicle to be inspected; The square hole lock abnormality judgment unit also includes: Preset bottom edge midline angle range; When the bottom edge centerline angle is within the preset bottom edge centerline angle range, the square hole lock determines that the abnormal condition exists.

Images