Intelligent Inspection System and Method for Finger Plates of Track Beams Based on Computer Vision

Abstract

The present invention discloses an intelligent inspection system and method for finger plates of track beams based on computer vision. The above system includes on-vehicle devices installed on an inspection vehicle and a ground analysis center. The on-vehicle devices include: a track beam number recognition module, a three-dimensional detection module for finger plates, a pulse control box, and an image analysis and processing system. The track beam number recognition module collects track beam number images as the inspection vehicle runs and recognizes the track beam numbers based on the track beam number images. When the track beam number recognition module recognizes the track beam numbers, the pulse control box triggers the three-dimensional detection module for finger plates to collect three-dimensional images and two-dimensional images of the finger plates, and the image analysis and processing system processes and analyzes the three-dimensional images and two-dimensional images of the finger plates in combination with the track beam numbers. Through this intelligent inspection system for finger plates, the inspection efficiency can be improved and the manual maintenance cost can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of track detection, and particularly to an intelligent inspection system and method for finger plates of track beams based on computer vision. Background Art

[0002] Rail transit has become one of the important means of transportation for urban people in China. At present, the total mileage of urban rail transit in China has exceeded 6,000 km, and the proportion of subways is about 85%. However, traditional subways have high investment and long construction periods, and it is difficult to meet the traffic construction needs of small and medium-sized cities. Compared with double-track rail subways, straddle-type monorails are low-cost, occupy less space, are light and flexible, and are more suitable for laying in cities with complex terrains and dense populations.

[0003] With the continuous expansion of the application scope of straddle-type monorail technology, higher requirements are put forward for the regular inspection of its components. As a key component of the straddle-type monorail, the loosening of the finger plate causes misalignment of two adjacent beams, which will affect the safe operation of rail transit. At present, the daily inspection (line inspection) of finger plates mainly relies on manual testing to check whether the finger plates are faulty. Due to factors such as insufficient light at night, too fast speed of the work vehicle, and poor vision of the inspection personnel, it is impossible to accurately detect the subtle diseases existing in the finger plates. The ideal method for finger plate inspection is to use machines to replace humans to achieve automatic identification of fault features. However, the currently commonly used identification methods still remain at the level of installing auxiliary sensors. Due to the large number of finger plates and limited installation space, it is difficult to achieve large-scale and high-efficiency finger plate inspection. Summary of the Invention

[0004] In view of the above-mentioned technical problems such as low inspection efficiency of existing finger plates, the present invention provides an intelligent inspection system and method for finger plates of track beams based on computer vision that can improve the inspection efficiency.

[0005] In a first aspect, an embodiment of the present application provides an intelligent inspection system for finger plates of track beams based on computer vision, including on-vehicle equipment installed on an inspection vehicle, and the on-vehicle equipment includes: a track beam number recognition module, a three-dimensional finger plate detection module, a pulse control box, and an image analysis and processing system;

[0006] The track beam number recognition module collects track beam number images during the operation of the inspection vehicle and recognizes the track beam number according to the track beam number images. When the track beam number recognition module recognizes the track beam number, the pulse control box triggers the three-dimensional finger plate detection module to collect three-dimensional images and two-dimensional images of the finger plates, and the image analysis and processing system processes and analyzes the three-dimensional images and two-dimensional images of the finger plates in combination with the track beam number.

[0007] The above-mentioned intelligent inspection system for the finger plates of the track beam. Among them, the image analysis and processing system includes an in-vehicle server and a display terminal. The in-vehicle server performs finger plate detection on the two-dimensional image of the finger plate through deep learning based on the YOLO model to obtain the position information of the finger plate, and judges whether the finger plate fails according to the three-dimensional image of the finger plate based on the position information; if a failure occurs, the display terminal displays the failure information and gives an alarm.

[0008] The above-mentioned intelligent inspection system for the finger plates of the track beam. Among them, the in-vehicle device also includes a mileage encoder for recording the driving mileage of the inspection vehicle; if the in-vehicle server does not obtain the position information of the finger plate when performing finger plate detection on the two-dimensional image of the finger plate, it is judged whether the finger plate is within the actual mileage range through the mileage encoder.

[0009] The above-mentioned intelligent inspection system for the finger plates of the track beam. Among them, the track beam number recognition module recognizes the track beam number through OCR character recognition technology and calibrates the mileage encoder according to the recognized track beam number.

[0010] The above-mentioned intelligent inspection system for the finger plates of the track beam. Among them, it further includes a ground analysis center. The ground analysis center receives the compressed three-dimensional image and two-dimensional image of the finger plate, extracts the characteristic data of the finger plate according to the three-dimensional image and two-dimensional image of the finger plate, and predicts the characteristic change trend of the finger plate by using the characteristic data of the finger plate.

[0011] The above-mentioned finger plate three-dimensional detection module includes multiple line laser 3D cameras and 2D cameras. After being triggered, the finger plate three-dimensional detection module collects three-dimensional images through the line laser 3D cameras and collects two-dimensional images through the 2D cameras. After collecting for a certain distance, the finger plate three-dimensional detection module automatically shuts down until the track beam number recognition module recognizes the track beam number again.

[0012] In a second aspect, an embodiment of the present application provides a computer vision-based intelligent inspection method for the finger plates of the track beam applied to the above-mentioned intelligent inspection system for the finger plates of the track beam, including:

[0013] Track beam number recognition step: Collect the track beam number image through the track beam number recognition module, and recognize the track beam number according to the track beam number image based on OCR character recognition technology;

[0014] Finger plate image acquisition step: When the track beam number is recognized, trigger the finger plate three-dimensional detection module to collect the three-dimensional image and two-dimensional image of the finger plate through the pulse control box;

[0015] Finger plate image processing steps: Send the three-dimensional image and two-dimensional image of the finger plate to the image analysis and processing system, and process and analyze the three-dimensional image and two-dimensional image of the finger plate by the image analysis and processing system in combination with the track beam number.

[0016] The above-mentioned intelligent inspection method for track beam finger plates, wherein the finger plate image processing steps include:

[0017] Two-dimensional image detection step: Detect the finger plate from the two-dimensional image of the finger plate through deep learning based on the YOLO model to obtain the position information of the finger plate;

[0018] Three-dimensional image detection step: Detect the three-dimensional image of the finger plate according to the position information to judge whether a fault occurs.

[0019] The above-mentioned intelligent inspection method for track beam finger plates, wherein the two-dimensional image detection step further includes:

[0020] If the position information of the finger plate is not detected, judge whether the finger plate is within the actual mileage range according to the driving mileage recorded by the mileage encoder. If not, record the fault information.

[0021] The above-mentioned intelligent inspection method for track beam finger plates, wherein the three-dimensional image detection step includes:

[0022] Detect the three-dimensional image of the finger plate according to the position information to obtain a detection result, judge whether the detection result meets the detection index. If it meets, return to the track beam number identification step. If it does not meet, record the fault information.

[0023] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0024] 1. The single-rail vehicle finger plate detection system based on the vision method proposed by the present invention is installed on the track detection vehicle, which not only improves the manual efficiency but also does not require a separately customized detection vehicle; and uses the point cloud map and color map methods based on the camera module for feature detection and fault measurement of the track beam, and performs real-time detection during the movement of the vehicle, realizing non-contact and high-efficiency finger plate status detection;

[0025] 2. By collecting finger plate information, globally analyzing the finger plate, giving a fault alarm for faulty components, and timely completing the maintenance tasks, effectively ensuring that the finger plates on the on-line operation lines are all in a normal and safe working state, and guaranteeing the normal operation of the train. Brief Description of the Drawings

[0026] Figure 1 It is a schematic diagram of the track beam finger plate;

[0027] Figure 2 Schematic diagram of the structure of the intelligent inspection system for finger plates of track beams based on computer vision provided by the present invention;

[0028] Figure 3 Schematic diagram of the layout of the system equipment provided by the present invention;

[0029] Figure 4 External view schematic diagram of the three-dimensional detection module for finger plates provided by the present invention;

[0030] Figure 5 Schematic diagram of the installation position of the three-dimensional detection module for finger plates provided by the present invention;

[0031] Figure 6 Flowchart of the OCR solution based on deep learning provided by the present invention;

[0032] Figure 7 Workflow diagram of the image analysis and processing system provided by the present invention;

[0033] Figure 8 Schematic diagram of the target detection result of the finger plate provided by the present invention;

[0034] Figure 9 Flowchart of the YOLO framework for target detection provided by the present invention;

[0035] Figure 10 Result diagram of the point cloud processing of the finger plate provided by the present invention;

[0036] Figure 11 Workflow diagram of the vehicle-mounted equipment provided by the present invention;

[0037] Figure 12 Schematic diagram of the process of the intelligent inspection method for finger plates of track beams based on computer vision provided by the present invention;

[0038] Among them, the reference numerals are:

[0039] 1. Three-dimensional detection module for finger plates; 2. Track beam number recognition module. Detailed implementation manners

[0040] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described and explained below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided by the present application without creative efforts shall fall within the scope of protection of the present application.

[0041] Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present application. For those of ordinary skill in the art, without creative efforts, the present application can also be applied to other similar scenarios based on these drawings. In addition, it can also be understood that although the efforts made in such a development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing, or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood as insufficient disclosure of the content of the present application.

[0042] In this application, the mention of "embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those of ordinary skill in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0043] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the ordinary meaning understood by those of ordinary skill in the technical field to which this application belongs. The terms "a", "an", "one", "the", and similar words involved in this application do not indicate a limitation in quantity and can represent a singular or plural number. The terms "including", "comprising", "having", and any variations thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include unlisted steps or units, or may further include other steps or units inherent to these processes, methods, products, or devices. The terms "connected", "coupled", and similar words involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0044] The present invention will be described in detail below in conjunction with the embodiments shown in the accompanying drawings. It should be noted, however, that these embodiments are not intended to limit the present invention, and any equivalent transformation or substitution in terms of function, method, or structure made by those of ordinary skill in the art based on these embodiments shall fall within the protection scope of the present invention.

[0045] The finger plate automatic inspection system for the track beam of the present invention is designed based on non-contact three-dimensional vision measurement technology. Through the detection system installed at the front end of the inspection vehicle, all finger plates on the top running surface, both side guiding surfaces, and stabilizing surfaces of the track beam can be detected simultaneously, such as Figure 1 shown, and real-time alarms can be issued for major diseases such as loose fixing bolts, misalignment, finger touching, and excessive gaps of the finger plates.

[0046] Embodiment 1:

[0047] Figure 2 is a structural schematic diagram of the intelligent inspection system for the finger plates of the track beam based on computer vision provided by the present invention; as Figure 2 shown, the intelligent inspection system for the finger plates of the track beam includes on-vehicle equipment installed on the inspection vehicle and a ground analysis center. The on-vehicle equipment includes: a track beam number recognition module, a three-dimensional finger plate detection module, a pulse control box, and an image analysis and processing system;

[0048] The track beam number recognition module collects track beam number images as the inspection vehicle runs and identifies the track beam number according to the track beam number images through OCR character recognition technology. When the track beam number recognition module recognizes the track beam number, the pulse control box triggers the three-dimensional finger plate detection module to collect three-dimensional images and two-dimensional images of the finger plates, and the image analysis and processing system processes and analyzes the three-dimensional images and two-dimensional images of the finger plates in combination with the track beam number.

[0049] The three-dimensional finger plate detection module includes multiple line laser 3D cameras and 2D cameras. After being triggered, the three-dimensional finger plate detection module collects three-dimensional images through the line laser 3D cameras and two-dimensional images through the 2D cameras. After collecting for a certain distance, the three-dimensional finger plate detection module automatically shuts down until the track beam number recognition module recognizes the track beam number again.

[0050] The image analysis and processing system includes an on-vehicle server and a display terminal. The on-vehicle server performs finger plate detection on the two-dimensional images of the finger plates through deep learning based on the YOLO model to obtain the position information of the finger plates, and judges whether the finger plates are faulty according to the position information through the three-dimensional images of the finger plates; if a fault occurs, the display terminal displays the fault information and issues an alarm.

[0051] The vehicle-mounted device further includes an odometer encoder for recording the driving mileage of the inspection vehicle; and the odometer encoder can be calibrated according to the track beam number identified by the track beam number identification module. If the vehicle-mounted server fails to obtain the position information of the finger plate when detecting the two-dimensional image of the finger plate, it is determined whether the finger plate is within the actual mileage range through the odometer encoder.

[0052] The ground analysis center receives the compressed three-dimensional image and two-dimensional image of the finger plate, extracts the feature data of the finger plate according to the three-dimensional image and two-dimensional image of the finger plate, and uses the feature data of the finger plate to predict the feature change trend of the finger plate.

[0053] The following describes the specific implementation manner of the intelligent inspection system for track beam finger plates based on computer vision proposed by the present invention in combination with specific embodiments.

[0054] In some embodiments, the vehicle-mounted device includes: 6 finger plate three-dimensional detection modules 1, 6 pulse control boxes, 1 track beam number identification module 2, 1 odometer encoder, and 1 set of image analysis and processing system. Among them, the image analysis and processing system includes 2 vehicle-mounted servers, 1 set of UPS, and 1 display terminal.

[0055] Among them, as Figure 3 shown, 6 finger plate three-dimensional detection modules 1 are installed inside the inspection vehicle's enclosure net, continuously collecting three-dimensional and two-dimensional image data of the finger plate during the operation of the inspection vehicle; as Figure 4 shown, the finger plate three-dimensional detection module is composed of multiple cameras, including a line laser 3D camera and a 2D camera, and the installation positions are as Figure 5 shown. The line laser 3D camera collects point cloud images, and the 2D camera collects color images. The three-dimensional detection module uploads the collected point cloud images (three-dimensional images) and color images (two-dimensional images) to the vehicle-mounted server of the image analysis and processing system. Since a line laser 3D camera is used, the imaging is less affected by the outside world and the imaging time is short.

[0056] The above odometer encoder is used to record the driving mileage of the inspection vehicle; the pulse control box is used for the trigger control of the finger plate three-dimensional detection module;

[0057] The above track beam number identification module 2 is installed inside the inspection vehicle's enclosure net, continuously collecting the track beam number images at specific positions of the track beam and outputting the character recognition results;

[0058] Specifically, the track beam number recognition module is installed on one side of the front end of the inspection vehicle at a fixed angle. It continuously collects images through a high-speed camera and recognizes the track beam number based on OCR character recognition technology. Through the recognized number information, the mileage encoder is calibrated to eliminate its cumulative error. When the number information is recognized, the pulse control box triggers the six finger plate three-dimensional detection modules to collect point cloud images, which are saved to the vehicle-mounted server together with the number. After shooting a certain distance, the finger plate three-dimensional detection module automatically shuts down until the track beam number is recognized again. Its installation position is as Figure 3 shown.

[0059] In the above embodiment, the process of the track beam number recognition module using OCR character recognition technology to recognize the track beam number mainly includes steps such as image preprocessing, text region detection, character recognition, and result processing. Among them, grid-based text region detection and character recognition are the key technologies of OCR recognition. In the image preprocessing process, when reading text, it is necessary to first determine the direction of the image. It is necessary to use the pre-trained VGG16 network for transfer learning to achieve the inclination correction of the text direction in the image. For the character recognition result, the key information in the recognition result is extracted by using the method of regular expression matching the keywords in the recognition result.

[0060] As Figure 6 shown, the process is as follows:

[0061] Image preprocessing: First, perform image enhancement and Gaussian filtering on the collected image to eliminate the influence of noise. Then, extract the edges of the text in the obtained image. Use the VGG16 network for transfer learning to judge the inclination direction of the text in the image before OCR character recognition, and rotate the image by the corresponding angle to prepare for text region detection and recognition;

[0062] Text region detection: By extracting the convolutional features of the text region, use YOLO V3 to perform grid object detection on the text region and accurately frame the edges where the text is located;

[0063] Character recognition: This model constructs a convolutional layer component using the convolutional layer and max pooling layer in the standard CNN model. After extracting the convolutional features of the text line, the feature vector sequence is used as the input of the recurrent layer, and the label distribution obtained from the previous layer is transformed through operations such as deduplication and integration by the CTC transcription technology;

[0064] Result processing: Based on regular expressions, screen and match the characters in the image data to obtain the final result.

[0065] In some embodiments, the image analysis and processing system is used to process and analyze the high-definition two-dimensional and three-dimensional image data of the finger plate, and automatically analyze and recognize the detection item point information such as bolt looseness, plate gap, height difference, and tooth gap spacing of the finger plate;

[0066] Among them, the image analysis and processing system includes 2 vehicle-mounted servers, 1 set of UPS, and 1 display terminal. The vehicle-mounted servers are used to control the system to collect and process all sensor signals, store data, and arrange the image analysis and processing system; the UPS is used to maintain the normal operation of the servers after a power outage; the display terminal is used to display the operating status of the system in real time, and can realize video playback and fault alarm. Specifically, in the client interface of the display terminal, video query and playback can be realized, and the detected faults can be alarmed and guidance can be provided to maintenance technicians for repair.

[0067] The working process of the image analysis and processing system is as Figure 7 shown. During the process of the camera collecting point cloud images, due to the influence of vehicle vibration, regular point cloud displacement will occur in the point cloud, and error correction is required. The system collects the corresponding point cloud images and color image data, and performs fingerboard detection on the color image data through deep learning based on the YOLO model, and the results are as Figure 8 shown.

[0068] Among them, YOLO solves the detection problem as a problem of bounding box and classification probability regression. Most detection algorithms rely on reusing classifiers for detection. YOLO removes the branch for generating candidate regions, directly performs candidate region classification and bounding box regression on the image, and reduces the multiple detections of the same target. All modules are completed in a convolutional neural network without branches, realizing an end-to-end framework. As Figure 9 shown. Therefore, the network becomes simple due to no branches, and the detection speed is significantly faster compared to the detection based on candidate regions. The specific approach of YOLO is for an image, place an S×S grid on the image. After the image is sent into the convolutional network to extract features, the fully connected layer outputs the target classification and bounding box. Each grid is responsible for detecting the targets falling within this grid, and a single grid predicts B bounding boxes, confidence levels, and target categories. Non-maximum suppression is used to remove the redundant bounding boxes to obtain the detection results.

[0069] Since the fingerboard model is known, after the fingerboard is located, the height differences between the locations of 12 circular screws on the fingerboard and the outer plane of the screw holes can be obtained by searching for preset fixed positions on the point cloud map, and whether the screws are loose can be judged by the height differences, as Figure 10 shown.

[0070] Through fingerboard positioning, searching the point cloud map along the CD direction can obtain the height difference between the two beams where the fingerboards are located, and judge whether there is a stepped phenomenon, as Figure 10 shown.

[0071] Through the positioning of the finger board, the point cloud map can be searched in the AB direction, and the finger spacing and the height difference between fingers can be obtained to determine whether the finger touching phenomenon occurs. The results are as Figure 10 shown.

[0072] That is, the present invention can cover the maintenance points of the finger board, such as the finger board state detection (broken finger, finger touching), the fastening bolt state detection (missing, loose), and the "step difference" of the finger board, etc.

[0073] In summary, the working process of the on-vehicle equipment is as Figure 11 shown: After the finger board inspection system runs, the track beam number recognition module acquires the track beam number image and recognizes the track beam number according to the track beam number image. When the track beam number recognition module recognizes the track beam number, the pulse control box triggers the finger board three-dimensional detection module to collect the three-dimensional image and two-dimensional image of the finger board. The image analysis and processing system processes and analyzes the three-dimensional image and two-dimensional image of the finger board in combination with the track beam number, and stores the processing results and gives a fault alarm.

[0074] In addition, the present invention can also judge the state of the current detection area according to each piece of collected image data, refer to the point cloud data in the area near the target detection point, and compare the point cloud data of the detection point, solving the false alarm and missed alarm phenomena caused by external environmental interference.

[0075] In some embodiments, the ground analysis center includes: 1 data storage and analysis server, 1 all-in-one machine, and client software.

[0076] After the inspection vehicle arrives at the station, the point cloud image and the color image are compressed and transmitted to the ground analysis center through the wireless network. The ground analysis center can process the historical data of the finger board collected based on the data preprocessing algorithm, extract the effective numerical statistical features in the finger board feature data by means of effective statistical analysis, and mine and analyze the finger board feature data based on various feature processing algorithms of machine learning to find the changing rules thereof, realizing the prediction of the changing trend of the finger board features, and providing a timely warning for the wear and change of the finger board. Among them, the data storage and analysis server is used for data storage and analysis; the all-in-one machine is used for the arrangement of the client software, the display of personnel operations and trend analysis results, etc.

[0077] Other installation accessories of the above intelligent inspection system include but are not limited to cables, cable troughs, equipment mounting seats, etc.

[0078] In summary, the intelligent inspection system can improve the line reliability. By researching the finger plate inspection system, collecting finger plate information, conducting a global analysis of the finger plates, alarming for faulty components, and promptly completing maintenance tasks, it effectively ensures that all finger plates on the in-service lines are in a normal and safe working state, guaranteeing the normal operation of trains. Secondly, it can improve the inspection efficiency. The existing daily inspection of finger plates is affected by insufficient night light, excessive working vehicle speed, and poor visual acuity of inspectors, resulting in low efficiency, long inspection time, and the inability to effectively detect subtle diseases of finger plates. This finger plate inspection system shortens the inspection time, expands the inspection scope, and can efficiently and reliably detect finger plate faults. It can also reduce the manual maintenance cost. Manual inspection has low work efficiency, high intensity, and does not meet the needs of mechanization and automation. Researching the intelligent inspection system for finger plates can greatly improve the detection efficiency. The original maintenance time for finger plates is reduced from the original 10 - 15 minutes to 3 - 5 minutes, improving the detection efficiency by 60%; and the inspection cycle is shortened from 30 days to 5 days, improving the finger plate detection efficiency by 80%. Deploying the finger plate inspection system to conduct systematic monitoring of finger plates, maintenance personnel only need to perform fixed-point maintenance and replacement according to the information provided by the computer, reducing the manual inspection frequency of finger plates and the number of maintenance personnel. After deploying the inspection system, the personnel configuration required for on-site finger plate detection can be optimized, reducing the labor cost by 2 million yuan per year.

[0079] Embodiment 2:

[0080] Combined with an intelligent inspection system for finger plates of track beams disclosed in Embodiment 1, this embodiment discloses a specific implementation example of an intelligent inspection method (hereinafter referred to as "the method") applied to the above intelligent inspection system for finger plates of track beams.

[0081] Referring to Figure 12 as shown, the method includes:

[0082] Step S1: Collect the track beam number image through the track beam number recognition module, and recognize the track beam number based on the OCR character recognition technology according to the track beam number image;

[0083] Step S2: When the track beam number is recognized, trigger the finger plate three-dimensional detection module through the pulse control box to collect the three-dimensional image and two-dimensional image of the finger plate;

[0084] Step S3: Send the three-dimensional image and two-dimensional image of the finger plate to the image analysis and processing system, and process and analyze the three-dimensional image and two-dimensional image of the finger plate by the image analysis and processing system in combination with the track beam number.

[0085] Among them, Step S3 specifically includes:

[0086] Step S31: Perform finger board detection on the two-dimensional image of the finger board through deep learning based on the YOLO model to obtain the position information of the finger board;

[0087] If the position information of the finger board is not detected, determine whether the finger board is within the actual mileage range according to the driving mileage recorded by the mileage encoder. If not, record the fault information.

[0088] Step S32: Detect the three-dimensional image of the finger board according to the position information to determine whether a fault has occurred.

[0089] Step S32 specifically includes detecting the three-dimensional image of the finger board according to the position information to obtain a detection result, and determining whether the detection result meets the detection index. If it meets, return to the track beam number identification step. If it does not meet, record the fault information.

[0090] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0091] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An intelligent inspection system for finger plates of track beams based on computer vision, characterized in that, It includes on-vehicle devices installed on the inspection vehicle, and the on-vehicle devices include: a track beam number recognition module, a finger plate three-dimensional detection module, a pulse control box, and an image analysis and processing system; The track beam number recognition module collects track beam number images as the inspection vehicle runs and recognizes the track beam number according to the track beam number images. When the track beam number recognition module recognizes the track beam number, the pulse control box triggers the finger plate three-dimensional detection module to collect the three-dimensional image and two-dimensional image of the finger plate, and the image analysis and processing system processes and analyzes the three-dimensional image and two-dimensional image of the finger plate in combination with the track beam number; Among them, the image analysis and processing system includes an on-vehicle server and a display terminal. The on-vehicle server performs finger plate detection on the two-dimensional image of the finger plate through deep learning based on the YOLO model to obtain the position information of the finger plate, and judges whether the finger plate fails according to the position information through the three-dimensional image of the finger plate; if a failure occurs, the display terminal displays the failure information and alarms; Among them, the on-vehicle device further includes a mileage encoder for recording the driving mileage of the inspection vehicle; if the on-vehicle server does not obtain the position information of the finger plate when performing finger plate detection on the two-dimensional image of the finger plate, it is judged whether the finger plate is within the actual mileage range through the mileage encoder.

2. The intelligent inspection system for the finger plate of the track beam according to claim 1, wherein The track beam number recognition module recognizes the track beam number through OCR character recognition technology and calibrates the mileage encoder according to the recognized track beam number.

3. The intelligent inspection system for the finger plate of the track beam according to claim 1, wherein It further includes a ground analysis center, which receives the compressed three-dimensional image and two-dimensional image of the finger plate, extracts the characteristic data of the finger plate according to the three-dimensional image and two-dimensional image of the finger plate, and predicts the characteristic change trend of the finger plate by using the characteristic data of the finger plate.

4. The intelligent inspection system for the finger plate of the track beam according to claim 1, characterized in that, The finger plate three-dimensional detection module includes a plurality of line laser 3D cameras and 2D cameras. After being triggered, the finger plate three-dimensional detection module collects three-dimensional images through the line laser 3D cameras and collects two-dimensional images through the 2D cameras. After collecting for a certain distance, the finger plate three-dimensional detection module automatically shuts down until the track beam number recognition module recognizes the track beam number again.

5. An intelligent inspection method for finger plates of track beams based on computer vision, characterized in that, Applied to the track beam finger plate intelligent inspection system according to any one of the above claims 1-4, the track beam finger plate intelligent inspection method includes: Track beam number recognition step: Collect track beam number images through the track beam number recognition module, and recognize the track beam number according to the track beam number images based on OCR character recognition technology; Finger plate image collection step: When the track beam number is recognized, trigger the finger plate three-dimensional detection module to collect the three-dimensional image and two-dimensional image of the finger plate through the pulse control box; Finger plate image processing step: Send the three-dimensional image and two-dimensional image of the finger plate to the image analysis and processing system, and process and analyze the three-dimensional image and two-dimensional image of the finger plate in combination with the track beam number through the image analysis and processing system; Among them, the finger plate image processing step includes: Two-dimensional image detection step: Detect the two-dimensional image of the finger board through deep learning based on the YOLO model to obtain the position information of the finger board; Three-dimensional image detection step: Detect the three-dimensional image of the finger board according to the position information to determine whether a failure has occurred; Among them, the two-dimensional image detection step further includes: If the position information of the finger board is not detected, it is judged whether the finger board is within the actual mileage range according to the driving mileage recorded by the mileage encoder. If not, the fault information is recorded.

6. The intelligent inspection method for the finger plate of the track beam according to claim 5, characterized in that, The three-dimensional image detection step includes: Detect the three-dimensional image of the finger board according to the position information to obtain a detection result, judge whether the detection result meets the detection index. If it meets, return to the track beam number recognition step. If it does not meet, record the fault information.

Images