Railway vehicle cloud edge cooperative detection method and system, and inspection equipment

Abstract

The invention relates to the technical field of rail traffic equipment maintenance and artificial intelligence image recognition, in particular to a rail vehicle cloud edge cooperative detection method and system and inspection equipment. The method comprises the following steps: acquiring image data of an area to be detected, synchronously recording attitude information of shooting equipment, carrying out image quality scoring on the image data, if the image quality score reaches a preset threshold value, calling a local target detection model to carry out defect identification on an image, and outputting an identification result containing a target type, a position and confidence; if the confidence coefficient of the recognition result is lower than a set threshold value, the image data and the posture information are uploaded to a cloud server, a cloud high-precision recognition model is called for secondary recognition, and a cloud recognition result is obtained. The real-time requirement is met through quick response of the local model, the cloud model rechecks a low-confidence result, efficiency and precision are both considered, and the contradiction that in the prior art, manual judgment standards are different, and speed and precision are difficult to consider is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rail transit equipment maintenance and artificial intelligence image recognition technology, in particular to a rail vehicle cloud-edge collaborative detection method and system and a patrol inspection device. BACKGROUND

[0002] With the rapid development of rail transit, the operation frequency and load of trains are increasing, which puts forward higher requirements for the maintenance of key components at the bottom of the vehicle. In order to ensure the safety of train operation, manual inspection is usually arranged at the bottom of the train during non-operation time at night. The maintenance personnel need to enter the working area at the bottom of the train, rely on portable lighting equipment and naked eye observation to visually inspect the rail components, bottom pipelines, fasteners and the like to determine whether there are cracks, rust, looseness, foreign matter attachment and other faults.

[0003] However, the existing manual inspection method mainly relies on the experience of the operator, and has the following disadvantages:

[0004] The space at the bottom of the train is narrow and the light is dim, the working environment is poor, and small defects are easy to be missed;

[0005] Relying on manual photographing, recording and rechecking, the working process is tedious and the efficiency is difficult to improve;

[0006] Different maintenance personnel have different judgment standards, the recognition accuracy is limited, and misjudgment or omission is easy to occur;

[0007] It is impossible to feedback the shooting angle, distance and the like according to the real-time shooting effect, resulting in poor image quality and affecting subsequent analysis.

[0008] In recent years, with the development of image recognition and artificial intelligence technology, some research has tried to apply target detection algorithms to industrial defect recognition to assist in judging surface cracks, rust and other problems. However, most of the current applications are concentrated in fixed scenes such as product quality inspection on the assembly line or static imaging systems. For complex, unstructured and space-limited rail transit inspection scenes, there is still a lack of related applications.

[0009] Therefore, it is urgent to design an intelligent detection scheme to assist maintenance personnel to efficiently complete the defect detection work at the bottom of the train, improve the fault detection rate, reduce manual misjudgment, improve the inspection efficiency and the safety guarantee level of rail transit. SUMMARY

[0010] To solve one of the above-mentioned defects of low recognition accuracy, low operation efficiency and lack of intelligent guidance in the existing manual inspection of the bottom of the rail transit train, the application provides a rail vehicle cloud-edge collaborative detection method, system and inspection equipment, which significantly simplifies the rail transit night manual inspection process, can quickly cover multiple key parts of the train bottom, avoids manual omission, improves the operation efficiency and regional coverage integrity; by accessing a salient object detection algorithm based on deep learning, intelligent recognition and classification of multiple typical fault targets such as cracks, rust, foreign matter and loose bolts are realized, human judgment errors and subjectivity are effectively reduced, and more stable and reliable defect detection is realized.

[0011] In a first aspect, the application provides a rail vehicle cloud-edge collaborative detection method:

[0012] The rail vehicle cloud-edge collaborative detection method comprises:

[0013] S1: acquiring image data of a to-be-detected area and synchronously recording attitude information of a shooting device, the attitude information comprising three-dimensional angle data and distance data between the device and the shooting target;

[0014] S2: performing image quality scoring on the image data, the scoring being performed based on a preset standard template library;

[0015] S3: if the image quality score reaches a preset threshold, calling a local target detection model to perform defect recognition on the image and outputting a recognition result containing target type, position and confidence;

[0016] S4: if the confidence of the recognition result is lower than a set threshold, uploading the image data and the attitude information to a cloud server and calling a cloud-end high-precision recognition model for secondary recognition to obtain a cloud-end recognition result.

[0017] By adopting the above technical solution, in view of the problems in the prior art that rail transit manual inspection relies on naked eye judgment, lacks standardized image quality control and intelligent recognition collaborative mechanism, and leads to uneven image quality and defect recognition relying on experience, the application filters qualified images through image quality scoring, solves the missing detection problem caused by image blur and angle deviation in the prior art, the local model quickly responds to meet the real-time requirement, the cloud-end model reviews the low-confidence result, and the efficiency and accuracy are taken into account, solving the contradiction between the inconsistent manual judgment standards and the difficulty in balancing speed and accuracy in the prior art.

[0018] Optionally, the creation process of the standard template library comprises:

[0019] The professional inspection personnel adopts multi-view to shoot the inspection point image at each inspection point, collects and stores image data, corresponding shooting posture information, target labeling information and image quality score, forms a standard template library containing multiple inspection reference groups, and the inspection reference group is used for subsequent image quality comparison and identification correction.

[0020] By adopting the technical scheme, the template library provides a unified reference for image quality score, region alignment and defect identification, avoids the problem of non-uniform standards caused by experience difference of the shooter, the inspection reference group containing posture data enables the subsequent shot image to be compared with the template in space posture, solves the problem that the same part image is difficult to reuse due to different shooting conditions in the prior art, and lays a foundation for historical trend analysis.

[0021] Optionally, the image quality score in step S2 includes a definition score, a target completeness score and a shooting angle score, and the total image quality score is calculated by the following formula:

[0022] Total score = ω1*definition score + ω2*target completeness score + ω3*shooting angle score

[0023] Wherein, ω1, ω2 and ω3 are weight coefficients, and ω1+ω2+ω3=1, and the definition score, the target completeness score and the shooting angle score are all normalized results.

[0024] By adopting the technical scheme, the subjective judgment is converted into a calculable quantitative index, the image quality evaluation standards of different scenes and different personnel are consistent, and the low-quality image is prevented from entering the identification process to affect the result accuracy.

[0025] Optionally, the definition score is calculated by Laplacian variance, and the formula is:

[0026] Definition score = Var(L(x, y))

[0027] Wherein, L(x, y) is a response map of the image processed by the Laplacian operator, and Var(·) is a function for calculating the variance of the response map.

[0028] The target completeness score is calculated by boundary coverage rate, and the formula is:

[0029] Target completeness score = Area(B∩B0) / Area(B0)

[0030] Wherein, B is a detection frame of the target region in the current image, B0 is a reference frame of the same part target in the standard template library, and Area(·) is a function for calculating the area of the region.

[0031] The shooting angle score is calculated by a Gaussian attenuation function, and the formula is:

[0032] Score of shooting angle = exp(-(Δθ) 2 / (2σ 2 ))

[0033] Wherein, Δθ is the deviation value of the current shooting angle and the reference angle in the standard template library, and σ is the angle deviation tolerance parameter.

[0034] Optionally, if the image quality score in step S2 is lower than the preset threshold, the system generates a shooting adjustment instruction, which includes:

[0035] The angle adjustment direction and distance adjustment value are displayed in a graphical manner on the touch display screen, and the corresponding prompt voice is output through the voice module to guide the user to adjust the shooting angle and the distance between the device and the target until the image quality score reaches the preset threshold.

[0036] Optionally, before the defect recognition in step S3 or S4, the key points of the target structure are detected by the region alignment technology, and the process includes:

[0037] The feature key points of the target structure are extracted from the current image, and the feature key points are the target edge contour, connection node or geometric feature point; the extracted feature key points are matched with the reference key points of the same part in the standard template library, and the spatial alignment of the target region is realized by adjusting the image angle and scale parameter to eliminate the interference of the shooting angle and distance difference on the recognition.

[0038] Optionally, after the key points of the target structure are detected by the region alignment technology before the defect recognition in step S3 or S4, it further includes:

[0039] A lightweight fully convolutional saliency detection network is called to process the image corresponding to the key points of the target structure, and a saliency map representing the attention degree of the region is output; the region where there is a significant difference between the visual features and the background in the image is located as a salient region according to the saliency map, and the salient region is sent to the defect detection model for defect recognition processing.

[0040] Optionally, the local target detection model in step S3 and the cloud high-precision recognition model in step S4 both contain a scene classification network and a specialized sub-model, and the process of scene classification implemented by the scene classification network includes:

[0041] The scene classification network extracts features from the input image corresponding to the salient region, and judges the scene to which the current region belongs based on the preset scene category label; according to the scene classification result, a specialized sub-model matched with the scene is called to perform defect recognition, and the specialized sub-model includes a bolt defect recognition sub-model, a crack recognition sub-model, a rust recognition sub-model and a foreign matter recognition sub-model.

[0042] By adopting the technical scheme,

[0043] In a second aspect, the application provides a track vehicle defect detection inspection device, which comprises various modules for executing the method in the first aspect or any possible implementation manner of the first aspect.

[0044] The track vehicle defect detection inspection device comprises:

[0045] An acquisition module is configured to acquire image data of a to-be-tested region and synchronously record attitude information of a shooting device, wherein the attitude information comprises three-dimensional angle data and distance data between the device and a shooting target.

[0046] A scoring module is configured to perform image quality scoring on the image data, wherein the scoring is performed based on a preset standard template library.

[0047] A defect detection module is configured to, if the image quality score reaches a preset threshold, call a local target detection model to perform defect identification on the image and output an identification result comprising a target type, a position and a confidence level.

[0048] A communication module is configured to, if the confidence level of the identification result is lower than a set threshold, upload the image data and the attitude information to a cloud server, call a cloud high-precision identification model to perform secondary identification, and acquire a cloud identification result.

[0049] In a third aspect, the application provides a track vehicle cloud-edge collaborative detection system, which comprises the inspection device and the cloud server.

[0050] In a fourth aspect, the application provides an inspection device, which comprises a processor, a memory and a communication bus, wherein the communication bus is configured to realize communication connection between the processor and the memory, and the processor is configured to execute a computer program stored in the memory to realize the track vehicle cloud-edge collaborative detection method.

[0051] In a fifth aspect, the application further provides a computer readable storage medium, which stores a computer program; the computer program can be executed by a processor to realize the track vehicle cloud-edge collaborative detection method.

[0052] In a sixth aspect, the application further provides a computer program product, which comprises a computer program; the computer program can be executed by a processor to realize the track vehicle cloud-edge collaborative detection method.

[0053] The application at least has the following beneficial technical effects:

[0054] 1. Significantly simplifies the nighttime manual inspection process for rail transit, enabling rapid coverage of multiple key parts on the bottom of trains, avoiding human omissions, and improving operational efficiency and the completeness of area coverage; by integrating a salient target detection algorithm based on deep learning, it achieves intelligent identification and classification of various typical fault targets such as cracks, corrosion, foreign objects, and loose bolts, effectively reducing human judgment errors and subjectivity, and achieving more stable and reliable defect detection.

[0055] 2. Through real-time image quality assessment and posture feedback mechanisms, the system guides inspection personnel to adjust shooting angles, distances, and lighting in the form of graphics and voice, reducing the number of repeated shots, quickly obtaining qualified images, and shortening the inspection time for a single area.

[0056] 3. Supports both online collaboration and offline caching modes. It can still collect data normally in environments with unstable networks, such as tunnels and garages, and automatically synchronize after the network is restored, ensuring that the inspection process is not interrupted.

[0057] 4. A two-level detection strategy is adopted, consisting of a local lightweight model and a cloud-based high-precision model: the local model quickly screens for obvious defects, while the cloud model verifies low-confidence results. Combined with a hybrid expert mechanism (MoE), specialized sub-models (such as bolt and crack expert models) are invoked to effectively identify minor defects (such as micro-cracks and slight loosening of bolts), reducing the risk of human error.

[0058] 5. Based on a standard template library, implement quantitative evaluation of image quality (including sharpness, completeness, and angle scoring). Eliminate the influence of shooting angle and distance differences through region alignment technology, provide standardized image input for defect identification, and enhance the robustness of model recognition.

[0059] 6. Introduce a salient target detection network to prioritize processing regions with significant visual features in the image (potential defect areas), reduce interference from complex backgrounds, and improve recognition accuracy in scenarios with multiple defects. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of a cloud-edge collaborative detection system for rail vehicles provided in an embodiment of the present invention;

[0061] Figure 2 A structural diagram of a handheld inspection device provided in an embodiment of the present invention;

[0062] Figure 3 A hardware block diagram of a cloud-edge collaborative detection system structure for rail vehicles provided in an embodiment of the present invention;

[0063] Figure 4 A flowchart of a cloud-edge collaborative detection method for rail vehicles provided in an embodiment of the present invention;

[0064] Figure 5A flowchart of another cloud-edge collaborative detection method for rail vehicles provided in an embodiment of the present invention;

[0065] Figure 6 A flowchart of another cloud-edge collaborative detection method for rail vehicles provided in an embodiment of the present invention;

[0066] Figure 7 This is a structural diagram of the inspection equipment module provided in an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items. The term “exemplary” means “serving as an example, embodiment, or illustration,” and any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. The terms “first” and “second” are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, “a plurality” means two or more.

[0069] Please see Figure 1 This invention provides a cloud-edge collaborative inspection system for rail vehicles, integrating high-definition image acquisition, intelligent lighting, human-computer interaction, posture perception, edge computing, and edge-cloud collaborative recognition functions. It is particularly suitable for efficient nighttime inspections of complex structures under trains. The system consists of a handheld inspection device and a cloud server, which work together to achieve accurate fault detection and feedback.

[0070] Specifically, the handheld inspection equipment includes, but is not limited to, the following components: an image acquisition module, a dimmable illumination module, an attitude sensing module, a ranging module, a local processing module, a display and voice interaction module, a communication module, a scoring module, a defect detection module, and a protective housing structure. The overall structure of the equipment meets IP65 or higher industrial protection standards, possessing water and dust resistance, shock and drop resistance, and is adaptable to the complex operation and maintenance environment of rail transit, including high humidity, high dust levels, and extreme temperatures.

[0071] Specifically, the cloud server platform mainly deploys the following modules: defect detection, feedback guidance, historical comparison, model management and update, and data encryption and archiving. This platform supports concurrent access from multiple devices, possesses distributed task scheduling and edge node synchronization capabilities, and allows for remote configuration and unified management of defect detection tasks through a dedicated cloud platform.

[0072] Please see Figure 2 and Figure 3 The handheld inspection device described in this invention has an overall rectangular strip structure, measuring approximately 200mm × 100mm × 25mm, making it suitable for one-handed operation and easy to use in the confined space under trains during maintenance. The device uses a built-in embedded local processor as its core, employing a modular design with functional components arranged around the processor, and achieving data interaction and unified control through standardized hardware interfaces.

[0073] Specifically, the device features a 5-inch high-brightness IPS touchscreen display 301 on the front, used to display images, target recognition results, and defect annotation information in real time, while also providing a user interface. Below the screen is a function button area 302, including a power button, a camera button, and a brightness adjustment button, facilitating physical operation by users wearing gloves or in bright light.

[0074] Specifically, the front-end center of the equipment is equipped with a high-sensitivity, high-definition industrial camera 303, supporting resolutions above 2K, frame rates above 30FPS, and global shutter speeds, and possessing high signal-to-noise ratio performance in low-light conditions. The camera module is connected to the local processor via the Mobile Industrial Processor Interface (MIPI) to acquire inspection image data and participate in subsequent image quality analysis and defect detection.

[0075] Specifically, multiple high-brightness LED illumination units 304 are evenly distributed around the camera, supporting PWM dimming control and equipped with a diffuser to improve illumination uniformity and soft light effect, ensuring image clarity in dark or complex background environments. The illumination module is directly controlled by the local processor and can automatically adjust its brightness according to ambient light conditions.

[0076] Specifically, to achieve spatial geometry perception and image capture distance control, a laser ranging module 305 is integrated around the camera. This module, together with a nine-axis IMU (Inertial Measurement Unit) attitude sensor, constitutes the attitude perception and target localization subsystem. The IMU sensor includes an accelerometer, gyroscope, and magnetometer, which can measure the device's orientation and angular velocity changes in real time. The laser ranging module directly provides precise distance information from the device to the target surface. By fusing the attitude angle provided by the IMU, the absolute distance provided by the laser ranging module, and the visual information provided by the camera through a local processor, the system can achieve high-precision vision-inertial-distance fusion positioning. This allows for a more accurate reconstruction of the relative spatial geometry between the device and the target, significantly improving the accuracy of assessments of detection angle, capture distance, and image quality.

[0077] Specifically, the inspection equipment features a voice interaction module 306 on its side, including a speaker and audio chip, for real-time broadcasting of recognition results, shooting angle adjustment suggestions, and system prompts, improving human-computer interaction efficiency. Below it is a multi-functional interface area 307, including a USB-C port (for device charging and data communication), a MicroSD expansion slot (for image and log storage expansion), etc. All interfaces are sealed with silicone plugs, meeting IP65 and above protection standards, and are dustproof and waterproof.

[0078] Specifically, the overall shell 308 of the inspection equipment is injection molded from industrial-grade polycarbonate and rubber composite materials, possessing high strength, anti-slip, and drop-resistant properties. A matte sandblasting process further enhances scratch resistance and handheld comfort. The rear shell of the equipment houses a high-capacity lithium battery module 309, supporting 4-6 hours of continuous inspection operations, and is equipped with a fast-charging management circuit to ensure long-term, high-intensity operation.

[0079] Specifically, the internal communication module of the inspection equipment supports Wi-Fi, Bluetooth, and 5G connectivity, enabling real-time data transmission with the cloud server. The local processor has task scheduling capabilities, automatically deciding whether to upload images to the cloud based on image quality scores and recognition confidence levels, allowing for further analysis and correction by the cloud's high-precision recognition model. The cloud server platform is responsible for receiving, analyzing, archiving, and feeding back the recognition results, forming an "edge-cloud" collaborative structure. This inspection equipment constitutes an intelligent hardware system with the local processor at its core, integrating multiple modules such as image acquisition, lighting, ranging, attitude control, interaction, and communication, enabling it to complete high-quality defect inspection tasks in complex environments.

[0080] Please see Figure 4This invention also provides a cloud-edge collaborative detection method for rail vehicles, applicable to refined inspection of train undercarriages or track foundation components. This method combines a two-layer MoE detection strategy of local intelligent judgment by the inspection equipment and deep cloud-based recognition. The main steps include:

[0081] S1: During the initial deployment phase of the system, professional personnel will sequentially capture images of each inspection point from multiple perspectives to establish a "standard template library." The templates include images, attitude sensor data, target annotations, and image quality scores, forming an "inspection reference group" that can be used for subsequent comparisons.

[0082] S2: During the maintenance phase, maintenance personnel turn on the handheld inspection device. The system completes self-tests and initializes various sensors, including IMU attitude module calibration, ranging module calibration, camera parameter loading, ambient light adaptation, network channel testing, etc., and then enters standby mode. During initialization, the current connection status and preparation progress can be displayed on the touchscreen.

[0083] S3: After the inspection equipment establishes a connection with the cloud server platform, it automatically synchronizes inspection task information, including vehicle number, inspection location, and inspection area number. This information is preset and issued by the background dispatch system or operation platform and displayed on the main interface of the inspection equipment.

[0084] S4: Maintenance personnel enter the bottom of the train or the designated work area, and according to the screen prompts or voice instructions, align the equipment with the structural parts to be inspected (such as couplers, air pipes, suspension springs, bolts, etc.) and begin the inspection.

[0085] S5: When the camera starts shooting, the device automatically turns on the LED lighting system to compensate for the brightness of the target area and begin image acquisition. The image data is transmitted in real time to the local processor for preprocessing via the image acquisition module (e.g., an industrial camera).

[0086] S6: After the acquired image data undergoes image enhancement processing such as edge enhancement and noise reduction filtering, the image is stored locally in JPEG / PNG compressed format.

[0087] S7: The local handheld inspection device calls the quality scoring module based on the standard template library and generative AI enhancement to determine whether the image meets the recognition conditions. The evaluation dimensions include sharpness, target completeness, shooting angle and brightness saturation, etc., and calculates the total score.

[0088] S8: If the score is lower than the preset threshold (e.g., blurry, incomplete area cropping, skewed angle, etc.), the inspection equipment will issue voice broadcasts or screen prompts such as "Please turn 10° to the left", "Get closer to the target", "Insufficient light, please increase brightness" to improve the collection quality.

[0089] S9: Once the score reaches a preset threshold, the image will undergo rapid defect classification and detection on the local processor. If the confidence level of the detection result reaches the set threshold, it indicates that the recognition result of the local inspection equipment has met the reliability requirements. The system will directly feed back the result (including target type, location, confidence level, etc.) to the inspection personnel, prompting them to perform confirmation and verification operations. After the operation information is uploaded to the cloud server in real time, all images and recognition results will be archived with timestamps, location information numbers, and operator IDs, which can be used for subsequent historical image comparison and analysis. If the confidence level of the detection result is lower than the set threshold, the image data and posture information need to be uploaded to the cloud server and input into the cloud high-precision recognition model for secondary detection. The identified targets include typical faults such as cracks, corrosion, foreign objects, and loose bolts. The system outputs the target type (defect category label), location (boundary box coordinates), and confidence level of the detection result. The secondary detection result supports visual overlay and is returned to the inspection equipment for display.

[0090] S10: If the confidence level in the secondary inspection results is lower than the set threshold (uncertain defect type), the inspection equipment will prompt "Please move closer and retake the photo" or "Suspected defect, please confirm", guiding the user to retake the target area from different angles to avoid omissions, or to allow the user to manually confirm the defect type.

[0091] S11: All captured images and inspection results will be cached locally on the device, with timestamps, inspection numbers, and geographic location tags. Images of critical defects are stored encrypted and added to the maintenance record history database.

[0092] S12: The cloud server performs position alignment and defect trend analysis on the current inspection results and historical images. If the corrosion degree of the same location increases or the crack expands, the system automatically marks "deterioration risk" and generates a trend chart in the cloud report.

[0093] S13: Maintenance personnel can manually select options such as "confirm defect" or "no obvious abnormality" on the inspection equipment screen. This feedback data is used to train and improve the accuracy of subsequent models.

[0094] S14: After the task is completed, the system will automatically organize all image numbers, detection results, coordinate information, anomaly types, number of retakes, operator IDs and other data, and generate a structured record file to be uploaded to the cloud.

[0095] S15: The cloud server converts the inspection results into standard format reports (such as PDF / HTML), summarizing information such as target number, number of faults, image examples, inspection time, and recommended re-inspection items. The reports are synchronized to the operation and maintenance platform for subsequent maintenance personnel to refer to.

[0096] Specifically, the inspection equipment supports two working modes: online collaborative mode and offline acquisition mode. In online collaborative mode, images are uploaded to a cloud server in real time, where the cloud server performs secondary defect identification and issues feedback commands; this mode is suitable for areas with high network coverage. In offline acquisition mode, image and sensor data are temporarily stored locally and uploaded to the cloud server in batches after the network is restored; this mode is suitable for environments with unstable signals, such as tunnels and parking garages. The system has fault identification capabilities in both modes, with only slight differences in identification accuracy and real-time feedback.

[0097] Specifically, this invention designs a real-time, multi-dimensional image quality assessment framework that transforms qualitative image quality into a calculable and comparable comprehensive score Q. score This score is the core basis for triggering the user operation guidance mechanism, ensuring the data input quality of the subsequent defect identification model. Image sharpness is mainly reflected in the sharpness of edges and details. This embodiment of the invention uses the Laplacian variance method to quantify this feature. The principle is that the Laplacian operator is sensitive to high-frequency information in an image; a sharp image has sharper edges, and after processing by the Laplacian operator, it will produce a larger response value, thus its response map has a larger variance. First, the input image I is converted to a grayscale image, and then a 3×3 Laplacian convolution kernel K is used. L Perform a convolution operation on it to obtain the Laplacian response map L(u,v):

[0098]

[0099] Where L(u,v) is the processed Laplacian response map, I is the input image, and (u,v) are the pixel coordinates. In this embodiment, the Laplacian kernel K... L It can be set as:

[0100]

[0101] Subsequently, the variance of the response map L is calculated, which is the sharpness score S. c :

[0102]

[0103] Where W and H are the width and height of the image, μ L S is the mean of the response plot L. c The value of S directly reflects the sharpness of the image; a low S score indicates a low sharpness. c This indicates that the image may be out of focus or have motion blur, and the system can prompt the user to make adjustments accordingly.

[0104] To ensure that the component under inspection is completely captured in the image, this embodiment of the invention introduces a target integrity score S. oThe scoring relies on a pre-set "standard template library" of the system. This library stores reference images of each key component under standard shooting conditions and their ideal bounding boxes (B). ref When a new image is acquired, a lightweight local object detection model quickly locates the bounding box B of the target in the current image. det Goal completeness score S o Defined as the ratio of the intersection area of ​​the detection box and the reference box to the area of ​​the reference box:

[0105]

[0106] If the shooting is incomplete, resulting in B det It only covered part of the target, which is related to B. ref The intersection will become smaller, thus causing S o Lowering the shooting angle is crucial for identifying specific defects. This invention utilizes the IMU sensor built into the inspection equipment to obtain the current shooting posture of the equipment and compares it with the optimal shooting posture preset for that component in the "standard template library" (as a reference posture). The posture is represented in three-dimensional space using a unit quaternion to avoid gimbal lock problems. Let the current equipment posture be q. curr The reference pose obtained from the template library is q. ref The angular deviation θ between the two err The relative rotation quaternion q can be calculated. rel To obtain:

[0107]

[0108] in, It is q ref The conjugate quaternion, This represents quaternion multiplication. Angle error θ err That is, the angle corresponding to this relative rotation:

[0109] θ err =2·arccos(q) rel ,ω)

[0110] Where q rel ω is q rel The scalar part. To map the angle error to a standardized score, this embodiment uses a Gaussian decay function to define the angle compliance score S. a :

[0111]

[0112] Where, σ a It is an adjustable parameter used to control the tolerance for angular deviation. When the deviation is zero, Sa =1, the larger the deviation, the more S a The closer the score is to 0, the better; finally, the three independent scores are combined into a comprehensive image quality score Q by weighted summation. score :

[0113] Q score =ω c ·S′ c +ω o ·S o +ω a ·S a

[0114] Where, ω c ,ω o ,ω a These are weighting coefficients that satisfy ω c +ω o +ω a =1. S c ′ is the original sharpness divided into equal parts S c The result after normalization. These weights are not fixed but rather system parameters that can be flexibly configured based on the actual situation. For example, when detecting fine cracks, the sharpness weight ω can be increased. c When confirming the existence of a component, the integrity weight ω can be increased. o This configurability demonstrates the adaptability and flexibility of the inspection equipment of this invention. Based on the comprehensive score, the system's decision function is defined as a simple threshold judgment. When the comprehensive image quality score is higher than the preset quality threshold, the system accepts the image and performs analysis; when the comprehensive image quality score is lower than the preset threshold, a guidance instruction is generated to retake the image. This closed-loop feedback mechanism, "from sensor data to quantitative scoring, and then to user behavior guidance," is key to ensuring the robustness and high efficiency of the entire system.

[0115] Specifically, the guidance commands are implemented based on a state estimation algorithm using an extended Kalman filter. This algorithm combines the high-frequency dynamic response of the IMU with the global accuracy of vision, providing the device with accurate real-time pose estimation. To model the device's motion, the system's state components x are first defined. k At time k, this vector contains the device's three-dimensional position in the world coordinate system. Three-dimensional velocity and unit quaternions representing direction

[0116]

[0117] The process model describes how the system state evolves over time, driven by measurements from the IMU. The acceleration a is measured by the IMU during the time interval Δt from time k-1 to k. m and angular velocity ω m It can predict the current state. This is a state transition function x. k =f(x) k＝1 ,u k ), where the control input u k =T. The specific prediction equation is as follows:

[0118]

[0119] v k =v k-1 +(R(q k-1 (a) m -b a )-g)Δt

[0120]

[0121] Where R(q) k-1 ) is a rotation matrix derived from the attitude quaternion of the previous moment, where g is the gravity vector, and b is the rotation matrix derived from the attitude quaternion of the previous moment. a and b g These are the biases of the accelerometer and gyroscope, respectively, and q(·) is a function that converts the rotation vector into a quaternion. The prediction step accumulates errors due to IMU noise, and the detection model uses camera observations to correct for these errors. When the camera observes a known 3D feature point P in the environment... j It is represented as two-dimensional pixels z in the image. j This constitutes a measurement. The measurement function z j =h(x k ,P j This projection process is described as follows:

[0122]

[0123] Where K is the camera's intrinsic parameter matrix. The update phase of the Extended Kalman Filter (EKF) calculates the measured value z. j and the measurement h(x) predicted based on the current state k ,P j The residual between the two functions is used, along with the Jacobian matrix of the measurement function h, to update the state vector x. k And reduce its uncertainty. To obtain accurate real-time pose estimation x k Then, the system can calculate the target pose x. ref The required adjustment amount is then converted into a user instruction, the target pose x. ref=T is obtained from the standard template library. Position error e p and attitude error e q The calculations are as follows:

[0124] e p =p ref -p k

[0125]

[0126] The system is based on the error vector e k =The magnitude and direction of T are determined by a control law C = g(e k This generates discrete user commands, enabling adjustments to shooting distance, angle, and direction. A complete mathematical chain from perception to decision-making to action is constructed, which is the core technology for realizing the human-machine collaboration and precise shooting functions of this invention.

[0127] Please see Figure 5 The defect detection system designed in this invention is based on the core idea of ​​divide and conquer and specialization. Through a two-stage pipeline consisting of "salience guidance" and "MoE", it achieves a balance between detection efficiency, accuracy and breadth under the condition of limited computing resources at the edge.

[0128] Specifically, based on saliency detection, regions of interest (ROIs) are quickly identified. Traditional defect detection methods typically require scanning the entire high-resolution image using a sliding window or directly inputting the entire image into a massive detection network, which is extremely time-consuming and energy-intensive on edge devices. This invention employs a more efficient strategy: first, a lightweight, fully convolutional visual saliency detection network processes the input image. This network's task is not to identify specific defects, but rather to mimic the attention mechanism of the human visual system, quickly identifying "abnormal" regions in the image that differ significantly from the surrounding background in color, texture, or structure. Defects, whether cracks, rust spots, or foreign objects, generally meet this "abnormal" definition in terms of visual characteristics. Therefore, the output of the visual saliency detection network is a "saliency map," where the highlighted areas are the regions of interest that the system deems most worthy of attention. In this way, the system focuses subsequent detailed analysis tasks from the entire image to a few key ROIs, reducing computational load by orders of magnitude and significantly improving processing speed.

[0129] Specifically, based on the refined identification using a hybrid expert model, after obtaining the ROIs, the system activates its core identification module, "a specially designed hybrid expert model." MoE does not refer to a single network structure, but rather an ensemble learning architecture. In this invention, it consists of a "gated network" and a set of "expert networks." The gated network here is functionally a component classification network. It receives an ROI extracted by a visual saliency detection network as input, and its task is to identify the main component types contained in this ROI, such as "this is a bolt assembly," "this is a weld," or "this is a spring steel plate." This classification result is the "routing decision" in the MoE architecture. The expert networks are a series of small, highly specialized deep learning models. Each "expert" is specifically trained to detect a specific type of defect on a particular component. For example: Bolt Expert (E... bolt This is specifically designed for detecting loose, missing, or broken bolts, as well as displacement of anti-loosening marking lines; Weld Expert (E weld ) is specifically designed for detecting surface cracks, porosity, and incomplete penetration in weld areas; Surface Expert (E surface It is specifically designed for detecting defects such as rust and paint peeling on large metal surfaces; Foreign Object Expert (E foreign It is specifically designed to identify foreign objects that are attached to the normal structure and do not belong to the vehicle itself.

[0130] Specifically, the entire MoE workflow can be described as a sparsely gated computation process. For a given ROI input x, the gating network G(x) calculates the probability that the ROI belongs to each component category. The system selects the category k with the highest probability, and then only activates and runs the corresponding expert network E. k The output y is the final detection result. This process can be formally represented as:

[0131]

[0132] y = E k (x)

[0133] This rigid gating selection mechanism ensures that for each ROI, only one most relevant small expert model is invoked, rather than running all sub-models as in traditional ensemble models, which meets the efficiency requirements of edge computing.

[0134] Specifically, the lightweight defect detection model deployed locally on the inspection equipment aims to achieve high-speed, efficient, and high-accuracy real-time preliminary screening with limited computing resources (such as embedded GPUs or NPUs). To achieve this goal, the model preferably employs a lightweight convolutional neural network (CNN) architecture designed specifically for mobile or edge computing scenarios. In a preferred embodiment, this local model can be built based on the MobileNetV3 architecture. The "lightweight" nature of the MobileNetV3 architecture stems from its unique combination of technologies: (1) Depthwise separable convolution: This structure decomposes the standard convolution into two steps: "depthwise convolution" and "pointwise convolution". Compared with traditional convolution, it can reduce the amount of computation and parameters to about one-tenth while maintaining the ability to extract similar features. This is the key to achieving model lightweighting; (2) Introduction of Squeeze-and-Excitation (SE) attention module: Lightweight SE modules are introduced at key nodes of the network, enabling them to adaptively learn the importance of different feature channels and strengthen the weight of key defect features, thereby improving detection accuracy with minimal computational overhead; (3) Adoption of advanced activation function and network structure search (NAS): The computationally more efficient h-swish activation function is used to replace the traditional ReLU, and the network structure search technology is used to automatically optimize the configuration of each layer of the network to achieve the best balance between speed and accuracy. Through the above technology, the local model of the present invention has the following quantifiable technical features: the number of model parameters can be controlled between 2 million and 5 million, the size of the compiled model file is about 5MB to 15MB, and the computational load (FLOPs) of a single inference on a typical edge computing chip is less than 1G, thereby ensuring the millisecond-level real-time processing capability of orbital images.

[0135] Specifically, the training dataset for the local model comes from a high-quality, standardized defect sample library that has been manually selected and labeled by experts. The training goal is to maximize accuracy, ensuring that the model gives a very high confidence score for typical defects it can identify. For anomalies outside the model's knowledge base, or those with ambiguous or atypical shapes, the model tends to output a lower confidence score, thus triggering a secondary detection process in the cloud. This is a "specialist education" training paradigm, designed to quickly and accurately handle the most frequent common defects.

[0136] Specifically, the high-precision defect detection model deployed on a cloud server aims to utilize the near-infinite computing resources of the cloud to make a refined final judgment on "suspected defects" or "low-confidence defects" reported by the local model with high recall and high analytical granularity. To achieve this goal, the model preferably adopts a new network architecture that is different from the local CNN paradigm and has stronger global modeling capabilities. In a preferred embodiment, the cloud model can be built based on the "Vision Transformer (ViT)" architecture. The "high precision" of the ViT architecture comes from its revolutionary working mechanism, which is fundamentally different from the local receptive field of CNN: (1) Self-Attention Mechanism: ViT first divides the input defect image into a series of fixed-size image patches, and then linearly embeds these image patches into the Transformer encoder. The multi-head self-attention mechanism in the encoder can calculate the relationship between any two image patches in the image, no matter how far apart they are in space; (2) Global context awareness: This global modeling capability gives ViT an unparalleled advantage in dealing with specific types of defects. For example, for thin, discontinuous cracks, or anomalies (such as foreign objects) that need to be judged in conjunction with the surrounding environment structure, ViT can accurately identify them by capturing global dependencies, which is difficult for traditional CNNs to achieve due to the limitations of local convolution kernel size. The cloud-based ViT model is a computationally intensive large model, and its technical characteristics are in stark contrast to the local model: the number of model parameters can reach 80 million to 300 million, and the computational cost of a single inference (FLOPs) exceeds 15G. This highly complex model must rely on the powerful computing power of the cloud server cluster to run and cannot be deployed locally on the inspection equipment.

[0137] Specifically, the training dataset for the cloud-based model is a dynamically growing aggregated large dataset that gathers data uploaded by all inspection equipment. It not only contains a massive number of standard defect samples, but more importantly, it continuously includes all challenging samples deemed "low confidence" and novel defect samples that were previously "unidentified" by the local model. The cloud-based model employs a continuous learning or incremental learning training strategy, constantly learning and iterating from this new and challenging data to achieve self-evolution. Its training objective is to maximize recall, ensuring that no potential safety hazard is missed. This is a "lifelong learning" paradigm of a "generalist expert," crucial for ensuring the upper limit of the entire system's detection capabilities.

[0138] In this embodiment, cracks in images typically appear as thin, directional dark or light lines. For effective detection, the expert network employs a multi-scale feature fusion architecture, such as a feature pyramid network. This structure can simultaneously capture deep semantic information and shallow high-resolution detail information of the image. The network is trained as a semantic segmentation model, and its output is not a simple bounding box, but a binary mask of the same size as the input ROI. Pixels with a value of 1 precisely identify the location, direction, and width of the crack. To address the class imbalance problem, the crack expert network is trained using a composite loss function L:

[0139] L=αL DICE +βL FOCAL

[0140] Among them, L DICE The Dice loss directly optimizes the overlap between the predicted mask and the ground truth mask. For the predicted result P and the ground truth label G, it is defined as... L FOCAL Focal loss is an improvement on standard cross-entropy loss. By reducing the weight of correctly classified simple samples, it allows the model to focus more on learning difficult-to-classify samples. By combining these two loss functions, the model can accurately delineate the outline of cracks at the pixel level, providing a foundation for subsequent quantitative analysis.

[0141] Specifically, for detecting rust on the undercarriage, the main characteristics of rust are color and texture. The expert network's processing flow consists of two steps: First, a color space transformation is performed, converting the input RGB image to the HSV or Lab color space, which are less sensitive to changes in lighting. In these spaces, the hue and saturation features of the rusted area are more prominent and stable. Second, the network extracts color and texture features from the image in parallel. Finally, the color and texture features are concatenated and fed into a classifier to determine the presence of rust. The segmentation network can then output the precise extent of the rusted area for calculating quantitative indicators such as the percentage of rust area. The joint extraction of multi-level texture and color features helps enhance the model's sensitivity to rusted areas.

[0142] Specifically, for foreign object detection under vehicles, the types and shapes of foreign objects are numerous and varied, making it difficult to exhaustively train using traditional supervised methods. Therefore, this invention adopts an unsupervised anomaly detection approach. The foreign object expert network only learns from images of "normal" parts during the training phase. It is trained as a variant of an autoencoder or generative adversarial network, with the goal of perfectly reconstructing the input "normal" image. During detection, when a ROIR containing a foreign object is detected... new Inputting the foreign object into the network, since the network has never seen this foreign object before, its reconstructed image AE(R) newThis will result in a significant reconstruction error E between the original input and the region where the foreign object is located. recon :

[0143]

[0144] The system can accurately locate foreign objects by detecting these high-error areas.

[0145] Specifically, regarding the detection of loose bolts, the visual characteristics of bolt loosening are often very subtle, such as a slight misalignment of the anti-loosening marking line or a tiny gap between the nut and the washer. To achieve reliable detection, bolt experts employ a sophisticated cascaded detection strategy. This involves using a key-point detection network f... kp Precisely locate all key elements K = {k1,…,k} in the bolt assembly. n For example, the center of the bolt head, several corner points of the nut, and the two endpoints of the anti-loosening mark. After obtaining the precise coordinates of these key points, the system enters a rule-based and machine learning-based analysis module. This module determines the bolt condition by calculating the geometric relationships between these points. For example, by calculating the coordinates of the two endpoints k of the anti-loosening mark. start ,k end The angle θ between the lines line and the reference angle θ in the standard template ref Comparison:

[0146] θ line =atan2(k end,y -k start,y ,k end,x -k start,x )

[0147] When the angle deviation is |θ line -θ ref | Exceeding the preset rotation threshold T rot When this happens, it can be determined that the bolt has become loose due to rotation.

[0148] Specifically, the models deployed on cloud servers prioritize detection accuracy, allowing for a certain degree of latency, and are mainly used for keyframe review, historical image comparison, and model retraining feedback.

[0149] Specifically, the model deployed on the cloud server is trained using a large number of on-site images of rail transit collected by this system. These images include those of cracks, corrosion, and loose bolts on the bottom components of trains under different working conditions. The training set includes automatically labeled and manually verified layers, exhibiting high robustness and industrial applicability. The training data for the local lightweight target detection model mainly consists of typical defect samples from the "standard template library." This includes: multi-view standard images (containing corresponding attitude sensor data, target labels, and quality scores) taken by professional inspectors at various inspection points during the initial deployment phase of the system; and structured samples that have been selected and represent common defects (such as obvious cracks, large areas of corrosion, and significant bolt loss). The data volume is relatively small and focuses on easily identifiable typical scenarios to adapt to the limited computing resources and real-time requirements of local equipment. Local model training data serves for rapid initial screening, focusing on optimizing the model's speed in recognizing high-confidence defects and adapting to the low-computing-power environment of edge devices. Cloud-based model training data, on the other hand, serves for high-precision verification, focusing on optimizing the model's accuracy in recognizing low-confidence defects in complex scenarios, relying on the ample computing power of the cloud to support deep feature learning. Local model training data is updated infrequently, primarily relying on a standard template library for initialization, and subsequently optimized indirectly through lightweight model update packages distributed from the cloud (rather than directly using massive amounts of new data). Cloud-based model training data is updated in real-time (continuously receiving new samples uploaded locally), supporting incremental training and continuously optimizing the ability to recognize new scenarios and defects, making model evolution more flexible.

[0150] Specifically, in any given image, the system supports the simultaneous detection of multiple defect targets of different categories. The output structure includes: target category, such as "crack", "rust", "foreign object", "missing bolt", etc.; bounding box position: [x min ,y min ,x max ,y max ]; Confidence score: between 0.0 and 1.0; Defect number and priority label: such as “CRK_01_P1” (crack No. 1, priority P1); Image number and timestamp: for easy result archiving and tracking.

[0151] Specifically, the system supports the following model management functions. The system treats all operator interactions on-site as valuable, high-quality labeled data. This data is automatically collected and fed into a dedicated "training dataset." The cloud-based model training platform is periodically launched, using this newly added data for incremental training or fine-tuning of the model. After thorough validation, the optimized new model is packaged and seamlessly pushed to all online handheld devices via over-the-air (OTA) updates. This process forms a complete, automated, continuous learning loop, enabling the system's detection capabilities to continuously evolve over time.

[0152] To better illustrate the application effects of this invention, the following describes the operation process of the handheld device and the practical application of the end-to-cloud collaboration mechanism in conjunction with typical inspection tasks, such as... Figure 6 The scenario shown depicts a nighttime inspection of the air springs and fastening bolts at the bottom of a train within the maintenance depot.

[0153] A1: After the maintenance personnel enter the work area, they start the handheld testing equipment, and the equipment automatically completes the initialization (including IMU calibration, network connection test, task synchronization, and loading of the standard template library).

[0154] A2: According to the task prompt, the touch screen will display "Current task: Spring and bolt area of ​​the 4th compartment at the bottom of vehicle number KD1436" and the voice prompt will say "Please align with the target area at the bottom of the vehicle body and keep the distance at more than 0.6 meters".

[0155] A3: The device automatically turns on the ring LED lighting to illuminate the target area, and the camera starts previewing. After the target enters the field of view, press the shutter button or the system will automatically recognize and capture the frame.

[0156] A4: After image acquisition is completed, the system evaluates the clarity, completeness, and shooting angle. If it prompts "The image is blurry, please move closer" or "The target is incomplete, please rotate 15° to the right", the user can fine-tune the device according to the prompt.

[0157] A5: The image is initially identified using a local lightweight model. If a suspected "missing bolt" is detected, the area on the device frame is highlighted, accompanied by a voice prompt: "Defect detected, please confirm."

[0158] A6: The detected image and confidence result are reported to the cloud. After secondary judgment by the high-precision model in the cloud, the final label is returned: "Location CR-04 Bolt loose (confidence level 0.86)".

[0159] A7: After the user clicks "Confirm," the image marker is locked, the data is archived, and the user can continue shooting the next target. The system prompts, "Please move to carriage 5 to continue shooting the spring area."

[0160] A8: After the task is completed, the inspection records are automatically organized and a PDF inspection report is generated and uploaded to the cloud. The report includes information such as inspection number, time, identified image number, defect type, and operator ID.

[0161] Through this process, users can complete efficient inspections of multiple areas in a short time, and achieve standardization, structuring and traceability of results with the help of device prompts and cloud recognition.

[0162] To better execute the above method, embodiments of this application also provide an inspection device, such as... Figure 7As shown, the device includes a processor, a memory, and a communication bus for enabling communication between the processor and the memory.

[0163] Inspection equipment can be implemented in various forms, including but not limited to mobile phones, tablets, PDAs, and other devices.

[0164] The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the methods provided in the above embodiments; the data storage area may store data involved in the methods provided in the above embodiments.

[0165] Optionally, the memory may be a read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), optical disc (including compact disc read-only memory (CD-ROM), compressed optical disc, laser disc, digital versatile optical disc, Blu-ray disc, etc.), magnetic disk storage medium, or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. The memory exists independently and is connected to the processor via a communication bus, or the memory is integrated with the processor.

[0166] A processor may include one or more processing cores. The processor executes instructions, programs, code sets, or instruction sets stored in memory, and calls data stored in memory to perform various functions and process data as described in this application. The processor may be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the above-described processor functions may also be other types, and the embodiments of this application do not specifically limit this.

[0167] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0168] In an alternative embodiment, the device may also include a communication interface (not shown) for communicating with other devices.

[0169] This application provides a computer-readable storage medium, including, for example, various media capable of storing program code such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. This computer-readable storage medium stores a computer program that can be loaded by a processor and execute the methods of the above embodiments.

[0170] This application also provides a computer program product comprising a computer program tangibly embodied on a readable medium thereof, the computer program containing program code for performing any of the methods described in any of the embodiments of this application, the computer program being downloadable and installable over a network, and / or installed from a removable medium (such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc.).

[0171] The above description of the embodiments is only used to provide a detailed introduction to the technical solutions of this application. However, the description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of this application, and should not be construed as a limitation of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A cloud-edge collaborative detection method for rail vehicles, characterized in that, The cloud-edge collaborative detection method for rail vehicles includes: S1: Acquire image data of the area to be tested and simultaneously record the posture information of the shooting device, including three-dimensional angle data and distance data between the device and the shooting target; S2: Perform image quality scoring on the image data, the scoring being performed based on a preset standard template library; S3: If the image quality score reaches the preset threshold, call the local target detection model to identify defects in the image and output the identification results including target type, location and confidence level; S4: If the confidence level of the recognition result is lower than the set threshold, the image data and posture information are uploaded to the cloud server, and the cloud high-precision recognition model is called for secondary recognition to obtain the cloud recognition result.

2. The cloud-edge collaborative detection method for rail vehicles as described in claim 1, characterized in that, The creation process of the standard template library includes: Professional inspectors take images of each inspection point from multiple perspectives, collect and store image data, corresponding shooting posture information, target annotation information and image quality scores, forming a standard template library containing multiple inspection reference groups. The inspection reference groups are used for subsequent image quality comparison and recognition correction.

3. The cloud-edge collaborative detection method for rail vehicles as described in claim 1, characterized in that, The image quality score in step S2 includes a sharpness score, a target completeness score, and a shooting angle score. The total image quality score is calculated using the following formula: Total score = ω1 × Sharpness score + ω2 × Target completeness score + ω3 × Shooting angle score Wherein, ω1, ω2, and ω3 are weighting coefficients, and ω1+ω2+ω3=1. The sharpness score, target integrity score, and shooting angle score are all results after normalization processing.

4. The cloud-edge collaborative detection method for rail vehicles as described in claim 3, characterized in that, The sharpness score is calculated using the Laplace variance formula: Sharpness score = Var(L(x,y)) Where L(x,y) is the response map of the image after processing by the Laplacian operator, and Var(·) is the function for calculating the variance of the response map; The target integrity score is calculated using the boundary coverage rate, and the formula is as follows: Goal completeness score = Area(B∩B0) / Area(B0) Where B is the detection box of the target region in the current image, B0 is the reference box of the target in the same location in the standard template library, and Area(·) is the function to calculate the area of ​​the region. The shooting angle score is calculated using a Gaussian decay function, and the formula is as follows: Shooting angle score = exp(-(Δθ)2 / (2σ) 2 )) Where Δθ is the deviation between the current shooting angle and the reference angle in the standard template library, and σ is the angle deviation tolerance parameter.

5. The cloud-edge collaborative detection method for rail vehicles as described in claim 1, characterized in that, If the image quality score in step S2 is lower than a preset threshold, the system generates a shooting adjustment instruction, which includes: The angle adjustment direction and distance adjustment values ​​are displayed graphically on the touch screen, and corresponding voice prompts are output through the voice module to guide the user to adjust the shooting angle and the distance between the device and the target until the image quality score reaches the preset threshold.

6. The cloud-edge collaborative detection method for rail vehicles as described in claim 1, characterized in that, Before defect identification in step S3 or S4, key points of the target structure are detected using region alignment technology. The process includes: Extract key feature points of the target structure from the current image. These key feature points are target edge contours, connection nodes, or geometric feature points. Match the extracted key feature points with reference key points of the same location in a standard template library. Spatial alignment of the target area is achieved by adjusting the image angle and scale parameters, thereby eliminating interference from differences in shooting angle and distance on recognition.

7. The cloud-edge collaborative detection method for rail vehicles as described in claim 6, characterized in that, Before defect identification in step S3 or S4, after detecting key points of the target structure using region alignment technology, the following steps are also included: A lightweight fully convolutional saliency detection network is invoked to process the image corresponding to the key points of the target structure, and a saliency map representing the attention of the region is output. Based on the saliency map, the regions in the image where the visual features are significantly different from the background are located as salient regions, and the salient regions are sent to the defect detection model for defect identification.

8. The cloud-edge collaborative detection method for rail vehicles as described in claim 7, characterized in that, Both the local target detection model in step S3 and the cloud-based high-precision recognition model in step S4 include a scene classification network and specialized sub-models. The scene classification network performs scene classification by: The scene classification network extracts features from the input image corresponding to the salient region and determines the scene to which the current region belongs based on the preset scene category label. According to the scene classification result, it calls the specialized sub-model that matches the scene to identify defects. The specialized sub-model includes a bolt defect identification sub-model, a crack identification sub-model, a rust identification sub-model, and a foreign object identification sub-model.

9. An inspection device for detecting defects in rail vehicles, characterized in that, include The acquisition module is used to acquire image data of the area to be tested and simultaneously record the posture information of the shooting device. The posture information includes three-dimensional angle data and distance data between the device and the shooting target. The scoring module is used to score the image quality of the image data, and the scoring is performed based on a preset standard template library; The defect detection module is used to call the local target detection model to identify defects in the image if the image quality score reaches a preset threshold, and output the recognition results including target type, location and confidence level. The communication module is used to upload the image data and posture information to the cloud server if the confidence level of the recognition result is lower than a set threshold, call the cloud high-precision recognition model for secondary recognition, and obtain the cloud recognition result.

10. A cloud-edge collaborative detection system for rail vehicles, characterized in that, It includes the inspection equipment as described in claim 9 and the cloud server as described in claim 1.

Images