Train interior intelligent inspection system, method and electronic device

Abstract

This invention provides an intelligent inspection system, method, and electronic device for train interiors. The system includes a computing device and an image acquisition device connected via a network. The image acquisition device acquires images of target areas inside the train, generating images to be inspected. The computing device invokes various detection models through multiple interfaces. These interfaces include a first invocation interface, and the detection models include an instance segmentation model. The computing device invokes the instance segmentation model through the first invocation interface, inputs the image to be inspected into the instance segmentation model, performs target segmentation using the instance segmentation model, and obtains pixel-level segmentation results. Based on the pixel-level segmentation results, it determines whether the target component in the target area has undergone positional movement and / or state change. This invention improves the efficiency of train interior inspection and saves on manual inspection costs.

Description

Technical Field

[0001] This invention relates to the field of rail transit equipment manufacturing technology, and provides an intelligent inspection system, method and electronic equipment for train interiors. Background Technology

[0002] Train interior inspection refers to the process of systematically inspecting, identifying, and evaluating the condition of all facilities, components, and surfaces inside a train carriage. Its core objectives are to ensure operational safety, guarantee service quality, and achieve preventative maintenance. In the interior inspection of trains (such as subway trains and high-speed trains), manual visual inspection remains the primary method. However, manual inspection is time-consuming, inefficient, and leads to extended maintenance cycles. Summary of the Invention

[0003] This invention provides an intelligent inspection system, method, and electronic device for train interiors to solve the problem of low efficiency in manual inspection of train interiors.

[0004] This invention proposes an intelligent inspection system for the interior of a train, including a computing device and an image acquisition device; the computing device and the image acquisition device are connected via a network; the image acquisition device is used to acquire images of a target area inside the train and generate an image to be inspected; the computing device is used to call various different detection models through multiple calling interfaces; the multiple calling interfaces include a first calling interface, and the various detection models include an instance segmentation model; the computing device is used to call the instance segmentation model through the first calling interface, input the image to be inspected into the instance segmentation model, perform target segmentation through the instance segmentation model, obtain pixel-level segmentation results; and, based on the pixel-level segmentation results, determine whether the target component in the target area has undergone positional movement and / or state change.

[0005] According to one embodiment of the present invention, the multiple calling interfaces further include a second calling interface, and the multiple different detection models further include a target detection model; the computing device is also used to call the target detection model through the second calling interface, input the image to be detected acquired by the image acquisition device into the target detection model, obtain the detection result through the target detection model, and determine whether the target component in the target area is missing based on the detection result.

[0006] According to an embodiment of the present invention, the target area includes multiple target components; the detection result includes multiple bounding rectangles corresponding to the multiple target components; when determining whether a target component in the target area is missing based on the detection result, the computing device is specifically used to: acquire a first reference image; the first reference image includes multiple reference bounding rectangles corresponding to the multiple target components that are pre-labeled; and compare and verify the multiple bounding rectangles one by one based on the multiple reference bounding rectangles to determine whether one or more target components are missing among the multiple target components.

[0007] According to an embodiment of the present invention, the target region includes a single target component; the detection result includes a mask image corresponding to the target region; when determining whether a target component in the target region is missing based on the detection result, the computing device is specifically used to: acquire a second reference image; the second reference image is a mask image of a single target component that is not missing; and verify the mask image in the detection result based on the second reference image to determine whether a single target component is missing.

[0008] According to one embodiment of the present invention, the multiple calling interfaces further include a third calling interface, and the multiple different detection models further include an anomaly detection model; the computing device is also used to call the anomaly detection model through the third calling interface, input the image to be detected acquired by the image acquisition device into the anomaly detection model, and detect whether the target area has surface defects or foreign object intrusion through the anomaly detection model.

[0009] According to one embodiment of the present invention, the anomaly detection model is a model trained using a weakly supervised learning training method.

[0010] According to one embodiment of the present invention, the multiple calling interfaces further include a fourth calling interface, and the multiple different detection models further include a semantic segmentation model; the computing device is also used to call the semantic segmentation model through the fourth calling interface, input the image to be detected acquired by the image acquisition device into the semantic segmentation model, extract the features of the crack edge distribution through the semantic segmentation model; and determine whether a crack has appeared in the target area based on the features of the crack edge distribution.

[0011] According to one embodiment of the present invention, the computing device includes an intelligent analysis host and / or an edge computing device; the intelligent analysis host is configured in a designated carriage of the train; each carriage of the train is equipped with at least one image acquisition device and at least one edge computing device; the image acquisition device is one or more of a zoom pan-tilt camera, a panoramic camera, a depth camera, an inspection robot, or a drone.

[0012] This invention also provides an intelligent inspection method for train interiors. The method is applied to a computing device, which calls various detection models through multiple calling interfaces. The multiple calling interfaces include a first calling interface, and the various detection models include an instance segmentation model. The method includes: acquiring an image to be inspected; calling the instance segmentation model through the first calling interface, inputting the image to be inspected into the instance segmentation model, performing target segmentation through the instance segmentation model, and obtaining pixel-level segmentation results; and determining whether the target component in the target area has undergone positional movement and / or state change based on the pixel-level segmentation results.

[0013] The present invention also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement an intelligent inspection system for the interior of a train as described above.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a train interior intelligent inspection system as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a train interior intelligent inspection system as described above.

[0016] This invention provides an intelligent inspection system for train interiors. The system includes a computing device and multiple image acquisition devices. These image acquisition devices can be deployed inside the train carriages and are connected to the computing device via a network. The image acquisition devices can transmit images of the target areas to be inspected within the train to the computing device. The computing device then calls an instance segmentation model to perform target segmentation, obtaining pixel-level segmentation results. Based on these results, it can determine whether the target components in the target area have shifted in position and / or changed in state. For example, it can determine whether the knob of a mechanical lock has rotated or whether the indicator needle of a fire extinguisher has shifted. Thus, multiple image acquisition devices in multiple carriages can synchronously and in real-time acquire images of the areas to be inspected (target areas) and transmit them to the computing device. The computing device, by calling relevant models, automatically detects whether the target components have undergone changes in position or state, achieving millisecond-level response. This solves the problems of missed and false detections in manual visual inspection of subway train interiors, saves on manual inspection costs, and improves inspection efficiency and accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the system architecture of an embodiment of the intelligent inspection system for train interiors provided by the present invention; Figure 2 This is a schematic diagram of the system architecture of another embodiment of the intelligent inspection system for train interiors provided by the present invention; Figures 3a to 3c This is an example diagram illustrating the detection of whether the knob of a mechanical lock is deflected in one embodiment of the intelligent inspection system for train interiors provided by the present invention. Figure 3d and Figure 3e This is an example diagram illustrating the pointer displacement detection of a pressure gauge for fire-fighting equipment in one embodiment of the intelligent inspection system for train interiors provided by the present invention. Figure 4a This is an example diagram illustrating the detection of whether any component is missing among multiple target components in one embodiment of the intelligent inspection system for train interiors provided by the present invention; Figures 4b to 4d This is an example diagram illustrating the detection of whether a safety hammer is missing in one embodiment of the intelligent inspection system for train interiors provided by the present invention; Figure 5a This is an example diagram of foreign object intrusion in one embodiment of the intelligent inspection system for train interiors provided by the present invention; Figure 5b and Figure 5c This is an example diagram of floor surface damage in one embodiment of the intelligent inspection system for train interiors provided by the present invention; Figure 6a and Figure 6b This is an example diagram of glass crack detection in one embodiment of the intelligent inspection system for train interiors provided by the present invention; Figure 7 This is an example diagram of character recognition in one embodiment of the intelligent inspection system for train interiors provided by the present invention; Figure 8 This is a flowchart illustrating the intelligent inspection system for train interiors provided by the present invention. Figure 9 This is a schematic diagram of the software module structure of the vehicle exterior image registration device provided by the present invention; Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0019] Figure label: 10. Computing equipment; 20. Image acquisition equipment; 31. Instance segmentation model; 32. Object detection model; 33. Anomaly detection model; 34. Semantic segmentation model; 35. Character recognition model; 901. Acquisition Module; 902. Call Module; 903. Judgment Module; 1010, Processor; 1020, Communication interface; 1030, Memory; 1040, Communication bus. Detailed Implementation

[0020] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0021] Train interior inspection refers to the systematic inspection, identification, and evaluation of the condition of facilities, components, and surfaces inside train carriages. Its core objectives are to ensure operational safety, guarantee service quality, and achieve preventative maintenance. The main inspection targets include: Passenger areas: seats (damage, stains, looseness), floors (cracks, warping, foreign objects), windows (cracks, scratches, damage), interior panels (scratches, deformation, detachment), handrails / hooks (looseness, missing). Functional facilities: doors (status indicator lights, rubber strips, anti-pinch devices), lighting systems (damaged lampshades, faulty lamps), air conditioning vents (damaged grilles, abnormal airflow), information displays (black screen, distorted display), fire extinguishers (in place, normal pressure). Hygiene and cleanliness: large areas of stains, graffiti, leftover litter or hazardous materials. Dynamic behavior: during train operation, detecting safety hazards such as passengers leaning against doors or luggage blocking passageways.

[0022] In the interior inspection of subway or high-speed trains, manual visual inspection remains the primary method. However, manual inspection methods generally suffer from the following problems: Manual visual inspection has limitations. It relies on the inspector's experience and attention and is easily affected by fatigue and environmental factors, leading to missed or false inspections. For example, inspections are usually conducted late at night, and fatigue and insufficient ambient light can cause a decline in the inspector's attention, potentially causing them to miss important defects.

[0023] Manual inspection is inefficient. It takes a lot of time, especially when the workload is heavy, which leads to longer maintenance cycles.

[0024] Among related technologies, some technical solutions exist that reduce human intervention by increasing the degree of automation in inspection, such as attempting to apply machine vision to train interior inspection. However, the following drawbacks still exist: Slow response speed: The image processing algorithm used has a slow response speed in real-time detection, which cannot meet the needs of rapid detection.

[0025] Insufficient adaptability: Many existing algorithms cannot effectively adapt to constantly changing inspection environments and emerging fault types. Some solutions utilize fixed monitoring equipment and image recognition algorithms for defect identification, but their limitation lies in their inability to adapt to the identification of novel faults.

[0026] Blind spots exist: The limited field of view of fixed cameras makes certain areas unmonitored, resulting in potential defects going undetected.

[0027] Higher cost: It requires complex camera setups and expensive image processing hardware, which increases the overall implementation cost.

[0028] In view of this, embodiments of the present invention propose an intelligent inspection system, method, and electronic device for train interiors. This system includes multiple image acquisition devices and computing devices. For example, in some embodiments, the image acquisition devices may be intelligent zoom PTZ cameras, and the computing devices may be intelligent analysis hosts. Based on the hardware architecture of the intelligent zoom PTZ cameras and intelligent analysis hosts, the computing devices invoke various detection models through multiple calling interfaces to achieve compatibility with different types of faults. This allows the system to adapt to different application environments and new fault types. This method overcomes the problems of missed detections and false detections in manual visual inspection, thereby achieving comprehensive and accurate inspection of the interior of subway trains, improving inspection efficiency and safety.

[0029] The intelligent zoom PTZ camera is set in multiple preset positions inside the carriage and can dynamically adjust the PTZ angle and focal length according to the inspection needs, so as to achieve all-round monitoring of different parts to be inspected.

[0030] The intelligent inspection system for train interiors proposed in this invention will now be described in detail with reference to the accompanying drawings. The system includes: as shown in the attached drawings. Figure 1 As shown, the system includes a computing device 10 and multiple image acquisition devices 20, which are connected via a network, for example, via a wireless network or via a wired network.

[0031] The image acquisition device 20 is used to acquire images of the target area (the part to be inspected) inside the train and generate an image to be inspected. The image acquisition device 20 can be one or more combinations of a zoom pan-tilt camera, a panoramic camera or a depth camera, an inspection robot, or a drone.

[0032] Computing device 10 is used to invoke various different detection models through multiple calling interfaces. For example, it can invoke models such as... Figure 1 The example shown includes one or more of the following: instance segmentation model 31, object detection model 32, anomaly detection model 33, semantic segmentation model 34, and character recognition model 35. For example, the interface can be an Application Programming Interface (API).

[0033] The computing devices may include intelligent analysis hosts and / or edge computing devices. Intelligent analysis hosts may be configured in designated carriages of the train, with one intelligent analysis host configured for the entire train. Each intelligent analysis host is networked with multiple image acquisition devices. At least one image acquisition device and at least one edge computing device are configured in each carriage of the train; for example, one edge computing device is configured in each carriage (passenger compartment).

[0034] In some embodiments, the multiple calling interfaces include at least a first calling interface, and the multiple different detection models include at least an instance segmentation model.

[0035] The computing device 10 is used to call the instance segmentation model through the first calling interface, input the image to be detected acquired by the image acquisition device into the instance segmentation model, perform target segmentation through the instance segmentation model, obtain pixel-level segmentation results, and determine whether the target component in the target area has undergone positional movement and / or state change based on the pixel-level segmentation results.

[0036] based on Figure 1 The system architecture shown enables automated intelligent inspection of the interior of subway or high-speed trains. The computing equipment can call upon various detection models to intelligently and automatically identify whether faults have occurred in the inspected parts of the train. Compared to manual inspection, this significantly improves inspection efficiency and reduces false positives and false negatives. Figure 1 The system architecture shown can intelligently detect fault conditions such as changes in the position / state of target components inside the train, missing target components, cracked glass, surface damage, or foreign object intrusion in target areas. Furthermore, when the application environment changes and new fault types need to be detected, only the corresponding API call needs to be added, making it more versatile.

[0037] It should be noted that in some embodiments, the number of image acquisition devices can be one, for example, the image acquisition device is an inspection robot, and one inspection robot can be configured for one train.

[0038] like Figure 2 As shown, in one specific embodiment, the intelligent inspection system inside the train includes an intelligent analysis host and multiple intelligent cameras. The intelligent cameras can be, for example, intelligent zoom pan-tilt cameras. Figure 2 Taking the train shown as an example, the entire train includes driver's cabs (driver's cars) located at the head and tail of the train, and multiple passenger compartments (passenger cars) located between the two driver's cabs. Each passenger car is equipped with multiple smart cameras, and the driver's car is equipped with one smart camera. Each car is equipped with a switch. The intelligent analysis host can be deployed in a designated car, such as a passenger car near the middle of the train. Multiple smart cameras are connected to the intelligent analysis host via a network. The intelligent analysis host receives image / video data collected by the cameras in real time and performs defect identification and analysis. The intelligent analysis host has adaptive iteration capabilities, and can continuously optimize the identification algorithm based on new data and feedback, improving the accuracy and efficiency of subsequent detection.

[0039] Optionally, the train's internal intelligent inspection system may also include a data storage module, which can periodically save inspection results and equipment status to achieve digital data management.

[0040] The following is based on Figure 1 or Figure 2 The system architecture example shown includes some specific implementations.

[0041] In one specific embodiment, the detection of positional changes and state changes of the components to be inspected can be achieved based on the intelligent inspection system inside the train. The computing device invokes an instance segmentation model through a first calling interface, inputs the image to be inspected into the instance segmentation model, performs target segmentation through the instance segmentation model to obtain pixel-level segmentation results, and determines, based on the pixel-level segmentation results, whether the target component in the target area has undergone positional movement and / or state change.

[0042] The instance segmentation model can be one or more of the following models in combination: Mask R-CNN (Mask Region-based Convolutional Neural Network), SOLO (Segmenting Objects by Locations), PolarMask (Single Shot Instance Segmentation with Polar Representation), Mask2Former (Masked-attention Mask Transformer for Universal Image Segmentation), CondInst (Conditional Convolutions for Instance Segmentation), and BoxInst (High-Performance Instance Segmentation with Box Annotations).

[0043] For example, the image to be detected is input into an instance segmentation model, which outputs the target category of the target part and the target part's location information in the image. This location information could be the bounding rectangle or an irregularly shaped bounding box of the target part. The location information can be accurate to the pixel level, resulting in pixel-level image segmentation results. This improves the approximation accuracy of the bounding rectangle, making high-precision displacement detection possible. For example... Figure 3a The displayed area is the target area. Figure 3bThis is an image to be inspected, captured by an intelligent zoom pan-tilt camera focusing on the local area where the mechanical lock is located. Figure 3c Here is an example of the detection results output by the instance segmentation model. Figure 3c The component marked with a triangle is the mechanical lock knob. Depending on the orientation of the triangle, the knob on the mechanical lock changes position. Figure 3c The angle shown on the left has rotated to the angle shown on the right, with the angle change being △α. The knob marked with a triangle is the target component. The rotation of the target component's position can cause a change in the state of the mechanical lock, such as changing it from a closed state to an open state, or vice versa. In other words, both the position and state of the target component's mechanical lock have changed. This can be achieved by accurately detecting the boundary of the triangle mark using an instance segmentation model. Thus, by activating the instance segmentation model, target segmentation results and category information (e.g., category information indicating that the specific target component is a mechanical lock) can be obtained. After image post-processing, a relatively accurate boundary position can be obtained. By calculating the change in the boundary position relative to the standard position (the standard position for normal train operation) or the previous position, it is possible to determine whether the target has moved, thereby judging whether the position or state of the target component has changed, achieving accurate detection of changes in the target component's position and / or state.

[0044] For example, such as Figure 3d and Figure 3e As shown, Figure 3d The image to be detected, acquired by the image acquisition device, is processed by the instance segmentation model, and outputs an image (detection result) as shown in 3e. The current position information of the pointer (target component) in the image can be compared with historical position information or standard position information used as a reference to determine whether the pointer's position has changed. If the pointer deflects significantly (exceeding the preset error angle), it indicates that the fire-fighting equipment may be in an abnormal malfunction state.

[0045] In one specific embodiment, the multiple calling interfaces also include a second calling interface, and the multiple different detection models also include a target detection model. The computing device is further configured to call the target detection model through the second calling interface, input the image to be detected acquired by the image acquisition device into the target detection model, obtain the detection result through the target detection model, and determine whether the target component in the target area is missing based on the detection result.

[0046] The loss of an object (target component) often leads to significant changes in image features, and applying object detection models can achieve good recognition results. When the target region includes multiple target components, due to the certain clustering of the target components to be detected, it is necessary to pre-encode the target components using a standard model and label them at standard locations in the image. During detection inference, the standard locations labeled by the standard model are used for verification to determine whether a single target component is missing. For example, ... Figure 4a As shown in the diagram, the yellow rectangles represent the inference results (detection results) output by the object detection model; the side pointed to by the arrows is a schematic diagram of the standard positions marked by the standard model, and the side opposite to the arrows represents the detection results of the object detection model. The yellow rectangles are the bounding rectangles of multiple target components, used to mark the positions of multiple target components. By comparing the positional relationships of the target components on both sides and performing a one-to-one correspondence topology verification, the existence of the target can be determined, and whether any target components are missing can be identified. For example, by comparing... Figure 4a The images shown on both sides can be used to determine whether key target components such as bolts are missing or have been displaced (e.g., bolts are loose).

[0047] Based on the above exemplary description, it can be seen that in one specific embodiment, the target region includes multiple target components; the detection result includes multiple bounding rectangles corresponding to the multiple target components, for example... Figure 4a The multiple yellow rectangles shown represent the bounding boxes corresponding to multiple target components, which are the detection results obtained by the target detection model. Based on the detection results, it is determined whether a target component is missing in the target region. Specifically, this can be achieved by acquiring a first reference image, which includes multiple reference bounding boxes corresponding to multiple pre-annotated target components. For example... Figure 4a The image pointing to the side of the arrow is the first reference image, which includes standard position information for multiple target components. Figure 4a The multiple red rounded rectangles shown represent multiple reference bounding rectangles corresponding to multiple target components, indicating the standard position of the bounding rectangles of the target components. Based on these reference bounding rectangles, each bounding rectangle is compared and verified. For example, the comparison can be performed by comparing the coordinates of the multiple bounding rectangles with the coordinates of the multiple reference bounding rectangles to see if they are consistent and within the allowable error range. If they are inconsistent or exceed the error range, it indicates that one or more target components are missing or have been displaced, which may lead to a malfunction.

[0048] For example, such as Figure 4b As shown, Figure 4b The image to be detected is captured by an image acquisition device focusing on the area where the safety hammer is located. The target detection model then outputs the detection result, for example, the detection result could be... Figure 4cThe mask image shown contains the boundary features when the target component (safety hammer) is not missing.

[0049] like Figure 4d , Figure 4d The image shown is a mask image obtained when the safety hammer is missing. (Comparison) Figure 4c and Figure 4d As can be seen, there are significant differences between the two. The mask image when the safety hammer is not missing can be used as a reference image to verify the image output by the target detection model. By comparing the differences in image features between the two, especially the differences in the boundary features of the target component, it can be determined whether the target component is missing. For example... Figure 4c As a reference image (second reference image), the image output by the object detection model is Figure 4d As shown, the safety hammer is missing.

[0050] Based on the above exemplary description, it can be seen that in one specific embodiment, the target region includes a single target component, and the detection result includes a mask image corresponding to the target region. Based on the detection result, it is determined whether the target component in the target region is missing, which may specifically involve obtaining a second reference image (e.g., Figure 4c The image shown is a mask image; the second reference image is a mask image of a single target component that is not missing. Based on the second reference image, the mask image in the detection result is verified, for example, by comparing the differences between the image features of the target components, and determining whether a single target component (e.g., a safety hammer) is missing based on the degree of difference.

[0051] The object detection model can be one or more of the following models in combination: Faster R-CNN (Faster Region-based Convolutional Neural Network), Cascade R-CNN (Cascade Region-based Convolutional Neural Network), RetinaNet, Deformable DETR (Deformable Detection Transformer, a Transformer-based object detector with deformable attention mechanism), and RT-DETR (Real-Time Detection Transformer, a real-time Transformer-based object detector).

[0052] In one specific embodiment, the multiple calling interfaces also include a third calling interface, and the multiple different detection models also include an anomaly detection model. The computing device calls the anomaly detection model through the third calling interface, inputs the image to be detected acquired by the image acquisition device into the anomaly detection model, and detects whether the target area has surface defects or foreign object intrusion through the anomaly detection model. The anomaly detection model is a model trained using a weakly supervised learning method.

[0053] Video anomaly detection tasks based on weakly supervised learning present both normal and abnormal videos in the training data. However, while the presence of anomalies is known, the specific locations of the anomalies are unknown; each video only possesses a weak label indicating whether it is abnormal. In this situation, the addition of abnormal videos to the training data makes single-class learning methods unsuitable, and the accuracy of the labels cannot be achieved using fully supervised learning methods. Therefore, this invention proposes using weakly supervised learning to address this issue. Specifically, an anomaly detection model is trained using weakly supervised learning. The trained anomaly detection model can detect surface damage or foreign object intrusion.

[0054] For example, such as Figure 5a As shown, after prediction by the anomaly detection model, the comparison... Figure 5a The two feature images shown on the right side of the image indicate the presence of a white foreign object intruding into the target area. For example, Figure 5b The image to be detected is captured by the image acquisition device focusing on the floor area inside the subway car. After passing through the anomaly detection model, the output is as follows: Figure 5c The image shown, Figure 5c The image clearly shows the detected abnormal areas, which may be floor damage or bulges.

[0055] Anomaly detection models can be one or more of the following models in combination: PaDiM (Patch Distribution Modeling Framework), PatchCore (Patch-wise Core-set Memory Bank Framework), STFPM (Student-Teacher Feature Pyramid Matching Framework), GANomaly (Generative Adversarial Network for Anomaly Detection), CFLOW-AD (Conditional Normalizing Flow-based Anomaly Detection Framework), and RD4AD (Reverse Distillation for Anomaly Detection).

[0056] In one specific embodiment, the multiple calling interfaces also include a fourth calling interface, and the multiple different detection models also include a semantic segmentation model. The computing device calls the semantic segmentation model through the fourth calling interface, inputs the image to be detected acquired by the image acquisition device into the semantic segmentation model, and extracts the features of the crack edge distribution through the semantic segmentation model; based on the features of the crack edge distribution, it determines whether a crack has appeared in the target area. For example, the target area is the glass of a carriage, and it determines whether the glass has a crack.

[0057] Semantic segmentation models are suitable for detecting objects with certain pixel distribution characteristics but varied shapes; at the same time, due to the upsampling process in the network structure, they still have good applicability to small-scale objects. The effect of using a semantic segmentation network (semantic segmentation model) to extract crack edge distribution shows that the image features of the crack are accurately detected, and the detected crack boundaries have good continuity, facilitating subsequent statistical processing and analysis.

[0058] For example, Figure 6a The image to be detected is obtained by focusing the image acquisition device on the car window and taking a picture. After semantic segmentation, the output image is as follows: Figure 6b As shown, the semantic segmentation model can also simultaneously output the identification result of whether it is a glass crack, that is... Figure 6b Does the image feature match the characteristics of a glass crack?

[0059] Semantic segmentation models can be one or more combinations of the following models: U-Net U-Shaped (Convolutional Neural Network for Biomedical Image Segmentation), DeepLabv3+ (Deep Labellingv3 Plus), PSPNet (Pyramid Scene Parsing Network), SegFormer (Segmenting Transformer with Hierarchical Multi-Scale Attention), and HRNet (High-Resolution Network for Semantic Segmentation).

[0060] In one specific embodiment, the various calling interfaces also include a fifth calling interface, and the various different detection models also include a character recognition model. The computing device calls the character recognition model through the fifth calling interface, inputting the image to be detected acquired by the image acquisition device into the character recognition model, and then using the character recognition model to recognize the characters in the image to be detected. The character recognition model is a model that supports Optical Character Recognition (OCR). For example, it can examine characters printed on paper on electronic devices, determine their shape by detecting dark and light patterns, and then translate the shape into computer text using a character recognition method. OCR includes four steps: preprocessing, feature extraction, character segmentation, and character recognition. In this embodiment of the invention, the character recognition model can be called to automatically recognize text markings within a target area inside the carriage. For example, such as... Figure 7 As shown, a character recognition model can be activated to automatically recognize text markings inside the carriage. Based on the recognition results, it can be determined whether the text markings are worn or covered by foreign objects, making them unrecognizable, or whether the text markings are correct, etc.

[0061] It should be noted that the above embodiments are merely illustrative examples, and other types of intelligent cameras, such as panoramic cameras or depth cameras, can be used instead of intelligent zoom PTZ cameras. Intelligent cameras can provide wider field of view coverage and prevent blind spots in monitoring.

[0062] Furthermore, the processing of computing devices can also be achieved through edge computing devices, without being limited to centralized intelligent analysis hosts. For example, by deploying processing units near intelligent cameras, data transmission latency can be reduced, and the response speed of real-time detection can be improved. Alternatively, in system design, the functions of the intelligent analysis host can be distributed across multiple modules, such as by introducing multiple small processors to share data processing tasks in different areas, forming a distributed processing system.

[0063] It should also be noted that the aforementioned object detection model, semantic segmentation model, instance segmentation model, anomaly detection model, and other models can be implemented based on deep learning networks. These models can be integrated into a single model or they can be different, independent models. In some embodiments, some of these models can be selected and combined with machine vision algorithms (such as edge detection, shape recognition, etc.) to achieve various defect recognitions. That is, replacing some models with machine vision algorithms that can achieve the corresponding functions can also yield some implementation examples. Combining machine vision algorithms with deep learning detection models may achieve high-precision detection results in certain specific scenarios, especially when there are sufficient defect samples.

[0064] In addition, image acquisition equipment can be inspection robots or drones, using drones or mobile robots for inspections instead of fixed cameras. Using drones for aerial inspections inside the carriage can cover a wider area and quickly adapt to environmental changes.

[0065] In some embodiments, the camera's installation method can be changed. The camera's installation direction and angle can be altered, employing a suspended or wall-mounted installation scheme to cover different inspection areas and increase viewing flexibility.

[0066] This invention also provides an intelligent inspection method for the interior of a train, which can be applied to computing devices. For example... Figure 8 As shown, the method may include the following steps: Step 801: Obtain the image to be detected.

[0067] Step 802: Call the instance segmentation model through the first call interface, input the image to be detected into the instance segmentation model, and perform target segmentation through the instance segmentation model to obtain pixel-level segmentation results.

[0068] Step 803: Based on the pixel-level segmentation results, determine whether the target component in the target area has undergone positional movement and / or state change.

[0069] This invention also provides an intelligent inspection device for the interior of a train, which can be integrated into a computing device. For example... Figure 9 As shown, the device may include the following modules: The acquisition module 901 is used to acquire the image to be detected.

[0070] The calling module 902 is used to call the instance segmentation model through the first calling interface, input the image to be detected into the instance segmentation model, and perform target segmentation through the instance segmentation model to obtain pixel-level segmentation results.

[0071] The discrimination module 903 is used to determine whether the target component in the target area has moved and / or changed its state based on the pixel-level segmentation results.

[0072] In summary, in the system and method proposed in this invention, the intelligent zoom PTZ camera can be dynamically adjusted. This camera can automatically adjust the PTZ angle and focal length according to the inspection requirements, achieving comprehensive monitoring of different parts to be inspected within the carriage. The intelligent analysis host supports real-time data processing. The host integrates advanced image processing algorithms, enabling real-time analysis of data collected by the camera and fault identification, and possesses adaptive iterative capabilities. Optionally, in some embodiments, automated camera inspection can significantly reduce the workload of manual inspection, improve inspection efficiency, and reduce human error.

[0073] Optionally, in some embodiments, digital storage and management functions can also be deployed: the system can periodically save test results and the status of internal equipment, promote the digital management of maintenance operations, and provide data basis for vehicle health management.

[0074] This invention also proposes a fault identification algorithm based on deep learning: using deep learning technology for fault identification can effectively improve the detection accuracy and overcome the limitations of traditional manual inspection.

[0075] In terms of system integration and flexibility, the integration of cameras, analysis host and data storage modules forms a highly efficient intelligent inspection system with the flexibility to adapt to different inspection environments.

[0076] The system and method proposed in the embodiments of the present invention solve at least one of the following technical problems: Limitations of manual visual inspection: Traditional manual visual inspection relies on manpower, which is prone to problems such as fatigue, subjectivity, missed detections, and false detections, especially in late-night inspection environments. This invention, through an intelligent machine vision system, replaces manual inspection, reducing interference from human factors.

[0077] Blind Spot Problem: During in-vehicle inspections, due to the unique layout of equipment and inspection points, blind spots may exist in certain areas, causing important defects to go undetected. This invention, by analyzing and breaking down the detection requirements for blind spots and combining them with equipment assembly principles, proposes an indirect detection scheme based on the cascading effect after a fault occurs, thereby improving detection coverage.

[0078] Lack of efficient data analysis methods: The current lack of effective data analysis tools leads to insufficient fault trend analysis in maintenance work. The intelligent analysis host introduced in this embodiment of the invention can quickly identify potential problems and support decision-making by analyzing the collected data in real time.

[0079] The system and method proposed in the embodiments of the present invention achieve the following technical effects: Improved inspection efficiency and accuracy: The automated inspection system reduces the time spent on manual inspections, improves the efficiency and accuracy of inspections, and enables the faster detection and reporting of potential faults.

[0080] Reduced manual labor intensity: The implementation of the system has significantly reduced the workload of maintenance personnel, especially during long night inspections, effectively reducing the risks associated with fatigue.

[0081] Improve safety: Precise automatic inspections can promptly identify potential safety hazards, reduce the risk of accidents caused by missed inspections, and improve the overall safety of operations.

[0082] Realization of digital management: The system can regularly save the digital status of the internal equipment, providing a reliable data foundation for subsequent maintenance work and promoting digital management and intelligent decision-making in operation and maintenance.

[0083] Promoting the transformation to intelligent operation and maintenance: The embodiments of this invention provide a brand-new intelligent solution for subway operation and maintenance or high-speed rail operation and maintenance, marking the transformation from traditional operation and maintenance mode to intelligent operation and maintenance mode, and improving the overall technical level of the industry.

[0084] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040. The processor 1010, communications interface 1020, and memory 1030 communicate with each other via the communication bus. The processor 1010 can call logical instructions from the memory 1030 to execute the following methods: Acquire the image to be detected; call the instance segmentation model through the first call interface, input the image to be detected into the instance segmentation model, perform target segmentation through the instance segmentation model, and obtain pixel-level segmentation results; based on the pixel-level segmentation results, determine whether the target component in the target region has undergone positional movement and / or state change.

[0085] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] This invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as including: Acquire the image to be detected; call the instance segmentation model through the first call interface, input the image to be detected into the instance segmentation model, perform target segmentation through the instance segmentation model, and obtain pixel-level segmentation results; based on the pixel-level segmentation results, determine whether the target component in the target region has undergone positional movement and / or state change.

[0087] On the other hand, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the transmission methods provided in the above embodiments, including, for example: Acquire the image to be detected; invoke the instance segmentation model through the first calling interface, input the image to be detected into the instance segmentation model, perform target segmentation through the instance segmentation model, and obtain pixel-level segmentation results; determine whether the target component in the target region has undergone positional movement and / or state change based on the pixel-level segmentation results.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A train interior intelligent inspection system, characterized in that, The system includes a computing device and an image acquisition device; the computing device and the image acquisition device are connected via a network. The image acquisition device is used to acquire images of target areas inside the train and generate images to be detected. The computing device is configured to invoke multiple different detection models through multiple calling interfaces; the multiple calling interfaces include a first calling interface, and the multiple different detection models include an instance segmentation model; the computing device is configured to invoke the instance segmentation model through the first calling interface, input the image to be detected into the instance segmentation model, perform target segmentation through the instance segmentation model, obtain pixel-level segmentation results; and, based on the pixel-level segmentation results, determine whether the target component in the target region has undergone positional movement and / or state change.

2. The system according to claim 1, characterized in that, The multiple calling interfaces also include a second calling interface, and the multiple different detection models also include a target detection model; The computing device is also used to call the target detection model through the second calling interface, input the image to be detected acquired by the image acquisition device into the target detection model, obtain the detection result through the target detection model, and determine whether the target component in the target area is missing based on the detection result.

3. The system according to claim 2, characterized in that, The target area includes multiple target components; the detection result includes multiple bounding rectangles corresponding to the multiple target components. When determining whether a target component is missing in the target area based on the detection results, the computing device is specifically used for: Obtain a first reference image; the first reference image includes multiple reference bounding rectangles corresponding to multiple pre-annotated target components; Based on the plurality of reference bounding rectangles, each bounding rectangle is compared and verified to determine whether one or more target components are missing.

4. The system according to claim 2, characterized in that, The target region includes a single target component; the detection result includes a mask image corresponding to the target region; When determining whether a target component is missing in the target area based on the detection results, the computing device is specifically used for: Obtain a second reference image; the second reference image is a mask image of the single target component where no part is missing; Based on the second reference image, the mask image in the detection result is verified to determine whether the individual target component is missing.

5. The system according to claim 1, characterized in that, The various calling interfaces also include a third calling interface, and the various different detection models also include an anomaly detection model; The computing device is also used to call the anomaly detection model through a third calling interface, input the image to be detected acquired by the image acquisition device into the anomaly detection model, and detect whether the target area has surface defects or foreign object intrusion through the anomaly detection model.

6. The system according to claim 5, characterized in that, The anomaly detection model is a model trained using a weakly supervised learning method.

7. The system according to claim 1, characterized in that, The various calling interfaces also include a fourth calling interface, and the various different detection models also include a semantic segmentation model; The computing device is also used to call the semantic segmentation model through the fourth calling interface, input the image to be detected acquired by the image acquisition device into the semantic segmentation model, and extract the features of the crack edge distribution through the semantic segmentation model; Based on the characteristics of the crack edge distribution, it is determined whether a crack has appeared in the target area.

8. The system according to claim 1, characterized in that, The computing device includes an intelligent analysis host and / or an edge computing device; the intelligent analysis host is configured in a designated carriage of the train; each carriage of the train is equipped with at least one image acquisition device and at least one edge computing device. The image acquisition device is one or more of the following: a zoom pan-tilt camera, a panoramic camera, a depth camera, an inspection robot, or a drone.

9. A method for intelligent inspection of the interior of a train, characterized in that, The method is applied to a computing device, which is used to invoke multiple different detection models through multiple calling interfaces; the multiple calling interfaces include a first calling interface, and the multiple different detection models include an instance segmentation model; the method includes: Acquire the image to be detected; The instance segmentation model is invoked through the first calling interface. The image to be detected is input into the instance segmentation model, and the target segmentation is performed by the instance segmentation model to obtain pixel-level segmentation results. Based on the pixel-level segmentation results, determine whether the target component in the target region has undergone positional movement and / or state change.

10. An electronic device, characterized in that, It includes a processor and a memory storing a computer program, wherein the processor executes the program to implement the intelligent inspection method for the train interior as described in claim 9.

Images