Method, device and medium for detecting a motor train unit floor component
By combining convolutional feature extraction and attention processing, the problem of low reliability in the detection of EMU base plate components was solved, achieving efficient detection of small targets and reducing the false negative rate.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUITIE TECH CO LTD
- Filing Date
- 2026-02-10
- Publication Date
- 2026-05-29
Smart Images
Figure CN122116015A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data analysis technology, and more specifically, to a method, equipment, and medium for testing components of a high-speed train's floor plate. Background Technology
[0002] The Dynamic Image Detection System (TEDS) for High-Speed Train Operation Faults is a key piece of equipment for ensuring railway transportation safety. It acquires high-speed images of the underside of running high-speed trains, allowing analysts to identify faults. However, manual inspection methods suffer from high labor intensity, low efficiency, and susceptibility to subjective factors leading to missed detections. This is especially true for minor faults such as missing bolts in the base plate, which appear small (typically less than 32×32 pixels) and highly similar to normal components in the images, resulting in a high rate of missed detections. Currently, deep learning-based target detection algorithms, particularly the YOLO series, have been attempted for industrial fault detection. However, directly applying general-purpose models to TEDS scenarios faces significant challenges: insufficient performance in small target detection. Research by the inventors has revealed that small targets have limited pixel information, which is easily lost during feature extraction, leading to relatively low detection reliability. Summary of the Invention
[0003] In view of this, the purpose of this application is to provide a method, equipment and medium for testing the floor components of a high-speed train, so as to improve the problem of relatively low reliability of the testing of the floor components of a high-speed train in the prior art.
[0004] To achieve the above objectives, this application adopts the following technical solution: A method for inspecting components of a high-speed train floor, comprising: Acquire the target floor image obtained by image acquisition of the target EMU floor; Convolutional feature extraction is performed on the target substrate image to form substrate image convolutional features. During the convolutional feature extraction process, the receptive field is adjusted at least by switchable dilated convolution to capture multi-scale contextual information in the target substrate image. The convolutional features of the base plate image are detected and output to obtain base plate component detection data. The base plate component detection data is used to reflect at least one of the following: whether there are missing components on the base plate of the target EMU and the location of the components on the base plate.
[0005] In a preferred embodiment of this application, in the above-mentioned method for detecting components on the undercarriage of a high-speed train, the step of extracting convolutional features from the target undercarriage image to form convolutional features of the undercarriage image includes: The target substrate image is downsampled and convolved through at least one first convolutional layer to form a first substrate convolutional feature; The first base plate convolutional feature is formed by performing switchable dilated convolution on the first base plate convolutional feature through at least one second convolutional layer. Based on the second base plate convolution features, the base plate image convolution features are determined.
[0006] In a preferred embodiment of this application, in the aforementioned method for detecting the floor plate components of a high-speed train, the step of performing switchable open convolution on the first floor plate convolutional features through at least one second convolutional layer to form second floor plate convolutional features includes: The first base plate convolution feature is dilated by multiple dilated convolution units included in the second convolution layer to obtain multiple base plate dilated convolution features, wherein the multiple dilated convolution units have different dilation rates. The first base plate convolutional features are subjected to global pooling and convolution processing to obtain the target weight distribution, wherein the number of weights in the target weight distribution is equal to the number of the plurality of dilated convolutional units, and there is a one-to-one correspondence between them; Based on the weights in the target weight distribution, the multiple base plate hole convolution features are weighted and summed to achieve gating adjustment, thereby obtaining the second base plate convolution feature.
[0007] In a preferred embodiment of this application, in the above-mentioned method for detecting components on the undercarriage of a high-speed train, the step of detecting and outputting the convolutional features of the undercarriage image to obtain undercarriage component detection data includes: Attention processing is applied to the convolutional features of the substrate image to enhance the features and form attention features of the substrate image. The attention processing includes global attention and local attention. The attention features of the base plate image are detected and output to obtain the base plate component detection data.
[0008] In a preferred embodiment of this application, in the above-mentioned method for detecting components on the undercarriage of a high-speed train, the step of performing attention processing on the convolutional features of the undercarriage image to achieve feature enhancement and form attention features of the undercarriage image includes: Global attention processing is performed on the convolutional features of the substrate image to form global attention features of the substrate, wherein the global attention processing includes multi-head self-attention calculation based on a moving window; The convolutional features of the substrate image are subjected to local attention processing to form substrate image attention features, wherein the local attention processing is implemented based on the coordinate attention mechanism.
[0009] In a preferred embodiment of this application, in the above-mentioned method for detecting the underbody components of a high-speed train, the step of performing local attention processing on the convolutional features of the underbody image to form attention features of the underbody image includes: The convolutional features of the base image are pooled along the row direction to form the first base image pooling feature, and the convolutional features of the base image are pooled along the column direction to form the second base image pooling feature. The first base plate pooling feature and the second base plate pooling feature are spliced together to form a spliced base plate pooling feature, and the spliced base plate pooling feature is subjected to convolution dimensionality reduction processing to obtain a base plate convolution dimensionality reduction feature. The base plate convolutional dimensionality reduction features are split to form a first base plate split feature and a second base plate split feature. The first base plate split feature and the second base plate split feature are then subjected to convolutional dimensionality increase processing to form a first base plate dimensionality increase feature and a second base plate dimensionality increase feature. The first and second base plate dimensionality-upgraded features are respectively subjected to attention mapping to form a first attention weight and a second attention weight; Based on the first attention weight and the second attention weight, attention adjustment is performed on the convolutional features of the substrate image to form substrate image attention features.
[0010] In a preferred embodiment of this application, the above-mentioned method for detecting the floor slab components of a high-speed train further includes a training step for a floor slab component detection model. The floor slab component detection model is a neural network model and is used to obtain the floor slab component detection data based on the target floor slab image. The training step includes: A sample base plate image is acquired and segmented to form multiple local base plate images. Each local base plate image is scaled to form a corresponding scaled base plate image. Based on the distribution relationship of the multiple local base plate images in the sample base plate image, the multiple scaled base plate images are combined to form multiple combined base plate images. Based on the multiple combined base plate images and corresponding base plate labels, the candidate component detection model is trained to form the base plate component detection model.
[0011] In a preferred embodiment of this application, in the above-mentioned method for detecting floor components of high-speed trains, the step of training a candidate component detection model based on the plurality of combined floor images and corresponding floor labels to form the floor component detection model includes: The candidate component detection model is used to extract convolutional features from the multiple combined base plate images to obtain the corresponding multiple combined image convolutional features. Based on the preset feature similarity judgment conditions, at least one similar feature combination is determined from the multiple combined image convolutional features. Each of the similar feature combinations includes at least two combined image convolutional features whose similarity relationship satisfies the preset conditions. For each of the similar feature combinations, the candidate component detection model performs feature fusion processing on the convolutional features of at least two combined images in the similar feature combination, and generates the restored base image corresponding to the similar feature combination based on the fused features. Based on the multiple combined base plate images, the restored base plate images corresponding to each similar feature combination, and the corresponding base plate labels, the candidate component detection model is trained to obtain a base plate component detection model.
[0012] Based on the above, this application also provides an electronic device, including: Memory, used to store computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned method for detecting components on the EMU floor.
[0013] Based on the above, this application also provides a computer-readable storage medium storing a computer program that, when executed, performs the various steps of the above-described method for detecting components on the EMU floor.
[0014] The method, equipment, and medium for detecting EMU (Electric Multiple Unit) floorboard components provided in this application first acquire a target floorboard image obtained by image acquisition of the target EMU floorboard; second, convolutional feature extraction is performed on the target floorboard image to form floorboard image convolutional features. During the convolutional feature extraction process, at least switchable dilated convolution is used to adjust the receptive field to capture multi-scale contextual information in the target floorboard image; then, the floorboard image convolutional features are detected and output to obtain floorboard component detection data. This floorboard component detection data reflects at least one of the following: whether components on the target EMU floorboard are missing, and the location of the components on the floorboard. Based on the above, since the receptive field is adjusted by at least switchable dilated convolution during the convolutional feature extraction process to capture multi-scale contextual information in the target floorboard image, more detailed features can be captured, improving the semantic representation accuracy of the obtained floorboard image convolutional features. This ensures that the detected floorboard component detection data has high reliability, thereby improving the problem of relatively low reliability in EMU floorboard component detection in existing technologies. Attached Figure Description
[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings.
[0016] Figure 1 A structural block diagram of an electronic device provided in an embodiment of this application.
[0017] Figure 2 This is a flowchart illustrating the method for detecting the undercarriage components of a high-speed train provided in an embodiment of this application.
[0018] Figure 3 This is a schematic diagram of convolutional feature extraction provided in an embodiment of this application.
[0019] Figure 4 This is a schematic diagram of switchable dilated convolution provided in an embodiment of this application.
[0020] Figure 5 This is a schematic diagram of local attention processing provided in an embodiment of this application. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0022] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0023] like Figure 1 As shown in the figure, this application provides an electronic device. The electronic device may include a memory, a processor, and a detection device for components on the EMU (Electric Multiple Unit) floor.
[0024] In detail, the memory and the processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, the memory and the processor can be electrically connected via one or more communication buses or signal lines. The EMU floor component detection device includes at least one software functional module stored in the memory in the form of software or firmware. The processor is used to execute executable computer programs stored in the memory, such as the software functional modules and computer programs included in the EMU floor component detection device, to implement the EMU floor component detection method provided in this application embodiment.
[0025] Optionally, the memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
[0026] Optionally, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0027] Optionally, the EMU floor component detection device may include the following software functional modules: The image acquisition module is used to acquire the target floor image obtained by image acquisition of the target EMU floor. The convolutional feature extraction module is used to extract convolutional features from the target substrate image to form convolutional features of the substrate image. In the process of convolutional feature extraction, the receptive field is adjusted at least by switchable dilated convolution to capture multi-scale contextual information in the target substrate image. The detection output module is used to detect and output the convolutional features of the base plate image to obtain base plate component detection data. The base plate component detection data is used to reflect at least one of the following: whether there are missing components on the base plate of the target EMU and the location of the components on the base plate.
[0028] Understandable. Figure 1 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown may include, for example, a communication unit for exchanging information with other devices (such as image acquisition devices).
[0029] Combination Figure 2This application also provides a method for inspecting EMU (Electric Multiple Unit) floor plate components applicable to the aforementioned electronic equipment. The method steps defined in the process of the EMU floor plate component inspection method can be implemented by the electronic equipment. The following will describe... Figure 2 The specific process shown will be explained in detail.
[0030] Step S110: Obtain the target floor image obtained by image acquisition of the target EMU floor.
[0031] In this embodiment of the application, the electronic device can acquire an image of the target train's undercarriage obtained by image acquisition. For example, a communication-connected image acquisition device can perform high-speed image acquisition of the undercarriage of a running train and send the acquired image to the electronic device, enabling the electronic device to acquire the target undercarriage image.
[0032] Step S120: Extract convolutional features from the target base plate image to form base plate image convolutional features.
[0033] In this embodiment, after obtaining the target substrate image, the electronic device can perform convolutional feature extraction on the target substrate image to form substrate image convolutional features. During the convolutional feature extraction process, at least switchable dilated convolution is used to adjust the receptive field to capture multi-scale contextual information in the target substrate image, thereby capturing more detailed features and enabling effective characterization of small components on the substrate of the target train.
[0034] Step S130: Detect and output the convolutional features of the base plate image to obtain base plate component detection data.
[0035] In this embodiment, after obtaining the convolutional features of the base plate image, the electronic device can perform detection output on the convolutional features of the base plate image to obtain base plate component detection data. The base plate component detection data is used to reflect at least one of the following: whether a component on the base plate of the target EMU is missing, and the location of the component on the base plate. For example, it can be used only to reflect whether a component is missing, or only to reflect the location of the component, or to reflect both whether a component is missing and the location of the component. Furthermore, the determination of whether a component is missing can be achieved through a classification task (i.e., the output channel includes a classification branch); the determination of the location of the component can be achieved through a regression task (i.e., the output channel includes a regression branch).
[0036] Based on the above, since the receptive field is adjusted by at least switchable dilated convolution during the convolution feature extraction process to capture multi-scale contextual information in the target base plate image, more detailed features can be captured, improving the semantic representation accuracy of the obtained base plate image convolution features. This ensures that the detection data of the base plate component has high reliability, thereby improving the problem of relatively low reliability of EMU base plate component detection in the existing technology.
[0037] Firstly, regarding step S110, it should be noted that the specific method for obtaining the target base plate image is not limited and can be selected according to actual needs.
[0038] For example, in an alternative implementation, in order to ensure the real-time detection of the base plate component, the image acquisition device can send the image of the target base plate to the electronic device in real time after acquiring and forming the image, so that the electronic device can acquire and perform the corresponding detection in real time.
[0039] For example, in another alternative implementation, after the image acquisition device acquires and forms the target base plate image, it can first send it to the database for storage. In this way, when it is necessary to inspect the base plate component, the electronic device can then obtain the target base plate image from the database.
[0040] Secondly, regarding step S120, it should be noted that the specific method for extracting convolutional features from the target base plate image is not limited and can be selected according to actual needs.
[0041] For example, in an alternative implementation, in order to fully capture the multi-scale contextual information in the target substrate image, the above step S120 may further include steps S121, S122 and S123, wherein the specific contents of each step are as follows.
[0042] Step S121: The target substrate image is downsampled and convolved through at least one first convolutional layer to form a first substrate convolutional feature.
[0043] In this embodiment of the application, the target substrate image can be downsampled and convolved using at least one first convolutional layer to form a first substrate convolutional feature. For example... Figure 3 As shown, the target substrate image can be downsampled and convolved sequentially by two first convolutional layers to obtain the first substrate convolutional features. For example, the first first convolutional layer downsamples and convolves the target substrate image to obtain intermediate convolutional features, and the second first convolutional layer downsamples and convolves the intermediate convolutional features to form the first substrate convolutional features.
[0044] Step S122: The first base plate convolutional feature is subjected to switchable dilated convolution through at least one second convolutional layer to form a second base plate convolutional feature.
[0045] In this embodiment of the application, after obtaining the first base plate convolutional feature, a switchable dilated convolution can be performed on the first base plate convolutional feature through at least one second convolutional layer to form a second base plate convolutional feature. For example... Figure 3 As shown, the first base plate convolutional feature can be sequentially subjected to switchable dilated convolution through two second convolutional layers to obtain the second base plate convolutional feature. For example, the first second convolutional layer performs switchable dilated convolution on the first base plate convolutional feature to obtain the intermediate convolutional feature, and the second second convolutional layer performs switchable dilated convolution on the intermediate convolutional feature to form the second base plate convolutional feature.
[0046] Step S123: Based on the second base plate convolution features, determine the base plate image convolution features.
[0047] In this embodiment, after obtaining the second base plate convolutional feature, the base plate image convolutional feature can be determined based on the second base plate convolutional feature. For example, the second base plate convolutional feature can be directly determined as the base plate image convolutional feature. Alternatively, the second base plate convolutional feature can be further downsampled and convolved by a convolutional layer (a third convolutional layer) to obtain the base plate image convolutional feature. The dimensions of the first base plate convolutional feature, the second base plate convolutional feature, and the base plate image convolutional feature can be 80*80, 40*40, and 20*20, respectively.
[0048] It is understood that in step S122 above, the specific method of performing switchable dilated convolution on the first substrate convolution feature is not limited. For example, in an alternative implementation, in order to adaptively select different dilation rates according to the input image (such as the target substrate image), and to flexibly expand the receptive field, capture multi-scale contextual information, and improve the detection accuracy of minor faults without excessively increasing the computational burden, step S122 above may further include steps S122a, S122b, and S122c, wherein the specific contents of each step are as follows.
[0049] Step S122a: The first base plate convolution feature is subjected to dilated convolution through the multiple dilated convolution units included in the second convolution layer to obtain multiple base plate dilated convolution features.
[0050] In this embodiment, the first substrate convolutional feature can be dilated by multiple dilated convolutional units included in the second convolutional layer (that is, each second convolutional layer includes multiple dilated convolutional units) to obtain multiple substrate dilated convolutional features. The multiple dilated convolutional units have different dilation rates. Combined with... Figure 4 As shown, the second convolutional layer may include three dilated convolutional units, which perform dilated convolution on the first base plate convolutional features respectively, to obtain three base plate dilated convolutional features with dilation rates of 1, 3, and 5 respectively.
[0051] Step S122b: Perform global pooling and convolution processing on the first base plate convolutional features to obtain the target weight distribution.
[0052] In this embodiment, the first base plate convolutional features can also be subjected to global pooling and convolution processing (or processed using functions such as softmax) to obtain the target weight distribution. The number of weights in the target weight distribution is equal to the number of the plurality of dilated convolutional units, and they have a one-to-one correspondence. For example, the global pooling can be implemented using a 5×5 global average pooling layer, and the convolution processing can be implemented using a 1×1 convolutional layer.
[0053] Step S122c: Based on the weights in the target weight distribution, perform weighted summation calculation on the multiple base plate hole convolution features to achieve gating adjustment and obtain the second base plate convolution feature.
[0054] In this embodiment, after obtaining the target weight distribution and the plurality of base plate dilated convolutional features, a weighted summation calculation can be performed on the plurality of base plate dilated convolutional features based on each weight in the target weight distribution to achieve gating adjustment and obtain the second base plate convolutional feature. For example, Output (second base plate convolutional feature) = W1 * Conv r=1 (x)+W2*Conv r=3 (x)+W3*Conv r=5 (x), thus, the receptive field can be automatically expanded based on the complexity of the background (e.g., whether mud or dirt obscures the view). Here, W1, W2, and W3 are the three weights in the target weight distribution, and Conv... r=1 (x), Conv r=3 (x) and Conv r=5 (x) represents three base plate void convolution features with void ratios of 1, 3, and 5, respectively.
[0055] Thirdly, regarding step S130, it should be noted that the specific method for detecting and outputting the convolutional features of the base plate image is not limited and can be selected according to actual needs.
[0056] For example, in an alternative implementation, considering that the mounting hole of the missing bolt is extremely similar to the tail of the normal bolt in terms of texture and color, it is necessary to have strong fine-grained feature recognition and global context reasoning capabilities. Based on this, the above step S130 can further include steps S131 and S132, wherein the specific contents of each step are as follows.
[0057] Step S131: Attention processing is performed on the convolutional features of the base plate image to achieve feature enhancement and form attention features of the base plate image.
[0058] In this embodiment, attention processing can be applied to the convolutional features of the base plate image to enhance features and form attention features of the base plate image. This attention processing includes global attention and local attention. It should be noted that global attention helps capture long-range contextual dependencies, such as the arrangement pattern of bolts, while local attention helps accurately locate the edge coordinates of small targets, preventing bounding box drift (i.e., the location of components on the base plate).
[0059] Step S132: Detect and output the attention features of the base plate image to obtain base plate component detection data.
[0060] In this embodiment, after obtaining the attention features of the base plate image, the attention features can be detected and output to obtain base plate component detection data. For example, a decoupled head can be used to separate classification (whether a component on the base plate is missing) and regression tasks (at least one of the locations of components on the base plate). The decoupled head is a commonly used technique in object detection tasks, especially in modern detectors (such as YOLOv4, RetinaNet, and DETR), used to separate and independently process classification and regression tasks. In classic object detection frameworks, a single network head is typically used to simultaneously perform object classification (i.e., predicting the object's category) and location regression (i.e., predicting the object's bounding box location). Furthermore, the decoupled head method, by separating the processing flow of these two tasks, allows for more independent and efficient optimization of each task.
[0061] It is understood that the specific method of attention processing on the convolutional features of the base image in step S131 above is not limited. For example, in an alternative implementation, in order to further improve the ability to capture semantic features and enable effective identification and differentiation of similar features, step S131 above may further include steps S131a and S131b, wherein the specific contents of each step are as follows.
[0062] Step S131a: Perform global attention processing on the convolutional features of the substrate image to form global attention features of the substrate.
[0063] In this embodiment, global attention processing can be performed on the convolutional features of the substrate image to form global attention features of the substrate. This global attention processing includes multi-head self-attention computation based on a moving window. It should be noted that global attention processing can be implemented using Swin-Transformer (Shifted Window Transformer), a variant of the self-attention mechanism for computer vision tasks. It introduces window partitioning and displacement strategies on top of the traditional Transformer, effectively processing both local and global image information, thereby improving performance and efficiency. For example, the convolutional features of the substrate image can be divided into 7×7 windows. Self-attention computation is performed on features within each window, while features outside the window are ignored. Swin-Transformer uses two types of self-attention computation: W-MSA (Window Multi-Head Self-Attention) and SW-MSA (Shifted Window Multi-Head Self-Attention). W-MSA (Window Multi-Head Self-Attention) is the basic self-attention computation method of Swin-Transformer. Within each window, Swin-Transformer executes a multi-head self-attention mechanism. Specifically, for each feature within a window, multiple different attention heads are used to compute the relationship between that feature and other features, resulting in multiple distinct representations. Each head performs computation within its local window, without considering information outside the window. In this way, Swin-Transformer can capture rich feature relationships within a local scope. To overcome the limitations of W-MSA (i.e., each window can only focus on local features, making it difficult to capture global information across windows), Swin-Transformer introduces a "window shifting" mechanism. Specifically, after performing one layer of W-MSA computation, Swin-Transformer shifts the window's position and then performs self-attention computation at the new position. This allows cross-window features to be captured because each new window position overlaps with a portion of the original position. Based on this, by alternating between W-MSA and SW-MSA, Swin-Transformer can compute attention within a local window while simultaneously introducing global information across windows at multiple stages, effectively distinguishing similar backgrounds in different regions and improving the accuracy of image understanding.
[0064] Step S131b: Perform local attention processing on the convolutional features of the substrate image to form substrate image attention features.
[0065] In this embodiment, after obtaining the convolutional features of the substrate image, local attention processing can be further applied to the convolutional features of the substrate image to form substrate image attention features. The local attention processing is implemented based on a coordinate attention mechanism.
[0066] It is understood that in step S131b above, the specific method of performing local attention processing on the convolutional features of the base image is not limited. For example, in an alternative implementation, in order to further improve the ability to capture edge information of small targets, step S131b above may further include steps b1, b2, b3, b4 and b5, wherein the specific contents of each step are as follows.
[0067] Step b1: Perform pooling processing on the convolutional features of the base image along the row direction to form the first base image pooling feature, and perform pooling processing on the convolutional features of the base image along the column direction to form the second base image pooling feature.
[0068] In the embodiments of this application, combined with Figure 5 As shown, the convolutional features of the base image can be pooled along the row direction (e.g., the X-axis direction) to form the first base image pooled features. For example, average pooling can be performed to obtain a feature vector of size C×H×1. Then, the convolutional features of the base image can be pooled along the column direction (e.g., the Y-axis direction) to form the second base image pooled features. For example, average pooling can be performed to obtain a feature vector of size C×1×W. Here, C represents the dimension, and H and W represent the height and width, respectively.
[0069] Step b2 involves splicing the first base plate pooling feature and the second base plate pooling feature to form a spliced base plate pooling feature, and then performing convolutional dimensionality reduction processing on the spliced base plate pooling feature to obtain a base plate convolutional dimensionality reduction feature.
[0070] In this embodiment, after obtaining the first and second base-plate pooling features, the first and second base-plate pooling features can be concatenated to form a concatenated base-plate pooling feature. Furthermore, the concatenated base-plate pooling feature is subjected to convolutional dimensionality reduction processing to obtain a base-plate convolutional dimensionality-reduced feature. For example, dimensionality reduction can be performed using a 1×1 convolutional layer, and the result of the dimensionality reduction processing can be subjected to BatchNorm (batch normalization) and Non-linear (non-linear) processing to obtain the base-plate convolutional dimensionality-reduced feature.
[0071] Step b3 involves splitting the base plate convolutional dimensionality reduction features to form a first base plate splitting feature and a second base plate splitting feature, and then performing convolutional dimensionality increase processing on the first base plate splitting feature and the second base plate splitting feature to form a first base plate dimensionality increase feature and a second base plate dimensionality increase feature.
[0072] In this embodiment of the application, after obtaining the base plate convolutional dimensionality reduction feature, the base plate convolutional dimensionality reduction feature can be split to form a first base plate split feature and a second base plate split feature. Then, the first base plate split feature and the second base plate split feature are respectively subjected to convolutional dimensionality increase processing (for example, dimensionality increase processing can be performed through a 1×1 convolutional layer) to form a first base plate dimensionality increase feature and a second base plate dimensionality increase feature.
[0073] Step b4: Perform attention mapping on the first substrate upscaling feature and the second substrate upscaling feature respectively to form the first attention weight and the second attention weight.
[0074] In this embodiment, after obtaining the first and second upgraded features of the substrate, attention mapping can be performed on the first and second upgraded features to form a first attention weight and a second attention weight. That is, attention mapping can be performed on the first upgraded feature to form a first attention weight, and attention mapping can be performed on the second upgraded feature to form a second attention weight. It should be noted that attention mapping can be implemented using functions such as the Sigmoid function to map each parameter in the feature to a weight at the corresponding position. It should also be noted that attention mapping can refer to gating mapping, and the resulting first and second attention weights can refer to first and second gated weights.
[0075] Step b5: Based on the first attention weight and the second attention weight, the convolutional features of the substrate image are adjusted for attention to form the substrate image attention features.
[0076] In this embodiment of the application, attention adjustment can be performed on the convolutional features of the substrate image based on the first attention weight and the second attention weight to form substrate image attention features. For example, ;in, The attention features of the base plate image, The convolutional features of the base image, and These refer to the first attention weight and the second attention weight, respectively.
[0077] Fourthly, regarding the method for detecting the floor components of the high-speed train, it should be further explained that the above steps S120 and S130 can be implemented by a trained neural network model, such as a pre-trained floor component detection model, which can be used to obtain the floor component detection data based on the target floor image.
[0078] It should be noted that the specific training method of the base plate component monitoring model is not limited and can be selected according to actual needs.
[0079] For example, in an alternative implementation, in order to address the problem that the convergence direction of the model during training is biased towards normal samples due to the extreme scarcity of fault samples in the scenario of the EMU floor, resulting in a low fault recall rate, the above-mentioned EMU floor component detection method may further include steps S140 and S150, wherein the specific contents of each step are as follows.
[0080] Step S140: Obtain a sample base plate image, segment the sample base plate image to form multiple local base plate images, scale each local base plate image to form a corresponding scaled base plate image, and combine the multiple scaled base plate images based on the distribution relationship of the multiple local base plate images in the sample base plate image to form multiple combined base plate images.
[0081] In this embodiment, a sample substrate image can be acquired and segmented to form multiple local substrate images. Each local substrate image is then scaled to form a corresponding scaled substrate image. Based on the distribution of the multiple local substrate images within the sample substrate image, the scaled substrate images are combined to form multiple combined substrate images. For example, the original high-resolution image (i.e., the sample substrate image, such as 1920x1080) can be divided into a 3x3 grid. Each grid cell image is scaled to its original size, and brightness and noise are randomly adjusted. Subsequently, the nine processed small images are stitched back together in their original order to form a single large image. Based on this, the relative pixel proportion of small targets in the image can be significantly increased without changing the absolute size of the target, and the data volume can be expanded.
[0082] Step S150: Based on the multiple combined base plate images and corresponding base plate labels, train the candidate component detection model to form the base plate component detection model.
[0083] In this embodiment of the application, after obtaining the plurality of combined base plate images, the candidate component detection model can be trained based on the plurality of combined base plate images and the corresponding base plate labels to form the base plate component detection model.
[0084] For example, during training, the loss function can be: Total Loss = λ1 * L_SIoU + λ2 * L_Foca, where L_SIoU is the localization loss, L_Focal is the classification loss (α = 0.25, a balancing factor used to adjust the weights of positive and negative samples, γ = 2.0, an adjustment factor used to control the loss weights of easily classified and difficult-to-classify samples), with λ1 = 0.05 and λ2 = 0.95. Additionally, the SGD optimizer is used with an initial learning rate of 0.01, momentum of 0.937, and weight decay of 0.0005. The batch size is 32, and training lasts for 300 epochs, using cosine annealing for learning rate scheduling.
[0085] It is understood that the specific method of training the candidate component detection model in step S150 above is not limited. For example, in an alternative implementation, in order to further improve the problem of sample imbalance, step S150 above may further include steps S151, S152 and S153, wherein the specific contents of each step are as follows.
[0086] Step S151: Using the candidate component detection model, perform convolution feature extraction on the multiple combined base plate images to obtain the corresponding multiple combined image convolution features, and, based on the preset feature similarity judgment conditions, determine at least one similar feature combination among the multiple combined image convolution features.
[0087] In this embodiment of the application, a candidate component detection model can be used to extract convolutional features from the multiple combined base plate images (refer to the relevant explanation of step S120 above) to obtain the corresponding multiple combined image convolutional features. Based on a preset feature similarity determination condition, at least one similar feature combination is determined from the multiple combined image convolutional features. Each of the similar feature combinations includes at least two combined image convolutional features whose similarity relationship satisfies the preset condition. For example, two combined image convolutional features with a cosine similarity greater than a threshold can be determined as a similar feature combination.
[0088] Step S152: For each of the similar feature combinations, the candidate component detection model is used to perform feature fusion processing on the convolutional features of at least two combined images in the similar feature combination, and based on the fused features, the restored base plate image corresponding to the similar feature combination is generated.
[0089] In this embodiment, after obtaining the at least one similar feature combination, for each similar feature combination, the candidate component detection model performs feature fusion processing on at least two combined image convolutional features in the similar feature combination, and based on the fused features, generates (restores or maps the features to the image space) the restored substrate image corresponding to the similar feature combination. Exemplarily, the specific method of fusing the combined image convolutional features is not limited; for example, it can be achieved through manifold interpolation: fused features generated in the feature space... .in To follow a Beta distribution The random coefficients.
[0090] Step S153: Based on the multiple combined base plate images, the restored base plate images corresponding to each similar feature combination, and the corresponding base plate labels, the candidate component detection model is trained to obtain the base plate component detection model.
[0091] In this embodiment, after obtaining the reconstructed base plate image, the candidate component detection model can be trained based on the multiple combined base plate images, the reconstructed base plate images corresponding to each similar feature combination, and the corresponding base plate labels to obtain a base plate component detection model. That is, the multiple combined base plate images and the reconstructed base plate images corresponding to each similar feature combination can be used as samples. Then, the candidate component detection model detects these samples to obtain corresponding output data. The error between the output data and the corresponding base plate label is then calculated to obtain the corresponding loss. The model parameters of the candidate component detection model are then adjusted along the direction of reducing this loss until the loss converges or the number of training iterations reaches a threshold, thereby obtaining the base plate component detection model.
[0092] In this embodiment of the application, corresponding to the above-described method for detecting EMU floor components applied to the electronic device, a computer-readable storage medium is also provided. This computer-readable storage medium stores a computer program, which, when executed, performs each step of the EMU floor component detection method. The steps executed by the aforementioned computer program are not described in detail here, but can be found in the preceding explanation of the EMU floor component detection method.
[0093] In summary, the method, equipment, and medium for detecting EMU (Electric Multiple Unit) floorboard components provided in this application first acquire a target floorboard image obtained by image acquisition of the target EMU floorboard; second, convolutional feature extraction is performed on the target floorboard image to form floorboard image convolutional features. During the convolutional feature extraction process, at least switchable dilated convolution is used to adjust the receptive field to capture multi-scale contextual information in the target floorboard image; then, the floorboard image convolutional features are detected and output to obtain floorboard component detection data. This floorboard component detection data reflects at least one of the following: whether components on the target EMU floorboard are missing, and the location of the components on the floorboard. Based on the above, since the receptive field is adjusted at least by switchable dilated convolution during the convolutional feature extraction process to capture multi-scale contextual information in the target floorboard image, more detailed features can be captured, improving the semantic representation accuracy of the obtained floorboard image convolutional features. This ensures that the detected floorboard component detection data has high reliability, thereby improving the problem of relatively low reliability in EMU floorboard component detection in the prior art.
[0094] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0095] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0096] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0097] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method for inspecting components of a high-speed train floor, characterized in that, include: Acquire the target floor image obtained by image acquisition of the target EMU floor; Convolutional feature extraction is performed on the target substrate image to form substrate image convolutional features. During the convolutional feature extraction process, the receptive field is adjusted at least by switchable dilated convolution to capture multi-scale contextual information in the target substrate image. The convolutional features of the base plate image are detected and output to obtain base plate component detection data. The base plate component detection data is used to reflect at least one of the following: whether there are missing components on the base plate of the target EMU and the location of the components on the base plate.
2. The method for detecting components of a high-speed train floor as described in claim 1, characterized in that, The step of extracting convolutional features from the target substrate image to form convolutional features of the substrate image includes: The target substrate image is downsampled and convolved through at least one first convolutional layer to form a first substrate convolutional feature; The first base plate convolutional feature is formed by performing switchable dilated convolution on the first base plate convolutional feature through at least one second convolutional layer. Based on the second base plate convolution features, the base plate image convolution features are determined.
3. The method for detecting components of a high-speed train floor as described in claim 2, characterized in that, The step of performing switchable dilated convolution on the first base plate convolutional features through at least one second convolutional layer to form second base plate convolutional features includes: The first base plate convolution feature is dilated by multiple dilated convolution units included in the second convolution layer to obtain multiple base plate dilated convolution features, wherein the multiple dilated convolution units have different dilation rates. The first base plate convolutional features are subjected to global pooling and convolution processing to obtain the target weight distribution, wherein the number of weights in the target weight distribution is equal to the number of the plurality of dilated convolutional units, and there is a one-to-one correspondence between them; Based on the weights in the target weight distribution, the multiple base plate hole convolution features are weighted and summed to achieve gating adjustment, thereby obtaining the second base plate convolution feature.
4. The method for detecting components of a high-speed train floor as described in claim 1, characterized in that, The step of detecting and outputting the convolutional features of the base plate image to obtain the base plate component detection data includes: Attention processing is applied to the convolutional features of the substrate image to enhance the features and form attention features of the substrate image. The attention processing includes global attention and local attention. The attention features of the base plate image are detected and output to obtain the base plate component detection data.
5. The method for detecting components of a high-speed train floor as described in claim 4, characterized in that, The step of performing attention processing on the convolutional features of the substrate image to achieve feature enhancement and form attention features of the substrate image includes: Global attention processing is performed on the convolutional features of the substrate image to form global attention features of the substrate, wherein the global attention processing includes multi-head self-attention calculation based on a moving window; The convolutional features of the substrate image are subjected to local attention processing to form substrate image attention features, wherein the local attention processing is implemented based on the coordinate attention mechanism.
6. The method for detecting components of a high-speed train floor as described in claim 5, characterized in that, The step of performing local attention processing on the convolutional features of the substrate image to form substrate image attention features includes: The convolutional features of the base image are pooled along the row direction to form the first base image pooling feature, and the convolutional features of the base image are pooled along the column direction to form the second base image pooling feature. The first base plate pooling feature and the second base plate pooling feature are spliced together to form a spliced base plate pooling feature, and the spliced base plate pooling feature is subjected to convolution dimensionality reduction processing to obtain a base plate convolution dimensionality reduction feature. The base plate convolutional dimensionality reduction features are split to form a first base plate split feature and a second base plate split feature. The first base plate split feature and the second base plate split feature are then subjected to convolutional dimensionality increase processing to form a first base plate dimensionality increase feature and a second base plate dimensionality increase feature. The first and second base plate dimensionality-upgraded features are respectively subjected to attention mapping to form a first attention weight and a second attention weight; Based on the first attention weight and the second attention weight, attention adjustment is performed on the convolutional features of the substrate image to form substrate image attention features.
7. The method for detecting components of a high-speed train floor according to any one of claims 1-6, characterized in that, The method for detecting floor components of a high-speed train also includes a training step for a floor component detection model. This model is a neural network model and is used to obtain floor component detection data based on the target floor image. The training step includes: A sample base plate image is acquired and segmented to form multiple local base plate images. Each local base plate image is scaled to form a corresponding scaled base plate image. Based on the distribution relationship of the multiple local base plate images in the sample base plate image, the multiple scaled base plate images are combined to form multiple combined base plate images. Based on the multiple combined base plate images and corresponding base plate labels, the candidate component detection model is trained to form the base plate component detection model.
8. The method for detecting components of a high-speed train floor as described in claim 7, characterized in that, The step of training the candidate component detection model based on the multiple combined base plate images and corresponding base plate labels to form the base plate component detection model includes: The candidate component detection model is used to extract convolutional features from the multiple combined base plate images to obtain the corresponding multiple combined image convolutional features. Based on the preset feature similarity judgment conditions, at least one similar feature combination is determined from the multiple combined image convolutional features. Each of the similar feature combinations includes at least two combined image convolutional features whose similarity relationship satisfies the preset conditions. For each of the similar feature combinations, the candidate component detection model performs feature fusion processing on the convolutional features of at least two combined images in the similar feature combination, and generates the restored base image corresponding to the similar feature combination based on the fused features. Based on the multiple combined base plate images, the restored base plate images corresponding to each similar feature combination, and the corresponding base plate labels, the candidate component detection model is trained to obtain a base plate component detection model.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the method for detecting components of the EMU floor as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a computer program that, when executed, performs the method for detecting components of the EMU floor as described in any one of claims 1-8.