Target detection method, system, electronic device and storage medium for vehicle running part

Abstract

The application discloses a vehicle running part target detection method and system, electronic equipment and storage medium, and belongs to the technical field of computer vision technology. The vehicle running part target detection method comprises the following steps: acquiring a sample image of a vehicle running part, and preprocessing the sample image to obtain a training data set; a double-branch network structure is built, and the double-branch network structure is trained by using the training data set to obtain an image detection model; wherein the double-branch network structure comprises a preprocessing module, a high-resolution branch, a low-resolution branch, a double-side interaction module, a unified fusion module and a feature detection head; if a target detection task is received, a running part image is obtained by photographing the vehicle running part, and the image detection model is used for target detection on the running part image to obtain a detection frame of a target component. The application can improve the target detection accuracy of the vehicle running part.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, system, electronic device and storage medium for target detection of the running gear of a vehicle. Background Technology

[0002] With the development of artificial intelligence, in order to reduce manual workload and improve train maintenance efficiency, more and more inspection equipment is being used in train maintenance. Related technologies typically involve object detection on images of the vehicle's running gear, followed by fault identification based on the detection results. However, because small targets such as bolts and discs exist in the running gear, these targets occupy even smaller areas, have less contour and color information, and are more easily lost during feature extraction, resulting in lower object detection accuracy.

[0003] Therefore, how to improve the accuracy of target detection on the running gear of a vehicle is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a target detection method, system, electronic device, and storage medium for the running gear of a vehicle, which can improve the accuracy of target detection on the running gear of a vehicle.

[0005] To address the aforementioned technical problems, this application provides a target detection method for the running gear of a vehicle, the method comprising:

[0006] Acquire sample images of the vehicle's running gear and preprocess the sample images to obtain a training dataset;

[0007] A dual-branch network structure is constructed, and the image detection model is obtained by training the dual-branch network structure using the training dataset. The dual-branch network structure includes a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head. The preprocessing module extracts image features from the input image and uses the processing result as input to the high-resolution branch and the low-resolution branch. The high-resolution branch includes n residual blocks with a step size of 1, and the low-resolution branch includes n residual blocks with a step size greater than 1. The bilateral interaction module enables feature interaction between the high-resolution branch and the low-resolution branch. The unified fusion module fuses the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map. The feature detection head extracts features from the fused feature map to obtain detection box information.

[0008] If a target detection task is received, the vehicle running gear is photographed to obtain an image of the running gear, and the image detection model is used to perform target detection on the running gear image to obtain the detection box of the target component.

[0009] Optionally, the sample images are preprocessed to obtain a training dataset, including:

[0010] Select training samples from the sample images;

[0011] Add annotation information in a preset format to the target component in the training sample; wherein, the annotation information in the preset format includes the center point of the detection box, the bounding vector of the detection box, the width and height of the minimum bounding rectangle of the detection box, and the rotation angle of the detection box;

[0012] The training dataset is obtained by performing data augmentation operations on the training samples.

[0013] Optionally, the image detection model is obtained by training the dual-branch network structure using the training dataset, including:

[0014] Select target images from the training dataset;

[0015] The preprocessing module is used to extract features from the target image to obtain a first feature map; wherein, the preprocessing module includes multiple residual blocks connected in sequence;

[0016] The first feature map is extracted using the high-resolution branch and the low-resolution branch respectively, and the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch are interacted using the bilateral interaction module.

[0017] The unified fusion module is used to fuse the second feature map and the third feature map to obtain a fused feature map; wherein, the second feature map is the output result of the high-resolution branch, and the third feature map is the output result of the low-resolution branch;

[0018] The feature detection head is used to extract features from the fused feature map to obtain detection box information; wherein, the detection box information includes the detection box center point, the number of detection box categories, the detection box center point offset, the detection box bounding vector, and the detection box rotation angle;

[0019] The loss function is used to calculate the loss value between the detection box information and the annotation information of the target image, and the network parameters of the feature detection head are adjusted according to the loss value;

[0020] Determine whether all images in the training dataset have been selected; if yes, set the trained dual-branch network structure as an image detection model; otherwise, proceed to the step of selecting target images from the training dataset.

[0021] Optionally, the dual-branch network structure includes n-1 bilateral interaction modules; each bilateral interaction module includes a first hop connection structure, a second hop connection structure, a downsampling structure, and an upsampling structure;

[0022] Accordingly, the bilateral interaction module is used to interact with the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch, including:

[0023] The first intermediate feature is output by the i-th residual block in the high-resolution branch. The first intermediate feature is processed by the first skip connection structure of the i-th bilateral interaction module to obtain the second intermediate feature. The second intermediate feature is processed by the downsampling structure of the i-th bilateral interaction module to obtain the third intermediate feature; 1≤i≤n-1;

[0024] The fourth intermediate feature is determined by the output of the i-th residual block in the low-resolution branch. The fourth intermediate feature is processed by the second skip connection structure of the i-th bilateral interaction module to obtain the fifth intermediate feature. The fifth intermediate feature is processed by the upsampling structure of the i-th bilateral interaction module to obtain the sixth intermediate feature.

[0025] The fusion result of the third intermediate feature and the fifth intermediate feature is set as the input of the (i+1)th residual block in the low-resolution branch;

[0026] The fusion result of the sixth intermediate feature and the second intermediate feature is set as the input of the (i+1)th residual block in the high-resolution branch.

[0027] Optionally, the unified fusion module includes a bilinear interpolation upsampling module, an attention module, and a weighting module;

[0028] Accordingly, the unified fusion module is used to fuse the second feature map and the third feature map to obtain a fused feature map, including:

[0029] The third feature map is upsampled using a bilinear interpolation upsampling module to obtain the fourth feature map;

[0030] The feature weights of the second and fourth feature maps are determined using an attention module.

[0031] The second feature map and the fourth feature map are weighted and fused according to the feature weights using a weighting module to obtain the fused feature map.

[0032] Optionally, determining the feature weights of the second feature map and the fourth feature map using the attention module includes:

[0033] The attention module is used to perform average pooling, max pooling, concat, convolution, and sigmoid functions on the second and fourth feature maps to obtain the feature weights of the second and fourth feature maps.

[0034] Optionally, the feature detection head includes a heatmap detection head, an offset regression head, a bounding box regression head, and a rotation factor regression head;

[0035] Accordingly, the feature detection head is used to extract features from the fused feature map to obtain detection box information, including:

[0036] The heatmap detection head is used to extract the center point of the detection box and the number of detection box categories from the fused feature map.

[0037] The offset regression head is used to extract the offset of the detection box center point from the fused feature map;

[0038] The bounding box vectors of the detection boxes are extracted from the fused feature map using the bounding box regression head;

[0039] The rotation angle of the detection box is extracted from the fused feature map using the rotation factor regression head.

[0040] This application also provides a target detection system for the running gear of a vehicle, the system comprising:

[0041] The training set construction module is used to acquire sample images of the vehicle's running gear and preprocess the sample images to obtain a training dataset.

[0042] A model training module is used to build a dual-branch network structure and train the dual-branch network structure using the training dataset to obtain an image detection model. The dual-branch network structure includes a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head. The preprocessing module extracts image features from the input image and uses the processing result as input to the high-resolution branch and the low-resolution branch. The high-resolution branch includes n residual blocks with a step size of 1, and the low-resolution branch includes n residual blocks with a step size greater than 1. The bilateral interaction module enables feature interaction between the high-resolution branch and the low-resolution branch. The unified fusion module fuses the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map. The feature detection head extracts features from the fused feature map to obtain bounding box information.

[0043] The detection task execution module is used to, upon receiving a target detection task, capture an image of the vehicle's running gear to obtain an image of the running gear, and then use the image detection model to perform target detection on the image of the running gear to obtain a detection box of the target component.

[0044] This application also provides a storage medium storing a computer program thereon, which, when executed, implements the steps of the target detection method for the vehicle running gear described above.

[0045] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the target detection method for the vehicle running gear described above.

[0046] This application provides a target detection method for a vehicle running gear, comprising: acquiring sample images of the vehicle running gear and preprocessing the sample images to obtain a training dataset; constructing a dual-branch network structure and training the dual-branch network structure using the training dataset to obtain an image detection model; wherein, the dual-branch network structure includes a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head; the preprocessing module is used to extract image features from the input image and use the processing result as the input to the high-resolution branch and the low-resolution branch; the high-resolution branch includes n residual blocks with a step size of 1, and the low-resolution branch includes n residual blocks with a step size greater than 1; the bilateral interaction module is used to realize feature interaction between the high-resolution branch and the low-resolution branch; the unified fusion module is used to fuse the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map; the feature detection head is used to extract features from the fused feature map to obtain detection box information; if a target detection task is received, the vehicle running gear is photographed to obtain an image of the running gear, and the image detection model is used to perform target detection on the image of the running gear to obtain a detection box of the target component.

[0047] The solution provided in this application constructs a dual-branch network structure. The high-resolution and low-resolution branches in this structure can extract features at different resolutions from the preprocessing module's results. Furthermore, the bilateral interaction module within the dual-branch network structure enables feature interaction between the high-resolution and low-resolution branches. After acquiring sample images of the vehicle's running gear, this application generates a training dataset, which is then used to train the dual-branch network structure to obtain an image detection model. Because the dual-branch network structure includes high-resolution and low-resolution branches, it can extract features from small target objects, thus enabling the trained image detection model to detect small targets. After training, if a target detection task is received, the trained image detection model can be used to detect images of the running gear. Therefore, this application can improve the accuracy of target detection in the vehicle's running gear. This application also provides a target detection system for the vehicle's running gear, a storage medium, and an electronic device, all possessing the aforementioned beneficial effects, which will not be elaborated upon further here. Attached Figure Description

[0048] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart of a target detection method for a vehicle running gear provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of a dual-branch network structure provided in an embodiment of this application;

[0051] Figure 3 A flowchart illustrating a method for detecting small targets in the running gear of rail transit vehicles, provided as an embodiment of this application.

[0052] Figure 4 A schematic diagram illustrating the result of adding annotations to a training sample provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of another dual-branch network structure provided in an embodiment of this application;

[0054] Figure 6 This is a schematic diagram of the structure of a bilateral interaction module provided in an embodiment of this application;

[0055] Figure 7 This is a schematic diagram of the structure of a unified fusion module provided in an embodiment of this application;

[0056] Figure 8 This is a schematic diagram of the structure of an attention module provided in an embodiment of this application;

[0057] Figure 9 A schematic diagram of a small target image detection device for the running gear of a rail transit vehicle, provided in an embodiment of this application;

[0058] Figure 10 A schematic diagram showing the installation location of an image acquisition module provided in an embodiment of this application;

[0059] Figure 11 This is a schematic diagram of the structure of a detection module provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Please see below. Figure 1 , Figure 1 This is a flowchart of a target detection method for a vehicle running gear provided in an embodiment of this application.

[0062] Specific steps may include:

[0063] S101: Acquire sample images of the vehicle's running gear and preprocess the sample images to obtain a training dataset;

[0064] This embodiment can be applied to electronic devices with model training capabilities. This step involves capturing sample images of the vehicle's running gear, and preprocessing these sample images to obtain a training dataset. The training dataset includes multiple samples used to train the model. The preprocessing operation can annotate target components in the sample images and add detection boxes for each target component.

[0065] S102: Construct a dual-branch network structure and train the dual-branch network structure using the training dataset to obtain an image detection model;

[0066] This embodiment can create a dual-branch network structure; please refer to [link / reference]. Figure 2 , Figure 2This is a schematic diagram of a dual-branch network structure provided in an embodiment of this application. The dual-branch network structure includes: a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head.

[0067] The preprocessing module can simultaneously extract and learn low-level features from the image, reducing the spatial resolution of the image so that subsequent layers can better understand and process the data. Specifically, the preprocessing module extracts image features from the input image and uses the processing results as input to the high-resolution branch and the low-resolution branch. The high-resolution branch includes n residual blocks with a stride of 1, and the low-resolution branch includes n residual blocks with a stride greater than 1. These residual blocks include convolutional layers, batch normalization layers, and activation function layers. As a feasible implementation, the convolutional layers can be 3×3 convolutional layers. In the high-resolution branch, the stride of the 3x3 convolutions is 2, and the number of channels is doubled in each convolution. In the low-resolution branch, the stride of the 3x3 convolutions is 1, and the number of channels remains constant in each convolution.

[0068] The bilateral interaction module is used to realize feature interaction between the high-resolution branch and the low-resolution branch. The unified fusion module is used to fuse the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map. The feature detection head is used to extract features from the fused feature map to obtain detection box information.

[0069] In this embodiment, the dual-branch network structure can be iteratively trained using a training dataset. The trained dual-branch network structure is the image detection model. The image detection model is able to detect targets of interest in an image.

[0070] S103: If a target detection task is received, the vehicle running gear is photographed to obtain an image of the running gear, and the image detection model is used to perform target detection on the image of the running gear to obtain the detection box of the target component.

[0071] After training the image detection model, if a target detection task is received, images of the vehicle's running gear can be captured. These images can then be input into the image detection model, which outputs detection bounding boxes for the target components. After obtaining the detection bounding boxes, component fault detection can be performed based on the images corresponding to the boxes to determine if any faults exist in the vehicle's running gear.

[0072] The solution provided in this embodiment constructs a dual-branch network structure. The high-resolution and low-resolution branches in this structure can extract features at different resolutions from the preprocessing module's results. Furthermore, the bilateral interaction module within the dual-branch network structure enables feature interaction between the high-resolution and low-resolution branches. This embodiment generates a training dataset after acquiring sample images of the vehicle's running gear, and then uses this dataset to train the dual-branch network structure to obtain an image detection model. Because the dual-branch network structure includes high-resolution and low-resolution branches, it can extract features from small target objects, thus enabling the trained image detection model to detect small targets. After training, if a target detection task is received, the trained image detection model can be used to detect images of the running gear. Therefore, this embodiment can improve the accuracy of target detection on the vehicle's running gear.

[0073] As for Figure 1 A further description of the corresponding embodiment is that the training dataset can be obtained in the following way: selecting training samples from the sample images; adding annotation information of a preset format to the target components in the training samples; wherein, the annotation information of the preset format includes the center point of the detection box, the bounding vector of the detection box, the width and height of the minimum bounding rectangle of the detection box, and the rotation angle of the detection box; and performing data augmentation operations on the training samples to obtain the training dataset.

[0074] Specifically, in this embodiment, the sample images can be divided into training samples and test samples according to a preset ratio (e.g., 7:3). This embodiment can also add preset format annotation information to the target components in the test samples and perform data augmentation operations to obtain a test dataset. After obtaining the image detection model, this embodiment can use the test sample set to perform inference testing on the image detection model.

[0075] As a feasible implementation method, the image detection model can be obtained by training the dual-branch network structure in the following way:

[0076] Step 1: Select target images from the training dataset;

[0077] Step 2: Use the preprocessing module to extract features from the target image to obtain a first feature map;

[0078] The preprocessing module includes multiple residual blocks connected in sequence; the step size of the residual blocks in the preprocessing module is greater than 1.

[0079] Step 3: Extract features from the first feature map using the high-resolution branch and the low-resolution branch respectively, and use the bilateral interaction module to interact with the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch;

[0080] Furthermore, the n residual blocks in the high-resolution branch and the low-resolution branch correspond one-to-one. The dual-branch network structure includes n-1 bilateral interaction modules. The i-th bilateral interaction module is used to perform feature interaction on the i residual blocks in the high-resolution branch and the low-resolution branch, and uses the interaction result as the input of the i+1 residual blocks in the high-resolution branch and the low-resolution branch, where 1≤i≤n-1.

[0081] Each of the aforementioned bilateral interaction modules includes a first hop connection structure, a second hop connection structure, a downsampling structure, and an upsampling structure. The first and second hop connection structures each include a 3×3 convolutional layer, a batch normalization layer, and an activation function. The downsampling structure includes a 1×1 convolutional layer and a batch normalization layer, and the upsampling structure includes a 1×1 convolutional layer, a batch normalization layer, and a bilinear upsampling layer.

[0082] Accordingly, the process of feature interaction using the bilateral interaction module is as follows:

[0083] In the high-resolution branch, the i-th residual block outputs a first intermediate feature. This first intermediate feature is processed using the first skip connection structure of the i-th bilateral interaction module to obtain a second intermediate feature. The second intermediate feature is then processed using the downsampling structure of the i-th bilateral interaction module to obtain a third intermediate feature. Similarly, in the low-resolution branch, the i-th residual block outputs a fourth intermediate feature. This fourth intermediate feature is processed using the second skip connection structure of the i-th bilateral interaction module to obtain a fifth intermediate feature. The fifth intermediate feature is then processed using the upsampling structure of the i-th bilateral interaction module to obtain a sixth intermediate feature. The fusion result of the third and fifth intermediate features is set as the input to the (i+1)-th residual block in the low-resolution branch. Finally, the fusion result of the sixth and second intermediate features is set as the input to the (i+1)-th residual block in the high-resolution branch.

[0084] Step 4: Use the unified fusion module to fuse the second feature map and the third feature map to obtain a fused feature map;

[0085] The second feature map is the output of the high-resolution branch, and the third feature map is the output of the low-resolution branch.

[0086] Specifically, the unified fusion module includes a bilinear interpolation upsampling module, an attention module, and a weighting module. Correspondingly, the process of using the unified fusion module to obtain a fused feature map includes: using the bilinear interpolation upsampling module to perform an upsampling operation on the third feature map to obtain a fourth feature map; using the attention module to determine the feature weights of the second and fourth feature maps; and using the weighting module to perform weighted fusion of the second and fourth feature maps according to the feature weights to obtain the fused feature map.

[0087] In this embodiment, the attention module can be used to perform average pooling, max pooling, concat (connection) operations, convolution operations, and sigmoid function (an activation function) processing on the second feature map and the fourth feature map to obtain the feature weights of the second feature map and the fourth feature map.

[0088] Step 5: Use the feature detection head to extract features from the fused feature map to obtain detection box information;

[0089] The detection box information includes the detection box center point, the number of detection box categories, the detection box center point offset, the detection box border vector, and the detection box rotation angle.

[0090] Step 6: Calculate the loss value between the detection box information and the annotation information of the target image using a loss function, and adjust the network parameters of the feature detection head according to the loss value.

[0091] The aforementioned feature detection head includes a heatmap detection head, an offset regression head, a bounding box regression head, and a rotation factor regression head. Correspondingly, the feature extraction process includes: using the heatmap detection head to extract the center point of the detection box and the number of detection box categories from the fused feature map; using the offset regression head to extract the offset of the center point of the detection box from the fused feature map; using the bounding box regression head to extract the bounding box vector from the fused feature map; and using the rotation factor regression head to extract the rotation angle of the detection box from the fused feature map.

[0092] Step 7: Determine whether all images in the training dataset have been selected; if yes, set the trained dual-branch network structure as an image detection model; if not, proceed to step 1.

[0093] The process described in the above embodiments is illustrated below through examples in practical applications.

[0094] As one of the key components of a train, the running gear mainly includes the wheel and axle system, motor system, braking system, and coupling system. It plays a role in load-bearing, running, and traction in the entire vehicle system. Its operating status is crucial to the safe operation of rail transit vehicles. Therefore, abnormal detection of the running gear is a particularly important part of daily locomotive and train maintenance.

[0095] Currently, the inspection of running gear in this field is still mainly done manually due to the complex structure of the running gear system and the numerous inspection items. With the development of the rail transit industry, traditional maintenance methods are no longer sufficient to meet the requirements of safe train operation, and relying solely on the experience of maintenance personnel is insufficient to promptly eliminate potential hazards. Compared with manual methods, computer vision-based methods have irreplaceable advantages such as speed and low cost. With the rapid development of deep learning, image detection methods represented by deep convolutional neural networks have become mainstream. The principle is to select an image segment by using a bounding box, input it into a convolutional neural network model to identify the image segment, and then automatically output the required target location and category. This method has advantages such as good detection performance and end-to-end automated learning, and can effectively achieve the localization of key components in rail transit scenarios. Therefore, by combining computer vision technology to change the fault diagnosis of the running gear of rail transit vehicles to an automatic detection mode, fault phenomena of the running gear of rail transit vehicles can be detected in a timely manner and automated diagnosis can be achieved. This is conducive to improving the maintenance efficiency of existing vehicles, discovering hidden dangers of components in advance and eliminating their potential threats to train safety, which is of great significance to the safety of rail transit. It can also reduce the manpower input in the fault diagnosis of the running gear of rail transit vehicles.

[0096] Currently, the mainstream method for detecting the running gear is to use deep neural networks. This method uses a line scan camera to take pictures of the side or bottom of the vehicle and then uses the obtained two-dimensional images to detect key components. However, in practical applications, although the mainstream method can detect conventional large target objects such as wheels and axles, the detection accuracy for small target objects such as bolts, air duct clamps, and cables is far from meeting the requirements. Therefore, it is necessary to improve the existing methods to increase the recognition rate of specific small targets and establish a device for automated detection of the running gear of rail transit vehicles to improve train maintenance efficiency and ensure train operation safety. Specifically, the technical problems to be solved are as follows: (1) For conventional small targets such as bolts and discs, unlike large target objects with obvious outlines, they occupy a smaller area, have less outline and color information, and are more likely to be lost during feature extraction. Therefore, the feature extraction network designed for large targets in the existing methods is no longer applicable to this scenario, and a high-resolution feature extraction network needs to be designed for small targets. (2) For small, tilted targets such as duct clips and cables, the tilt angles vary. Conventional horizontal detection boxes not only fail to accurately delineate the object's location but also label redundant background information, further reducing the network's recognition performance. Therefore, existing horizontal detection boxes cannot effectively detect this target, and an adaptive tilt angle detection method needs to be designed for small, tilted targets.

[0097] To address the technical problems existing in the aforementioned related technologies, this application provides a small target image detection scheme for the running gear of rail transit vehicles. To address the weak feature extraction capability of small target objects, this embodiment proposes a dual-branch network solution. While maintaining the original single-branch feature extraction, a continuously refined high-resolution branch is introduced to extract detailed features of small target objects, thereby enhancing the network's ability to recognize small targets. In the dual-branch fusion stage, a unified attention fusion module is used to handle the resolution and feature differences between the two branches, thereby promoting information interaction and achieving feature fusion. To address the problem that horizontal detection boxes are not suitable for detecting tilted small targets, this embodiment proposes a single-stage rotating small target detection method without anchor boxes. A rotating Gaussian kernel is used to coarsely locate the center point of the rotating small target object, and then the precise target object center is obtained by regressing the offset of the center point. Finally, the final rotating detection box is obtained by regressing the bounding box vectors in the four directions based on the center point. This method significantly improves the accuracy of detecting tilted objects compared to directly detecting horizontal boxes. This embodiment acquires image data of the running gear of rail transit vehicles, inputs the image data into an image detection model, and outputs the detection results of small targets in the scene to be detected. In this method, a dual-branch network is constructed to enhance the feature extraction capability for common small targets, and a rotated detection box is used to accurately locate tilted small targets, thereby improving the recognition rate of specific small targets. Based on this, an automated detection device for the running gear of rail transit vehicles is established, reducing the labor costs in fault diagnosis and improving train maintenance efficiency.

[0098] To realize the above technical concept, this embodiment proposes a small target image detection device and method for the running gear of rail transit vehicles, so as to improve the detection accuracy of small targets (bolts, clips, etc.) in the running gear.

[0099] like Figure 3 As shown, Figure 3 A flowchart of a method for detecting small targets in the running gear of rail transit vehicles, provided in this application embodiment, specifically includes the following steps:

[0100] S301: Obtain the raw dataset of images of the running gear of rail transit vehicles and divide it into a training sample set and a test sample set;

[0101] Specifically, in this embodiment, a line scan camera can be used to continuously photograph the side or underside of a rail transit vehicle. After the photographs are taken, they are placed in the same folder according to the train number. Then, an image stitching algorithm based on OpenCV is used to stitch the images in each folder into a panoramic image to generate the original image dataset (i.e., sample images of the vehicle running gear). Finally, the dataset is divided into a training sample set and a test sample set in a 7:3 ratio.

[0102] S302: Preprocess the above training sample set of images of the traverse section to construct a training dataset.

[0103] Specifically, the process of annotating the training sample set to generate training labels in this step includes: using a data labeling format based on center coordinate bounding vector regression, and using data augmentation methods such as standardization, random cropping, random scaling, and random horizontal flipping to generate the training dataset.

[0104] This embodiment uses GeoLabel software to label the required detection parts in the training sample set. In this scenario, a bounding box vector regression method based on center coordinates is used, i.e., the data labeling format xy[t, r, b, l, H, W]α. Please refer to... Figure 4 , Figure 4 This diagram illustrates the results of adding annotations to training samples provided in an embodiment of this application. The detection box is defined with a center point, which is located using the heatmap and offset of the regressed target. In the diagram, x and y represent the horizontal and vertical coordinate axes. Boundary vectors are regressed based on the center point in all four quadrants of the Cartesian coordinate system. The network can locate the final detection box by learning the center coordinates of the detection box (i.e., the target box) and the four boundary vectors t, r, b, l (i.e., the bounding vectors of the detection box). However, when the detection box is exactly horizontal, or the vectors are very close to the coordinate axes, the bounding vectors cannot detect the target. Therefore, during model regression, a confidence α (i.e., the rotation angle of the detection box) is added to determine whether the rotated detection box is horizontal. Furthermore, the width W and height H of the minimum bounding rectangle of the rotated box are added to the previously defined parameters. This solves the problem that the bounding vectors cannot locate the target when the detection box is horizontal, thus transforming the detection task into traditional horizontal target detection.

[0105] After annotation, data augmentation is performed on the training sample set to generate the final training dataset. First, the images are standardized to linearly change the pixel values ​​to the range of 0-1, as follows:

[0106]

[0107] Where, ρ i，j This represents the pixel value at position (i, j) in the image, where max and min represent the maximum and minimum values, respectively.

[0108] This embodiment can use a sliding window to perform batch random cropping of images, and then perform random rotation of the cropped pieces by 90°, 180°, 270°, horizontal flipping, vertical flipping, etc. After the processing is completed, the training dataset can be constructed.

[0109] S303: Construct a dual-branch network structure for small target image detection.

[0110] Based on the existing low-resolution branch used for extracting high-level features, a high-resolution branch for extracting small target objects is added. This embodiment achieves deep feature coupling through a bilateral interaction module and a unified fusion module.

[0111] Specifically, step S303 above includes the following 5 sub-steps:

[0112] Sub-step 1: Construct a dual-branch network structure, the complete dual-branch network as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of another dual-branch network structure provided in this application embodiment. This embodiment adds an additional high-resolution branch to a ResNet (a network composed of a single low-resolution branch). The preprocessing module includes two residual blocks with a stride of 2, used to process the input image. The high-resolution branch uses the residual blocks in the ResNet as the basic unit, and each residual block consists of sequentially connected 3×3 convolutions, batch normalization layers, and activation function layers. Specifically, in terms of network structure, the high-resolution branch and the low-resolution branch have a one-to-one correspondence, both consisting of three stages of residual blocks; in terms of implementation details, the residual blocks in the high-resolution branch do not include downsampling operations, always maintaining the image resolution at 1 / 4 of the input image resolution to maintain high-resolution representation, thus enhancing the network's sensitivity to details in the image. The dual-branch network structure also includes a unified fusion module. The feature detection head of the dual-branch network structure includes a heatmap detection head, an offset regression head, a bounding box regression head, and a rotation factor regression head. After processing by the feature detection head, the part detection result (i.e., the detection box of the target part) can be output. In the diagram, 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 represent the resolution ratios.

[0113] Sub-step 2: Construct the bilateral interaction module of the dual-branch network. Please refer to [link / reference]. Figure 6 , Figure 6This is a schematic diagram of the structure of a bilateral interaction module provided in an embodiment of this application. Each bilateral interaction module includes a first hop connection structure, a second hop connection structure, a downsampling structure, and an upsampling structure. The first hop connection structure corresponding to the high-resolution branch includes a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function; the second hop connection structure corresponding to the low-resolution branch includes a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function; the downsampling structure includes a 3×3 convolutional layer and a batch normalization layer; and the upsampling layer includes a 1×1 convolutional layer, a batch normalization layer, and a bilinear interpolation upsampling layer. This represents the SUM (summation) operation. The bilateral interaction module includes high-to-low and low-to-high interactions. For high-to-low interactions, the feature maps in the high-resolution branch are first processed through a skip-connected structure (including 3×3 convolutions, batch normalization layers, ReLU activation functions, and the SUM operation), then downsampled (specifically, 3×3 convolutions with a stride of 2), and finally fused using the SUM operation to serve as the input for the next residual block in the low-resolution branch. For low-to-high interactions, the feature maps in the low-resolution branch are first processed through a skip-connected structure, then upsampled through 1×1 convolutions, batch normalization, and bilinear interpolation, and finally fused using the same SUM operation to serve as the input for the next residual block in the high-resolution branch.

[0114] Sub-step 3: Construct a unified fusion module for the dual-branch network. Please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of a unified fusion module provided in an embodiment of this application. The unified fusion module takes the outputs of a high-resolution branch and a low-resolution branch as inputs. The output of the high-resolution branch serves as the input of low-level features, while the output of the low-resolution branch, after upsampling through bilinear interpolation, serves as the input of high-level features (where high-level features refer to more abstract features, and low-level features refer to more primitive basic features). The unified fusion module utilizes the weights Q generated by the attention module and uses the weights Q to guide the two features through the Mul operation. Finally, the two features are fused through the SUM operation. In the diagram, Mul represents the multiplication operation.

[0115] Sub-step 4: Construct the attention module, please refer to [link / reference]. Figure 8 , Figure 8This is a schematic diagram of an attention module provided in an embodiment of this application. The outputs of the high-resolution branch and the low-resolution branch are input into the attention module, and then subjected to average pooling, max pooling, concat, 3×3 convolution, and sigmoid function processing to obtain the weights Q. Specifically, in this embodiment, the two inputs are processed using max pooling and average pooling to obtain four single-channel data points. These four data points are then concatenated using a concat operation, and finally, convolution and the sigmoid function are used to generate the final weights Q.

[0116] Sub-step 5: Construct the heatmap detection head, offset regression head, bounding box regression head, and rotation factor regression head. These four modules each consist of sequentially connected 3×3 convolutional layers, batch normalization layers, and activation function layers, but they differ in their output dimensions. The heatmap detection head outputs the number of categories and their center positions; the offset regression head outputs the offset of the center position; the bounding box regression head outputs the offset of the bounding box; and the rotation factor regression head outputs the rotation angle. Based on this information, the detection results for the component can be obtained, but only the target's bounding box can be obtained.

[0117] Specifically, the feature map output from sub-step 4 is predicted using a heatmap detection head, offset regression head, bounding box regression head, and rotation factor regression head. The heatmap detection head classifies the target category while regressing the target object's center position (x0, y0), with the confidence level being highest at the target's center position using a Gaussian kernel rotation. The offset regression head regresses the offset (Δx, Δy) from the center position coordinates (x0, y0) obtained from the heatmap detection head, thus obtaining the corrected target center coordinates.

[0118] (x, y)=(x0+Δx, y0+Δy);

[0119] The method of regressing bounding box vectors based on the target center is key to detecting small, rotating targets. After obtaining the center coordinates of the target using a heatmap detection head and an offset regression head, the vector parameters [t, r, b, l] of the four sides of the rotating target bounding box need to be regressed based on these center coordinates. Since the method of regressing bounding box vectors based on the target center can handle target object detection in any direction, the rotation factor regression head determines whether the target bounding box is horizontal by regressing the rotation angle α. When α = 0, it represents a horizontal target bounding box. The bounding box regression head, in addition to regressing the bounding box vectors of the four sides of the rotating rectangle, also regresses the width W and height H of the minimum horizontal bounding rectangle of the target bounding box. The parameter regression of the rotation factor and the minimum horizontal bounding rectangle of the target bounding box solves the problem that directly regressing bounding box vectors based on the target center cannot locate the target bounding box, enabling the network model to detect target objects in any direction. Conventional target detection methods can only obtain horizontal detection boxes, which contain a lot of redundant information, and the labeled component positions are inaccurate. This embodiment introduces a rotation angle on top of the standard four parameters, which can accurately label the position of the rotating target. The introduction of a rotation angle of 0 here is intended to illustrate that the horizontal box appears as a special type of non-horizontal box, and its processing method is completely the same as that of a non-horizontal box.

[0120] The task of the heatmap detection head is to output the number of categories C and the center position (x0, y0). Therefore, the output dimension is three-dimensional, [C, x0, y0], requiring no additional operations. The number of categories C is the sum of all component categories in all images of the traveling parts. This is actually defined manually through annotation; for example, if there are three categories—bolt, disc, and cable—then C should be 3. Each detection result will output one category, such as cable. The center position (x0, y0) is the coordinate of the target component's center position. The task of the offset regression head is to output the offset of the target's center position, so the output dimension is two-dimensional, [Δx, Δy]. The dimensions need to be changed through a reshape (matrix transformation) operation, where Δx is the X-axis coordinate offset and Δy is the Y-axis coordinate offset. The offset regression head is used to correct the center position of the target component. The task of the bounding box regression head is to output the offsets of the four sides of the bounding box relative to the center position, so the output dimension is four-dimensional, [t, r, b, l]. The dimensions need to be changed through a reshape operation. The bounding box regression head determines the center position of the target component and then uses the vector parameters of the four sides to determine the position of the detection box. The rotation factor regression head outputs the rotation angle of the detection box, so the output dimension is one-dimensional, [α], and needs to be changed through a reshape operation. The rotation factor regression head is used to determine the tilt angle of the detection box.

[0121] S304: The training dataset constructed using S302 is used to supervise the training of the dual-branch network for small object image detection, resulting in the corresponding image detection model.

[0122] Specifically, S304 may include the following four sub-steps:

[0123] Sub-step 1: Input the training dataset into the dual-branch network structure for small target image detection described above to obtain the corresponding image detection prediction results.

[0124] Sub-step 2: Based on the prediction results obtained in sub-step 1 and the corresponding true labels in the training set, the difference between the predicted value and the true value is calculated using a loss function. The heatmap detection head uses the Focal loss function to alleviate the problem of imbalance between positive and negative samples, while the other regression heads use Smooth L1 Loss as the loss function. This allows the network to maintain gradient stability when the difference is large during the fitting process and reduce the gradient when the difference is small, which is beneficial to the learning of the network model.

[0125] Sub-step 3: Based on the loss function selected in sub-step 2, the model parameters are iteratively updated through backpropagation.

[0126] Sub-step 4: Repeat sub-steps 1 to 3 above until the model converges and the final image detection model is obtained.

[0127] S305: Use the image detection model obtained in S304 to perform inference tests on the test sample set and output the final image detection results.

[0128] Please see Figure 9 , Figure 9 This is a schematic diagram of a small target image detection device for the running gear of a rail transit vehicle, provided in an embodiment of this application. The device includes:

[0129] The image acquisition module is used to acquire original images of the running gear of rail transit vehicles.

[0130] The data storage module is used to store the original images and other data information mentioned above.

[0131] A detection module pre-trained with training samples is used to obtain the location and classification information of small targets based on the original image. It should be noted that the training process of the recognition module pre-trained with training samples includes:

[0132] S1: Obtain the raw dataset of images of the running gear of rail transit vehicles.

[0133] S2: Preprocess the above training sample set of images of the traverse section to construct a training dataset.

[0134] S3: Construct a dual-branch network structure for small target image detection.

[0135] S4: Use the above training dataset to conduct supervised training on the dual-branch network for small target image detection to obtain the corresponding image detection model.

[0136] The output display module is used to display the location and classification information of the small targets obtained above.

[0137] Specifically, the image acquisition module can use a line scan camera to capture images of the side or underside of the vehicle to obtain original images of the running gear of the rail transit vehicle.

[0138] When rail transit vehicles are under maintenance, the image acquisition module collects image information of the vehicle's running gear; the image acquisition module is connected to the data storage module via a signal connection; in actual use, such as... Figure 10 As shown, Figure 10 This is a schematic diagram illustrating the installation location of an image acquisition module provided in an embodiment of this application. Image acquisition module A can be directly installed on one side of track B. Due to the multiple line scan cameras integrated inside, it can completely detect the actual image of the entire vehicle running gear. Furthermore, in this embodiment, the image acquisition module can also be installed on both sides of the same rail to achieve mutual cooperation. The specific installation method can be selected according to the actual conditions of the construction site.

[0139] The data storage module stores the original images and other data, and interacts with the image acquisition module and the detection module pre-trained with training samples. The detection module pre-trained with training samples is used to obtain the location information of small targets based on the original images.

[0140] Specifically, the detection module structure pre-trained with training samples is as follows: Figure 11 As shown, Figure 11 This is a schematic diagram of a detection module provided in an embodiment of this application. The detection module may include at least one processor, at least one internal memory, a non-volatile storage medium, a communication interface, an I / O interface, and a system bus. The non-volatile storage medium includes a database, a computer program, and an operating system. The computer program is loaded and executed by the processor to implement the relevant steps in the target detection method disclosed in any of the foregoing embodiments.

[0141] In this embodiment, the communication interface can create a data transmission channel between the detection module and external devices. The communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here. The I / O interface is used to acquire external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs, and is not specifically limited here. Furthermore, the internal memory, as a carrier of resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it can include operating systems, computer programs, and data, and the storage method can be temporary storage or permanent storage.

[0142] The operating system manages and controls the various hardware devices and computer programs on the detection module to enable the processor to perform calculations and processing on the massive amounts of data in the memory. It can be Windows Server, Netware, Unix, Linux, etc. In addition to computer programs capable of performing the target detection method executed by the detection module as disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The data may include the two-dimensional image data to be detected collected by the detection module.

[0143] The output display module displays the location and classification information of the small targets acquired above, providing more detailed data for subsequent precise positioning of key components, thereby enabling the monitoring of the safety status of the rail transit system. Each module in the target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the target device in hardware form or independent of the target device, or stored in the target device's memory in software form, so that the target device can invoke and execute the corresponding operations of each module.

[0144] This embodiment proposes a small target image detection device and scheme for the running gear of rail transit vehicles. Addressing the low recognition rate of small target objects by mainstream image detection methods when detecting components in the running gear of rail transit vehicles, this invention introduces a continuously refined high-resolution branch to form a dual-branch network and uses a unified fusion module to couple the features of the two branches, enhancing the network's ability to recognize small target objects. To address the problem that horizontal detection boxes are not suitable for detecting tilted small targets, this embodiment proposes a single-stage rotating small target detection scheme without anchor boxes. It uses a rotating Gaussian kernel to coarsely locate the center point of the rotating small target object, and obtains the final rotating detection box by regressing the bounding box vectors in four directions based on the center point. This method significantly improves the accuracy of detecting tilted objects compared to directly detecting horizontal boxes. This embodiment can be applied to component detection in the running gear of rail transit vehicles, improving the detection accuracy of small target objects and providing more detailed data for subsequent precise positioning of key components, thereby enabling the monitoring of the safety status of the rail transit system. It can greatly reduce the workload of subway inspection personnel and provide a partial solution for the whole-vehicle inspection of rail transit vehicles.

[0145] This application also provides a target detection system for the running gear of a vehicle, including:

[0146] The training set construction module is used to acquire sample images of the vehicle's running gear and preprocess the sample images to obtain a training dataset.

[0147] A model training module is used to build a dual-branch network structure and train the dual-branch network structure using the training dataset to obtain an image detection model. The dual-branch network structure includes a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head. The preprocessing module extracts image features from the input image and uses the processing result as input to the high-resolution branch and the low-resolution branch. The high-resolution branch includes n residual blocks with a step size of 1, and the low-resolution branch includes n residual blocks with a step size greater than 1. The bilateral interaction module enables feature interaction between the high-resolution branch and the low-resolution branch. The unified fusion module fuses the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map. The feature detection head extracts features from the fused feature map to obtain bounding box information.

[0148] The detection task execution module is used to, upon receiving a target detection task, capture an image of the vehicle's running gear to obtain an image of the running gear, and then use the image detection model to perform target detection on the image of the running gear to obtain a detection box of the target component.

[0149] The solution provided in this embodiment constructs a dual-branch network structure. The high-resolution and low-resolution branches in this structure can extract features at different resolutions from the preprocessing module's results. Furthermore, the bilateral interaction module within the dual-branch network structure enables feature interaction between the high-resolution and low-resolution branches. This embodiment generates a training dataset after acquiring sample images of the vehicle's running gear, and then uses this dataset to train the dual-branch network structure to obtain an image detection model. Because the dual-branch network structure includes high-resolution and low-resolution branches, it can extract features from small target objects, thus enabling the trained image detection model to detect small targets. After training, if a target detection task is received, the trained image detection model can be used to detect images of the running gear. Therefore, this embodiment can improve the accuracy of target detection on the vehicle's running gear.

[0150] Optionally, the training set construction module preprocesses the sample images to obtain the training dataset, including: selecting training samples from the sample images; adding annotation information of a preset format to the target components in the training samples; wherein the annotation information of the preset format includes the center point of the detection box, the bounding vector of the detection box, the width and height of the minimum bounding rectangle of the detection box, and the rotation angle of the detection box; and performing data augmentation operations on the training samples to obtain the training dataset.

[0151] Furthermore, the process by which the model training module trains the dual-branch network structure using the training dataset to obtain the image detection model includes:

[0152] Target images are selected from the training dataset; features are extracted from the target images using the preprocessing module to obtain a first feature map; wherein the preprocessing module includes multiple sequentially connected residual blocks; features are extracted from the first feature map using the high-resolution branch and the low-resolution branch respectively, and the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch are interacted using the bilateral interaction module; the second feature map and the third feature map are fused using the unified fusion module to obtain a fused feature map; wherein the second feature map is the output result of the high-resolution branch, and the third feature map is the output result of the low-resolution branch. The output of the rate branch; the feature detection head is used to extract features from the fused feature map to obtain detection box information; wherein, the detection box information includes the detection box center point, the number of detection box categories, the detection box center point offset, the detection box bounding vector, and the detection box rotation angle; the loss function is used to calculate the loss value between the detection box information and the annotation information of the target image, and the network parameters of the feature detection head are adjusted according to the loss value; it is determined whether the images in the training dataset have been selected; if so, the trained dual-branch network structure is set as an image detection model; if not, the step of selecting target images from the training dataset is initiated.

[0153] Furthermore, the dual-branch network structure includes n-1 bilateral interaction modules; each bilateral interaction module includes a first hop connection structure, a second hop connection structure, a downsampling structure, and an upsampling structure;

[0154] Accordingly, the process by which the model training module interacts with the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch using the bilateral interaction module includes:

[0155] The i-th residual block in the high-resolution branch outputs a first intermediate feature. The first intermediate feature is processed using the first skip connection structure of the i-th bilateral interaction module to obtain a second intermediate feature. The second intermediate feature is then processed using the downsampling structure of the i-th bilateral interaction module to obtain a third intermediate feature; 1 ≤ i ≤ n-1. The i-th residual block in the low-resolution branch outputs a fourth intermediate feature. The fourth intermediate feature is processed using the second skip connection structure of the i-th bilateral interaction module to obtain a fifth intermediate feature. The fifth intermediate feature is then processed using the upsampling structure of the i-th bilateral interaction module to obtain a sixth intermediate feature. The fusion result of the third and fifth intermediate features is set as the input of the (i+1)-th residual block in the low-resolution branch. The fusion result of the sixth and second intermediate features is set as the input of the (i+1)-th residual block in the high-resolution branch.

[0156] Furthermore, the unified fusion module includes a bilinear interpolation upsampling module, an attention module, and a weighting module;

[0157] Accordingly, the process by which the model training module fuses the second and third feature maps using the unified fusion module to obtain a fused feature map includes:

[0158] The third feature map is upsampled using a bilinear interpolation upsampling module to obtain a fourth feature map; the feature weights of the second and fourth feature maps are determined using an attention module; and the second and fourth feature maps are weighted and fused using a weighting module based on the feature weights to obtain the fused feature map.

[0159] Furthermore, the process by which the model training module determines the feature weights of the second feature map and the fourth feature map using the attention module includes:

[0160] The attention module is used to perform average pooling, max pooling, concat, convolution, and sigmoid functions on the second and fourth feature maps to obtain the feature weights of the second and fourth feature maps.

[0161] Furthermore, the feature detection head includes a heatmap detection head, an offset regression head, a bounding box regression head, and a rotation factor regression head;

[0162] Accordingly, the process by which the model training module uses the feature detection head to extract features from the fused feature map to obtain detection box information includes:

[0163] The heatmap detection head is used to extract the center point of the detection box and the number of detection box categories from the fused feature map; the offset regression head is used to extract the offset of the center point of the detection box from the fused feature map; the bounding box regression head is used to extract the bounding box vector of the detection box from the fused feature map; and the rotation factor regression head is used to extract the rotation angle of the detection box from the fused feature map.

[0164] Since the embodiments of the system part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the system part, and they will not be repeated here.

[0165] This application also provides a storage medium on which a computer program is stored, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0166] This application also provides an electronic device that may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the electronic device may also include various network interfaces, power supplies, and other components.

[0167] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0168] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A target detection method for a vehicle running gear, characterized in that, include: Acquire sample images of the vehicle's running gear and preprocess the sample images to obtain a training dataset; A dual-branch network structure is constructed, and the image detection model is obtained by training the dual-branch network structure using the training dataset. The dual-branch network structure includes a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head. The preprocessing module extracts image features from the input image and uses the processing result as input to the high-resolution branch and the low-resolution branch. The high-resolution branch includes n residual blocks with a step size of 1, and the low-resolution branch includes n residual blocks with a step size greater than 1. The bilateral interaction module enables feature interaction between the high-resolution branch and the low-resolution branch. The unified fusion module fuses the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map. The feature detection head extracts features from the fused feature map to obtain detection box information. If a target detection task is received, the vehicle running gear is photographed to obtain an image of the running gear, and the image detection model is used to perform target detection on the running gear image to obtain the detection box of the target component.

2. The target detection method for the running gear of a vehicle according to claim 1, characterized in that, The sample images are preprocessed to obtain a training dataset, including: Select training samples from the sample images; Add annotation information in a preset format to the target component in the training sample; wherein, the annotation information in the preset format includes the center point of the detection box, the bounding vector of the detection box, the width and height of the minimum bounding rectangle of the detection box, and the rotation angle of the detection box; The training dataset is obtained by performing data augmentation operations on the training samples.

3. The target detection method for the running gear of a vehicle according to claim 1, characterized in that, The image detection model is obtained by training the dual-branch network structure using the training dataset, including: Select target images from the training dataset; The preprocessing module is used to extract features from the target image to obtain a first feature map; wherein, the preprocessing module includes multiple residual blocks connected in sequence; The first feature map is extracted using the high-resolution branch and the low-resolution branch respectively, and the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch are interacted using the bilateral interaction module. The unified fusion module is used to fuse the second feature map and the third feature map to obtain a fused feature map; wherein, the second feature map is the output result of the high-resolution branch, and the third feature map is the output result of the low-resolution branch; The feature detection head is used to extract features from the fused feature map to obtain detection box information; wherein, the detection box information includes the detection box center point, the number of detection box categories, the detection box center point offset, the detection box bounding vector, and the detection box rotation angle; The loss function is used to calculate the loss value between the detection box information and the annotation information of the target image, and the network parameters of the feature detection head are adjusted according to the loss value; Determine whether all images in the training dataset have been selected; if yes, set the trained dual-branch network structure as an image detection model; otherwise, proceed to the step of selecting target images from the training dataset.

4. The target detection method for the running gear of a vehicle according to claim 3, characterized in that, The dual-branch network structure includes n-1 bilateral interaction modules; each bilateral interaction module includes a first hop connection structure, a second hop connection structure, a downsampling structure, and an upsampling structure; Accordingly, the bilateral interaction module is used to interact with the features extracted from the residual blocks in the high-resolution branch and the low-resolution branch, including: The first intermediate feature is output by the i-th residual block in the high-resolution branch. The first intermediate feature is processed by the first skip connection structure of the i-th bilateral interaction module to obtain the second intermediate feature. The second intermediate feature is processed by the downsampling structure of the i-th bilateral interaction module to obtain the third intermediate feature; 1≤i≤n-1; The fourth intermediate feature is determined by the output of the i-th residual block in the low-resolution branch. The fourth intermediate feature is processed by the second skip connection structure of the i-th bilateral interaction module to obtain the fifth intermediate feature. The fifth intermediate feature is processed by the upsampling structure of the i-th bilateral interaction module to obtain the sixth intermediate feature. The fusion result of the third intermediate feature and the fifth intermediate feature is set as the input of the (i+1)th residual block in the low-resolution branch; The fusion result of the sixth intermediate feature and the second intermediate feature is set as the input of the (i+1)th residual block in the high-resolution branch.

5. The target detection method for the running gear of a vehicle according to claim 3, characterized in that, The unified fusion module includes a bilinear interpolation upsampling module, an attention module, and a weighting module; Accordingly, the unified fusion module is used to fuse the second feature map and the third feature map to obtain a fused feature map, including: The third feature map is upsampled using a bilinear interpolation upsampling module to obtain the fourth feature map; The feature weights of the second and fourth feature maps are determined using an attention module. The second feature map and the fourth feature map are weighted and fused according to the feature weights using a weighting module to obtain the fused feature map.

6. The target detection method for the running gear of a vehicle according to claim 5, characterized in that, The step of determining the feature weights of the second feature map and the fourth feature map using an attention module includes: The attention module is used to perform average pooling, max pooling, concat, convolution, and sigmoid functions on the second and fourth feature maps to obtain the feature weights of the second and fourth feature maps.

7. The target detection method for the running gear of a vehicle according to claim 3, characterized in that, The feature detection head includes a heatmap detection head, an offset regression head, a bounding box regression head, and a rotation factor regression head; Accordingly, the feature detection head is used to extract features from the fused feature map to obtain detection box information, including: The heatmap detection head is used to extract the center point of the detection box and the number of detection box categories from the fused feature map. The offset regression head is used to extract the offset of the detection box center point from the fused feature map; The bounding box vectors of the detection boxes are extracted from the fused feature map using the bounding box regression head; The rotation angle of the detection box is extracted from the fused feature map using the rotation factor regression head.

8. A target detection system for a vehicle running gear, characterized in that, include: The training set construction module is used to acquire sample images of the vehicle's running gear and preprocess the sample images to obtain a training dataset. A model training module is used to build a dual-branch network structure and train the dual-branch network structure using the training dataset to obtain an image detection model. The dual-branch network structure includes a preprocessing module, a high-resolution branch, a low-resolution branch, a bilateral interaction module, a unified fusion module, and a feature detection head. The preprocessing module extracts image features from the input image and uses the processing result as input to the high-resolution branch and the low-resolution branch. The high-resolution branch includes n residual blocks with a step size of 1, and the low-resolution branch includes n residual blocks with a step size greater than 1. The bilateral interaction module enables feature interaction between the high-resolution branch and the low-resolution branch. The unified fusion module fuses the outputs of the high-resolution branch and the low-resolution branch to obtain a fused feature map. The feature detection head extracts features from the fused feature map to obtain bounding box information. The detection task execution module is used to, upon receiving a target detection task, capture an image of the vehicle's running gear to obtain an image of the running gear, and then use the image detection model to perform target detection on the image of the running gear to obtain a detection box of the target component.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the target detection method for the vehicle running gear as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the target detection method for the vehicle running gear as described in any one of claims 1 to 7.

Images