Track line foreign matter intrusion detection method and system

Abstract

The invention provides a track line foreign matter intrusion detection method and system, and the method comprises the steps: carrying out the pre-training of a preset YOLOV9 network frame through a track line foreign matter intrusion data set, so as to generate a corresponding feature extraction network model, carrying out the multi-scale feature extraction of image information through a backbone network in the feature extraction network model, and obtaining a feature extraction result; obtaining a plurality of corresponding multi-scale feature maps; calculating bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time through a preset detection module network according to the plurality of multi-scale feature maps; constructing a corresponding foreign matter detection network in real time according to the network weight file; and predicting bounding box information and classification information corresponding to the object in real time according to the image information through the foreign matter detection network so as to correspondingly complete detection of the foreign matter on the track line. The bounding box information and the classification information of the invading foreign matter can be efficiently and accurately obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of foreign object detection, and in particular to a method and system for detecting foreign object intrusion into a track. Background Art

[0002] With the rapid development of my country's railway transportation industry, track safety monitoring has become particularly important. Among them, track foreign body intrusion detection is one of the key links in railway safety operations. It involves real-time monitoring of the environment around the railway track in order to promptly detect and deal with foreign objects that may pose a threat to train operation safety. Traditional track foreign body intrusion detection methods mainly rely on manual inspections or automated systems based on traditional machine vision algorithms. These methods often have problems such as low detection efficiency, high false alarm rate, and inability to adapt to complex environmental changes.

[0003] In recent years, with the rapid development of deep learning technology, deep learning-based target detection algorithms have made significant progress in fields such as image recognition and video analysis. Among them, the YOLO (You Only Look Once) series of algorithms, as one of the outstanding algorithms in the field of target detection, has attracted widespread attention for its efficient detection speed and good detection performance. Specifically, the YOLO algorithm can complete the detection task of multiple targets in an image through a single forward propagation, which can significantly improve detection efficiency while improving detection accuracy.

[0004] However, although the YOLO algorithm performs well in general target detection tasks, there are still some challenges in directly applying the existing YOLO model to specific application scenarios, such as track foreign object intrusion detection. These challenges include insufficient sensitivity for small target detection, insufficient robustness in complex backgrounds, and limited recognition of specific types of foreign objects.

[0005] Based on this, it is necessary to propose a new track foreign object detection method based on YOLO to improve the accuracy and robustness of the algorithm in order to meet the special needs of track foreign object intrusion detection. Summary of the invention

[0006] In view of the above situation, the present invention provides a track foreign object intrusion detection method to solve the problem that the foreign object intrusion detection method in the prior art has low sensitivity and insufficient robustness in complex backgrounds, resulting in limited ability to recognize specific types of foreign objects.

[0007] An embodiment of the present invention provides a method for detecting foreign body intrusion on a track line, wherein the method comprises the following steps: The image information on the track line is collected in real time by the on-board imaging device on the train, and the corresponding track line foreign object intrusion data set is generated in real time according to the image information; The preset YOLOV9 network framework is pre-trained through the track line foreign body intrusion dataset to generate a corresponding feature extraction network model, and the image information is subjected to multi-scale feature extraction through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps; Calculating in real time through a preset detection module network a bounding box loss, a classification loss, and a confidence loss corresponding to an object on the track according to a plurality of the multi-scale feature maps; Performing gradient descent processing on the bounding box loss, the classification loss, and the confidence loss to output a corresponding network weight file, and constructing a corresponding foreign object detection network in real time according to the network weight file; The foreign object detection network predicts the bounding box information and classification information corresponding to the object in real time according to the image information, so as to complete the detection of foreign objects on the track line.

[0008] The track line foreign body intrusion detection method proposed by the present invention first introduces the compact inverted block into the backbone network GELAN in its feature extraction network model. The network model uses cheap deep convolution for spatial mixing and efficient point convolution for channel mixing. It can be embedded into the GELAN structure in the preset YOLOV9 network framework as an efficient basic module, so that the structure can achieve optimal efficiency while maintaining the same performance. Then, the partial self-attention is introduced into the detection module network model. The attention divides the feature map extracted from the feature extraction network model into two parts and inputs them into two branches. One branch obtains the attention weight of a part of the feature map through multi-head self-attention and feedforward neural network, and the other branch does not process. The feature maps obtained by the two branches are fused to obtain the final attention result. The partial self-attention integrates the global representation learning ability into the preset YOLOV9 network framework at a low computational cost through the above operation, thereby greatly enhancing the model's capabilities and improving the model performance. Finally, three different loss functions are used to describe the bounding box loss, classification loss, and confidence loss of foreign objects in the image; In addition, adaptively changing the learning rate according to the number of iterations and adaptively ending the training early according to the current accuracy can not only ensure the convergence speed of the algorithm and reduce the loss function to the lowest point, but also end the training early after the loss function has reached the lowest point, thus speeding up the training. When performing foreign object intrusion detection, the final weight file obtained by pre-training is loaded and the captured image is input into the network to obtain the bounding box position, shape information and classification information of the intruding foreign object to be detected, so as to complete the detection of the intruding object.

[0009] Furthermore, the step of pre-training the preset YOLOV9 network framework through the track foreign body intrusion dataset to generate a corresponding feature extraction network model includes: Initialize the initial parameters in the preset YOLOV9 network framework, wherein the initial parameters of the preset YOLOV9 network framework include an initial learning rate, a final learning rate, an initial weight, a preset number of learning rate rounds, a maximum number of training rounds, and an early stopping parameter; Assign values ​​to the iterative training rounds and perform iterative training calculations, where the number of rounds for the first iterative training is 0; When it is detected in real time that the number of rounds is less than the preset number of rounds, the initial learning rate will gradually increase from a number close to 0 to the preset learning rate; Real-time judgment of whether the downward gradient of the loss generated during training is greater than the preset loss threshold; If it is determined that the descending gradient of the generated loss is greater than the preset loss threshold, the preset YOLOV9 network framework continues to iterate effectively, and the early stopping parameter is reset to zero, and the iteration of the preset YOLOV9 network framework is continued until the maximum number of training rounds is reached, and the final weight file is saved; Before the maximum number of training rounds is reached, if it is determined that the descending gradient of the generated loss is less than the preset loss threshold, and the number of continuous rounds exceeds the early stopping parameter, the training is terminated in advance and the final weight file is saved.

[0010] Furthermore, the step of calculating the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps through a preset detection module network includes: Determine in real time the predicted bounding box information and the real bounding box information corresponding to the plurality of multi-scale feature maps, and determine corresponding positive samples and negative samples according to the predicted bounding box information and the real bounding box information; The bounding box loss is calculated based on the determined bounding box information of the positive sample and the true bounding box information; The classification loss is calculated according to the positive sample information contained in the positive sample; The confidence loss is calculated according to the positive sample information included in the positive sample and the negative sample information included in the negative sample.

[0011] Furthermore, the expression of the algorithm for calculating the bounding box loss is:

[0012] Among them, L Bbox represents the bounding box loss, CIoU is Complete-IoU, and its function is expressed as:

[0013] Wherein, IoU is the intersection over union ratio of the predicted bounding box and the true bounding box, α is the weight function, It is used to measure the consistency of the aspect ratio of the bounding box, ρ represents the Euclidean distance between the two center points, b and b gt represents the center point of the predicted bounding box and the true bounding box, c represents the diagonal distance of the minimum bounding box that can contain both the predicted box and the true box, and the expression of IoU is:

[0014]

[0015]

[0016] Among them, Area of ​​Overlap is the area of ​​the overlap between the predicted bounding box and the true bounding box, Area of ​​Union is the total area after the predicted bounding box and the true bounding box are combined, which is equal to the sum of the areas of the two bounding boxes minus the area of ​​the overlap area, w gt is the width of the true bounding box, h gt is the height of the true bounding box, w is the width of the predicted bounding box, and h is the height of the predicted bounding box.

[0017] Furthermore, the algorithm for calculating the classification loss is expressed as:

[0018] in, is the regulating factor, ≥0 is an adjustable focus parameter, α t is the category weight factor, which is used to solve the category imbalance problem, p t is the prediction result of each category, L focal represents the classification loss, α t 、p t They are expressed as the following functions:

[0019]

[0020] Among them, p is the prediction result and y is the label value.

[0021] Furthermore, the algorithm for calculating the confidence loss is expressed as:

[0022] Where N is the number of samples, L is confrepresents the confidence loss, i is the serial number of the detected target, y i is the true label of the i-th sample, indicating whether there is a target object. is the prediction confidence of the i-th sample.

[0023] Furthermore, after the step of calculating the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps by using a preset detection module network, the method further includes: Back-propagation updates the parameters of the preset YOLOV9 network framework according to the calculated loss, and dynamically updates the initial learning rate according to preset rules; Among them, dynamically updating the initial learning rate according to the preset rules includes two stages, the first stage is the preset stage, wherein the current round number is less than the preset round number of the learning rate, and the formula for calculating the learning rate of the training round is:

[0024] Among them, lr warmup (t) is the learning rate of the tth training round, lr start is the learning rate at the beginning of the preset, lr0 is the initial learning rate after the preset ends, t is the current number of training rounds, warmup e pochs is the preset number of rounds for the learning rate; The second stage is the learning rate scheduling stage, in which the current round number is greater than the preset round number of the learning rate, and the learning rate scheduling strategy adopts the cosine annealing method, which is expressed as follows:

[0025] Among them, lr cosinc (t) is the learning rate of the tth training round, lr min is the minimum learning rate, lr max is the maximum learning rate, t is the current number of training rounds, max_epoch is the maximum number of training rounds, where the maximum number of iterations max_epoch is set to 300, and the initial learning rate lr0 is set to 0.01.

[0026] Furthermore, the step of performing multi-scale feature extraction on the image information through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps includes: When the image information is acquired, calling out the convolutional neural network included in the backbone network, and performing multi-scale feature extraction processing on the image information through the convolutional neural network; Extracting a number of initial multi-scale feature maps in real time through the convolutional neural network; Fusing a plurality of the initial multi-scale feature maps through a feature pyramid network to generate the multi-scale feature map accordingly; The expression of the algorithm for performing multi-scale feature extraction processing on the image information is as follows:

[0027] in, is the input feature map of the l-1th layer, W l is the convolution kernel weight of layer l, b l is the bias term of the lth layer, σ is the activation function, and * represents the convolution calculation; The expression of the algorithm for fusing several of the initial multi-scale feature maps through the feature pyramid network is:

[0028]

[0029]

[0030] Among them, C3, C4, and C5 are feature maps obtained at different levels of the feature extraction network model, P3, P4, and P5 are multi-scale feature maps generated by processing the feature maps obtained at different levels through the feature pyramid network, and W 5 1x1 , W 4 1x1 , W 3 1x1 is a 1x1 convolution, which is used for dimensionality reduction of feature maps. Upsample() is an upsampling operation of feature maps. 5 1x1 , b 4 1x1 , b 3 1x1 is the bias term of each layer; in:

[0031] Among them, FPN represents the feature pyramid network, P3, P4, and P5 represent different multi-scale feature maps, and each multi-scale feature map corresponds to a different downsampling ratio.

[0032] Another aspect of the present invention provides a track foreign body intrusion detection system, the system comprising: The acquisition module is used to acquire image information on the track line in real time through the on-board imaging device on the train, and generate a corresponding track line foreign body intrusion data set in real time according to the image information; A training module is used to pre-train a preset YOLOV9 network framework through the track foreign body intrusion dataset to generate a corresponding feature extraction network model, and perform multi-scale feature extraction on the image information through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps; A calculation module, used to calculate the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps through a preset detection module network; A processing module, used for performing gradient descent processing on the bounding box loss, the classification loss and the confidence loss to output a corresponding network weight file, and constructing a corresponding foreign body detection network in real time according to the network weight file; The execution module is used to predict the bounding box information and classification information corresponding to the object in real time according to the image information through the foreign object detection network, so as to complete the detection of foreign objects on the track line.

[0033] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or will be learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of a track foreign body intrusion detection method proposed in an embodiment of the present invention; Figure 2 A flow chart of a track foreign body intrusion detection method proposed in an embodiment of the present invention; Figure 3 is a schematic diagram of a compact inverted block according to an embodiment of the present invention; Figure 4 Schematic diagram of some self-attention modules in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a track foreign object intrusion detection system proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar originals or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0036] These and other aspects of the embodiments of the present invention will become clear with reference to the following description and drawings. In these descriptions and drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto. On the contrary, the embodiments of the present invention include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0037] Under the YOLOV9 network framework, the track foreign body intrusion detection method mainly includes the following two network models: a track foreign body intrusion detection feature extraction network model based on compact inverted blocks, and a track foreign body intrusion detection module network model based on a partial self-attention network model. Among them, the track foreign body intrusion detection feature extraction network is used to efficiently extract the multi-scale features of the images captured by the on-board RGB camera of the running train, and the track foreign body intrusion detection module network model is used to efficiently aggregate the global context information in the multi-scale feature map extracted by the above feature extraction network model; Using a large-scale data set, the network model of the track foreign object intrusion detection method is pre-trained, wherein the classification loss function, bounding box loss function and confidence loss function in the detection module network model will calculate the bounding box loss, classification loss and confidence loss corresponding to the foreign object, and adjust the weight parameters of the above model through back propagation on the main branch and auxiliary branch in the preset YOLOV9 network framework.

[0038] See also Figure 1 The present invention proposes a method for detecting foreign body intrusion on a track line, wherein the method comprises the following steps: S101, collecting image information on the track line in real time through the on-board imaging equipment on the train, and generating a corresponding track line foreign object intrusion data set in real time according to the image information.

[0039] S102, pre-training the preset YOLOV9 network framework through the track line foreign body intrusion dataset to generate a corresponding feature extraction network model, and performing multi-scale feature extraction on the image information through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps.

[0040] In this step, the method of pre-training the YOLOV9 network framework using the track line foreign body intrusion dataset includes: S1021, initializing the parameters of the preset YOLOV9 network framework, wherein the initial parameters of the preset YOLOV9 network framework include an initial learning rate lr0, a final learning rate lr f, initial weight weight0, learning rate preset number of rounds warmup_epochs, maximum number of training rounds max_epoch and early stopping parameter patience; It should be noted here that if the initial weight weight0 is not given, the convolutional layer of the YOLOV9 network will be initialized with He normal to determine the initial weights and biases of the convolutional layer in the YOLOV9 network.

[0041] S1022, assigning values ​​to iterative training rounds and performing iterative training calculations, wherein Epoch=0 for the first iterative training.

[0042] S1023, when the current round epoch is less than the preset round number warmup_epochs, the learning rate lr will gradually increase from a number close to 0 to the preset initial learning rate lr0.

[0043] S1024, determining in real time whether the descending gradient of the loss generated during the training process is greater than a preset loss threshold.

[0044] If it is determined that the descending gradient of the generated loss is greater than the preset loss threshold, the preset YOLOV9 network framework continues to iterate effectively, the count of the parameter patience will be reset, and the iteration of the preset YOLOV9 network framework will continue until the maximum number of training rounds max_epoch is reached, and the final weight file is saved. Before the maximum number of training rounds max_epoch is reached, if it is determined that the descending gradient of the generated loss is less than the preset loss threshold, and the number of continuous rounds epoch exceeds the early stopping parameter patience, the training is terminated in advance and the final weight file is saved.

[0045] It should be noted that after the step of calculating the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the multi-scale feature maps through the preset detection module network, the method further includes: The parameters of the YOLOV9 improved network are back-propagated and updated according to the calculated loss, and the initial learning rate is dynamically updated according to preset rules.

[0046] Among them, dynamically updating the initial learning rate according to the preset rules includes two stages, the first stage is the preset stage, wherein the current round number is less than the preset round number of the learning rate, and the formula for calculating the learning rate of the training round is:

[0047] Among them, lr warmup (t) is the learning rate of the tth training round, lr startis the learning rate at the beginning of the preset, lr0 is the initial learning rate after the preset ends, t is the current number of training rounds, warmup e pochs is the preset number of rounds of the learning rate. The second stage is the learning rate scheduling stage, in which the current round number is greater than the preset number of rounds of the learning rate, and its learning rate scheduling strategy adopts the cosine annealing method, which is expressed as follows:

[0048] Among them, lr cosinc (t) is the learning rate of the tth training round, lr min is the minimum learning rate, lr max is the maximum learning rate, t is the current number of training rounds, max_epoch is the maximum number of training rounds, where the maximum number of iterations max_epoch is set to 300, and the initial learning rate lr0 is set to 0.01.

[0049] Among them, the backbone network GELAN in the feature extraction network is used to perform multi-scale feature extraction on the collected image information, and the specific steps include: when the image information is acquired, calling out the convolutional neural network contained in the backbone network, and performing multi-scale feature extraction processing on the image information through the convolutional neural network; Extracting a number of initial multi-scale feature maps in real time through the convolutional neural network; Fusing a plurality of the initial multi-scale feature maps through a feature pyramid network to generate the multi-scale feature map accordingly; The expression of the algorithm for performing multi-scale feature extraction processing on the image information is as follows:

[0050] Among them, F l-1 is the input feature map of the l-1th layer, W l is the convolution kernel weight of layer l, b l is the bias term of the lth layer, σ is the activation function, and * represents the convolution calculation; The expression of the algorithm for fusing several of the initial multi-scale feature maps through the feature pyramid network is:

[0051]

[0052]

[0053] Among them, C3, C4, and C5 are feature maps obtained at different levels of the feature extraction network model, P3, P4, and P5 are multi-scale feature maps generated by processing the feature maps obtained at different levels through the feature pyramid network, and W 5 1x1 , W 4 1x1 , W 3 1x1 is a 1x1 convolution, which is used for dimensionality reduction of feature maps. Upsample() is an upsampling operation of feature maps. 5 1x1 、b 4 1x1 、b 3 1x1 is the bias term of each layer; in:

[0054] Among them, FPN represents the feature pyramid network, P3, P4, and P5 represent different multi-scale feature maps, and each multi-scale feature map corresponds to a different downsampling ratio. Further, as Figure 2 , Figure 3 as well as Figure 4 As shown, this embodiment further specifically includes the following contents: S103, calculating in real time the bounding box loss, classification loss and confidence loss corresponding to the object on the track according to the plurality of multi-scale feature maps through a preset detection module network.

[0055] In this step, the calculation methods of bounding box loss, classification loss, and confidence loss include: S1031, determining in real time predicted bounding box information and true bounding box information corresponding to the plurality of multi-scale feature maps, and determining corresponding positive samples and negative samples according to the predicted bounding box information and the true bounding box information; S1032, calculating the bounding box loss according to the determined bounding box information of the positive sample and the true bounding box information; S1033, calculating the classification loss according to the positive sample information included in the positive sample; S1034: Calculate the confidence loss according to the positive sample information included in the positive sample and the negative sample information included in the negative sample.

[0056] Specifically, we use an IoU-based strategy to determine positive and negative samples. When the IoU between the predicted box and a real box is greater than 0.5, the predicted box is considered a positive sample. In addition, to ensure that each real box has at least one corresponding positive sample, even if no predicted box has an IoU exceeding the threshold of 0.5, the predicted box with the highest IoU will be selected as a positive sample. When the IoU between the predicted box and all real boxes is less than 0.4, the predicted box is considered a negative sample. When the IoU between the predicted box and the real box is between 0.4 and 0.5, the predicted box will be ignored and will not be used as a positive sample or a negative sample to reduce uncertainty in training.

[0057] In this embodiment, ρ represents the Euclidean distance between two center points, b and bgt represent the center points of the predicted bounding box and the true bounding box, and c represents the diagonal distance of the minimum bounding box that can contain both the predicted box and the true box. The above bounding box loss function is expressed as:

[0058] Among them, CIoU is Complete-IoU, and its function is expressed as:

[0059] Wherein, IoU is the intersection over union ratio of the predicted bounding box and the true bounding box, α is the weight function, It is used to measure the consistency of the aspect ratio of the bounding box. Indicates calculating the Euclidean distance between two center points. and represents the center point of the predicted bounding box and the true bounding box, c represents the diagonal distance of the minimum bounding box that can contain both the predicted box and the true box, and the expression of IoU is:

[0060]

[0061]

[0062] Among them, Area of ​​Overlap is the area of ​​the overlap between the predicted bounding box and the true bounding box, Area of ​​Union is the total area after the predicted bounding box and the true bounding box are combined, which is equal to the sum of the areas of the two bounding boxes minus the area of ​​the overlap area, w gt is the width of the true bounding box, h gt is the height of the true bounding box, w is the width of the predicted bounding box, and h is the height of the predicted bounding box.

[0063] The above classification loss function adopts the Focal loss loss function, which is expressed as:

[0064] in, is the regulating factor, ≥0 is an adjustable focus parameter, α t is the category weight factor, which is used to solve the category imbalance problem, p t is the prediction result of each category. t 、p t They are expressed as the following functions:

[0065]

[0066] Among them, p is the prediction result and y is the label value.

[0067] The above confidence loss function is expressed as:

[0068] Among them, N is the number of samples, i is the serial number of the detection target, and y i is the true label of the i-th sample, indicating whether there is a target object. is the prediction confidence of the i-th sample.

[0069] S104, performing gradient descent processing on the bounding box loss, the classification loss and the confidence loss to output a corresponding network weight file, and constructing a corresponding foreign body detection network in real time according to the network weight file.

[0070] In this embodiment, when the iterative training rounds reach the maximum number of training rounds max_epoch, the bounding box loss, classification loss and confidence loss are all minimized. The final weight file is output.

[0071] S105, predicting the bounding box information and classification information corresponding to the object in real time according to the image information through the foreign object detection network, so as to complete the detection of foreign objects on the track line.

[0072] The track line foreign body intrusion detection method based on the improved YOLOV9 network proposed in the present invention utilizes the backbone network GELAN introduced with a compact inverted block to extract the multi-scale feature map of the input image, achieving higher efficiency while ensuring the same feature extraction performance, saving the overall computational complexity of the foreign body detection network. At the same time, partial self-attention is introduced into the detection module network model, which completes the self-attention operation while reducing the computational complexity and enhancing the detection performance of the model; In addition, the bounding box loss function, classification loss function, and confidence loss function are used to describe the current bounding box loss, classification loss, and confidence loss of the intruding foreign object, and the current learning rate is adaptively adjusted according to the current number of training rounds, which can not only ensure the convergence speed of the algorithm, but also ensure that the algorithm will not oscillate near the lowest point of loss, so that it cannot converge effectively; When detecting intruding foreign objects, it is only necessary to input the images collected by the on-board imaging device into the network and load the final weight file during pre-training to obtain the bounding box information and classification information of the intruding foreign objects to be detected. In an embodiment of the present invention, based on the YOLOV9 network, an improved YOLOV9 network is proposed in combination with a compact inverted block and partial self-attention, and it is used in track line foreign object intrusion detection, and the bounding box information and classification information of the intruding foreign objects in the track line can be obtained to ensure the safety and efficiency of railway operations.

[0073] See also Figure 5 The present invention also proposes a track foreign body intrusion detection system, wherein the system comprises: The acquisition module is used to acquire image information on the track line in real time through the on-board imaging device on the train, and generate a corresponding track line foreign body intrusion data set in real time according to the image information; A training module is used to pre-train a preset YOLOV9 network framework through the track foreign body intrusion dataset to generate a corresponding feature extraction network model, and perform multi-scale feature extraction on the image information through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps; A calculation module, used to calculate the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps through a preset detection module network; A processing module, used for performing gradient descent processing on the bounding box loss, the classification loss and the confidence loss to output a corresponding network weight file, and constructing a corresponding foreign body detection network in real time according to the network weight file; The execution module is used to predict the bounding box information and classification information corresponding to the object in real time according to the image information through the foreign object detection network, so as to complete the detection of foreign objects on the track line.

[0074] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0076] The technical principles of the present invention are described above in conjunction with specific embodiments. These descriptions are only for explaining the principles of the present invention and cannot be interpreted in any way as limiting the scope of protection of the present invention. Based on the explanations here, those skilled in the art can associate other specific implementations of the present invention without creative work, and these methods will fall within the scope of protection of the present invention.

Claims

1. A method for detecting foreign body intrusion on a track, characterized in that: The steps include: The image information on the track line is collected in real time by the on-board imaging device on the train, and the corresponding track line foreign object intrusion data set is generated in real time according to the image information; The preset YOLOV9 network framework is pre-trained through the track line foreign body intrusion dataset to generate a corresponding feature extraction network model, and the image information is subjected to multi-scale feature extraction through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps; Calculating in real time through a preset detection module network a bounding box loss, a classification loss, and a confidence loss corresponding to an object on the track according to a plurality of the multi-scale feature maps; Performing gradient descent processing on the bounding box loss, the classification loss, and the confidence loss to output a corresponding network weight file, and constructing a corresponding foreign object detection network in real time according to the network weight file; The foreign object detection network predicts the bounding box information and classification information corresponding to the object in real time based on the image information, so as to complete the detection of foreign objects on the track line.

2. The method for detecting foreign body intrusion on a track according to claim 1, characterized in that: The step of pre-training the preset YOLOV9 network framework through the track foreign body intrusion data set to generate a corresponding feature extraction network model includes: Initialize the initial parameters in the preset YOLOV9 network framework, wherein the initial parameters of the preset YOLOV9 network framework include an initial learning rate, a final learning rate, an initial weight, a preset number of learning rate rounds, a maximum number of training rounds, and an early stopping parameter; Assign values ​​to the iterative training rounds and perform iterative training calculations, where the number of rounds for the first iterative training is 0; When it is detected in real time that the number of rounds is less than the preset number of rounds, the initial learning rate will gradually increase from a number close to 0 to the preset learning rate; Real-time judgment of whether the downward gradient of the loss generated during training is greater than the preset loss threshold; If it is determined that the descending gradient of the generated loss is greater than the preset loss threshold, the preset YOLOV9 network framework continues to iterate effectively, and the early stopping parameter is reset to zero, and the iteration of the preset YOLOV9 network framework is continued until the maximum number of training rounds is reached, and the final weight file is saved; Before the maximum number of training rounds is reached, if it is determined that the descending gradient of the generated loss is less than the preset loss threshold, and the number of continuous rounds exceeds the early stopping parameter, the training is terminated in advance and the final weight file is saved.

3. The method for detecting foreign body intrusion on a track according to claim 1, characterized in that: The step of calculating the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps through the preset detection module network comprises: Determine in real time the predicted bounding box information and the real bounding box information corresponding to the plurality of multi-scale feature maps, and determine corresponding positive samples and negative samples according to the predicted bounding box information and the real bounding box information; The bounding box loss is calculated based on the determined bounding box information of the positive sample and the true bounding box information; The classification loss is calculated according to the positive sample information contained in the positive sample; The confidence loss is calculated according to the positive sample information included in the positive sample and the negative sample information included in the negative sample.

4. The method for detecting foreign body intrusion on a track according to claim 3, characterized in that: The expression of the algorithm for calculating the bounding box loss is: Among them, L Bbox represents the bounding box loss, CIoU is Complete-IoU, and its function is expressed as: Wherein, IoU is the intersection over union ratio of the predicted bounding box and the true bounding box, α is the weight function, It is used to measure the consistency of the aspect ratio of the bounding box, ρ represents the Euclidean distance between the two center points, b and b gt represents the center point of the predicted bounding box and the true bounding box, c represents the diagonal distance of the minimum bounding box that can contain both the predicted box and the true box, and the expression of IoU is: Among them, Area of ​​Overlap is the area of ​​the overlap between the predicted bounding box and the true bounding box, Area of ​​Union is the total area after the predicted bounding box and the true bounding box are combined, which is equal to the sum of the areas of the two bounding boxes minus the area of ​​the overlap area, w gt is the width of the true bounding box, h gt is the height of the true bounding box, w is the width of the predicted bounding box, and h is the height of the predicted bounding box.

5. The method for detecting foreign body intrusion on a track according to claim 3, characterized in that: The expression of the algorithm for calculating the classification loss is: in, is the regulating factor, ≥0 is an adjustable focus parameter, α t is the category weight factor, which is used to solve the category imbalance problem, p t is the prediction result of each category, L focal represents the classification loss, α t 、p t They are expressed as the following functions: Among them, p is the prediction result and y is the label value.

6. The method for detecting foreign body intrusion on a track according to claim 3, characterized in that: The expression of the algorithm for calculating the confidence loss is: Where N is the number of samples, L is conf represents the confidence loss, i is the serial number of the detected target, y i is the true label of the i-th sample, indicating whether there is a target object. is the prediction confidence of the i-th sample.

7. The method for detecting foreign body intrusion on a track according to claim 2, characterized in that: After the step of calculating the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps through the preset detection module network, the method further comprises: Back-propagation updates the parameters of the preset YOLOV9 network framework according to the calculated loss, and dynamically updates the initial learning rate according to preset rules; Among them, dynamically updating the initial learning rate according to the preset rules includes two stages, the first stage is the preset stage, wherein the current round number is less than the preset round number of the learning rate, and the formula for calculating the learning rate of the training round is: Among them, lr warmup (t) is the learning rate of the tth training round, lr start is the learning rate at the beginning of the preset, lr0 is the initial learning rate after the preset ends, t is the current number of training rounds, warmup e pochs is the preset number of rounds for the learning rate; The second stage is the learning rate scheduling stage, in which the current round number is greater than the preset round number of the learning rate, and the learning rate scheduling strategy adopts the cosine annealing method, which is expressed as follows: Among them, lr cosinc (t) is the learning rate of the tth training round, lr min is the minimum learning rate, lr max is the maximum learning rate, t is the current number of training rounds, max_epoch is the maximum number of training rounds, where the maximum number of iterations max_epoch is set to 300, and the initial learning rate lr0 is set to 0.

01.

8. The method for detecting foreign body intrusion on a track according to claim 1, characterized in that: The step of performing multi-scale feature extraction on the image information through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps comprises: When the image information is acquired, calling out the convolutional neural network included in the backbone network, and performing multi-scale feature extraction processing on the image information through the convolutional neural network; Extracting a number of initial multi-scale feature maps in real time through the convolutional neural network; Fusing a plurality of the initial multi-scale feature maps through a feature pyramid network to generate the multi-scale feature map accordingly; The expression of the algorithm for performing multi-scale feature extraction processing on the image information is as follows: in, is the input feature map of the l-1th layer, W l is the convolution kernel weight of layer l, b l is the bias term of the lth layer, σ is the activation function, and * represents the convolution calculation; The expression of the algorithm for fusing several of the initial multi-scale feature maps through the feature pyramid network is: Among them, C3, C4, and C5 are feature maps obtained at different levels of the feature extraction network model, P3, P4, and P5 are multi-scale feature maps generated by processing the feature maps obtained at different levels through the feature pyramid network, and W 5 1x1 , W 4 1x1 , W 3 1x1 is a 1x1 convolution, which is used for dimensionality reduction of feature maps. Upsample() is an upsampling operation of feature maps. 5 1x1 、b 4 1x1 、b 3 1x1 is the bias term of each layer; in: Among them, FPN represents the feature pyramid network, P3, P4, and P5 represent different multi-scale feature maps, and each multi-scale feature map corresponds to a different downsampling ratio.

9. A track foreign body intrusion detection system, characterized in that: The system comprises: The acquisition module is used to acquire image information on the track line in real time through the on-board imaging device on the train, and generate a corresponding track line foreign body intrusion data set in real time according to the image information; A training module is used to pre-train a preset YOLOV9 network framework through the track foreign body intrusion dataset to generate a corresponding feature extraction network model, and perform multi-scale feature extraction on the image information through the backbone network in the feature extraction network model to obtain a number of corresponding multi-scale feature maps; A calculation module, used to calculate the bounding box loss, classification loss and confidence loss corresponding to the object on the track in real time according to the plurality of multi-scale feature maps through a preset detection module network; A processing module, used for performing gradient descent processing on the bounding box loss, the classification loss and the confidence loss to output a corresponding network weight file, and constructing a corresponding foreign body detection network in real time according to the network weight file; The execution module is used to predict the bounding box information and classification information corresponding to the object in real time according to the image information through the foreign object detection network, so as to complete the detection of foreign objects on the track line.

Images