Railway train number detection method and device, computer equipment and storage medium

Abstract

The invention relates to a railway train number detection method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring continuous multi-frame target video frames containing a train; performing space-time alignment processing on the feature maps of the continuous multiple frames of target video frames through a feature space-time alignment module in the vehicle number detection model to obtain aligned feature maps corresponding to all the frames of target video frames; through a multi-branch sensing module in a vehicle number detection model, performing vehicle number detection processing of a plurality of branch modules on the aligned feature map of each frame of target video frame to obtain a vehicle number detection result of each frame of target video frame under each branch module; and performing fusion processing on the vehicle number detection result of each target video frame under each branch module to obtain a target vehicle number detection result of each target video frame. By adopting the method, the accuracy and robustness of a vehicle number detection result can be improved.

Description

Technical Field

[0001] The present application relates to the field of heavy-load railway ground monitoring and detection technology, and in particular to a railway train number detection method, device, computer equipment, storage medium and computer program product. Background Art

[0002] Throughout the history of railway transportation, as heavy-load freight lines continue to expand and deepen, the requirements for vehicle operational safety are becoming increasingly stringent. Therefore, reliable, regular, and timely maintenance and repair of railway freight cars is crucial. This requires collecting the truck's license plate number to obtain basic vehicle information.

[0003] With the rapid development of integrated railway video surveillance systems, automatic detection of freight train license plates using surveillance cameras and machine vision technology installed along the railway has gradually become an important part of the intelligent construction of railway systems.

[0004] Freight train carriages in railway on-site environments are easily affected by complex natural environments such as bad weather, lighting changes, and damage to the carriages. At the same time, some characters on the train license plates are prone to fading, loss, pollution, obstruction, distortion, and other losses. In addition, there are difficulties such as low on-site video quality and insufficient video clarity. These situations pose greater challenges to the detection task. Summary of the Invention

[0005] Based on this, it is necessary to provide a railway train number detection method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of railway train number recognition in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for detecting a train number on a railway. The method comprises:

[0007] Obtain multiple consecutive target video frames containing trains;

[0008] Through the feature spatiotemporal alignment module in the vehicle number detection model, the feature maps of the target video frames of the consecutive frames are spatiotemporally aligned to obtain the aligned feature maps corresponding to the target video frames of each frame;

[0009] Through the multi-branch perception module in the vehicle number detection model, the aligned feature map of each frame of the target video frame is processed by multiple branch modules for vehicle number detection, and the vehicle number detection result of each frame of the target video frame under each branch module is obtained;

[0010] The vehicle number detection results of each target video frame under each branch module are fused to obtain the target vehicle number detection results of each target video frame.

[0011] In one embodiment, the multi-branch sensing module includes at least a first branch module and a second branch module;

[0012] The multi-branch perception module in the vehicle number detection model performs vehicle number detection processing on the aligned feature map of each frame of the target video frame using multiple branch modules, and obtains the vehicle number detection results of each frame of the target video frame under each branch module, including:

[0013] Performing vehicle license plate area detection on the aligned feature maps of the target video frames through the region proposal network in the first branch module to obtain candidate region feature maps of each aligned feature map;

[0014] Based on the known train model number features in the external feature memory, each candidate region feature map is enhanced to obtain an enhanced feature map of each candidate region feature map;

[0015] Performing vehicle number detection processing on each of the enhanced feature maps using the first vehicle number detection model in the first branch module to obtain a vehicle number detection result of each target video frame under the first branch module;

[0016] Performing semantic segmentation processing on each target video frame through the second branch module to obtain a vehicle number foreground image of each target video frame;

[0017] The second vehicle number detection model in the second branch module is used to perform vehicle number detection processing on each of the vehicle number foreground images to obtain the vehicle number detection results of each frame of the target video frame under the second branch module; the detection accuracy of the second vehicle number detection model is greater than the detection accuracy of the first vehicle number detection model.

[0018] In one embodiment, based on the vehicle number feature in the external feature memory, each candidate region feature map is enhanced to obtain an enhanced feature map of the candidate region feature map, including:

[0019] If it is detected that the similarity between the candidate region feature map and the vehicle number feature of the known train model reaches a preset similarity condition, then based on the vehicle number feature, the vehicle number candidate region in the candidate region feature map is enhanced to obtain an enhanced feature map of the candidate region feature map;

[0020] The method further comprises:

[0021] If it is detected that the similarity does not meet the preset similarity condition, the external feature memory is updated based on the vehicle number candidate area in the candidate area feature map.

[0022] In one embodiment, the vehicle license plate detection results of each target video frame under each branch module are fused to obtain the target vehicle license plate detection results of each target video frame, including:

[0023] Determine a first confidence level of a vehicle license plate detection result of each target video frame under the first branch module, and determine a second confidence level of a vehicle license plate detection result of each target video frame under the second branch module;

[0024] Based on the first confidence level and the second confidence level, the vehicle number detection results of the target video frames in each frame under the first branch module and the vehicle number detection results under the second branch module are fused to obtain the target vehicle number detection results of the target video frames in each frame.

[0025] In one embodiment, a feature spatiotemporal alignment module in a vehicle license plate detection model is used to perform spatiotemporal alignment processing on feature maps corresponding to multiple consecutive target video frames to obtain aligned feature maps corresponding to each target video frame, including:

[0026] Through the backbone network in the vehicle number detection model, feature extraction processing of the vehicle number text feature is performed on each frame of the target video frame to obtain a feature map of each frame of the target video frame;

[0027] The feature spatiotemporal alignment module performs temporal alignment and spatial alignment on the feature maps of multiple consecutive target video frames to obtain aligned feature maps corresponding to the target video frames of each frame.

[0028] In one embodiment, obtaining a plurality of consecutive target video frames containing a train includes:

[0029] Performing target detection processing on multiple consecutive current video frames of the railway for the train through the target detection model to obtain target detection results of the current video frames;

[0030] Based on the target detection result, a target video frame containing the train is determined from multiple consecutive frames of the current video frame.

[0031] In one embodiment, after fusing the vehicle license plate detection results of each target video frame under each branch module to obtain the target vehicle license plate detection results of each target video frame, the method further includes:

[0032] Based on the target vehicle number detection results of each target video frame, respectively extracting the vehicle number images corresponding to each target vehicle number detection result from each target video frame;

[0033] The plurality of vehicle number images are sent to the cloud, so that the cloud performs vehicle number recognition processing on the vehicle number images.

[0034] In a second aspect, the present application further provides a railway train number detection device. The device comprises:

[0035] A video frame acquisition module is used to acquire multiple consecutive target video frames containing trains;

[0036] A spatiotemporal alignment module is used to perform spatiotemporal alignment processing on the feature maps of multiple consecutive target video frames through the feature spatiotemporal alignment module in the vehicle license plate detection model to obtain an aligned feature map corresponding to each target video frame;

[0037] A vehicle number detection module is configured to perform vehicle number detection processing on the aligned feature maps of each target video frame using multiple branch modules through the multi-branch perception module in the vehicle number detection model, and obtain vehicle number detection results of each target video frame under each branch module;

[0038] The result fusion module is used to fuse the vehicle license plate detection results of each target video frame under each branch module to obtain the target vehicle license plate detection results of each target video frame.

[0039] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0040] Obtain multiple consecutive target video frames containing trains;

[0041] Through the feature spatiotemporal alignment module in the vehicle number detection model, the feature maps of the target video frames of the consecutive frames are spatiotemporally aligned to obtain the aligned feature maps corresponding to the target video frames of each frame;

[0042] Through the multi-branch perception module in the vehicle number detection model, the aligned feature map of each frame of the target video frame is processed by multiple branch modules for vehicle number detection, and the vehicle number detection result of each frame of the target video frame under each branch module is obtained;

[0043] The vehicle number detection results of each target video frame under each branch module are fused to obtain the target vehicle number detection results of each target video frame.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0045] Obtain multiple consecutive target video frames containing trains;

[0046] Through the feature spatiotemporal alignment module in the vehicle number detection model, the feature maps of the target video frames of the consecutive frames are spatiotemporally aligned to obtain the aligned feature maps corresponding to the target video frames of each frame;

[0047] Through the multi-branch perception module in the vehicle number detection model, the aligned feature map of each frame of the target video frame is processed by multiple branch modules for vehicle number detection, and the vehicle number detection result of each frame of the target video frame under each branch module is obtained;

[0048] The vehicle number detection results of each target video frame under each branch module are fused to obtain the target vehicle number detection results of each target video frame.

[0049] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0050] Obtain multiple consecutive target video frames containing trains;

[0051] Through the feature spatiotemporal alignment module in the vehicle number detection model, the feature maps of the target video frames of the consecutive frames are spatiotemporally aligned to obtain the aligned feature maps corresponding to the target video frames of each frame;

[0052] Through the multi-branch perception module in the vehicle number detection model, the aligned feature map of each frame of the target video frame is processed by multiple branch modules for vehicle number detection, and the vehicle number detection result of each frame of the target video frame under each branch module is obtained;

[0053] The vehicle number detection results of each target video frame under each branch module are fused to obtain the target vehicle number detection results of each target video frame.

[0054] The above-mentioned railway train car number detection method, device, computer equipment, storage medium and computer program product obtain continuous multiple-frame target video frames containing trains; through the feature spatiotemporal alignment module in the car number detection model, the feature maps of the continuous multiple-frame target video frames are spatiotemporally aligned to obtain the aligned feature maps corresponding to each frame of the target video frame; through the multi-branch perception module in the car number detection model, the aligned feature maps of each frame of the target video frame are subjected to car number detection processing of multiple branch modules to obtain the car number detection results of each frame of the target video frame under each branch module; the car number detection results of each frame of the target video frame under each branch module are fused separately to obtain the target car number detection results of each frame of the target video frame. By adopting this method, the continuity between consecutive target video frames can be further enhanced through spatiotemporal alignment technology, so that the vehicle license plate detection model has stronger analysis capabilities for complex natural environments and damaged vehicle license plates when processing the aligned feature maps, thereby improving the accuracy and robustness of the vehicle license plate detection results. It also further improves the accuracy of the target vehicle license plate detection results obtained by the final processing by fusing the vehicle license plate detection results output by multiple different branch modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 2. A diagram showing an application environment of a method for detecting a railway train number in one embodiment;

[0056] Figure 2 1 is a flow chart of a method for detecting a railway train number in one embodiment;

[0057] Figure 3 1. A flow chart of steps for obtaining vehicle license plate detection results for each target video frame under each branch module in one embodiment;

[0058] Figure 4 A schematic diagram of the network structure of a vehicle license plate detection model in one embodiment;

[0059] Figure 5 A schematic flow chart of a method for detecting a railway train number in another embodiment;

[0060] Figure 6 1 is a flow chart of a method for detecting a railway train number in another embodiment;

[0061] Figure 7 1 is a structural block diagram of a railway train number detection device in one embodiment;

[0062] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0065] The railway train number detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, video capture equipment is deployed in the railway scenario. The video capture equipment is used to monitor trains passing through the railway. The video capture equipment is usually installed in key sections, station entrances and exits, and other locations to ensure comprehensive coverage of every important node that the train passes. At the same time, edge processors are also deployed in the railway scenario. The edge processor can be deployed independently in the railway scenario or inside the video capture equipment. The edge processor has powerful computing power and can analyze the video data collected by the video capture equipment in real time locally, avoiding the delay caused by transmitting large amounts of data to the cloud. The edge processor can also upload the processing results to the cloud, so that the processing center can dispatch the processing results in the cloud for subsequent vehicle number recognition tasks.

[0066] In one embodiment, Figure 2 As shown, a method for detecting the train number of a railway train is provided. Figure 1 The edge processor in the example is used to illustrate the process, including the following steps:

[0067] Step S201: Acquire multiple consecutive target video frames containing a train.

[0068] Herein, the term "train" refers to a railway train, including but not limited to ordinary trains, express trains, and high-speed trains.

[0069] Specifically, video capture equipment deployed on the railway collects railway video data in real time and sends it to the edge processor. The edge processor detects the presence of a train in the video to determine whether a train has passed through the railway. If a train is detected in a video frame, the edge processor extracts multiple consecutive frames in which the train appears and sets these frames as the target video frames.

[0070] In step S202, the feature spatiotemporal alignment module in the vehicle license plate detection model is used to perform spatiotemporal alignment processing on the feature maps of multiple consecutive target video frames to obtain aligned feature maps corresponding to each target video frame.

[0071] The vehicle number detection model detects the location of train vehicle numbers in video frames. It primarily consists of a backbone network for feature extraction, a spatiotemporal feature alignment module for feature-level deformable alignment, and a multi-branch perception module for multi-level feature aggregation.

[0072] Specifically, the edge processor inputs multiple consecutive target video frames into the vehicle license plate detection model, first performs feature extraction processing on each target video frame through the backbone module to obtain the feature map of each target video frame, and then uses the feature spatiotemporal alignment model to perform dual alignment of time and space on the feature maps of multiple consecutive target video frames, and finally outputs the aligned feature map corresponding to each target video frame.

[0073] In step S203, the vehicle number detection results of each target video frame under each branch module are obtained by performing vehicle number detection processing on the aligned feature maps of each target video frame through the multi-branch perception module in the vehicle number detection model.

[0074] Among them, the multi-branch perception module includes at least two branch modules. Different branch modules perceive the position of the vehicle license plate in the detection video frame in different ways, and output vehicle license plate detection results with different accuracies to improve the comprehensiveness, accuracy and reliability of vehicle license plate detection.

[0075] Specifically, the edge processor inputs the aligned feature maps corresponding to each frame of the target video frame into each branch module included in the multi-branch perception module, so that each branch module can perform vehicle number detection processing on the aligned feature maps corresponding to each frame of the target video frame. For example, the vehicle number detection effect can be enhanced by using the vehicle number features of known train models. For example, a pixel-level prediction network can be used to perform more accurate vehicle number detection, thereby outputting the vehicle number detection results of each frame of the target video frame under different branch modules.

[0076] In step S204, the vehicle license plate detection results of each target video frame under each branch module are fused to obtain the target vehicle license plate detection results of each target video frame.

[0077] The train number is a unique identifier for each train carriage. It typically consists of a string of numbers, letters, or a combination of both, used to distinguish between different carriages, models, or trains.

[0078] The target vehicle license plate detection result refers to the information describing the location of the vehicle license plate in the target video frame. It should be noted that in this application, the target vehicle license plate detection result does not include the specific vehicle license plate text information. That is, this application detects the location information of the vehicle license plate in the edge processor, rather than detecting the specific vehicle license plate content.

[0079] Specifically, for each target video frame, the edge processor fuses the vehicle license plate detection results of each target video frame under each branch module. During the fusion process, the vehicle license plate detection results with low confidence can also be eliminated to improve the accuracy of the vehicle license plate detection results. Finally, the edge processor obtains the target vehicle license plate detection results of each target video frame.

[0080] In the above-mentioned railway train number detection method, multiple consecutive target video frames containing a train are obtained; the feature spatiotemporal alignment module in the number detection model performs spatiotemporal alignment on the feature maps of the consecutive target video frames to obtain aligned feature maps corresponding to each target video frame; the multi-branch perception module in the number detection model performs number detection on the aligned feature maps of each target video frame using multiple branch modules to obtain number detection results for each target video frame under each branch module; and the number detection results of each target video frame under each branch module are fused to obtain target number detection results for each target video frame. This method can further enhance the continuity between consecutive target video frames through spatiotemporal alignment technology, enabling the number detection model to have stronger analytical capabilities for complex natural environments and damaged number plates when processing the aligned feature maps, thereby improving the accuracy and robustness of the number detection results. Furthermore, the accuracy of the target number detection results obtained by the final processing is further improved by fusing the number detection results output by multiple different branch modules.

[0081] In one embodiment, the multi-branch sensing module includes at least a first branch module and a second branch module. Figure 3 As shown, in the above step S203, the multi-branch perception module in the vehicle license plate detection model performs vehicle license plate detection processing on the aligned feature maps of each target video frame using multiple branch modules, and obtains the vehicle license plate detection results of each target video frame under each branch module, which specifically include the following contents:

[0082] In step S301, the region proposal network in the first branch module is used to detect the vehicle license plate area on the aligned feature maps of each target video frame to obtain the candidate region feature maps of each aligned feature map.

[0083] Among them, the Region Proposal Network (RPN) is a deep learning model component for target detection, which is used to automatically generate candidate regions that may contain targets (such as vehicle license plates).

[0084] The candidate region feature map refers to a feature map that marks regions that may contain vehicle license plates.

[0085] Figure 4 The following is a schematic diagram of the network structure of the license plate detection model. Specifically, for each target video frame, the edge processor inputs the corresponding aligned feature map into the region proposal network in the first branch module (also known as the long-term feature regression perception branch). The region proposal network detects the license plate area in the aligned feature map and obtains candidate license plate regions that may contain the license plate in the aligned feature map. The candidate license plate regions are annotated in the aligned feature map, and the edge processor then obtains feature maps of the candidate regions in the aligned feature map.

[0086] Step S302 : Based on the vehicle number features of the known train models in the external feature memory, the feature maps of each candidate region are enhanced to obtain enhanced feature maps of each candidate region feature map.

[0087] Among them, the external feature memory stores the vehicle number features of multiple known train models.

[0088] Specifically, the edge processor can introduce an external feature memory into the first branch module, and perform instance-level feature enhancement processing on each candidate area feature map through the vehicle number features of known train models in the external feature memory that have a high similarity with the candidate area feature map, so as to enhance the clarity, character characteristics and other features of the vehicle number in the candidate area feature map, thereby solving the defect of unclear train number during train driving, and then obtaining an enhanced feature map of the candidate area feature map.

[0089] Step S303: Perform vehicle number detection processing on each enhanced feature map through the first vehicle number detection model in the first branch module to obtain the vehicle number detection result of each frame of the target video frame under the first branch module.

[0090] The first and second vehicle license plate detection models are constructed using detection heads with different structures and are used to detect the location of the vehicle license plate. Because the first and second vehicle license plate detection models use different detection heads, the detection accuracy of the first and second vehicle license plate detection models is also different.

[0091] Specifically, the edge processor can build a first license plate detection model based on the regression detection head. The first license plate detection model performs appearance-level feature analysis on the enhanced feature map, calculates a preset anchor box to predict the license plate area, and outputs the license plate detection result of the enhanced feature map under the first branch module. The license plate detection result of the target video frame under the first branch module can describe the location and approximate size of the detected license plate, for example, by marking the license plate detection result of the target video frame under the first branch module with a rectangular box.

[0092] It's important to note that in real-world railway scenarios, train number text can exhibit issues such as letter size variations, inconsistent train number lengths, and overlapping train numbers. Appearance-level feature analysis alone cannot accurately capture these details. Therefore, a second, adjacent branch module is built in parallel to perform pixel-level feature analysis on the feature map, achieving multi-level feature fusion analysis across "instance-appearance-pixel" levels to further improve train number detection accuracy.

[0093] Step S304: Perform semantic segmentation processing on each target video frame through the second branch module to obtain a vehicle license plate foreground image of each target video frame.

[0094] Specifically, the edge processor introduces a pixel-level prediction network based on segmentation to construct a deconvolution module in the second branch module (also known as the adjacent feature semantic perception branch). The edge processor uses the deconvolution module in the second branch module to perform semantic segmentation processing on the foreground of the vehicle license plate text on the target video frame, outputs whether each pixel in the target video frame belongs to the vehicle license plate text, and obtains the vehicle license plate foreground image of each target video frame. It can be understood that by segmenting the foreground area and background area of ​​the vehicle license plate text in the target video frame, the vehicle license plate detection process can be performed only on the foreground image of the vehicle license plate when performing vehicle license plate detection in the subsequent step, avoiding the influence of the background area on the vehicle license plate detection process, and obtaining more detailed information about the vehicle license plate text, which helps to improve the precision and accuracy of vehicle license plate detection.

[0095] Among them, the pixel-level prediction network can generate more delicate output and achieve semantic segmentation of the foreground by enlarging the low-resolution feature map and restoring it to a higher spatial size.

[0096] Step S305: Perform vehicle number detection processing on each vehicle number foreground image through the second vehicle number detection model in the second branch module to obtain the vehicle number detection result of each target video frame under the second branch module; the detection accuracy of the second vehicle number detection model is greater than the detection accuracy of the first vehicle number detection model.

[0097] Specifically, the edge processor can construct a second vehicle license plate detection model in the second branch module based on the segmentation detection head. This second vehicle license plate detection model performs pixel-level vehicle license plate detection on the vehicle license plate foreground image, generating a vehicle license plate detection result for the target video frame generated by the second branch module. This vehicle license plate detection result for the target video frame generated by the second branch module accurately describes the outline of the detected vehicle license plate.

[0098] It should be noted that the segmentation detection head and the regression detection head are two key components in the target detection task, each responsible for different tasks: (1) The role of the segmentation detection head is to generate a pixel-level segmentation mask for the target (such as the train car number) (that is, to determine the specific area of ​​the target in the image) and generate an accurate "contour" for each detected target. (2) The role of the regression detection head is to predict the parameters of the bounding box of the target (such as the train car number), output the position and size of the target (such as the center point coordinates, width and height) based on the feature map, and locate the specific position of the target in the image. It can be seen that the segmentation detection head performs pixel-level target detection, while the regression detection head performs appearance-level (the overall appearance of the target) target detection. Therefore, the target detection accuracy of the segmentation detection head is higher than that of the regression detection head.

[0099] In this embodiment, at least two branch modules in the vehicle license plate detection model are used to perform appearance-level vehicle license plate detection processing and pixel-level vehicle license plate detection processing on the aligned feature maps of each target video frame, and the output is vehicle license plate detection results of different accuracies for each target video frame under different branch modules, thereby improving the accuracy and robustness of vehicle license plate detection.

[0100] In one embodiment, the above step S302 enhances the feature maps of each candidate area based on the vehicle number features of the known train model in the external feature memory to obtain an enhanced feature map of each candidate area feature map, specifically including the following contents: if it is detected that the similarity between the candidate area feature map and the vehicle number features of the known train model reaches a preset similarity condition, then based on the vehicle number features, the vehicle number candidate area in the candidate area feature map is enhanced to obtain an enhanced feature map of the candidate area feature map.

[0101] It should be noted that, since the feature spatiotemporal alignment module aligns the temporal features and spatial features of the feature maps of multiple consecutive target video frames, although it can enhance the temporal consistency of multiple consecutive target video frames, it may also cause the aligned feature maps of adjacent target video frames to have the same appearance deterioration (such as blurring, deformation of vehicle license plates, and other poor appearance conditions), which weakens the advantages of feature aggregation. Therefore, the present application also provides more reliable and richer feature details by expanding the scope of feature aggregation and selecting vehicle license plate features in a long-term and global external feature memory.

[0102] The similarity condition is used to determine the degree of feature similarity between a candidate region in the candidate region feature map that may contain a vehicle number and vehicle number features of each known train model in the external feature memory. For example, the similarity condition may be that the feature similarity between the candidate region and the vehicle number features of each known train model is not less than a preset similarity threshold.

[0103] Specifically, the edge processor calculates the feature similarity between each vehicle number candidate area that may contain the vehicle number in the candidate area feature map and the vehicle number features of each known train model in the external feature memory; if the feature similarity is detected to be not less than the preset similarity threshold, it can be considered that the train belongs to a known train model, and then the edge processor can enhance the vehicle number candidate area in the candidate area feature map based on the vehicle number features corresponding to the feature similarity, which can be to strengthen the detailed features of the vehicle number, correct the deformation of the vehicle number, provide richer feature information, and then the edge processor obtains an enhanced feature map.

[0104] The above-mentioned railway train car number detection method also includes: if it is detected that the similarity does not meet the preset similarity condition, the external feature memory is updated based on the car number candidate area in the candidate area feature map.

[0105] Specifically, if the feature similarity is detected to be less than a preset similarity threshold, it can be considered that the train does not belong to a known train model, and then the edge processor can store the vehicle number candidate area as a new vehicle number feature in the external feature memory to expand the vehicle number feature in the external feature memory.

[0106] In this embodiment, by detecting whether the similarity between the candidate area feature map and the vehicle number features of the known train model reaches a preset similarity condition, it is selected whether to enhance the vehicle number candidate area in the candidate area feature map based on the vehicle number features, thereby obtaining an enhanced feature map of the candidate area feature map, or to update the external feature memory based on the vehicle number candidate area. The rich vehicle number features in the external feature memory can be used to enhance the feature information of the candidate area feature map, and the vehicle number features in the external feature memory can be expanded, so that the accuracy of vehicle number detection is further improved.

[0107] In one embodiment, the above-mentioned step S204 performs fusion processing on the vehicle number detection results of each frame of target video frame under each branch module to obtain the target vehicle number detection result of each frame of target video frame, which specifically includes the following contents: determining the first confidence level of the vehicle number detection result of each frame of target video frame under the first branch module, and determining the second confidence level of the vehicle number detection result of each frame of target video frame under the second branch module; based on the first confidence level and the second confidence level, the vehicle number detection result of each frame of target video frame under the first branch module and the vehicle number detection result under the second branch module are fused to obtain the target vehicle number detection result of each frame of target video frame.

[0108] The first confidence level and the second confidence level are used to measure the reliability of the vehicle license plate detection results output by each branch module.

[0109] Specifically, for each target video frame, the edge processor can calculate the confidence of the vehicle number detection result of the target video frame under each branch module separately, or combine the vehicle number detection results of the target video frame under all branch modules to calculate the confidence of each vehicle number detection result, and then the edge processor calculates the first confidence of the vehicle number detection result of each target video frame under the first branch module, and obtains the second confidence of the vehicle number detection result of each target video frame under the second branch module. Based on the first confidence and the second confidence, the edge processor can retain the vehicle number detection results whose first confidence or second confidence exceeds the preset confidence threshold, and can also fuse the vehicle number detection results whose position information represented by the vehicle number detection results overlap, and can also eliminate the vehicle number detection results whose first confidence or second confidence does not exceed the preset confidence threshold, and set the vehicle number detection result obtained by the final processing as the target vehicle number detection result of each target video frame.

[0110] In this embodiment, based on the first confidence level and the second confidence level, the vehicle license plate detection results of each target video frame under the first branch module and the vehicle license plate detection results under the second branch module are fused. This can remove the vehicle license plate detection results with lower confidence levels, and can also fuse the vehicle license plate detection results with overlapping position information into one, ultimately obtaining a more accurate and concise target vehicle license plate detection result.

[0111] In one embodiment, the above step S202, through the feature spatiotemporal alignment module in the vehicle license plate detection model, performs spatiotemporal alignment processing on the feature maps of multiple consecutive target video frames to obtain aligned feature maps corresponding to each target video frame, specifically including the following contents: through the backbone network in the vehicle license plate detection model, performs feature extraction processing on the vehicle license plate text features of each target video frame to obtain the feature map of each target video frame; through the feature spatiotemporal alignment module, performs temporal alignment processing and spatial alignment processing on the feature maps of multiple consecutive target video frames to obtain the aligned feature maps corresponding to each target video frame.

[0112] Among them, the feature map is used to represent the vehicle license plate text features of the video frame.

[0113] Specifically, the edge processor can use a multi-layer (for example, 50-layer) ResNet as the backbone network, and use the backbone network to extract the vehicle license plate text features of each input target video frame to obtain a feature map of each target video frame.

[0114] It should be noted that, since the trains in the railway scene are often in motion, and the video acquisition equipment is in a complex and harsh line environment, the train number is prone to motion blur and camera defocusing, which causes great trouble for the positioning of the train number text area. The deterioration of the picture quality can be alleviated by the contextual information in the video, but there are also differences in spatial information between the train number text targets at different timestamps. Therefore, the present application proposes a feature spatiotemporal alignment module including a parallel expansion module, a multi-scale cascade structure and a deformable convolution. Before the multi-frame feature maps are fused and enhanced, the features of the feature maps of each frame at different times and spaces are aligned. The temporal features and spatial features of the feature maps of each frame can be aligned by a multi-scale cascade structure and a deformable convolution, and finally the aligned feature maps corresponding to the target video frames of each frame are obtained.

[0115] Among them, the temporal feature is used to describe the dynamic changes between frames of the feature map, such as the motion trajectory of the vehicle number, etc. The spatial feature is used to describe the static information in each frame of the feature map, such as the shape and color of the vehicle number, etc.

[0116] Furthermore, the edge processor can pre-train the vehicle license plate detection model to be trained using video frame samples to obtain a trained vehicle license plate detection model. Figure 4As shown, during the training process of the vehicle number detection model to be trained, the edge processor can extract the current key frame and the adjacent reference frames of the current key frame from the video frame samples, such as determining the current key frame from multiple consecutive video frame samples, and then using the previous video frame and the next video frame of the current key frame as adjacent reference frames, and then inputting the current key frame and the two adjacent reference frames of the current key frame into the vehicle number detection model to be trained; through the backbone network in the vehicle number detection model to be trained, the current key frame and the two adjacent reference frames of the current key frame are subjected to feature extraction processing, and the feature map of the current key frame and the feature map of the adjacent reference frames of each frame are obtained respectively; through the feature spatiotemporal alignment module in the vehicle number detection model to be trained, the previous adjacent reference frame, the current key frame and the subsequent adjacent reference frame are subjected to spatiotemporal alignment processing. The method performs time alignment processing and space alignment processing to obtain the aligned feature map of the previous adjacent reference frame, the aligned feature map of the current key frame and the aligned feature map of the subsequent adjacent reference frame; through the multi-branch perception module in the vehicle number detection model to be trained, the aligned feature map of the previous adjacent reference frame, the aligned feature map of the current key frame and the aligned feature map of the subsequent adjacent reference frame are subjected to vehicle number detection processing of multiple branch modules to obtain the vehicle number detection results of the previous adjacent reference frame under each branch module, the vehicle number detection results of the current key frame under each branch module and the vehicle number detection results of the subsequent adjacent reference frame under each branch module; according to the difference between the vehicle number detection result and the actual vehicle number position, the model parameters of the vehicle number detection model to be trained are iteratively updated to obtain the trained vehicle number detection model.

[0117] In this embodiment, the backbone network in the vehicle license plate detection model is first used to perform feature extraction processing on the vehicle license plate text features of each frame of the target video frame to preliminarily extract the feature map of each frame of the target video frame; then, the feature spatiotemporal alignment module is used to perform more accurate temporal alignment processing and spatial alignment processing on the feature maps of multiple consecutive frames of target video frames, thereby greatly improving the accuracy and feature quality of the aligned feature maps corresponding to each frame of the target video frame, and also improving the accuracy of vehicle license plate detection using the aligned feature maps in subsequent steps.

[0118] In one embodiment, the above-mentioned step S201, obtaining a continuous multi-frame target video frame containing a train, specifically includes the following contents: using a target detection model, performing target detection processing for the train on the continuous multi-frame current video frame of the railway to obtain the target detection result of the current video frame; based on the target detection result, determining the target video frame containing the train from the continuous multi-frame current video frame.

[0119] An object detection model is an AI model that detects whether a specified object (such as a train) appears in a video frame. For example, a lightweight YOLOV5 detection network can be used to build an object detection model.

[0120] Specifically, the video data of the railway collected in real time by the monitoring equipment includes multiple consecutive frames of current video frames. The edge processor uses a lightweight target detection model to perform target detection processing for the train on the received multiple consecutive frames of current video frames, so as to quickly detect in real time whether a train appears in the video data and obtain the target detection results of each frame of the current video frame. If the target detection result indicates that a train appears in the current video frame, the current video frame is set as the target video frame. Moreover, it can be understood that a train is composed of multiple consecutive carriages, so when a train passes by, a train will appear in multiple consecutive frames of video data until the train completely leaves, so the edge processor can obtain multiple consecutive frames of target video frames.

[0121] In this embodiment, a lightweight target detection model is used to perform target detection processing for trains on multiple consecutive frames of current video frames of the railway, so as to quickly detect in real time whether a train appears in the current video frame, thereby quickly determining whether a train passes through the railway, and then extracting target video frames containing trains from multiple consecutive frames of current video frames based on the target detection results, providing a reliable data source for subsequent steps of detecting the train number.

[0122] In one embodiment, in the above step S204, after the vehicle number detection results of each target video frame under each branch module are fused and processed to obtain the target vehicle number detection results of each target video frame, it also includes: based on the target vehicle number detection results of each target video frame, extracting the vehicle number images corresponding to each target vehicle number detection result from each target video frame; sending multiple vehicle number images to the cloud for the cloud to perform vehicle number recognition processing on the vehicle number images.

[0123] Specifically, the edge processor can also extract the vehicle number image of each target video frame from each target video frame based on the target vehicle number detection result of each target video frame. For example, according to the vehicle number position information corresponding to the target vehicle number detection result, the image corresponding to the vehicle number position information is cropped out from the target video frame to obtain the vehicle number image corresponding to the target vehicle number detection result of each frame. The edge processor sends the target vehicle number detection result, the vehicle number image corresponding to the target vehicle number detection result, and the confidence of the target vehicle number detection result to the cloud, so that the cloud can perform subsequent vehicle number recognition tasks and other processing tasks on the received vehicle number image, such as identifying the specific vehicle number content. The vehicle number recognition result can also be sent to the processing center for the staff of the processing center to view the train vehicle number recognition result.

[0124] In this embodiment, the edge processor has powerful computing capabilities and can quickly analyze video data locally, avoiding delays caused by transmitting large amounts of data to the cloud. By sending the target vehicle license plate detection results and their confidence levels, as well as the vehicle license plate images in each target video frame to the cloud, the cloud can perform more specific vehicle license plate recognition processing on the vehicle license plate images based on the received data, thereby reducing the task processing workload on the cloud.

[0125] In one embodiment, Figure 5 As shown, another railway train number detection method is provided, which is applied to Figure 1 The edge processor in the example is used to illustrate the process, including the following steps:

[0126] Step S501 : Using a target detection model, target detection processing is performed on multiple consecutive frames of current video of the railway to obtain target detection results of the current video frames.

[0127] Step S502 : Based on the target detection result, a target video frame containing a train is determined from the multiple consecutive current video frames.

[0128] In step S503, the backbone network in the vehicle license plate detection model is used to extract the vehicle license plate text features of each target video frame to obtain a feature map of each target video frame.

[0129] In step S504, a feature spatiotemporal alignment module is used to perform temporal alignment and spatial alignment on the feature maps of multiple consecutive target video frames to obtain aligned feature maps corresponding to each target video frame.

[0130] In step S505, the region proposal network in the first branch module is used to detect the vehicle license plate area on the aligned feature maps of each target video frame to obtain candidate region feature maps of each aligned feature map.

[0131] Step S506: Based on the vehicle number features of the known train models in the external feature memory, the feature maps of each candidate region are enhanced to obtain enhanced feature maps of each candidate region feature map.

[0132] Step S507, performing vehicle number detection processing on each enhanced feature map through the first vehicle number detection model in the first branch module of the multi-branch perception module of the vehicle number detection model, and obtaining the vehicle number detection result of each frame of the target video frame under the first branch module.

[0133] Step S508: Perform semantic segmentation processing on each target video frame through the second branch module in the multi-branch perception module to obtain a vehicle license plate foreground image of each target video frame.

[0134] In step S509, the second vehicle number detection model in the second branch module is used to perform vehicle number detection processing on each vehicle number foreground image to obtain the vehicle number detection result of each target video frame under the second branch module; the detection accuracy of the second vehicle number detection model is greater than the detection accuracy of the first vehicle number detection model.

[0135] In step S510 , the vehicle license plate detection results of each target video frame under each branch module are fused to obtain the target vehicle license plate detection results of each target video frame.

[0136] The above-mentioned railway train vehicle number detection method can achieve the following beneficial effects: it can further enhance the continuity between continuous target video frames through spatiotemporal alignment technology, so that the vehicle number detection model has stronger analysis capabilities for complex natural environments and vehicle number damage when processing the aligned feature map, thereby improving the accuracy and robustness of the vehicle number detection results, and further improving the accuracy of the target vehicle number detection results obtained by the final processing by fusing the vehicle number detection results output by multiple different branch modules.

[0137] In order to more clearly illustrate the railway train number detection method provided by the embodiment of the present disclosure, the railway train number detection method is specifically described below with a specific embodiment. Figure 6 As shown, another railway train number detection method is provided, which can be applied to Figure 1 The edge processor in the , specifically includes the following:

[0138] (1) Start detection

[0139] Deploy video acquisition equipment to collect railway video data in real time, load the vehicle number detection model into the edge processor, introduce external feature memory, and trigger the "start detection" command after completing multiple initialization tasks.

[0140] (2) Is there a train passing by?

[0141] A lightweight YOLOV5 detection network is used to quickly and in real time detect whether a train appears in the video frames captured by the video acquisition device.

[0142] (3) Vehicle license plate detection model

[0143] If a vehicle is detected passing by, the license plate detection model based on multi-level aggregation of video spatiotemporal features simultaneously considers the spatial and temporal features in the video frame; through the multi-scale deformable convolution, dual-branch feature perception network and dynamic adaptive external feature memory with proposal quality evaluation of the license plate detection model, it realizes the aggregation of appearance-instance-pixel multi-level features, accurately locates the license plate position in the video frame in all aspects, and outputs the license plate position recognition result.

[0144] (4) Update external feature memory

[0145] When an external feature memory is introduced to assist the vehicle number detection model in vehicle number detection processing, feature matching processing can also be performed on the feature map of the video frame and the vehicle number features of known train models stored in the external feature memory to determine the similarity between the feature map of the video frame and the vehicle number features of the known train models. If it is detected that the feature map of the video frame is highly consistent with the vehicle number features of the known train models, it is considered that it belongs to the known train models, and the feature map of the current video frame can be enhanced by the content in the external feature memory. If there is no corresponding similar vehicle number feature in the external feature memory, the vehicle number feature of this type is updated and stored, and the content of the vehicle number feature in the external feature memory is expanded.

[0146] (5) Output target detection results

[0147] After the video frames pass through the network, the final detection results are output. This result includes the confidence level of the detected license plate number and its location information. The system uses this information to identify the specific license plate area in the image. After the current vehicle passes, the license plate area information is uploaded to the cloud, where the processing center performs the next step of identification and processing.

[0148] In this embodiment, the following beneficial effects can be achieved: First, the spatiotemporal feature deformable alignment module proposed in the vehicle license plate detection model of this application realizes feature-level alignment between video frames. This approach enhances the features of the current keyframe through temporal and spatial contextual information, enabling more accurate capture of vehicle license plate information in complex environments such as severe weather, changing lighting, and damaged vehicles. Compared with traditional single-frame image-based methods, this spatiotemporal feature alignment technology can fully utilize the continuity of the video, significantly improving the robustness and accuracy of detection.

[0149] Secondly, this application designs a dual-branch perception structure of instance-appearance-pixel multi-level feature aggregation in the vehicle license plate detection model. This structure fully utilizes the rich information in the video through multi-level aggregation of instance-level features, appearance-level features and pixel-level features. This multi-level feature aggregation method can not only effectively overcome problems such as fading, pollution, occlusion and deformation of vehicle license plate characters, but also aggregate instance-level long-term features through the dynamic adaptive external feature memory of the proposal area quality evaluation index, thereby enhancing the appearance perception of the vehicle license plate target in the current frame. This method ensures that the vehicle license plate can be accurately identified even when the characters are discontinuous or missing.

[0150] Compared to existing methods such as Faster R-CNN (Faster Region-based Convolutional Neural Network) and GSO-BP (Group Search Optimization-Backpropagation) algorithms, this application not only achieves more refined feature extraction but also, by introducing a dynamic adaptive external feature memory, enables edge processors to perform better when handling complex backgrounds and changing lighting. Furthermore, the introduction of a pixel-level foreground semantic feature perception branch further refines the prediction of text box areas, further improving the accuracy of vehicle license plate detection.

[0151] In summary, the present invention has higher robustness and accuracy in processing the task of train number detection in railway scene videos under harsh environments. Through innovative spatiotemporal feature alignment and multi-level feature aggregation methods, it significantly improves the overall performance of the system and solves many problems existing in the existing technology.

[0152] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0153] Based on the same inventive concept, embodiments of the present application further provide a railway train number detection device for implementing the aforementioned railway train number detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more railway train number detection device embodiments provided below can be found in the above-described limitations of the railway train number detection method and will not be further elaborated here.

[0154] In one embodiment, Figure 7 As shown, a railway train number detection device 700 is provided, comprising: a video frame acquisition module 701, a spatiotemporal alignment module 702, a train number detection module 703 and a result fusion module 704, wherein:

[0155] The video frame acquisition module 701 is used to acquire multiple consecutive target video frames containing trains.

[0156] The spatiotemporal alignment module 702 is used to perform spatiotemporal alignment processing on the feature maps of multiple consecutive target video frames through the feature spatiotemporal alignment module in the vehicle license plate detection model to obtain the aligned feature maps corresponding to each target video frame.

[0157] The vehicle number detection module 703 is used to perform vehicle number detection processing on the aligned feature maps of each frame of target video frame using the multi-branch perception module in the vehicle number detection model using multiple branch modules to obtain the vehicle number detection results of each frame of target video frame under each branch module.

[0158] The result fusion module 704 is used to fuse the vehicle license plate detection results of each target video frame under each branch module to obtain the target vehicle license plate detection results of each target video frame.

[0159] In one embodiment, the multi-branch perception module includes at least a first branch module and a second branch module. The vehicle number detection module 703 is further configured to perform vehicle number region detection processing on the aligned feature maps of each target video frame through the region proposal network in the first branch module to obtain candidate region feature maps of each aligned feature map; enhance each candidate region feature map based on the vehicle number features of the known train model in the external feature memory to obtain enhanced feature maps of each candidate region feature map; perform vehicle number detection processing on each enhanced feature map through the first vehicle number detection model in the first branch module to obtain vehicle number detection results of each target video frame under the first branch module; perform semantic segmentation processing on each target video frame through the second branch module to obtain vehicle number foreground maps of each target video frame; perform vehicle number detection processing on each vehicle number foreground map through the second vehicle number detection model in the second branch module to obtain vehicle number detection results of each target video frame under the second branch module; the detection accuracy of the second vehicle number detection model is greater than the detection accuracy of the first vehicle number detection model.

[0160] In one embodiment, the railway train vehicle number detection device 700 further includes a feature enhancement module for enhancing the vehicle number candidate region in the candidate region feature map based on the vehicle number feature if the similarity between the candidate region feature map and the vehicle number feature of a known train model meets a preset similarity condition. The railway train vehicle number detection device 700 further includes a memory update module for updating an external feature memory based on the vehicle number candidate region in the candidate region feature map if the similarity between the candidate region feature map and the vehicle number feature of a known train model meets a preset similarity condition.

[0161] In one embodiment, the result fusion module 704 is also used to determine the first confidence level of the vehicle license plate detection results of each target video frame under the first branch module, and to determine the second confidence level of the vehicle license plate detection results of each target video frame under the second branch module; based on the first confidence level and the second confidence level, the vehicle license plate detection results of each target video frame under the first branch module and the vehicle license plate detection results under the second branch module are fused to obtain the target vehicle license plate detection results of each target video frame.

[0162] In one embodiment, the spatiotemporal alignment module 702 is also used to perform feature extraction processing of the vehicle license plate text features on each target video frame through the backbone network in the vehicle license plate detection model to obtain a feature map of each target video frame; and to perform temporal alignment processing and spatial alignment processing on the feature maps of multiple consecutive target video frames through the feature spatiotemporal alignment module to obtain an aligned feature map corresponding to each target video frame.

[0163] In one embodiment, the video frame acquisition module 701 is also used to perform target detection processing for trains on multiple consecutive frames of current video frames of the railway through a target detection model to obtain target detection results of the current video frames; based on the target detection results, determine the target video frame containing the train from the multiple consecutive frames of current video frames.

[0164] In one embodiment, the railway train vehicle number detection device 700 also includes a result uploading module, which is used to extract the vehicle number images corresponding to each target vehicle number detection result from each target video frame based on the target vehicle number detection result of each frame of the target video frame; and send multiple vehicle number images to the cloud for the cloud to perform vehicle number recognition processing on the vehicle number images.

[0165] Each module in the above-mentioned railway train number detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0166] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data such as target video frames, aligned feature maps, and target vehicle number detection results. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting vehicle numbers of railway trains is implemented.

[0167] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0168] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0170] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0171] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0172] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting a railway train number, characterized in that: The method comprises: Obtain multiple consecutive target video frames containing trains; Through the feature spatiotemporal alignment module in the vehicle number detection model, the feature maps of the target video frames of the consecutive frames are spatiotemporally aligned to obtain the aligned feature maps corresponding to the target video frames of each frame; Through the multi-branch perception module in the vehicle number detection model, the aligned feature map of each frame of the target video frame is processed by multiple branch modules for vehicle number detection, and the vehicle number detection result of each frame of the target video frame under each branch module is obtained; The vehicle number detection results of each target video frame under each branch module are fused to obtain the target vehicle number detection results of each target video frame.

2. The method according to claim 1, characterized in that The multi-branch sensing module includes at least a first branch module and a second branch module; The multi-branch perception module in the vehicle number detection model performs vehicle number detection processing on the aligned feature map of each frame of the target video frame using multiple branch modules, and obtains the vehicle number detection results of each frame of the target video frame under each branch module, including: Performing vehicle license plate area detection on the aligned feature maps of the target video frames through the region proposal network in the first branch module to obtain candidate region feature maps of each aligned feature map; Based on the known train model number features in the external feature memory, each candidate region feature map is enhanced to obtain an enhanced feature map of each candidate region feature map; Performing vehicle number detection processing on each of the enhanced feature maps using the first vehicle number detection model in the first branch module to obtain a vehicle number detection result of each target video frame under the first branch module; Performing semantic segmentation processing on each target video frame through the second branch module to obtain a vehicle number foreground image of each target video frame; The second vehicle number detection model in the second branch module is used to perform vehicle number detection processing on each of the vehicle number foreground images to obtain the vehicle number detection results of each frame of the target video frame under the second branch module; the detection accuracy of the second vehicle number detection model is greater than the detection accuracy of the first vehicle number detection model.

3. The method according to claim 2, characterized in that The method of performing enhancement processing on each of the candidate region feature maps based on the vehicle number feature in the external feature memory to obtain an enhanced feature map of the candidate region feature map includes: If it is detected that the similarity between the candidate region feature map and the vehicle number feature of the known train model reaches a preset similarity condition, then based on the vehicle number feature, the vehicle number candidate region in the candidate region feature map is enhanced to obtain an enhanced feature map of the candidate region feature map; The method further comprises: If it is detected that the similarity does not meet the preset similarity condition, the external feature memory is updated based on the vehicle number candidate area in the candidate area feature map.

4. The method according to claim 2, characterized in that The vehicle number detection results of each target video frame under each branch module are respectively fused to obtain the target vehicle number detection results of each target video frame, including: Determine a first confidence level of a vehicle license plate detection result of each target video frame under the first branch module, and determine a second confidence level of a vehicle license plate detection result of each target video frame under the second branch module; Based on the first confidence level and the second confidence level, the vehicle number detection results of the target video frames in each frame under the first branch module and the vehicle number detection results under the second branch module are fused to obtain the target vehicle number detection results of the target video frames in each frame.

5. The method according to claim 1, wherein The feature spatiotemporal alignment module in the vehicle number detection model performs spatiotemporal alignment processing on the feature maps corresponding to the target video frames of the consecutive frames to obtain the aligned feature maps corresponding to the target video frames of each frame, including: Through the backbone network in the vehicle number detection model, feature extraction processing of the vehicle number text feature is performed on each frame of the target video frame to obtain a feature map of each frame of the target video frame; The feature spatiotemporal alignment module performs temporal alignment and spatial alignment on the feature maps of multiple consecutive target video frames to obtain aligned feature maps corresponding to the target video frames of each frame.

6. The method according to claim 1, characterized in that The step of obtaining a plurality of continuous target video frames including a train includes: Performing target detection processing on multiple consecutive current video frames of the railway for the train through the target detection model to obtain target detection results of the current video frames; Based on the target detection result, a target video frame containing the train is determined from multiple consecutive frames of the current video frame.

7. The method according to claim 1, characterized in that After fusing the vehicle number detection results of each target video frame under each branch module to obtain the target vehicle number detection results of each target video frame, the method further includes: Based on the target vehicle number detection results of each target video frame, respectively extracting the vehicle number images corresponding to each target vehicle number detection result from each target video frame; The plurality of vehicle number images are sent to the cloud, so that the cloud performs vehicle number recognition processing on the vehicle number images.

8. A railway train number detection device, characterized in that: The device comprises: A video frame acquisition module is used to acquire multiple consecutive target video frames containing trains; A spatiotemporal alignment module is used to perform spatiotemporal alignment processing on the feature maps of multiple consecutive target video frames through the feature spatiotemporal alignment module in the vehicle license plate detection model to obtain an aligned feature map corresponding to each target video frame; A vehicle number detection module is configured to perform vehicle number detection processing on the aligned feature maps of each target video frame using multiple branch modules through the multi-branch perception module in the vehicle number detection model, and obtain vehicle number detection results of each target video frame under each branch module; The result fusion module is used to fuse the vehicle license plate detection results of each target video frame under each branch module to obtain the target vehicle license plate detection results of each target video frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images