Image detection method, system and equipment for accurately positioning foreign matters on rail surface and medium

Abstract

The invention provides an image detection method, system and device for accurately positioning foreign matters on a rail surface and a medium. The method comprises the following steps: acquiring and decoding a track real-time video stream to obtain a track image; performing change detection on the track image, and identifying candidate foreign matters; tracking the motion trail of each candidate foreign matter, and extracting an effective trail; and inputting the track image into a target identification network, and screening out the target foreign matter in the track image. Change detection is carried out on the track image, and interference of dynamic transformation of a local background on monitoring is effectively avoided. The motion trail of the candidate foreign matter is further tracked, and the influence of motion changes on candidate foreign matter recognition is eliminated. The target recognition network is trained by generating a large number of various sample sets, and the accuracy of target foreign matter recognition is improved. According to the invention, the monitoring cost is reduced, the accuracy of track foreign matter monitoring is improved, the scene adaptability is very high, accurate foreign matters can be quickly identified, and the line driving safety is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and railway track transportation safety, and in particular to a method, system, equipment and medium for accurately locating rail surface foreign body image detection. Background Art

[0002] In recent years, my country's railway industry has developed rapidly, especially high-speed rail, passenger dedicated lines, freight, etc., which have played an important role in people's daily lives. However, railway lines cover a wide range and often need to pass through geologically unstable areas and extreme weather areas. In this scenario, disasters such as rockfalls and mudslides are prone to occur, causing economic losses to the line and endangering the safety of people's lives. Therefore, track rockfall intrusion monitoring has important safety and economic significance in the railway transportation system.

[0003] At present, there are three common monitoring methods for rockfall intrusion on railway lines: (I) Surveillance video monitoring: Manual monitoring of abnormalities in surveillance videos is conducted continuously. This method is subject to subjective influences, and abnormal judgments are prone to ambiguity. In addition, the amount of line video is large, and labor costs and labor status are challenged.

[0004] (II) Designated target analysis and identification technology. Deep learning target detection technology is used to identify falling rocks on the line. However, the line is an open scene, and most of the abnormalities that occur are rare events. How to identify falling rocks from a few samples or zero samples has always been one of the challenges faced by the line.

[0005] (III) Geological sensor technology. By installing it on a mountain or slope, it monitors its internal parameters and then determines whether there is a risk of rockfall / mudslide. This method is affected by the installation location and soil conditions, the accuracy of the signal source is limited, and the installation cost is high, requiring regular maintenance and calibration. Summary of the invention

[0006] The present invention provides a method, system, device and medium for accurately locating rail surface foreign body image detection, which solves the problems of existing railway rockfall intrusion monitoring methods, such as high monitoring cost, great influence by environment and weather conditions, low monitoring accuracy, and scarcity of sample data that cannot be identified.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is to provide a method for accurately locating rail surface foreign object image detection, comprising the steps of: collecting and acquiring a real-time track video stream, decoding the real-time track video stream, and obtaining a plurality of continuous track images; sequentially performing change detection on the plurality of track images to identify all dynamic candidate foreign objects; using the plurality of track images, respectively tracking and analyzing the motion trajectory of each candidate foreign object, and extracting the effective trajectory of each candidate foreign object; selecting each of the track images having the candidate foreign object, sequentially inputting them into a trained target recognition network, and using the effective trajectory to screen out the target foreign object from the plurality of candidate foreign objects.

[0008] In some embodiments, the step of sequentially detecting changes in a plurality of the track images and identifying all dynamic candidate foreign objects specifically includes: calculating local similarity features of the plurality of the track images to establish an initial background image; setting an automatic update strategy to update the initial background image according to the track images to obtain a target background image; respectively calculating the first foreground value of all pixels in each of the track images and the second foreground value of all pixels in the target background image, comparing the first foreground value with its corresponding second foreground value, and extracting a plurality of initial foreground images from the plurality of the track images; post-processing the initial foreground image to obtain a target foreground image, and identifying all the candidate foreign objects from the plurality of the track images according to the target foreground image.

[0009] In some embodiments, in the step of setting an automatic update strategy, updating the initial background image according to the track image, and obtaining the target background image, specifically includes: the automatic update strategy includes a comprehensive threshold and a background update rate; according to the track image, adjusting the comprehensive threshold and background update rate of each pixel in the initial background image to obtain the target background image.

[0010] In some embodiments, the step of using multiple track images to track and analyze the motion trajectory of each candidate foreign object and extracting the effective trajectory of each candidate foreign object specifically includes: downsampling and matching the candidate foreign objects in two adjacent track images in turn to obtain a first matching result; predicting and matching the position and motion state of the candidate foreign object in each track image in turn to obtain a second matching result; associating the first matching result with the second matching result, updating the final position of each candidate foreign object according to the association result, and obtaining the candidate trajectory associated with each candidate foreign object; formulating a trajectory management strategy, and according to the trajectory management strategy, filtering out the effective trajectory uniquely corresponding to each candidate foreign object from the candidate trajectories.

[0011] In some embodiments, before the step of predicting and matching the position and motion state of the candidate foreign object in each of the track images in turn, the step of performing camera motion compensation operation and correction on the candidate foreign object in each of the track images is also included.

[0012] In some embodiments, in the step of selecting each of the track images having the candidate foreign object and inputting them into the trained target recognition network in sequence, the training method of the target recognition network includes the following steps: a first step, constructing a guidance network and an initial recognition network to obtain a plurality of different historical target foreign object images; a second step, inputting the historical target foreign object images into the guidance network and the initial recognition network respectively to obtain guidance prediction results and initial prediction results respectively; a third step, adjusting the parameters of the initial recognition network using the knowledge distillation loss function according to the guidance prediction results and the initial prediction results; a fourth step, repeating the second and third steps until the knowledge distillation loss function converges, and using the initial recognition network with adjusted parameters as the trained target recognition network.

[0013] In some embodiments, the second step also includes the steps of: constructing a pseudo image generation network, inputting the historical target foreign body image into the pseudo image generation network to generate a pseudo target image, and inputting the pseudo target image into the guidance network and the initial recognition network respectively.

[0014] The present invention also provides a system for accurately locating rail surface foreign body image detection, comprising: an acquisition and decoding unit, used for acquiring and obtaining a real-time track video stream, decoding the real-time track video stream, and obtaining a plurality of continuous track images; a change detection unit, used for sequentially performing change detection on a plurality of the track images, and identifying all dynamic candidate foreign bodies; a tracking and analysis unit, used for using a plurality of the track images to respectively track and analyze the motion trajectory of each of the candidate foreign bodies, and extracting the effective trajectory of each of the candidate foreign bodies; a screening and identification unit, used for selecting each of the track images having the candidate foreign bodies, sequentially inputting them into a trained target recognition network, and using the effective trajectory to screen out a target foreign body from a plurality of the candidate foreign bodies.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the computer program.

[0016] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0017] The beneficial effects of the present invention are as follows: the present application provides a method, system, device and medium for accurately locating the image detection of foreign objects on the rail surface. The method comprises the steps of: collecting and acquiring the real-time video stream of the track, decoding the real-time video stream of the track, and obtaining multiple continuous track images; performing change detection on multiple track images in turn to identify all dynamic candidate foreign objects; using multiple track images, tracking and analyzing the motion trajectory of each candidate foreign object respectively, and extracting the effective trajectory of each candidate foreign object; selecting each track image with a candidate foreign object, inputting it into the trained target recognition network in turn, and using the effective trajectory to screen out the target foreign object from multiple candidate foreign objects. The present application effectively avoids the interference of the dynamic transformation of the local background of the image on the monitoring by performing change detection on multiple track images, and at the same time filters out the interference items of the complex environment. The motion trajectory of the candidate foreign object is further tracked to eliminate the influence of motion changes on the identification of the candidate foreign object. The target recognition network is trained by generating a large number of diverse sample sets, thereby improving the accuracy of the target foreign object identification. This application not only reduces monitoring costs and improves the accuracy of track foreign object monitoring, but also has strong scene adaptability, can quickly identify accurate foreign objects and ensure line driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a method for accurately locating rail surface foreign matter image detection in the present application; Figure 2 It is a schematic diagram of a target background image in a method for accurately locating rail surface foreign matter image detection of the present application; Figure 3 It is a schematic diagram of a target foreground image in a method for accurately locating rail surface foreign matter image detection of the present application; Figure 4 It is a schematic diagram of one of the track images in a method for accurately locating foreign matter on a track surface in the present application; Figure 5 It is a schematic diagram of downsampling processing in a method for accurately locating rail surface foreign body image detection in the present application; Figure 6 It is a data flow diagram of S3 in a method for accurately locating rail surface foreign matter image detection in the present application; Figure 7 This is a schematic diagram of the training of the initial recognition network in a method for accurately locating rail surface foreign body image detection in the present application; Figure 8 This is a schematic diagram of the composition of a system for accurately locating rail surface foreign matter image detection in the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application; Fig.10 It is a schematic block diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0019] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0020] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0021] like Figure 1 As shown, the present invention provides a method for accurately locating rail surface foreign body image detection, comprising the steps of: S1: Collect and obtain the track real-time video stream, decode the track real-time video stream, and obtain multiple continuous track images.

[0022] S2: Perform change detection on multiple track images in sequence to identify all dynamic candidate foreign objects.

[0023] S3: Using multiple track images, the motion trajectory of each candidate foreign object is tracked and analyzed respectively, and the effective trajectory of each candidate foreign object is extracted.

[0024] S4: Select each track image with candidate foreign objects, input them into the trained target recognition network in sequence, and use the effective track to screen out the target foreign object from multiple candidate foreign objects.

[0025] This application effectively avoids the interference of dynamic changes in the local background of the image on monitoring by performing change detection on multiple track images, and at the same time filters out interference items in complex environments. The motion trajectory of the candidate foreign object is further tracked to eliminate the impact of motion changes on the identification of the candidate foreign object. The target recognition network is trained by generating a large number of diverse sample sets, which improves the accuracy of target foreign object identification. This application not only reduces monitoring costs and improves the accuracy of track foreign object monitoring, but also has strong scene adaptability, can quickly identify accurate foreign objects, and ensure line driving safety.

[0026] Combine the following Figures 1 to 10 , the present application is further described in detail with specific embodiments.

[0027] Figure 1 The following is a flow chart of a method for accurately locating rail surface foreign matter image detection provided by an embodiment of the present application, which is described in detail as follows: S1: Collect and obtain the track real-time video stream, decode the track real-time video stream, and obtain multiple continuous track images.

[0028] Among them, the present application uses multiple cameras to monitor the track line in real time and generate a real-time video stream of the track.

[0029] The acquired real-time track video stream is further decoded by a video analysis unit to obtain a plurality of continuous track images.

[0030] In this embodiment, the video analysis unit is a domestic intelligent analysis device equipped with a high-performance video processing chip that can perform hardware decoding on the real-time video stream of the track. Hardware decoding not only speeds up the decoding process, but also can decode the real-time video streams of the track of multiple cameras at the same time, further improving the foreign body recognition speed of this application.

[0031] S2: Perform change detection on multiple track images in sequence to identify all dynamic candidate foreign objects.

[0032] Specifically, firstly, local similarity features of multiple track images are calculated to establish an initial background image.

[0033] In this embodiment, when establishing the initial background image, only the first few consecutive track images (not all track images) decoded in S1 are used for local similarity feature calculation. It should be noted that the first few consecutive track images refer to track images that do not contain any intruding foreign objects (i.e., multiple images when foreign objects have not yet entered the track line). The initial background image can be established by using the mean filter or median filter establishment method.

[0034] The local similarity feature calculation refers to dividing a track image into multiple 5×5 pixel grids and assigning binary codes to the pixels in the pixel grid of the track image according to the similarity. The local similarity feature calculation process can be expressed by the following formula: ;

[0035] in, Represents the center pixel in one of the pixel grids The binary encoding of represents a binary code of one of the pixel grids, Represents the center pixel in the pixel grid The pixel value of Represents the center pixel No. The pixel values ​​of the neighbors, Represents the similarity threshold.

[0036] Furthermore, an automatic update strategy is set to update the initial background image according to the track image to obtain the target background image.

[0037] In this embodiment, the automatic update strategy includes a comprehensive threshold and a background update rate.

[0038] Specifically, according to the track image, the comprehensive threshold and background update rate of each pixel in the initial background image are adjusted to obtain the target background image.

[0039] As time goes by, the subsequent track images continue to change. After the initial background image is established, it is necessary to update the initial background image according to the subsequent track images to capture different features. In this embodiment, the updating method can be weighted average or mixed Gaussian.

[0040] In this embodiment, the comprehensive threshold and the background update rate are respectively expressed by the following formulas: ;

[0041] in, represents the comprehensive threshold, Represents the center pixel The background update rate, represents the Hamming distance, represents the Euclidean distance, Represents the center pixel The last background update rate, Represents the center pixel The update rate, Represents the center pixel The minimum value of the combined characteristic of Represents a candidate foreign body.

[0042] By adjusting the comprehensive threshold and background update rate of each pixel, the interference of the external environment (such as weather changes, flickering lights, etc.) on the image can be effectively avoided. The updated target background image is as follows: Figure 2 As shown, Figure 2 The target background image shown is an orbital scene at night.

[0043] Furthermore, the first foreground values ​​of all pixels in each track image and the second foreground values ​​of all pixels in the target background image are calculated respectively, and the first foreground values ​​are compared with their corresponding second foreground values ​​to extract multiple initial foreground images from the multiple track images.

[0044] Specifically, taking one of the track images as an example, the track image includes a target background image with the track as the background and at least one candidate foreign object. The first foreground value of all pixels in the track image is calculated, and the second foreground value of all pixels in the target background image is calculated. If the first foreground value of one of the pixels in the track image is the same as the second foreground value of the pixel at the corresponding position in the target background image, or the difference between the first foreground value and the second foreground value is less than the comprehensive threshold, it means that the pixel is not the pixel corresponding to the candidate foreign object; if the first foreground value of one of the pixels in the track image is greater than the comprehensive threshold, it means that the pixel is the pixel corresponding to the candidate foreign object, then the track image where the pixel is located is extracted to obtain an initial foreground image containing the candidate foreign object.

[0045] Furthermore, the initial foreground image is post-processed to obtain a target foreground image, and all candidate foreign objects are identified from the multiple track images according to the target foreground image.

[0046] Among them, post-processing includes morphological operations such as corrosion and expansion. Post-processing the initial foreground image can remove interference noise and help improve the accuracy of candidate foreign body recognition. The processed target foreground image is as follows Figure 3 As shown, the candidate foreign object 1 in the target foreground image can be clearly seen.

[0047] Combination Figure 4 ,according to Figure 3 The target foreground image shown is used to detect the corresponding track image to identify the candidate foreign object 1 and the specific position of the candidate foreign object 1 on the track 2.

[0048] S3: Using multiple track images, the motion trajectory of each candidate foreign object is tracked and analyzed respectively, and the effective trajectory of each candidate foreign object is extracted.

[0049] First, candidate foreign objects in two adjacent track images are downsampled and matched in turn to obtain a first matching result.

[0050] Specific, combined Figure 5 , Figure 6 As shown, multiple track images 3 are downsampled respectively to extract feature sequences 32 of candidate foreign objects 1 in the track images 3. Figure 5, taking two (kth and k-1th) adjacent track images 3 as examples, the candidate foreign bodies 1 in them are respectively intercepted, and the intercepted images are input into the pre-built feature sampling network 31. The feature sampling network 31 performs down-sampling processing on them, and obtains the feature sequences 32 corresponding to the candidate foreign bodies 1 in the two (kth and k-1th) track images 3.

[0051] The feature sequence 32 includes appearance features (such as color, texture, etc.) and motion features (such as speed, acceleration, etc.) of the candidate foreign body 1 .

[0052] Furthermore, the feature sequences 32 of the candidate foreign objects 1 in two adjacent track images 3 are matched in sequence to obtain a first matching result (ie, a successful match or a failed match).

[0053] In this embodiment, the matching is performed using a similar feature measurement function, and the specific process is expressed by the following formula: ;

[0054] in, represents the cosine distance between the feature sequences 32 of the candidate foreign object 1 in the two track images 3 (i.e., the difference in appearance features), represent The threshold value, The difference between the feature sequences 32 of the candidate foreign objects 1 in the two track images 3 io distance (i.e., the difference in motion characteristics), represent The threshold value, Represents the cost matrix.

[0055] By matching the candidate foreign objects 1 of the two track images 3 from the feature aspect, deep associations can be mined, which is of great significance for the accuracy of determining whether the candidate foreign objects 1 in the two track images 3 are the same object.

[0056] At the same time, reference Figure 6 As shown, camera motion compensation operation is performed and correction is performed on the candidate foreign object 1 in each track image 3.

[0057] Specifically, an offset calculation is performed on two adjacent track images 3 to obtain a displacement parameter of the camera, and the motion state of the candidate foreign object 1 is corrected according to the displacement parameter.

[0058] In this embodiment, when the camera in S1 collects the real-time video stream of the track, the camera is easily disturbed by external factors and causes jitter or slight movement, resulting in errors in the captured image. Therefore, this application uses camera motion compensation calculation to compensate for the errors caused by camera movement, thereby improving the accuracy of tracking the candidate foreign object 1. The displacement parameters of the camera are calculated using the following formula: ; ;

[0059] in, Representatives from Track images 3 to 4 The affine transformation matrix of the orbital image 3, represents the scale and rotation matrix, represents the translation matrix, Represents the state prediction vector after the camera motion compensation operation, Representatives from Track images 3 to 4 The scale and rotation matrix of the orbital image 3, Represents the state prediction vector before the camera motion compensation operation, Representatives from Track images 3 to 4 The translation matrix of the orbital image 3, Represents the predicted covariance matrix after camera motion compensation operation, Represents the predicted covariance matrix before the camera motion compensation operation, represent The transpose of .

[0060] According to the calculated displacement parameters, the motion state of the candidate foreign body 1 is corrected.

[0061] Next, the position and motion state of the candidate foreign object 1 in each track image 3 are predicted and matched in turn to obtain a second matching result.

[0062] Specifically, the track image 3 after the camera motion compensation operation is input into the Kalman filter 33. The Kalman filter 33 extracts the previous track image 3 corresponding to the input track image 3 from the tracking library 34, and predicts the next position and motion state of the candidate foreign object 1 therein according to the previous track image 3. The prediction result is matched with the position and motion state of the candidate foreign object 1 of the input track image 3 to obtain a second matching result (i.e., matching success or matching failure).

[0063] The above process can be specifically expressed by the following formula: ; ;

[0064] in, represents the Kalman filter gain, represents the observation model, represents the noise covariance, represents the corrected state prediction vector, Represents the depth information of the camera, represents the corrected prediction covariance matrix, represent The transpose of .

[0065] Furthermore, the first matching result and the second matching result are associated, and according to the association result, the final position of each candidate foreign object 1 is updated, and a candidate trajectory associated with each candidate foreign object 1 is obtained.

[0066] Specifically, when both the first matching result and the second matching result are successfully matched, it means that the association result is associated. Further, according to the association result, the final position and motion state of each candidate foreign object 1 are updated in the tracking library 34, and the track images 3 corresponding to the same candidate foreign object 1 are combined in chronological order to obtain the candidate track associated with each candidate foreign object 1.

[0067] Furthermore, a trajectory management strategy is formulated. According to the trajectory management strategy, the valid trajectory uniquely corresponding to each candidate foreign object 1 is screened out from the candidate trajectories, and redundant interfering trajectories are removed to achieve effective tracking of each candidate foreign object.

[0068] S4: Select each track image with candidate foreign objects, input them into the trained target recognition network in sequence, and use the effective track to screen out the target foreign object from multiple candidate foreign objects.

[0069] It should be noted that the candidate foreign objects in this application include fallen rocks, light objects, fallen trees, animals, and other intrusive objects that appear accidentally or intentionally on the railway track. In this embodiment, the target foreign objects specifically refer to fallen rocks. Fallen rocks staying on the track pose a great hidden danger to driving safety. Therefore, monitoring them by the method of this application can effectively ensure driving safety. In some other embodiments, the target foreign objects can also be set and monitored as needed.

[0070] Specifically, in this embodiment, combined with Figure 7 As shown, the training method of the target recognition network includes the steps: In the first step, a guidance network 41 and an initial recognition network 42 are constructed to obtain a plurality of different historical target foreign body images 43.

[0071] The guidance network 41 is used to guide the initial recognition network 42 to learn, so that the initial recognition network 42 can accurately recognize the required target foreign objects after learning. The historical target foreign object image 43 refers to the image of the real target foreign object recognized in the past, and in this embodiment, it refers to the track image of the falling rock in different scenarios (i.e., each track, each scene, and each weather condition).

[0072] In the second step, the historical target foreign body image 43 is input into the guidance network 41 and the initial recognition network 42 respectively to obtain the guidance prediction result and the initial prediction result respectively.

[0073] Specifically, first, a pseudo image generation network 44 is constructed, and the historical target foreign body image 43 is input into the pseudo image generation network 44 to generate a pseudo target image 45, and the pseudo target image 45 is input into the guidance network 41 and the initial recognition network 42 respectively.

[0074] The style of the historical target foreign body image 43 is transferred through the pseudo image generation network 44 to generate a new pseudo target image 45, which expands the training set and enhances the adaptability of different domains.

[0075] Furthermore, different historical target foreign body images 43 are respectively input into the initial recognition network 42 .

[0076] In combination with the above contents, a guidance prediction result is obtained from the guidance network 41 , and an initial prediction result is obtained from the initial recognition network 42 .

[0077] In the third step, according to the guidance prediction results and the initial prediction results, the parameters of the initial recognition network 42 are adjusted using the knowledge distillation loss function.

[0078] In this embodiment, the knowledge distillation loss function can be expressed by the following formula:

[0079] in, represents the knowledge distillation loss function, Represents the loss of the historical target foreign body image 43 and the label of the initial recognition network 42, represents the loss of the pseudo target image 45 and the label of the initial recognition network 42, represents the alignment loss of the guidance network 41 and the initial recognition network 42, Represents the detection parameters, The historical target foreign body image 43 represents one of the scenarios, Represents a pseudo target image 45 in one of the scenarios, Represents the historical target foreign body image 43 in another scenario, Represents a pseudo target image 45 in another scenario, Represents different tags.

[0080] Furthermore, according to the result of the above-mentioned knowledge distillation loss function, the initial recognition network 42 is optimized and the parameters of the initial recognition network 42 are adjusted.

[0081] In the fourth step, repeat the operations in the second and third steps until the knowledge distillation loss function converges, and use the initial recognition network with adjusted parameters as the trained target recognition network.

[0082] Furthermore, each track image with candidate foreign objects is selected and input into the trained target recognition network in turn, and the target foreign object is screened out from multiple candidate foreign objects using effective trajectories. That is, from the track images of multiple candidate foreign objects, the track image corresponding to the rockfall is found, and its specific location on the track is determined, and the staff is arranged to clean it up in time to ensure the safety of the train during travel.

[0083] Corresponding to the method for accurately locating rail surface foreign matter image detection in the above embodiment, Figure 8 A schematic diagram of the composition of the precise positioning rail surface foreign body image detection system provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0084] refer to Figure 8 The precise positioning rail surface foreign body image detection system includes: The acquisition and decoding unit 101 is used to acquire and obtain the track real-time video stream, decode the track real-time video stream, and obtain a plurality of continuous track images.

[0085] The change detection unit 102 is used to perform change detection on multiple track images in sequence to identify all dynamic candidate foreign objects.

[0086] The tracking and analyzing unit 103 is used to use multiple track images to track and analyze the motion trajectory of each candidate foreign object respectively, and extract the effective trajectory of each candidate foreign object.

[0087] The screening and identification unit 104 is used to select each track image with candidate foreign objects, input them into the trained target recognition network in sequence, and use the effective track to screen out the target foreign object from multiple candidate foreign objects.

[0088] It should be noted that since the above-mentioned precise positioning rail surface foreign object image detection system is based on the same concept as the embodiment of the method of the present application, its specific functions, the process of each module realizing its respective function and the technical effects brought about can be specifically referred to the description of the embodiment shown in the aforementioned precise positioning rail surface foreign object image detection method, and will not be repeated here.

[0089] Based on the same inventive concept, the present application also provides an electronic device, which includes a processor, a memory and a communication circuit, wherein the processor is connected to the memory and the communication circuit respectively; wherein the communication circuit is used for communication connection, the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-mentioned precise positioning rail surface foreign body image detection method.

[0090] See also Fig. 9 The electronic device described in the embodiment of the present application may specifically include a processor 210 and a memory 220. The memory 220 is coupled to the processor 210.

[0091] The processor 210 is used to control the operation of the electronic device. The processor 210 may also be referred to as a CPU (Central Processing Unit). The processor 210 may be an integrated circuit chip with signal processing capabilities. The processor 210 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0092] The memory 220 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 220 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 220 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 220 is used to store at least one program code.

[0093] The processor 210 is used to execute the computer program stored in the memory 220 to implement the methods described in the various method embodiments of the present application.

[0094] In some embodiments, the electronic device may further include: a peripheral terminal interface 230 and at least one peripheral terminal. The processor 210, the memory 220 and the peripheral terminal interface 230 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 230 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 240, a display screen 250, an audio circuit 260 and a power supply 270.

[0095] The peripheral terminal interface 230 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 210 and the memory 220. In some embodiments, the processor 210, the memory 220, and the peripheral terminal interface 230 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 210, the memory 220, and the peripheral terminal interface 230 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0096] The radio frequency circuit 240 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 240 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 240 is the communication circuit of the electronic device. The radio frequency circuit 240 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 240 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 240 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 240 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0097] The display screen 250 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 250 is a touch display screen, the display screen 250 also has the ability to collect touch signals on the surface or above the surface of the display screen 250. The touch signal can be input to the processor 210 as a control signal for processing. At this time, the display screen 250 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 250 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 250 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 250 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 250 can also be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 250 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0098] The audio circuit 260 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 210 for processing, or input them into the radio frequency circuit 240 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 210 or the radio frequency circuit 240 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 260 may also include a headphone jack.

[0099] The power supply 270 is used to power various components in the electronic device. The power supply 270 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 270 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0100] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0101] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] Based on the same inventive concept, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above-mentioned method for accurately positioning rail surface foreign body image detection.

[0105] See also Fig.10 , if the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 300. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage media.

[0106] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, and will not be repeated here.

[0107] It can be seen that the present invention discloses a method, system, device and medium for accurately locating foreign body images on the rail surface. The method includes the steps of: collecting and acquiring real-time video streams of the rail, decoding the real-time video streams of the rail, and obtaining multiple continuous rail images; performing change detection on multiple rail images in turn to identify all dynamic candidate foreign bodies; using multiple rail images, tracking and analyzing the motion trajectory of each candidate foreign body respectively, and extracting the effective trajectory of each candidate foreign body; selecting each rail image with a candidate foreign body, inputting them into the trained target recognition network in turn, and using the effective trajectory to screen out the target foreign body from multiple candidate foreign bodies. The present application effectively avoids the interference of the dynamic transformation of the local background of the image on the monitoring by performing change detection on multiple rail images, and at the same time filters out the interference items of the complex environment. The motion trajectory of the candidate foreign body is further tracked to eliminate the influence of motion changes on the identification of the candidate foreign body. The target recognition network is trained by generating a large number of diverse sample sets, thereby improving the accuracy of target foreign body identification. This application not only reduces monitoring costs and improves the accuracy of track foreign object monitoring, but also has strong scene adaptability, can quickly identify accurate foreign objects and ensure line driving safety.

[0108] The above are only embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for accurately locating rail surface foreign body image detection, characterized in that: Includes steps: Collecting and acquiring a real-time track video stream, decoding the real-time track video stream, and obtaining a plurality of continuous track images; Performing change detection on the plurality of track images in sequence to identify all dynamic candidate foreign objects; Using the plurality of track images, the motion trajectory of each candidate foreign object is tracked and analyzed respectively, and the effective trajectory of each candidate foreign object is extracted; Each of the track images having the candidate foreign objects is selected and sequentially input into the trained target recognition network, and the target foreign object is screened out from the plurality of the candidate foreign objects using the effective track.

2. The method for accurately locating rail surface foreign matter image detection according to claim 1 is characterized in that: The step of sequentially performing change detection on the plurality of track images to identify all dynamic candidate foreign objects specifically includes: Calculating local similarity features of the plurality of track images to establish an initial background image; Setting an automatic update strategy, updating the initial background image according to the track image, and obtaining a target background image; respectively calculating the first foreground value of all pixels in each of the track images and the second foreground value of all pixels in the target background image, comparing the first foreground value with the corresponding second foreground value, and extracting a plurality of initial foreground images from the plurality of track images; The initial foreground image is post-processed to obtain a target foreground image, and all the candidate foreign objects are identified from the plurality of track images according to the target foreground image.

3. The method for accurately locating rail surface foreign matter image detection according to claim 2 is characterized in that: In the step of setting the automatic update strategy, updating the initial background image according to the track image, and obtaining the target background image, the following steps are specifically included: The automatic update strategy includes a comprehensive threshold and a background update rate; According to the track image, the comprehensive threshold and background update rate of each pixel in the initial background image are adjusted to obtain the target background image.

4. The method for accurately locating rail surface foreign matter image detection according to claim 1, characterized in that: The step of using the plurality of track images to track and analyze the motion trajectory of each candidate foreign object and extracting the effective trajectory of each candidate foreign object specifically includes: Downsampling and matching the candidate foreign objects in two adjacent track images in sequence to obtain a first matching result; Predicting and matching the position and motion state of the candidate foreign object in each of the track images in turn to obtain a second matching result; Associating the first matching result with the second matching result, updating the final position of each candidate foreign object according to the association result, and obtaining a candidate trajectory associated with each candidate foreign object; A trajectory management strategy is formulated, and according to the trajectory management strategy, the valid trajectory uniquely corresponding to each candidate foreign object is respectively screened out from the candidate trajectories.

5. The method for accurately locating rail surface foreign body image detection according to claim 4, characterized in that: Before the step of predicting and matching the position and motion state of the candidate foreign object in each track image in turn, the step further includes: A camera motion compensation operation is performed on each candidate foreign object in the track image and corrected.

6. The method for accurately locating rail surface foreign matter image detection according to claim 1, characterized in that: In the step of selecting each of the track images having the candidate foreign objects and sequentially inputting them into the trained target recognition network, the training method of the target recognition network includes the steps of: The first step is to build a guidance network and an initial recognition network to obtain multiple different historical target foreign body images; The second step is to input the historical target foreign body image into the guidance network and the initial recognition network respectively to obtain the guidance prediction result and the initial prediction result respectively; The third step is to adjust the parameters of the initial recognition network using the knowledge distillation loss function according to the guidance prediction result and the initial prediction result; The fourth step is to repeat the operations of the second and third steps until the knowledge distillation loss function converges, and use the initial recognition network with adjusted parameters as the trained target recognition network.

7. The method for accurately locating rail surface foreign body image detection according to claim 6, characterized in that: The second step further includes the following steps: A pseudo image generation network is constructed, the historical target foreign body image is input into the pseudo image generation network to generate a pseudo target image, and the pseudo target image is respectively input into the guidance network and the initial recognition network.

8. A precise positioning rail surface foreign body image detection system, characterized in that: include: The acquisition and decoding unit is used to acquire and obtain the real-time track video stream, decode the real-time track video stream, and obtain a plurality of continuous track images; a change detection unit, used to perform change detection on the plurality of track images in sequence, and identify all dynamic candidate foreign objects; A tracking and analysis unit, configured to use the plurality of track images to respectively track and analyze the motion trajectory of each candidate foreign object, and extract a valid trajectory of each candidate foreign object; The screening and identification unit is used to select each of the track images having the candidate foreign objects, input them into the trained target recognition network in sequence, and use the effective track to screen out the target foreign object from the multiple candidate foreign objects.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images