A steel rail shunting automatic tracking method and system based on visual detection

Abstract

The application discloses a steel rail shunting automatic tracking method and system based on visual detection, and the method comprises the following steps: installing an image acquisition module on the alignment side of a steel rail at a transverse moving rack; acquiring images of a steel rail shunting area in real time by using the image acquisition module to obtain steel rail images; forming a running track of the steel rail by a preset target tracking model based on the obtained steel rail images; numbering each mirror room corresponding roller way and setting a buffer sequence; when a track is lost, storing the steel rail information corresponding to the lost track into a corresponding buffer queue according to the roller way number; when the steel rail is reversed from the mirror room, directly adding the latest record in the corresponding buffer queue to the attribute of the new track according to the roller way number corresponding to the new track. The application can solve the tracking confusion problem caused by the unquantifiable target position in the traditional tracking logic, and provides a feasible solution for the accurate tracking of the steel rail under a complex scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision detection, and in particular to a steel rail shunting automatic tracking method and system based on visual detection. BACKGROUND

[0002] Steel rail shunting refers to the process of diverting the steel rails from the detection center exit roller to different mirror rooms at the horizontal moving rack. Currently, the steel rail shunting is manually operated by on-site personnel in the secondary system, and the material tracking system cannot obtain the shunting information, so the tracking process stops here and cannot be further followed up. To realize the full-line connection of the production line, it is necessary to ensure the automatic shunting tracking at the steel shunting table.

[0003] Currently, the commonly used tracking technology mainly relies on the roller speed and the thermal detection signals of different position points to comprehensively predict the corresponding tracking data. However, this type of scheme has the problems of unstable tracking data and incomplete coverage, and therefore, the tracking of the steel rail shunting information is a difficult problem in the enterprise tracking process, which needs to be solved urgently. SUMMARY

[0004] The present application provides a steel rail shunting automatic tracking method and system based on visual detection to solve the technical problems of unstable tracking data and incomplete coverage in the prior art.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] On the one hand, the present application provides a steel rail shunting automatic tracking method based on visual detection, which comprises:

[0007] An image acquisition module is installed on the alignment side of the steel rail at the horizontal moving rack of the steel rail shunting; wherein the acquisition area of the image acquisition module covers the steel rail shunting area at the horizontal moving rack;

[0008] The image acquisition module is used to acquire the image of the steel rail shunting area in real time to obtain the steel rail image.

[0009] Based on the obtained steel rail image, a preset target tracking model is used to form the running track of the steel rail.

[0010] Further, the image acquisition module comprises a plurality of cameras.

[0011] The plurality of cameras are evenly distributed and fixed on the alignment side of the steel rail at the horizontal moving rack of the steel rail shunting; each camera can shoot a preset length of distance range along the length direction of the steel rail, and in the horizontal moving direction of the horizontal moving rack, the range shot by all the cameras is combined to cover the area from the detection center to each mirror room.

[0012] Further, the image acquisition module is used to acquire images of the rail diversion area in real time to obtain rail images, including:

[0013] Each camera is used to capture images of the rail diversion area in real time, respectively;

[0014] The images captured by each camera are subjected to perspective transformation;

[0015] The images subjected to perspective transformation are cropped according to the overlapping area of the shooting ranges of the cameras;

[0016] The cropped images are spliced into a complete rail diversion area monitoring image to obtain the rail images.

[0017] Further, the target tracking model includes an object detection module and a matching tracking module; wherein,

[0018] The object detection module is used to detect the position of each rail in each frame of rail image;

[0019] The matching tracking module is used to perform rail matching and correlation on each frame of image based on the detection result of the object detection module to form the running track of each rail in time sequence; wherein, the track corresponding to each rail takes the rail number corresponding thereto as a unique distinguishing attribute.

[0020] Further, the object detection model includes an instance segmentation network and a post-processing module; wherein,

[0021] The instance segmentation network is used to detect the rail image to output complete rail contour pixel points and the size and position coordinates of the rail;

[0022] The post-processing module is used to determine whether a recognized rail object needs to be split into multiple rail objects based on the output of the instance segmentation network according to the pixel area occupied by the rail in the image and the size of the rail; wherein, the number of splitting is calculated by the following formula:

[0023]

[0024] Wherein, N represents the number of splitting when one rail object is split into multiple rail objects; A is the pixel area occupied by the rail in the image; L is the pixel length occupied by the rail in the image; is a field calibration parameter, representing the pixel width of the rail in the image when the camera captures a rail of the corresponding specification;. represents the down rounding operation.

[0025] Further, after the running track of the rail is formed by the preset target tracking model based on the obtained rail image, the rail diversion automatic tracking method based on visual detection further includes:

[0026] numbering the corresponding rectangular region position in the image for each mirror room corresponding roll table, and setting a buffer sequence corresponding to each number respectively;

[0027] When the target tracking model detects that a certain track is lost, the rail information corresponding to the lost track is stored in the buffer queue corresponding to the number of the roll table corresponding to the position before the track is lost, wherein the rail information includes rail number, rail steel grade and rail specification;

[0028] When the rail is reversed from the mirror room, the last record in the buffer queue corresponding to the number of the roll table corresponding to the position of the new track is added to the attribute of the new track as the rail information corresponding to the rail reversed from the mirror room, and the corresponding record is removed from the buffer queue.

[0029] Further, the buffer queue is used to retain the latest preset number of rail information;

[0030] When new rail information is added to the buffer queue, if data is squeezed out of the buffer queue, the squeezed-out data is stored in the background database for reference.

[0031] On the other hand, the present application also provides a rail shunting automatic tracking system based on visual detection, which comprises:

[0032] An image acquisition module is arranged on the rail alignment side of the rail shunting cross-moving gantry; wherein the acquisition area of the image acquisition module covers the rail shunting area of the cross-moving gantry;

[0033] A server module is used for:

[0034] The image acquisition module is used to acquire the image of the rail shunting area in real time to obtain a rail image;

[0035] Based on the obtained rail image, a preset target tracking model is used to form the running track of the rail.

[0036] In another aspect, the present application also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above method.

[0037] In another aspect, the present application also provides a computer readable storage medium, which stores at least one instruction, which is loaded and executed by the processor to implement the above method.

[0038] The technical scheme provided by the present application has at least the following beneficial effects:

[0039] The technical scheme of the present application covers the rail shunting area at the cross-moving rack by arranging the camera, forms the running track of the rail through the target tracking model, completes the monitoring of the rail flow direction, and helps to realize the full-line penetration of material tracking. The present scheme can realize the tracking confusion problem caused by the unquantifiable target position in the traditional tracking logic, and provides a feasible solution for accurate tracking in a complex scene and large-area monitoring environment. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is an execution flow diagram of the rail shunting automatic tracking method based on visual detection provided by the embodiment of the present application;

[0042] Figure 2 is a layout diagram of the rail shunting automatic tracking system based on visual detection provided by the embodiment of the present application;

[0043] Figure 3 is a rail tracking flow and process principle diagram provided by the embodiment of the present application;

[0044] Figure 4 is a working principle diagram of the post-processing module provided by the embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0046] First embodiment

[0047] In view of the technical problems of unstable tracking data and incomplete coverage existing in the prior art, the present embodiment provides a rail shunting automatic tracking method based on visual detection, which completes the shunting tracking of the rail by installing a monitoring camera at the cross-moving rack where the rail is shunted from the detection center to different mirror rooms; the execution logic of the method can be realized by an electronic device. The execution flow of the method is as shown in Figure 1 , including the following steps:

[0048] S1, an image acquisition module is installed at the rail alignment side of the cross-moving rack where the rail is shunted; wherein the acquisition area of the image acquisition module covers the rail shunting area at the cross-moving rack;

[0049] In the embodiment, the image acquisition module includes a plurality of cameras; the plurality of cameras are fixedly arranged on the rail alignment side of the rail diverging cross-moving gantry; each camera can irradiate a range of 8-12 m from the end in the length direction of the rail; the ranges captured by all the cameras in the cross-moving direction of the cross-moving gantry are combined to cover the area from the detection center to each mirror room, and the range covered by the cameras is the visual tracking area.

[0050] Specifically, the method of the application is applied to a certain rail beam factory, the length of the gantry to be monitored is 100 m, the width is 45 m, the cameras are fixedly arranged on the rail alignment side, each camera can irradiate a range of 10 m from the end in the length direction of the rail, each camera covers a range of 12 m in the width direction of the gantry, and the ranges captured by the four cameras in the cross-moving direction of the cross-moving gantry cover the area from the detection center to each mirror room, and the range covered by the cameras is the visual tracking area. Figure 2 As shown in FIG. 4, the real-time collected images are sent to the server for processing.

[0051] The rail enters the visual tracking area from the exit roller of the detection center, at this time, the information of the current rail is obtained from L2 and is attached to the object attribute of the new track formed by the rail, and the rail in the visual tracking area is subjected to track recognition by the target tracking model. L2 is a secondary system of enterprise automation, and the rail information received by L2 includes rail number, steel type and specification.

[0052] S2, acquiring images of the rail diverging area by using the image acquisition module to obtain rail images;

[0053] Specifically, in the embodiment, the process of obtaining rail images by using the image acquisition module is as follows: the images of the rail diverging area are respectively captured in real time by using each camera; the images captured by all the cameras are subjected to perspective transformation first; then the perspective-transformed images are cropped according to the overlapping area of the capture ranges of the cameras; finally, the cropped images are spliced into a complete rail diverging area monitoring image to obtain the rail images.

[0054] S3, forming the running track of the rail by using a preset target tracking model based on the obtained rail images;

[0055] Wherein, in the embodiment, the target tracking model is composed of an object detection module and a matching tracking module; wherein, the object detection module is used to detect the position of each rail in each frame of rail image; the matching tracking module is used to perform rail matching and association on each frame of image based on the detection result of the object detection module, using Kalman filtering and Hungarian algorithm, to form the running track of each rail in time sequence; the output of the target tracking model is the running track of the rail in time sequence, wherein the track corresponding to each rail takes the rail number corresponding thereto as the unique distinguishing attribute.

[0056] Further, the object detection model is composed of an instance segmentation network and a post-processing module; wherein, the instance segmentation network is used to detect the rail image, output the complete rail contour pixel points, and the size and position coordinates of the rail; specifically, the instance segmentation network in the embodiment is a SOLOv2 network; the post-processing module is used to determine whether it is necessary to split one rail object into multiple rail objects according to the pixel area occupied by the rail in the image and the size of the rail based on the output of the instance segmentation network, as shown in Figure 4 The number of splitting is calculated by the following formula:

[0057]

[0058] Wherein, N represents the number of splitting when one rail object is split into multiple rail objects; A is the pixel area of the rail in the image; L is the pixel length of the rail in the image; is the on-site calibration parameter, representing the pixel width of the rail in the image when the camera shoots the rail of the corresponding specification;. represents the down rounding operation.

[0059] S4, the corresponding roller bed in each mirror room is numbered in the corresponding rectangular area position in the image, and a buffer sequence is set for each number respectively;

[0060] S5, when the target tracking model detects that a certain track is lost, the rail information corresponding to the lost track is stored in the buffer queue corresponding to the number according to the number of the roller bed corresponding to the position before the track is lost, as shown in Figure 3 ; wherein, the rail information includes: rail number, rail steel grade and rail specification; the buffer queue is used to retain the latest preset number (3 in the embodiment) of rail information; when new rail information is added to the buffer queue, if data is squeezed out from the buffer queue, the squeezed-out data is stored in the background database for checking.

[0061] S6, when the rail is returned from the mirror room, according to the number of the roller corresponding to the position of the new track, the latest record in the buffer queue corresponding to the number is added to the rail information corresponding to the rail returned from the mirror room as the attribute of the new track, and the record is cleared from the buffer queue.

[0062] It should be noted that when the rail is returned from the mirror room, the embodiment determines the source of the new track by judging the position of the new track. When it is determined that it is not from the detection center exit roller, the rail information is no longer obtained from L2, but the latest record in the buffer queue corresponding to the mirror room roller number is directly added to the rail information as the attribute of the track, and the record is cleared from the buffer queue. Thus, the risk of track label confusion caused by repeated entry of the rail into the tracking area can be effectively reduced.

[0063] In summary, the embodiment provides a rail shunting automatic tracking method based on visual detection. The scheme uses multiple groups of cameras arranged in the transverse moving cooling bed area to monitor the real-time state of the area, analyzes the action track of different rails from the image through a target tracking model, and realizes target tracking of the area. And a corresponding buffer queue is set at the entrance of each mirror room roller to retain the lost target track, thereby reducing the risk of track label confusion caused by repeated entry of the rail into the tracking area. Thus, it helps to realize the full-line penetration of material tracking. It provides a practical solution for accurate tracking in complex scenes and large area monitoring environments.

[0064] Second embodiment

[0065] The embodiment provides a rail shunting automatic tracking system based on visual detection. The network structure of the rail shunting automatic tracking system based on visual detection is as shown in Figure 2 The rail shunting automatic tracking system based on visual detection comprises the following modules:

[0066] An image acquisition module is arranged on the rail alignment side of the rail shunting transverse moving rack. The acquisition area of the image acquisition module covers the rail shunting area of the transverse moving rack.

[0067] A server module is used for:

[0068] The image acquisition module is used to acquire images of the rail shunting area in real time, and obtain rail images.

[0069] Based on the obtained rail images, a preset target tracking model is used to form the running track of the rail.

[0070] The steel rail shunting automatic tracking system based on visual detection of the embodiment corresponds to the steel rail shunting automatic tracking method based on visual detection of the first embodiment; wherein the functions realized by each functional module in the steel rail shunting automatic tracking system based on visual detection of the embodiment correspond to each process step in the steel rail shunting automatic tracking method based on visual detection of the first embodiment; therefore, no further elaboration is made here.

[0071] Third embodiment

[0072] The electronic device can be quite different in configuration or performance, and can include one or more processors (central processing units, CPUs) and one or more memories, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment.

[0073] The electronic device can be quite different in configuration or performance, and can include one or more processors (central processing units, CPUs) and one or more memories, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment.

[0074] Fourth embodiment

[0075] The embodiment provides a computer readable storage medium, which stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment. The computer readable storage medium can be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein can be loaded and executed by the processor in the terminal to implement the above method.

[0076] In addition, it should be noted that the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media containing computer usable program code.

[0077] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general purpose computer, embedded processor or other programmable data processing terminal device processor to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device processor produce a machine that implements the functions described in the flowcharts and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] It should also be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0080] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A steel rail automatic tracking method based on visual detection, characterized in that, The application relates to a rail shunting automatic tracking method based on visual detection, which comprises the following steps: a rail alignment side image acquisition module is arranged at a rail shunting cross-moving rack; wherein the acquisition area of the image acquisition module covers the rail shunting area at the cross-moving rack; an image of the rail shunting area is acquired in real time by using the image acquisition module to obtain a rail image; an operation track of the rail is formed by using a preset target tracking model based on the obtained rail image; the target tracking model comprises an object detection module and a matching tracking module; wherein the object detection module is used for detecting the position of each rail in each frame of the rail image; the matching tracking module is used for matching and correlating the rails in each frame of the image based on the detection result of the object detection module to form the operation track of each rail in time sequence; wherein the track corresponding to each rail is uniquely distinguished by the rail number corresponding to the track; the object detection module comprises an instance segmentation network and a post-processing module; wherein the instance segmentation network is used for detecting the rail image to output complete rail contour pixel points and the size and position coordinates of the rail; the post-processing module is used for determining whether a recognized rail object needs to be split into multiple rail objects based on the output of the instance segmentation network according to the pixel area of the rail in the image and the size of the rail; wherein the number of the split is calculated by the following formula: Wherein, N represents the number of splitting when one steel rail object is split into multiple steel rail objects; A is the pixel area occupied by the steel rail in the image; L is the pixel length occupied by the steel rail in the image; is a field calibration parameter, representing the pixel width of the steel rail in the image when the camera shoots the corresponding specification steel rail;. represents the floor operation.

2. The visual detection based rail diverging automatic tracking method according to claim 1, wherein, the image acquisition module comprises a plurality of cameras; the plurality of cameras are uniformly distributed and fixed at the rail alignment side of the rail shunting cross-moving rack; each camera can shoot a preset length of distance range along the length direction of the rail; in the cross-moving direction of the cross-moving rack, the range shot by all the cameras is combined to cover the area from the detection center to each mirror room.

3. The visual detection based rail diverging automatic tracking method according to claim 2, wherein, The image acquisition module is used for acquiring the image of the rail shunting area in real time to obtain the rail image, which comprises the following steps: each camera is used for shooting the image of the rail shunting area in real time respectively; perspective transformation is performed on the image shot by each camera; the image after the perspective transformation is cut according to the overlapping area of the shooting range of each camera; the cut image is spliced into a complete rail shunting area monitoring image to obtain the rail image.

4. The automatic tracking method of visual detection-based rail diverging according to any one of claims 1 to 3, characterized in that, After the operation track of the rail is formed by using the preset target tracking model based on the obtained rail image, the rail shunting automatic tracking method based on visual detection further comprises the following steps: the rectangular area position corresponding to each mirror room in the image is numbered, and a buffer sequence is set for each number respectively; when the target tracking model detects that a certain track is lost, the rail information corresponding to the lost track is stored in the buffer queue corresponding to the number of the roller corresponding to the position before the track is lost; wherein the rail information comprises the rail number, the rail type and the rail specification; when the rail is returned from the mirror room, the most recent record in the buffer queue corresponding to the number of the roller corresponding to the position of the newly added track is taken as the rail information corresponding to the rail returned from the mirror room and is added to the attribute of the newly added track, and the corresponding record is removed from the buffer queue.

5. The visual detection based rail diverging automatic tracking method of claim 4, wherein, The buffer queue is used to reserve a preset number of latest rail information; When new rail information is added to the buffer queue, if data is squeezed out from the buffer queue, the squeezed-out data is stored in a background database for checking.

6. A visual inspection based rail diverging automatic tracking system, characterized in that, Comprise: An image acquisition module is arranged on the rail alignment side of the rail diverging cross-moving gantry; wherein the acquisition area of the image acquisition module covers the rail diverging area of the cross-moving gantry; A server module is used to: Acquire images of the rail diverging area in real time by using the image acquisition module to obtain rail images; Based on the obtained rail images, form the running track of the rail by using a preset target tracking model; The target tracking model comprises an object detection module and a matching tracking module; wherein, The object detection module is used to detect the position of each rail in each frame of rail image; The matching tracking module is used to perform rail matching correlation on each frame of image based on the detection result of the object detection module to form the running track of each rail in time sequence; wherein the track corresponding to each rail takes the corresponding rail number as the unique distinguishing attribute; The object detection module comprises an instance segmentation network and a post-processing module; wherein, The instance segmentation network is used to detect the rail image to output complete rail contour pixel points and the size and position coordinates of the rail; The post-processing module is used to determine whether the identified rail object needs to be split into multiple rail objects according to the pixel area occupied by the rail in the image and the size of the rail based on the output of the instance segmentation network; wherein the number of splitting is calculated by the following formula: Wherein, N represents the number of splitting when one steel rail object is split into multiple steel rail objects; A is the pixel area occupied by the steel rail in the image; L is the pixel length occupied by the steel rail in the image; is a field calibration parameter, representing the pixel width of the steel rail in the image when the camera shoots the corresponding specification steel rail;. represents the floor operation.

Images