Imaging system and method for state detection image of coal transporting train

Abstract

The invention belongs to the technical field of image recognition, and discloses an imaging system and method for a coal transporting train state detection image, and the system comprises an image collection system and a monitoring center server. The image acquisition system is used for acquiring an appearance image of a carriage of a to-be-imaged coal transporting train by using an area array scanning camera, and acquiring and sending an original acquired picture to the monitoring center server; the monitoring center server is used for storing the originally collected pictures, splicing the originally collected pictures by using a preset image splicing algorithm, and outputting an imaging result of the carriage image of the coal transporting train to be imaged; according to the invention, splicing of adjacent area images is realized, filtering of repeated areas is ensured, accurate and efficient output of complete train carriage images is ensured, and overlapping of collected images under a parking or backing working condition of a coal conveying train is effectively avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and particularly relates to an imaging system and method for images of the state detection of coal-carrying trains. Background Art

[0002] With the comprehensive promotion and in-depth application of coal mine intelligent technologies, the market demand for the automatic detection of the appearance state of coal-carrying train vehicles is increasing day by day; since the length of coal-carrying trains far exceeds their width and height, traditional detection methods are difficult to meet the requirements of efficient and accurate detection; currently, for the acquisition of images for the state detection of coal-carrying trains, linear array cameras are usually used.

[0003] However, when imaging based on a linear array camera, the accuracy and efficiency of the state detection of coal-carrying trains decrease due to poor image quality; specifically, since there is image repetition and stacking in the pictures collected by the linear array camera for coal-carrying trains in the parking or reversing working conditions, the imaging effect is poor, greatly reducing the accuracy and efficiency of the state detection of coal-carrying trains; specifically, in the parking working condition, the phenomenon of continuous stacking in the same area will occur, as shown in the attachment Figure 1 shown; in the reversing working condition, the phenomenon of mirror stacking of images in the same area will occur, as shown in the attachment Figure 2 shown; it can be directly seen from the attachment Figure 2 that there is a serious phenomenon of image repetition and stacking. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention provides an imaging system and method for images of the state detection of coal-carrying trains to solve the technical problem that the accuracy and efficiency of the state detection of coal-carrying trains decrease due to poor image quality when imaging based on a linear array camera.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: The present invention provides an imaging system for images of the state detection of coal-carrying trains, including an image acquisition system and a monitoring center server; The image acquisition system is used to collect the exterior image of the carriage of the coal-carrying train to be imaged by using a area array scanning camera, and obtain and send the original collected pictures to the monitoring center server; The monitoring center server is used to store the original collected pictures and splice the original collected pictures by using a preset image stitching algorithm, and output the imaging result of the carriage image of the coal-carrying train to be imaged.

[0006] Further, the image acquisition system includes two image acquisition subsystems, and the two image acquisition subsystems are symmetrically arranged on both sides of the coal-carrying train track; wherein, the structures of the two image acquisition subsystems are the same, and each includes a column, a side-top imaging unit, a side-middle imaging unit and a side-bottom imaging unit; The vertical column is fixedly installed on the side of the coal-carrying train track. The side-top imaging unit, the side-middle imaging unit, and the side-bottom imaging unit are sequentially installed on the vertical column from top to bottom. The output ends of the side-top imaging unit, the side-middle imaging unit, and the side-bottom imaging unit are all connected to the input end of the monitoring server; The side-top imaging unit is used to collect the top-side image of the coal-carrying train to be imaged; the side-middle imaging unit is used to collect the middle-side image of the coal-carrying train to be imaged; the side-bottom imaging unit is used to collect the bottom-side image of the coal-carrying train to be imaged. Among them, the side-top imaging unit, the side-middle imaging unit, and the side-bottom imaging unit all include an equipment box and a line-scan camera. The equipment box is installed on the vertical column, and the line-scan camera is installed inside the equipment box.

[0007] Further, the depression angle of the line-scan camera in the side-top imaging unit is 45°. The angle of the line-scan camera in the side-middle imaging unit is parallel to the side of the carriage. The elevation angle of the line-scan camera in the bottom imaging unit is 45°.

[0008] Further, the image acquisition system further includes an image acquisition trigger subsystem; The image acquisition trigger subsystem includes a speed measurement module and an outdoor cabinet. The speed measurement module is arranged on the side of the vertical column and is used to collect and send the train speed of the coal-carrying train to be imaged to the outdoor cabinet. The outdoor cabinet is used to generate and send an image acquisition instruction to the side-top imaging unit, the side-middle imaging unit, and the side-bottom imaging unit in the image acquisition subsystem according to the collected train speed of the coal-carrying train to be imaged.

[0009] Further, the original acquired picture is a 2D picture, and the storage format of the original acquired picture is JPG format.

[0010] Further, the process of splicing the original acquired pictures by using a preset image stitching algorithm and outputting the imaging result of the carriage image of the coal-carrying train to be imaged is as follows: Extract two consecutive frames of the original acquired pictures from the stored original acquired pictures to obtain input images; Perform feature extraction and feature description processing on the input images to obtain the feature description information of the input images; Based on the feature description information of the input images, perform feature matching between the input images and the train reference image of the coal-carrying train to be imaged to obtain a feature matching result; According to the feature matching result, perform transformation matrix estimation on the input images to obtain a transformation matrix estimation result; Based on the estimated result of the transformation matrix, perform image transformation on the input image to transform the input image into the coordinate system of the train reference image of the coal-carrying train to be imaged, and obtain the input image transformation result; Fuse two adjacent frames of images in the input image transformation result to obtain a fused image; Use the fused image and the next frame of the original captured image as the input image again, and repeat the above operations until the train is in a non-stop state, and output the complete train image, that is, obtain the imaging result of the carriage image of the coal-carrying train to be imaged.

[0011] Further, use the SIFT algorithm to perform feature extraction and feature description processing on the input image to obtain the feature description information of the input image.

[0012] Further, based on the feature description information of the input image, when performing feature matching between the input image and the train reference image of the coal-carrying train to be imaged to obtain the feature matching result, use the normalized correlation method to determine the corresponding relationship of the key features between the input image and the train reference image of the coal-carrying train to be imaged; use mutual information to evaluate the similarity of the amount of shared information between the input image and the train reference image of the coal-carrying train to be imaged.

[0013] Further, according to the feature matching result, use the RANSAC algorithm to estimate the transformation matrix of the input image to obtain the estimated result of the transformation matrix; When fusing two adjacent frames of images in the input image transformation result to obtain a fused image, use feathering to fuse overlapping pixels with weighted average color values.

[0014] The present invention also provides an imaging method for the coal-carrying train status detection image, using the imaging system for the coal-carrying train status detection image; Among them, the imaging method for the coal-carrying train status detection image includes: Use a area array scanning camera to collect the appearance image of the carriage of the coal-carrying train to be imaged to obtain the original captured image; Use a preset image stitching algorithm to stitch the original captured images, and output the imaging result of the carriage image of the coal-carrying train to be imaged.

[0015] Compared with the prior art, the beneficial effects of the present invention are: An imaging system and method for images of the state detection of coal-carrying trains provided by the present invention collect the appearance images of the carriages of the coal-carrying trains to be imaged by using a planar array scanning camera, and splice the original collected pictures in combination with a preset image splicing algorithm to achieve seamless splicing of adjacent area images and ensure filtering of overlapping areas, ensuring accurate and efficient output of complete carriage images and effectively avoiding image overlap in the case of image collection when the coal-carrying train is parked or reversing; specifically, by using a planar array scanning, it can be seen that the image area obtained in a single case is large, and high frame rate and high pixel imaging effects can also be achieved in a low-frequency state; secondly, by using the preset image splicing algorithm, accurate splicing of adjacent area images can be achieved, and overlapping areas can be filtered, completely avoiding image overlap, improving the accuracy of the imaging result, and thus effectively improving the overall accuracy and efficiency of the state detection of coal-carrying trains. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is an imaging effect diagram of a coal-carrying train in a parking condition based on a linear array camera; Figure 2 It is an imaging effect diagram of a coal-carrying train in a reversing condition based on a linear array camera; Figure 3 It is a structural block diagram of an imaging system for images of the state detection of coal-carrying trains provided by the present invention; Figure 4 It is a structural schematic diagram of an image acquisition system and an image acquisition trigger system in the present invention; Figure 5 It is a distribution schematic diagram of imaging units in the present invention; Figure 6 It is an imaging principle diagram of a planar array scanning camera in the present invention; Figure 7 It is an imaging effect diagram of a coal-carrying train in a parking condition based on a planar array scanning camera in the present invention; Figure 8 It is an imaging effect diagram of a coal-carrying train in a reversing condition based on a planar array scanning camera in the present invention; Figure 9 It is a flowchart of an imaging method for images of the state detection of coal-carrying trains provided by the present invention; Figure 10 It is a flowchart of a preset image splicing algorithm in the present invention; Figure 11This is a schematic diagram of the process of circular splicing of the original captured images in the present invention; Figure 12 This is a schematic diagram of the imaging result of the carriage image of the coal-carrying train to be imaged output in the present invention.

[0018] Among them, 10 is the left column, 11 is the left roof imaging unit, 12 is the left middle imaging unit, 13 is the left lower imaging unit; 20 is the right column, 21 is the right roof imaging unit, 22 is the right middle imaging unit, 23 is the right lower imaging unit; 30 is the speed measurement module, 31 is the outdoor cabinet; 40 is the area array scanning camera; 100 is the coal-carrying train to be imaged. Detailed implementation manners

[0019] In order to make the technical problems, technical solutions and beneficial effects solved by the present application clearer and more understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application; obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0020] As shown in the attached Figure 3 As shown in the figure, the present invention provides an imaging system for coal-carrying train status detection images, including an image acquisition system and a monitoring center server, and the output end of the image acquisition system is connected to the input end of the monitoring center server; the image acquisition system is used to collect the outer tube images of the carriages of the coal-carrying train to be imaged by using an area array scanning camera, obtain and send the original captured images to the monitoring center server; the monitoring center server is used to store the original captured images and splice the original captured images by using a preset image splicing algorithm, and output the imaging structure of the carriage images of the coal-carrying train to be imaged.

[0021] In the present invention, the image acquisition system includes two image acquisition subsystems and an image acquisition trigger subsystem. The two image acquisition subsystems are symmetrically arranged on both sides of the coal-carrying train track, and the structures of the two image acquisition subsystems are the same; the output end of the image acquisition trigger subsystem is connected to the input ends of the two image acquisition subsystems, and the output ends of the two image acquisition subsystems are both connected to the input end of the central detection server.

[0022] As shown in the attached Figures 4 - 5 As shown in the figure, the two image acquisition subsystems are specifically a left image acquisition subsystem and a right image acquisition subsystem. The left image acquisition subsystem is arranged on the left side of the coal-carrying train track, and the right image acquisition subsystem is arranged on the right side of the coal-carrying train track.

[0023] The left image acquisition subsystem includes a left vertical column 10, a left roof imaging unit 11, a left middle imaging unit 12, and a left lower imaging unit 13. The left vertical column 10 is vertically fixed on the left side of the side of the coal transport train track. The left roof imaging unit 11, the left middle imaging unit 12, and the left lower imaging unit 13 are sequentially installed on the left vertical column 10 from top to bottom. Specifically, the left roof imaging unit 11 is arranged at the top of the left vertical column 10 and is used to acquire the left top image of the coal transport train 100 to be imaged. The left middle imaging unit 12 is arranged in the middle of the left vertical column 10 and is used to acquire the left middle image of the coal transport train 100 to be imaged. The left lower imaging unit 13 is arranged at the bottom of the left vertical column 10 and is used to acquire the left bottom image of the coal transport train 100 to be imaged. Among them, the depression angle of the left roof imaging unit 11 is 45°, the angle of the left middle imaging unit 12 is parallel to the side of the carriage, and the elevation angle of the left lower imaging unit 13 is 45°.

[0024] The right image acquisition subsystem includes a right vertical column 20, a right roof imaging unit 21, a right middle imaging unit 22, and a right lower imaging unit 23. The right vertical column 20 is vertically fixed on the right side of the side of the coal transport train track. The right roof imaging unit 21, the right middle imaging unit 22, and the right lower imaging unit 23 are sequentially installed on the right vertical column 20 from top to bottom. Specifically, the right roof imaging unit 21 is arranged at the top of the right vertical column 20 and is used to acquire the right top image of the coal transport train 100 to be imaged. The right middle imaging unit 22 is arranged in the middle of the right vertical column 20 and is used to acquire the right middle image of the coal transport train 100 to be imaged. The right lower imaging unit 23 is arranged at the bottom of the right vertical column 20 and is used to acquire the right bottom image of the coal transport train 100 to be imaged. Among them, the depression angle of the right roof imaging unit 21 is 45°, the angle of the right middle imaging unit 22 is parallel to the side of the carriage, and the elevation angle of the right lower imaging unit 23 is 45°.

[0025] Both the left vertical column 10 and the right vertical column 20 adopt H-shaped steel columns. The cross-sectional dimension characteristics of the H-shaped steel column are: length × width = 300 × 200 mm, and the column height of the H-shaped steel column is 7200 mm; the H-shaped steel column is located inside the catenary guy wire; among them, the horizontal distance between the left vertical column 10 and the right vertical column 20 is 6570 mm; the bottom end of the H-shaped steel column is fixed on the ground through a column foundation, and the dimension characteristics of the column foundation are: length × width × height = 1200 × 1200 × 2000 mm; it should be noted that the top end of the H-shaped steel column is provided with a horizontal installation beam. One end of the horizontal installation beam is perpendicular to the top end of the H-shaped steel column, and the other end of the horizontal installation beam extends horizontally towards the side of the coal-carrying train 100 to be imaged. Both the left roof imaging unit 11 and the right roof imaging unit 21 are installed at the end of the horizontal installation beam; among them, the installation height of the horizontal installation beam is 6500 mm, and the horizontal extension length of the horizontal installation beam is 300 mm.

[0026] The structures of the left roof imaging unit 11, the left middle imaging unit 12, the left lower imaging unit 13, the right roof imaging unit 21, the right middle imaging unit 22, and the right lower imaging unit 23 are the same, and all adopt 2D image acquisition modules; among them, the 2D image acquisition module includes a area array scanning camera 40 and an equipment box. The equipment is installed at a preset position on the left vertical column 10 or the right vertical column 20, and the area array scanning camera is installed in the equipment box; preferably, the area array scanning camera 40 adopts a 2D color area array scanning camera, and the working principle of the 2D color area array scanning camera is as shown in the appendix Figure 6 shown to achieve pixel matrix shooting and acquisition; among them, the imaging effect of the area array scanning camera on the coal-carrying train under the parking condition is as shown in the appendix Figure 7 shown, and the imaging effect of the area array scanning camera on the coal-carrying train under the reverse condition is as shown in the appendix Figure 8 shown.

[0027] It should be noted that the image acquisition process of the 2D color area array scanning camera is continuous. The image acquisition port is planar, enabling fast and accurate acquisition of two-dimensional image information. Moreover, it uses a CMOS chip and adopts the ROCC technology to achieve a high frame rate. At the same time, in combination with the intelligent LED stroboscopic technology, high-definition acquisition of railway train images is realized. Specifically, the 2D color area array scanning camera integrates an LED light source component, a 2D imaging component, and a trigger control component. Among them, the LED light source component and the 2D imaging component are integrated into an integrated structure and are triggered and controlled integrally. The LED light source component is white, high-brightness, and intelligent stroboscopic, with an operating temperature of -35°C to +70°C. The power supply voltage of the 2D imaging component is DC24V, the maximum pixel is 12 million, the data layer communication interface is Gigabit Ethernet, the control signal interface supports differential signals of ±6V, and the waterproof and dustproof level is IP65. The output end of the trigger control component is connected to the control ends of both the LED light source component and the 2D imaging component, and the input end of the trigger control component is connected to the output end of the image acquisition trigger system. The trigger control component is used to receive and respond to the image acquisition instruction sent by the image acquisition trigger system to control the opening and closing of the LED light source component and the 2D imaging component, and then use the LED light source component and the 2D imaging component to complete the shooting and acquisition of the appearance image of the moving coal-carrying train.

[0028] The image acquisition trigger subsystem includes a speed measurement module and an outdoor cabinet. The speed measurement module is arranged on the side of the left column 10 or the right column 20, and is used to collect the train speed of the coal-carrying train 100 to be imaged in real time and send the train speed of the coal-carrying train 100 to be imaged to the outdoor cabinet. The outdoor cabinet is used to dynamically generate an image acquisition instruction according to the received train speed of the coal-carrying train 100 to be imaged and send it to the trigger control component in the 2D color area array scanning camera, so as to ensure accurate and efficient image acquisition under different vehicle speed conditions.

[0029] Specifically, the speed measurement module includes a magnetic steel axle counting system and a speed measurement radar system. Among them, the magnetic steel axle counting system can accurately monitor the oncoming vehicle direction, the start time of passing the vehicle, the vehicle speed, the vehicle separation, and the end time of passing the vehicle through cooperation with the system clock, so as to achieve high-precision acquisition of the train speed. Multiple core devices are integrated inside the outdoor cabinet, including a control industrial computer, an acquisition industrial computer, a power signal box, a KVM all-in-one machine, and a UPS. When the magnetic steel axle counting system detects a passing vehicle signal, the system will immediately collect the signal and control the industrial computer to output an image acquisition instruction. At the same time, the system will also collect data on vehicle information, vehicle speed, and driving direction to ensure that the area array scanning camera 40 can be accurately triggered and complete the image acquisition work.

[0030] In the present invention, the monitoring center server includes a storage module and an image stitching module; the storage module is used to stitch the original captured pictures; wherein, the original image captured pictures are 2D pictures, and the storage format of the original captured pictures is JPG format; the picture stitching module is used to stitch the original captured pictures by using a preset image stitching algorithm, and output the imaging result of the carriage image of the coal-carrying train to be imaged.

[0031] Specifically, the process of stitching the original captured pictures by using a preset image stitching algorithm and outputting the imaging result of the carriage image of the coal-carrying train to be imaged is as follows: Select two consecutive frames of the original captured pictures stored in the storage module as the input to obtain the input image; use the SIFT (Scale-Invariant Feature Transform) algorithm to perform feature extraction and description processing on the input image to obtain the feature description information of the input image; based on the feature description information of the input image, perform feature matching between the input image and the train reference image of the coal-carrying train to be imaged to obtain the feature matching result; wherein, use the normalized correlation method to determine the correspondence of the key features between the input image and the train reference image of the coal-carrying train to be imaged; use mutual information to evaluate the similarity of the shared information quantity between the input image and the train reference image of the coal-carrying train to be imaged; according to the feature matching result, use the RANSAC algorithm to estimate the transformation matrix of the input image to obtain the transformation matrix estimation result; based on the transformation matrix estimation result, perform image transformation on the input image to convert the input image into the coordinate system of the train reference image of the coal-carrying train to be imaged to obtain the input image transformation result; fuse two adjacent frames of images in the input image transformation result to obtain the fused image; wherein, use feathering to fuse the overlapping pixels by using the weighted average color value; use the fused image and the next frame of the original captured picture as the input image again, and repeat the above operations until the train is no longer in the parking state, and finally output the complete train image, that is, obtain the imaging result of the carriage image of the coal-carrying train to be imaged.

[0032] In the present invention, the image acquisition technology of the area array camera is applied to the field of image acquisition of the appearance state of train vehicles, and combined with the image stitching algorithm, it effectively solves the problem of repeated stacking of the images captured by the traditional line array camera in the case of the coal-carrying train parking and backing; compared with the line array camera, the area array camera obtains a large image area in a single case, and can achieve high frame rate and high pixel imaging even in the low-frequency state. The image stitching algorithm can realize the stitching of adjacent area images and ensure the filtering of the repeated areas, and realize the output of the complete list of carriage images.

[0033] The working principle and imaging method are specifically described as follows: Taking the process of imaging the carriage image of a coal-carrying train under the conditions of parking or reversing using the above-mentioned imaging system for the state detection image of the coal-carrying train as an example; as shown in the appendix Figure 9 As shown, the specific process of the imaging method is as follows: Step 1: Use the area array scanning camera 40 in the image acquisition subsystem to collect the appearance image of the carriage of the coal-carrying train to be imaged, and obtain the original collected picture.

[0034] Step 2: Use a preset image stitching algorithm to stitch the original collected pictures, and output the imaging result of the carriage image of the coal-carrying train to be imaged.

[0035] As shown in the appendix Figure 10 As shown, using a preset image stitching algorithm to stitch the original collected pictures, and output the imaging result of the carriage image of the coal-carrying train to be imaged. The specific steps are as follows: Step 21: Extract the current frame of the original collected picture and the previous frame of the original collected picture from the original collected pictures, and use the current frame of the original collected picture and the previous frame of the original collected picture as the input image together, that is, obtain the input image; as shown in the appendix Figure 11 As shown, in the appendix Figure 11 P1 in it is the previous frame of the original collected picture, and P2 is the next frame of the original collected picture.

[0036] Step 22: Perform feature extraction and feature description processing on the input image to obtain the feature description information of the input image; it should be noted that for the extraction of feature points in the overlapping area under the conditions of parking and reversing of the coal-carrying train, the Scale-Invariant Feature Transform (SIFT) algorithm is used for feature extraction and feature description processing; among them, the image registration process based on SIFT point features includes feature extraction, feature description, feature matching, solving the transformation model parameters, and image transformation registration.

[0037] Specifically, the process of obtaining the feature description information of the input image is as follows: Step 221: Perform scale-space extreme value detection on the input image; specifically, search for the image positions at all scales for the input image, and identify the potential key points that are invariant to scale and rotation in the input image through the Gaussian differential function to obtain the candidate positions of the key points.

[0038] Step 222: Precise localization of key points; specifically, at the candidate position of each key point, determine the position and scale of the key point through a preset fitting fine model to obtain the precise localization result of the key point; among them, the key points are selected according to their stability.

[0039] Step 223: Determine the main direction of the key points; specifically, based on the local gradient method of the input image, assign one or more directions to each key point. It should be noted that in subsequent steps, the operations on the image data are transformed with respect to the direction, scale, and position of the key points to provide invariance to the above transformations.

[0040] Step 224: Generate a SIFT feature vector to obtain the feature description information of the input image; specifically, measure the local gradient of the input image within the neighborhood around each key point and at a predetermined scale to obtain the descriptor of the key point, that is, the SIFT feature vector; and draw the key points of the input image according to the descriptor of the key point to obtain the feature description information of the input image. Among them, the local gradient is used to represent the deformation of the local shape and the change of illumination that are allowed to be relatively large.

[0041] Step 225: After obtaining the method, position, and scale information of the key points, perform feature description on the train reference image of the coal-carrying train to be imaged to obtain the feature description information of the train reference image.

[0042] Step 23: Based on the feature description information of the input image, perform feature matching between the input image and the train reference image of the coal-carrying train to be imaged to obtain a feature matching result; specifically, perform feature matching on the key points of the input image and the train reference image of the coal-carrying train to be imaged according to the feature description information of the input image and the feature description information of the train reference image to obtain a feature matching result. Among them, in the feature matching process, the normalized cross-correlation (NCC) method is used to confirm the correspondence of the key feature points, and the mutual information (MI) is used to evaluate and measure the similarity of the amount of shared information between the input image and the train reference image of the coal-carrying train to be imaged.

[0043] Step 24: According to the feature matching result, estimate the transformation matrix for the input image to obtain the transformation matrix estimation result; specifically, use the RANSAC algorithm to estimate the homography matrix, and use the RANSAC algorithm to estimate the homography matrix from the matching point pairs, that is, obtain the transformation matrix estimation result. It should be noted that based on the feature matching result, obtain the stitchable area of the reference image and the registered image, register the registered image and then stitch it with the reference image, and place the two images on a common stitching plane. Since there are transformation relationships such as displacement, rotation, similarity, affine, and perspective for the same content in the two images, it is necessary to calculate the transformation structure of the two images and calculate the homography matrix, that is, obtain the transformation matrix estimation result.

[0044] Step 25: Based on the estimated result of the transformation matrix, perform image transformation on the input image, and use the affine transformation parameters for image interpolation to convert the input image into the coordinate system of the train reference image of the coal-carrying train to be imaged, obtaining the input image transformation result.

[0045] Step 26: For two adjacent frames of images in the input image transformation result, fuse the overlapping pixels by using weighted average color values through feathering to obtain the fused image A1.

[0046] Step 27: Determine whether the coal-carrying train to be imaged is in the parking condition; if so, perform the picture stitching loop operation; specifically, construct a new input image with the fused image A1 in Step 26 and the next frame of the original captured image P3 extracted from the original captured images, and repeat the operations of the above Steps 22 - 26 on the new input image until it is determined that the coal-carrying train to be imaged is in a non-parking condition, and output the complete train image, that is, obtain the imaging result of the carriage image of the coal-carrying train to be imaged, as shown in the appendix Figure 12 as follows.

[0047] It should be noted that the loop process of picture stitching is specifically as follows: when P1 and P2 are stitched through the above steps to obtain A1, use A1 as the input and stitch it with P3 to obtain A2, and so on to obtain A3 until the coal-carrying train to be imaged is in a non-parking condition, as shown in the appendix Figure 11 as follows; when the coal-carrying train to be imaged is in the reverse condition, after mirror processing the pictures in the input image, perform the operations of the above Steps 22 - 26.

[0048] The imaging system and method of the present invention use a planar array scanning camera to collect the train image of the coal-carrying train to be imaged, and filter the image overlapping area through an image stitching algorithm, and output the imaging structure of the complete train carriage image, which can efficiently and accurately solve the problem of repeated stacking of images collected by the camera in the parking and reversing situations, realize the stitching of adjacent area images and ensure the filtering of the overlapping area, and realize the output of the complete train carriage image; among them, using a planar array scanning camera can collect and obtain a larger image area at one time, and realize high frame rate and high pixel imaging in the low-frequency state of the planar array camera; in the process of filtering the image overlapping area by using the image stitching algorithm, feature point detection and matching algorithms are used to achieve feature matching, geometric transformation of the image is realized through affine transformation, and filtering and stitching of the image are realized through picture alignment and fusion stitching, and the imaging of the train carriage image is realized.

[0049] The above embodiments are only one of the implementation manners that can implement the technical solution of the present invention. The scope of protection required by the present invention is not limited only by this embodiment, but also includes any changes, substitutions and other implementation manners that are easily conceivable by those skilled in the art within the technical scope disclosed by the present invention.

Claims

1. An imaging system for detecting the state of a coal transport train, characterized in that: Including image acquisition system and monitoring center server; The image acquisition system is used to acquire the exterior image of the carriage of the coal transport train to be imaged by using an area scan camera, and obtain and send the original acquired image to the monitoring center server; The monitoring center server is used to store the original collected pictures, and use a preset image stitching algorithm to stitch the original collected pictures, and output an imaging result of the carriage image of the coal transport train to be imaged.

2. The imaging system for detecting the state of a coal train according to claim 1, characterized in that: The image acquisition system comprises two image acquisition subsystems, which are symmetrically arranged on both sides of the coal transport train track; wherein the two image acquisition subsystems have the same structure, and both comprise a column, a side top imaging unit, a side middle imaging unit and a side bottom imaging unit; The column is vertically fixed on the side of the coal transport train track, the side top imaging unit, the side middle imaging unit and the side lower imaging unit are sequentially installed on the column from top to bottom, and the output ends of the side top imaging unit, the side middle imaging unit and the side lower imaging unit are all connected to the input end of the monitoring server; The side top imaging unit is used to capture the top image of the side of the coal train to be imaged; the side middle imaging unit is used to capture the middle image of the side of the coal train to be imaged; the side lower imaging unit is used to capture the bottom image of the side of the coal train to be imaged; wherein the side top imaging unit, the side middle imaging unit and the side lower imaging unit all include an equipment box and an area array scan camera, the equipment box is installed on the column, and the area array scan camera is installed in the equipment box.

3. The imaging system for detecting the state of a coal train according to claim 2, characterized in that: The depression angle of the area scan camera in the side top imaging unit is 45°, the angle of the area scan camera in the side middle imaging unit is parallel to the side of the carriage, and the elevation angle of the area scan camera in the lower imaging unit is 45°.

4. The imaging system for detecting the state of a coal train according to claim 2, characterized in that: The image acquisition system further includes an image acquisition trigger subsystem; The image acquisition trigger subsystem includes a speed measurement module and an outdoor cabinet. The speed measurement module is arranged on the side of the column, and is used to collect and send the train speed of the coal train to be imaged to the outdoor cabinet; the outdoor cabinet is used to generate and send image acquisition instructions to the side top imaging unit, the side middle imaging unit and the side lower imaging unit in the image acquisition subsystem according to the collected train speed of the coal train to be imaged.

5. The imaging system for detecting the state of a coal train according to claim 1, characterized in that: The original collected picture is a 2D picture, and the storage format of the original collected picture is JPG format.

6. The imaging system for detecting the state of a coal train according to claim 1, characterized in that: The process of stitching the original collected pictures by using a preset image stitching algorithm to output an imaging result of the carriage image of the coal transport train to be imaged is as follows: Extracting two consecutive frames of original collected pictures from the stored original collected pictures to obtain an input image; Performing feature extraction and feature description processing on the input image to obtain feature description information of the input image; Based on the feature description information of the input image, feature matching is performed between the input image and a train reference image of the coal transport train to be imaged to obtain a feature matching result; According to the feature matching result, performing transformation matrix estimation on the input image to obtain a transformation matrix estimation result; Based on the transformation matrix estimation result, performing image transformation on the input image to convert the input image into a coordinate system of a train reference image of the coal transport train to be imaged, and obtaining an input image transformation result; Fusing two adjacent frames of images in the input image transformation result to obtain a fused image; The fused image and the next frame of original captured image are used as input images again, and the above operation is repeated until the train is in a non-stop state, and a complete train image is output, that is, an imaging result of the carriage image of the coal transport train to be imaged is obtained.

7. The imaging system for detecting the state of a coal train according to claim 6, characterized in that: The SIFT algorithm is used to perform feature extraction and feature description processing on the input image to obtain feature description information of the input image.

8. The imaging system for detecting the state of a coal train according to claim 6, characterized in that: Based on the feature description information of the input image, feature matching is performed between the input image and a train reference image of the coal transport train to be imaged. In the process of obtaining the feature matching result, the normalized correlation method is used to determine the correspondence between the key features of the input image and the train reference image of the coal transport train to be imaged; and the mutual information is used to evaluate the similarity in the amount of shared information between the input image and the train reference image of the coal transport train to be imaged.

9. The imaging system for detecting the state of a coal train according to claim 6, characterized in that: According to the feature matching result, using the RANSAC algorithm to estimate the transformation matrix of the input image to obtain a transformation matrix estimation result; Two adjacent frames of images in the input image transformation result are fused. In the process of obtaining the fused image, the overlapping pixels are fused by feathering using the weighted average color value.

10. An imaging method for detecting the state of a coal transport train, characterized in that: Utilizing the imaging system for detecting the state of a coal transport train as claimed in any one of claims 1 to 9; The imaging method for detecting the state of a coal transport train comprises: An area scan camera is used to collect the exterior image of the carriage of the coal transport train to be imaged, and an original collected image is obtained; The original collected pictures are stitched using a preset image stitching algorithm, and an imaging result of the carriage image of the coal transport train to be imaged is output.

Images