A method, device, electronic device and storage medium for detecting foreign matter on a track bed

Abstract

The present invention discloses a foreign body detection method, device, electronic device and storage medium for the track bed. The present invention uses the height information of the foreign body for preliminary screening to obtain at least one connected domain, and then uses the image grayscale information for accurate screening, and determines the real area of ​​each small foreign body on the track bed based on the expansion operation of the image; thus, the present invention can effectively separate the small foreign bodies in the grayscale image, thereby realizing the accurate detection of small foreign bodies on the track bed and ensuring the driving safety of the train.

Description

Technical Field

[0001] The present invention belongs to the technical field of ballast foreign body detection, and in particular relates to a ballast foreign body detection method, device, electronic equipment and storage medium. Background Art

[0002] Foreign objects on the railway track bed are foreign objects that appear on the track bed, usually including broken spring bars caused by rail vibration, broken stones caused by the impact of track plates, iron sheets and metal parts for fixing electrical cables. During train operation, foreign objects on the track bed will pose a great hidden danger to the driving safety of the train. At present, foreign objects on the track bed are mainly detected and removed through manual inspection. This method is costly and inefficient, and often cannot eliminate safety hazards in time. Therefore, the visual detection algorithm of track bed foreign objects came into being. The visual detection algorithm can detect foreign objects on the track bed in time, thereby ensuring the safe operation of the train.

[0003] At present, the visual inspection algorithm of the track bed generally uses a calibrated 3D camera to capture a depth image of the track bed, and then performs image recognition on the captured depth image to obtain the foreign object detection result by detecting the area with abnormal height. However, in actual working conditions, there will be flat foreign objects and small foreign objects in the track bed, such as iron sheets and metal parts for fixing electrical cables. The height characteristics of these small foreign objects are not obvious. Therefore, the existing visual inspection algorithm cannot accurately detect the aforementioned small and flat foreign objects, thereby increasing the safety hazard of train driving. Therefore, it is urgent to provide a method for detecting small foreign objects on the track bed. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, electronic device and storage medium for detecting foreign objects on the track bed, so as to solve the problem that the existing track bed visual detection algorithm cannot accurately detect small and flat foreign objects, thereby increasing the potential safety hazards of train operation.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting foreign matter on a ballast bed, comprising:

[0007] Acquire a grayscale image of the roadbed to be inspected, and perform height difference preprocessing on the grayscale image to obtain a preprocessed image for representing height information of the roadbed and all foreign objects on the roadbed in the grayscale image;

[0008] Screening out pixels whose grayscale values ​​are greater than a grayscale threshold in the preprocessed image, and forming at least one connected domain using all pixels whose grayscale values ​​are greater than the grayscale threshold;

[0009] Performing small foreign body detection on each of the at least one connected domain to determine the connected domain containing the small foreign body, so as to use the connected domain containing the small foreign body as the detection area;

[0010] In the detection area, pixels whose grayscale values ​​are within a preset grayscale range are screened out, and at least one detection connected domain is formed using all pixels whose grayscale values ​​are within the preset grayscale range;

[0011] A dilation operation is performed on the at least one detected connected domain to obtain at least one final connected region, wherein each final connected region in the at least one final connected region is used to represent a real region of a small foreign object in the grayscale image.

[0012] Based on the above disclosed content, the present invention first performs height difference preprocessing on the grayscale image of the roadbed to be detected, so as to obtain a preprocessed image containing height information of the roadbed and each foreign object, and then screens out pixels whose grayscale values ​​are greater than the grayscale threshold in the preprocessed image, so as to use the screened pixels to form at least one connected domain. This step is equivalent to screening out the area with abnormal height in the preprocessed image. Then, foreign object detection can be performed on at least one connected domain to obtain a connected domain containing small foreign objects (small foreign objects are foreign objects whose maximum height is less than the height threshold or whose surface area is less than the area threshold), so as to accurately locate the foreign objects in the connected domain containing small foreign objects, that is, in the connected domain containing small foreign objects, pixels whose grayscale values ​​are within a preset grayscale range are screened out, so as to form at least one detection connected domain. Finally, an expansion operation is performed on at least one detection connected domain, so as to use the expansion operation to connect the adjacent detection connected domains in any detection connected domain, so as to restore the shape of the noise point (that is, the small foreign object), and then obtain the real area of ​​each small foreign object in the grayscale image.

[0013] Through the above design, the present invention uses the height information of foreign objects for preliminary screening to obtain at least one connected domain, and then uses the image grayscale information for accurate screening, and determines the real area of ​​each small foreign object on the track bed based on the expansion operation of the image; thus, the present invention can effectively separate small foreign objects in the grayscale image, thereby realizing accurate detection of small foreign objects on the track bed and ensuring the driving safety of the train.

[0014] In a possible design, the grayscale image is preprocessed with a height difference to obtain a preprocessed image for representing the height information of the roadbed and all foreign objects on the roadbed in the grayscale image, including:

[0015] Calculating the global average grayscale value of the grayscale image, and adjusting the grayscale value of each pixel in the grayscale image to the global average grayscale value, to obtain a constant grayscale image;

[0016] Superimposing the grayscale image and the constant grayscale image to obtain a cached image, wherein the cached image is a double-layer image, one layer is the grayscale image, and the other layer is the constant grayscale image;

[0017] Performing an opening operation on the cached image to obtain an average grayscale template image and a constant grayscale template image respectively;

[0018] Subtracting the grayscale of the average grayscale template image from the constant grayscale template image to obtain the preprocessed image.

[0019] Based on the above disclosed content, the present invention discloses a specific process of height difference preprocessing, namely, first calculating the global average grayscale value of the grayscale image, and then adjusting the grayscale value of each pixel in the grayscale image to the global average grayscale to obtain a constant grayscale image, then superimposing the constant grayscale image and the grayscale image to obtain a double-layer cache image, and finally, performing an open operation on the cache image, and subtracting the grayscale of the two images obtained by the open operation to obtain a preprocessed image containing the height information of the road bed and each foreign object.

[0020] In a possible design, performing small foreign matter detection on each connected domain of the at least one connected domain to determine the connected domain containing the small foreign matter includes:

[0021] Performing width screening on each connected domain in the at least one connected domain, screening out connected domains with widths within a preset width range as pre-selected connected domains;

[0022] In the preselected connected domains, a preselected connected domain whose area is within a preset area range is selected as a pre-detection area;

[0023] In the pre-detection area, the pre-detection area whose length or width is not within the preset threshold range is screened out, so that the screened pre-detection area is used as a connected domain containing small foreign matter.

[0024] Based on the above disclosed content, the present invention discloses a specific process for detecting small foreign objects, which is essentially to perform size screening of connected domains, that is, to perform width screening, area screening, and length or width screening in sequence. The essence of this detection is to filter out connected domains containing large foreign objects in at least one connected domain, so as to retain only connected domains containing small foreign objects, so as to facilitate the subsequent precise positioning of small foreign objects.

[0025] In a possible design, the preset width range is 100px to 2048px, and the preset area range is 300px. 2 ~1000000px 2, the preset threshold range is 20px~100000px.

[0026] In a possible design, after obtaining the average grayscale template image, the method further includes:

[0027] Performing shadow height field processing on the average grayscale template image to obtain an average grayscale template image with image noise removed, wherein the image noise is used to characterize image information generated by the small foreign matter;

[0028] The average grayscale template image with image noise removed is input into the trained deep learning model for foreign body detection to obtain a foreign body detection result, wherein the foreign body detection result includes an image area for characterizing large foreign bodies, area information of the image area, and the coordinates of the center point of the image area.

[0029] Based on the above disclosed content, the present invention also discloses a method for detecting large foreign objects (large foreign objects are foreign objects with a maximum height greater than or equal to a height threshold, or a surface area greater than or equal to an area threshold), that is, the average grayscale template image is first subjected to shadow height field processing to filter out each small foreign object in the average grayscale template image, and then the processed average grayscale template image is input into the trained deep learning model to obtain the detection results of each large foreign object in the grayscale image; thus, combined with the aforementioned small foreign object detection method, accurate detection of different foreign objects on the roadbed can be achieved, thereby meeting the foreign object detection needs of the roadbed and further improving the driving safety of the train.

[0030] In a possible design, the shadow height field processing is performed on the average grayscale template image to obtain the average grayscale template image with image noise removed, including:

[0031] Performing pixel type conversion on the average grayscale template image, converting the average grayscale template image into a floating point type image, wherein the pixel value of the floating point type image is between 0 and 200;

[0032] Acquire acquisition parameters of the grayscale image, wherein the acquisition parameters include the angle Slant between the central axis of the light source beam and the central axis of the camera, the angle Tilt between the central axis of the light source beam and the positive axis of the X-axis of the plane rectangular coordinate system in the grayscale image, the reflectivity Albedo of the roadbed to be detected, and the ambient light Ambient;

[0033] Calibrate the acquisition parameters, and calibrate the values ​​of Slant, Tilt, Albedo and Ambient to 0, 0, 1 and 0 respectively, so as to obtain calibrated acquisition parameters;

[0034] Based on the calibrated acquisition parameters, the average grayscale template image is corrected to obtain an average grayscale template image with image noise removed after the correction is completed.

[0035] Based on the above disclosed content, the present invention discloses a specific process of shadow height field processing, the essence of which is to adjust various parameters during image acquisition to perform image correction on the average grayscale template image, thereby weakening the image information of small foreign objects in the grayscale image to achieve the purpose of filtering small foreign objects; thus, after the above-mentioned processing, large foreign objects in the average grayscale image can be effectively separated, thereby improving the recognition accuracy of large foreign objects.

[0036] In one possible design, the length and width of the structural element used for the dilation operation of the at least one detected connected domain are both 30 pixels, the length and width of the structural element used for the opening operation of the cached image are 20 pixels and 10 pixels respectively, the grayscale threshold is 3, and the preset grayscale range is 8 to 10000.

[0037] In a second aspect, the present invention provides a foreign body detection device for a track bed, comprising:

[0038] An acquisition unit, used for acquiring a grayscale image of the roadbed to be detected, and performing height difference preprocessing on the grayscale image to obtain a preprocessed image for representing height information of the roadbed and all foreign objects on the roadbed in the grayscale image;

[0039] A segmentation unit, used to screen out pixels whose grayscale values ​​are greater than a grayscale threshold in the preprocessed image, and use all pixels whose grayscale values ​​are greater than the grayscale threshold to form at least one connected domain;

[0040] A foreign body detection unit, configured to detect small foreign bodies in each of the at least one connected domain, determine the connected domain containing the small foreign body, and use the connected domain containing the small foreign body as a detection area;

[0041] A grayscale screening unit, used to screen out pixels whose grayscale values ​​are within a preset grayscale range in the detection area, and use all the pixels whose grayscale values ​​are within the preset grayscale range to form at least one detection connected domain;

[0042] The image processing unit is used to perform an expansion operation on the at least one detected connected domain to obtain at least one final connected area, wherein each of the at least one final connected area is used to represent a real area of ​​a small foreign object in the grayscale image.

[0043] In a third aspect, the present invention provides another foreign object detection device for a track bed. Taking the device as an electronic device as an example, the device includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the foreign object detection method for the track bed as in the first aspect or any possible design of the first aspect.

[0044] In a fourth aspect, the present invention provides a storage medium having instructions stored thereon, which, when executed on a computer, executes the foreign body detection method for the track bed as in the first aspect or any possible design of the first aspect.

[0045] In a fifth aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the foreign body detection method for the track bed as in the first aspect or any possible design of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic flow chart of the steps of the foreign body detection method for the ballast provided by the present invention;

[0047] Figure 2 A schematic diagram of the structure of a foreign body detection device for a ballast provided by the present invention;

[0048] Figure 3 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that although the description of these embodiments is used to help understand the present invention, it does not constitute a limitation of the present invention. The specific structures and functional details disclosed herein are only used to describe the exemplary embodiments of the present invention. However, the present invention can be embodied in many alternative forms, and it should not be understood that the present invention is limited to the embodiments set forth herein.

[0050] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0051] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0052] Example

[0053] See also Figure 1 As shown, the foreign body detection method for the track bed provided in the first aspect of the present embodiment uses the height information of the foreign body to perform preliminary screening of the foreign body to obtain at least one connected domain, and then uses the image grayscale information to accurately screen small foreign bodies, and determines the real area of ​​each small foreign body on the track bed based on the expansion operation of the image, thereby effectively separating the small foreign bodies in the grayscale image, thereby completing the accurate detection of small foreign bodies on the track bed, wherein the method provided in the present embodiment can be but not limited to running on the detection terminal side or the detection server side, and the detection terminal can be but not limited to a personal computer (personal computer, PC), a tablet computer, a smart phone and / or a personal digital assistant (personal digital assistant, PDA), etc. It can be understood that the aforementioned execution subject does not constitute a limitation on the embodiments of the present application, and accordingly, the operation steps of the present method are shown in the following steps S1 to S5.

[0054] S1. Obtain a grayscale image of the roadbed to be detected, and perform height difference preprocessing on the grayscale image to obtain a preprocessed image for characterizing the height information of the roadbed and all foreign objects on the roadbed in the grayscale image; in specific application, it can be but not limited to using a 3D camera to shoot the roadbed area to obtain a 3D point cloud image of the roadbed, and then convert the 3D point cloud image into a grayscale image. Optionally, it can be but not limited to realizing the image conversion based on Matlab software. At the same time, in this embodiment, the grayscale image is first preprocessed with height difference to obtain an image containing the height information of each object (i.e., the roadbed and each foreign object) in the grayscale image, that is, to obtain a preprocessed image, so as to subsequently screen small foreign objects based on the preprocessed image. Furthermore, for example, a small foreign object is a foreign object whose maximum height is less than a height threshold (e.g., less than 15 mm) or whose surface area is less than an area threshold (e.g., less than 2 square centimeters). Of course, the height threshold and area threshold defined therein can be specifically set according to the foreign objects detected in actual use, and are not specifically limited here.

[0055] Optionally, for example, performing height difference preprocessing on the grayscale image to obtain a preprocessed image may include but is not limited to the following steps S11 to S14.

[0056] S11. Calculate the global average grayscale value of the grayscale image, and adjust the grayscale value of each pixel in the grayscale image to the global average grayscale value to obtain a constant grayscale image; in specific application, the calculation method of the global average grayscale value is: sum the grayscale value of each pixel in the grayscale image to obtain the total grayscale value, and then divide the total grayscale value by the total number of pixels in the grayscale image, and the result obtained is the global average grayscale value; after obtaining the global average grayscale value, the grayscale image can be grayscale adjusted, that is, the grayscale value of each pixel in the grayscale image is adjusted to the global average grayscale value; for example, for example, the global average grayscale value is 155, then the grayscale value of each pixel in the grayscale image is adjusted by 155, so that a constant grayscale image is obtained after the adjustment is completed. Of course, the aforementioned global average grayscale value is only an example and is only used to explain the principle of grayscale adjustment.

[0057] After the grayscale image is grayscale adjusted based on the global average grayscale value, image superposition can be performed to facilitate subsequent image logic operations, as shown in the following step S12.

[0058] S12. Superimpose the grayscale image and the constant grayscale image to obtain a cached image, wherein the cached image is a dual-layer image, and one layer is the grayscale image, and the other layer is the constant grayscale image; in specific application, a dual-channel image is created as the cached image, wherein each pixel in the dual-channel image corresponds to two values, one value is the pixel value of the pixel in the grayscale image, and the other value is the grayscale value of the pixel in the constant grayscale image, that is, the global average grayscale value. Therefore, the created dual-channel image is actually a dual-layer image.

[0059] After the grayscale image and the constant grayscale image are superimposed, image logic operations can be performed, as shown in the following step S13.

[0060] S13. Perform an opening operation on the cached image to obtain an average grayscale template image and a constant grayscale template image respectively; in specific application, since the cached image is a double-layer image, it is equivalent to performing an opening operation on both layers, so the result is the image obtained after the opening operation on each layer; in this embodiment, the opening operation is to first perform an erosion operation on the image and then perform an expansion operation, which is used to smooth the shape boundary and can remove small particle noise in the image and disconnect the adhesion between objects in the image without changing the area, thereby providing an image basis for the subsequent recognition of small foreign objects.

[0061] Furthermore, for example, the length and width of the structure element used in the opening operation may be, but are not limited to, 20 pixels and 10 pixels respectively.

[0062] After the cached image is opened, a preprocessed image containing height information of the roadbed and all foreign objects on the roadbed in the grayscale image can be obtained based on the image obtained by the operation, wherein the acquisition step is shown in the following step S14.

[0063] S14. Subtract the grayscale of the average grayscale template image from the constant grayscale template image to obtain the preprocessed image; in specific application, it is equivalent to subtracting the grayscale value between each pixel in the average grayscale template image and the constant grayscale template image, and the grayscale difference obtained by the subtraction is used to characterize the height information of the roadbed and all foreign objects on the roadbed in the grayscale image. Therefore, after the grayscale of the above two images is subtracted, the preprocessed image after height difference preprocessing can be obtained.

[0064] After preprocessing the grayscale image by height difference to obtain a preprocessed image containing the height information of the roadbed and each foreign object in the grayscale image, small foreign objects can be screened. In this embodiment, the height information is first used for preliminary screening, and then the grayscale information and logical operations of the image are used for precise screening. The screening steps are shown in the following steps S2 to S5.

[0065] S2. Filter out pixels whose grayscale values ​​are greater than the grayscale threshold in the preprocessed image, and use all the pixels whose grayscale values ​​are greater than the grayscale threshold to form at least one connected domain. In specific applications, filtering out pixels whose grayscale values ​​are greater than the grayscale threshold is equivalent to filtering out highly abnormal objects. Optionally, for example, the grayscale threshold is 3. Therefore, step S2 is to filter out pixels whose grayscale values ​​are greater than 3 from the preprocessed image, and use the pixels whose grayscale values ​​are greater than 3 to form at least one connected domain. Furthermore, the method for obtaining the aforementioned connected domain is: for any pixel among the pixels whose grayscale values ​​are greater than 3, use the area composed of pixels in the preprocessed image that have the same grayscale value as the any pixel and are adjacent in position as the connected domain. Of course, the aforementioned method can also be used for the remaining pixels to form their corresponding connected domains. Therefore, based on the aforementioned filtered pixels whose grayscale values ​​are greater than 3, at least one connected domain can be formed, and the obtained connected domain is used as a highly abnormal area. Therefore, it can provide a data basis for the subsequent accurate screening of small foreign objects.

[0066] After completing the preliminary screening of foreign objects based on the preprocessed image (i.e., screening out highly abnormal areas), small foreign objects can be accurately located in at least one screened connected domain, wherein the accurate screening process is shown in the following step S3.

[0067] S3. Perform small foreign body detection on each of the at least one connected domains to determine the connected domain containing the small foreign body, and use the connected domain containing the small foreign body as the detection area; in specific application, the connected domain containing the small foreign body is determined in at least one connected domain based on the size of each connected domain, wherein the size operations are width screening, area screening, and length or width screening of the connected domain in sequence. Optionally, the aforementioned size screening process is shown in the following steps S31 to S33.

[0068] S31. Perform width screening on each connected domain in the at least one connected domain, and screen out connected domains with a width within a preset width range as pre-selected connected domains; in specific application, the preset width range can be, but is not limited to, 100px to 2048px (pixel value), and therefore, connected domains with a width between 100px and 2048px are used as pre-selected connected domains. In the present embodiment, the purpose of setting the maximum width to 2048px is to ensure that after the appearance of foreign objects that span the size of a single image, the foreign objects that span the single image can still be detected, thereby improving the detection accuracy.

[0069] After completing the width screening of each connected domain, the area screening can be performed on the screened connected domains (ie, the pre-selected connected domains), and the screening process is shown in the following step S32.

[0070] S32. In the pre-selected connected domain, the pre-selected connected domain whose area is within the preset area range is selected as the pre-detection area; in specific applications, the preset area range may be, but is not limited to, 300px. 2 ~1000000px 2 , that is, select the areas between 300px in the pre-selected connected domain 2 ~1000000px 2 The connected domain between them is used as the pre-detection area.

[0071] In this embodiment, the aforementioned preset width range and preset area range may be specifically set according to the hardware device used or the size of the captured image, and are not limited to the aforementioned examples.

[0072] After the area screening is completed, the length or width screening can be performed, and the screening process is shown in the following step S33.

[0073] S33. In the pre-detection area, the pre-detection area whose length or width is not within the preset threshold range is screened out, so as to use the screened pre-detection area as a connected domain containing small foreign objects; in specific application, the preset threshold range can be, but is not limited to, 20px to 100000px, therefore, in the pre-detection area, the pre-detection area whose length is less than 20px or whose length is greater than 100000px is screened out as a connected domain containing small foreign objects, and the pre-detection area whose width is less than 20px or whose width is greater than 100000px is screened out as a connected domain containing small foreign objects, that is, the pre-detection area whose length is not within the aforementioned preset threshold range or whose width is not within the aforementioned preset threshold range is used as a connected domain containing small foreign objects.

[0074] Therefore, through the aforementioned steps S31 to S33, the area where the small foreign matter is located can be accurately located from the highly abnormal area, so as to provide a data basis for the subsequent extraction of the real area of ​​the small foreign matter in the grayscale image, wherein the positioning process of the real area of ​​the small foreign matter is shown in the following steps S4 and S5.

[0075] S4. In the detection area, pixel points whose grayscale values ​​are within the preset grayscale range are screened out, and all pixel points whose grayscale values ​​are within the preset grayscale range are used to form at least one detection connected domain; in the present embodiment, the preset grayscale range may be, but is not limited to, 8 to 10000, that is, pixel points with grayscale values ​​between 8 and 10000 are screened out in the detection area, so that the pixel points screened out in this step can be used to form at least one detection connected domain, and at least one detection connected domain serves as a small foreign body area. In the present embodiment, the composition principle of the detection connected domain is the same as the composition principle of the connected domain in the aforementioned step S2, which will not be repeated here.

[0076] After obtaining the detection connected domain (i.e., the small foreign body area) through step S4, it is only necessary to use image logic operations to screen out images with the same structure in the detection connected domain and connect them to obtain the real area of ​​the small foreign body, that is, to segment the area in the grayscale image used to characterize the small foreign body. The image logic operation process is shown in the following step S5.

[0077] S5. Perform a dilation operation on the at least one detected connected domain to obtain at least one final connected area, wherein each of the at least one final connected area is used to characterize the real area of ​​a small foreign object in the grayscale image; in specific application, the dilation operation can connect the adjacent detected connected domains in any detected connected domain, thereby restoring the shape of the noise point (that is, the small foreign object), thereby accurately obtaining the real area of ​​each small foreign object in the grayscale image. Optionally, the length and width of the structural element used in the dilation operation can be, but not limited to, 30 pixels.

[0078] In this embodiment, if no connected area can be obtained after the expansion operation is performed on at least one detected connected domain, it means that there is no small foreign object in the grayscale image of the roadbed to be detected. Therefore, it can be judged whether there is a foreign object on the roadbed to be detected based on whether the expansion operation obtains a connected area, and after the final connected area is obtained, the position of the small foreign object in the grayscale image is accurately located based on the final connected area.

[0079] Therefore, through the method for detecting foreign objects on the track bed described in detail in the aforementioned steps S1 to S5, the present invention uses the height information of the foreign objects for preliminary screening to obtain at least one connected domain, and then uses the image grayscale information for accurate screening, and determines the real area of ​​each small foreign object on the track bed based on the expansion operation of the image; thus, the present invention can effectively separate small foreign objects in the grayscale image, thereby realizing accurate detection of small foreign objects on the track bed and ensuring the driving safety of the train.

[0080] In a possible design, the second aspect of this embodiment provides a method for detecting large foreign objects on the roadbed based on the first aspect of the embodiment, wherein the large foreign objects are foreign objects whose maximum height is greater than or equal to a height threshold (such as less than 15 mm) or whose surface area is greater than or equal to an area threshold. Accordingly, the detection method may include but is not limited to the following steps S6 and S7.

[0081] S6. Perform shadow height field processing on the average grayscale template image to obtain an average grayscale template image with image noise removed, wherein the image noise is used to characterize the image information generated by the small foreign objects; in specific applications, performing shadow height field processing on the average grayscale template image can remove the image information of small foreign objects in the image, thereby avoiding interference from small foreign objects when identifying large foreign objects. Optionally, the shadow height field processing process can include but is not limited to the following steps S61 to S64.

[0082] S61. Perform pixel type conversion on the average grayscale template image, converting the average grayscale template image into a floating-point type image, wherein the pixel value of the floating-point type image is between 0 and 200; in this embodiment, step S61 is equivalent to converting an int type image into a floating-point type image, so as to perform subsequent shadow height field processing of the image.

[0083] After the image type conversion, shadow height field processing can be performed. In this embodiment, the shadow height field processing is essentially to adjust various parameters during image acquisition to perform image correction on the average grayscale template image, so as to weaken the image information of small foreign objects in the grayscale image, so as to achieve the purpose of filtering small foreign objects. The correction process is shown in the following steps S62 to S64.

[0084] S62. Obtain acquisition parameters of the grayscale image, wherein the acquisition parameters include the angle Slant between the central axis of the light source beam and the central axis of the camera, the angle Tilt between the central axis of the light source beam and the positive axis of the X-axis of the plane rectangular coordinate system in the grayscale image, the reflectivity Albedo of the roadbed to be detected, and the ambient brightness Ambient.

[0085] In this embodiment, the correction process is equivalent to simulating the image acquisition process, and then the image with small foreign objects filtered out is obtained by continuously adjusting the acquisition parameters, thereby achieving image correction, wherein the parameter adjustment process is shown in the following step S63.

[0086] S63. The acquisition parameters are calibrated, and the values ​​of Slant, Tilt, Albedo and Ambient are calibrated to 0, 0, 1 and 0 respectively to obtain the calibrated acquisition parameters; in this embodiment, the reflectivity of the roadbed to be detected is set to 1, the central axis of the simulated light source is set to coincide with the positive z-axis (that is, the angle between the central axis of the light source beam and the central axis of the camera is 0), the angle between the light source and the X-axis after projection onto the XY plane is set to 0 (that is, the angle between the central axis of the light source beam and the positive axis of the X-axis of the plane rectangular coordinate system in the grayscale image is 0), and the amount of ambient light is set to 0. At this time, the obtained image can effectively filter out small foreign objects, and the denoising effect is the best. Therefore, the influence of small foreign objects on the subsequent large foreign object detection can be minimized.

[0087] S64. Based on the calibrated acquisition parameters, the average grayscale template image is corrected to obtain an average grayscale template image with image noise removed after the correction is completed; in this embodiment, after obtaining the calibrated acquisition parameters, the parameters can be used as a reference to adjust the image, so as to filter out small foreign objects after the adjustment to obtain an average grayscale template image with image noise removed.

[0088] After removing the noise of small foreign objects in the average grayscale template image, image recognition can be performed to obtain large foreign object detection results, as shown in the following step S7.

[0089] S7. Input the average grayscale template image with image noise removed into the trained deep learning model for foreign body detection to obtain a foreign body detection result, wherein the foreign body detection result includes an image area representing a large foreign body, area information of the image area, and the coordinates of the center point of the image area; in specific applications, the trained deep learning model can be, but is not limited to, a trained halcon anomaly (singular value detection) model.

[0090] Optionally, the training process of the halcon anomaly model can be, but is not limited to:

[0091] The first step is to obtain a training data set, wherein the training data set includes a number of average grayscale template images of the roadbed after being processed by the shadow height field.

[0092] Step 2: Take the training data set as input and the detection results of large foreign objects in the average grayscale template image of each roadbed in the training data as output, train the Halcon Anomaly model until the loss function of the model converges, and obtain the trained Halcon Anomaly model, in which the detection results include the image area used to characterize large foreign objects, the area information of the image area, and the coordinates of the center point of the image area.

[0093] Of course, in this embodiment, other deep learning networks may also be used, such as CNN (Convolutional Neural Networks) neural network or RNN (recursive neural network) neural network, etc., and are not limited to the aforementioned examples.

[0094] Therefore, through the aforementioned steps S6 and S7, the present invention uses the shadow height field algorithm to process the average grayscale template image, so as to filter out the edge contour features of small foreign objects. Therefore, the image preprocessed in this way can make the deep learning detection method using the halcon anomaly model more effective, and can effectively separate large foreign objects in the image.

[0095] like Figure 2 As shown, the third aspect of this embodiment provides a hardware device for implementing the foreign body detection method of the track bed described in the first aspect and the second aspect of the embodiment, including:

[0096] The acquisition unit is used to acquire a grayscale image of the roadbed to be detected, and perform height difference preprocessing on the grayscale image to obtain a height image used to characterize the height information of the roadbed and all foreign objects on the roadbed in the grayscale image.

[0097] The segmentation unit is used to screen out pixel points whose grayscale values ​​are greater than the grayscale threshold in the height image, and use all pixel points whose grayscale values ​​are greater than the grayscale threshold to form at least one connected domain.

[0098] The foreign body detection unit is used to detect small foreign bodies in each connected domain of the at least one connected domain, determine the connected domain containing the small foreign body, and use the connected domain containing the small foreign body as the detection area.

[0099] The grayscale screening unit is used to screen out the pixel points whose grayscale values ​​are within a preset grayscale range in the detection area, and use all the pixel points whose grayscale values ​​are within the preset grayscale range to form at least one detection connected domain.

[0100] The image processing unit is used to perform an expansion operation on the at least one detected connected domain to obtain at least one final connected area, wherein each of the at least one final connected area is used to represent a real area of ​​a small foreign object in the grayscale image.

[0101] The working process, working details and technical effects of the device provided in this embodiment can be found in the first and second aspects of the embodiments, and will not be described in detail here.

[0102] like Figure 3 As shown, the fourth aspect of this embodiment provides another device for detecting foreign objects on the track bed. Taking the device as an electronic device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the foreign object detection method for the track bed as described in the first and second aspects of the embodiments.

[0103] For example, the memory may include, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory, first input first output (FIFO) and / or first in last out (FILO), etc. Specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). At the same time, the processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0104] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may not be limited to a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (General Packet Radio Service, GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0105] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first and second aspects of the embodiment, and will not be described in detail here.

[0106] A fifth aspect of this embodiment provides a storage medium storing instructions for the method for detecting foreign objects in a track bed as described in the first and second aspects of the embodiment, that is, the storage medium stores instructions, and when the instructions are run on a computer, the method for detecting foreign objects in a track bed as described in the first and second aspects is executed.

[0107] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0108] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first and second aspects of the embodiments, and will not be described in detail here.

[0109] A sixth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the foreign body detection method for the track bed as described in the first and second aspects of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0110] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting foreign matter on a ballast bed, characterized in that: include: Acquire a grayscale image of the roadbed to be inspected, and perform height difference preprocessing on the grayscale image to obtain a preprocessed image for representing height difference information of the roadbed and all foreign objects on the roadbed in the grayscale image; Screening out pixels whose grayscale values ​​are greater than a grayscale threshold in the preprocessed image, and forming at least one connected domain using all pixels whose grayscale values ​​are greater than the grayscale threshold; Performing small foreign body detection on each of the at least one connected domain to determine the connected domain containing the small foreign body, so as to use the connected domain containing the small foreign body as the detection area; In the detection area, pixels whose grayscale values ​​are within a preset grayscale range are screened out, and at least one detection connected domain is formed using all pixels whose grayscale values ​​are within the preset grayscale range; Performing a dilation operation on the at least one detected connected domain to obtain at least one final connected region, wherein each final connected region in the at least one final connected region is used to represent a real region of a small foreign object in the grayscale image; Performing height difference preprocessing on the grayscale image to obtain a preprocessed image for representing height difference information of the roadbed and all foreign objects on the roadbed in the grayscale image, including: Calculating the global average grayscale value of the grayscale image, and adjusting the grayscale value of each pixel in the grayscale image to the global average grayscale value, to obtain a constant grayscale image; Superimposing the grayscale image and the constant grayscale image to obtain a cached image, wherein the cached image is a double-layer image, one layer is the grayscale image, and the other layer is the constant grayscale image; Performing an opening operation on the cached image to obtain an average grayscale template image and a constant grayscale template image respectively; Subtracting the grayscale of the average grayscale template image from the constant grayscale template image to obtain the preprocessed image.

2. The method according to claim 1, characterized in that Performing small foreign matter detection on each connected domain of the at least one connected domain to determine a connected domain containing small foreign matter includes: Performing width screening on each connected domain in the at least one connected domain, screening out connected domains with widths within a preset width range as pre-selected connected domains; In the preselected connected domains, a preselected connected domain whose area is within a preset area range is selected as a pre-detection area; In the pre-detection area, the pre-detection area whose length or width is not within the preset threshold range is screened out, so that the screened pre-detection area is used as a connected domain containing small foreign matter.

3. The method according to claim 2, characterized in that The preset width range is 100px to 2048px, and the preset area range is 300px 2 ~1000000px 2 , the preset threshold range is 20px~100000px.

4. The method according to claim 1, characterized in that After obtaining the average grayscale template image, the method further includes: Performing shadow height field processing on the average grayscale template image to obtain an average grayscale template image with image noise removed, wherein the image noise is used to characterize image information generated by the small foreign matter; The average grayscale template image with image noise removed is input into the trained deep learning model for foreign body detection to obtain a foreign body detection result, wherein the foreign body detection result includes an image area for characterizing large foreign bodies, area information of the image area, and the coordinates of the center point of the image area.

5. The method according to claim 4, characterized in that Performing shadow height field processing on the average grayscale template image to obtain an average grayscale template image with image noise removed includes: Performing pixel type conversion on the average grayscale template image, converting the average grayscale template image into a floating point type image, wherein the pixel value of the floating point type image is between 0 and 200; Acquire acquisition parameters of the grayscale image, wherein the acquisition parameters include the angle Slant between the central axis of the light source beam and the central axis of the camera, the angle Tilt between the central axis of the light source beam and the positive axis of the X-axis of the plane rectangular coordinate system in the grayscale image, the reflectivity Albedo of the roadbed to be detected, and the ambient light Ambient; Calibrate the acquisition parameters, and calibrate the values ​​of Slant, Tilt, Albedo and Ambient to 0, 0, 1 and 0 respectively, so as to obtain calibrated acquisition parameters; Based on the calibrated acquisition parameters, the average grayscale template image is corrected to obtain an average grayscale template image with image noise removed after the correction is completed.

6. The method according to claim 1, characterized in that The length and width of the structural element used for the dilation operation of the at least one detected connected domain are both 30 pixels, the length and width of the structural element used for the opening operation of the cached image are 20 pixels and 10 pixels respectively, the grayscale threshold is 3, and the preset grayscale range is 8 to 10000.

7. A device for detecting foreign matter on a track bed, characterized in that: include: An acquisition unit is used to acquire a grayscale image of the roadbed to be detected, and perform height difference preprocessing on the grayscale image to obtain a preprocessed image for representing height difference information of the roadbed and all foreign objects on the roadbed in the grayscale image; A segmentation unit, used to screen out pixels whose grayscale values ​​are greater than a grayscale threshold in the preprocessed image, and use all pixels whose grayscale values ​​are greater than the grayscale threshold to form at least one connected domain; A foreign body detection unit, configured to detect small foreign bodies in each of the at least one connected domain, determine the connected domain containing the small foreign body, and use the connected domain containing the small foreign body as a detection area; A grayscale screening unit, used to screen out pixels whose grayscale values ​​are within a preset grayscale range in the detection area, and use all the pixels whose grayscale values ​​are within the preset grayscale range to form at least one detection connected domain; An image processing unit, configured to perform a dilation operation on the at least one detected connected domain to obtain at least one final connected region, wherein each of the at least one final connected region is used to represent a real region of a small foreign object in the grayscale image; Performing height difference preprocessing on the grayscale image to obtain a preprocessed image for representing height difference information of the roadbed and all foreign objects on the roadbed in the grayscale image, including: Calculating the global average grayscale value of the grayscale image, and adjusting the grayscale value of each pixel in the grayscale image to the global average grayscale value, to obtain a constant grayscale image; Superimposing the grayscale image and the constant grayscale image to obtain a cached image, wherein the cached image is a double-layer image, one layer is the grayscale image, and the other layer is the constant grayscale image; Performing an opening operation on the cached image to obtain an average grayscale template image and a constant grayscale template image respectively; Subtracting the grayscale of the average grayscale template image from the constant grayscale template image to obtain the preprocessed image.

8. An electronic device, characterized in that: include: A memory, a processor and a transceiver which are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for detecting foreign matter on the track bed as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed on a computer, the method for detecting foreign matter on a track bed according to any one of claims 1 to 6 is executed.

Images