Rail sleeper image hole filling method and system

Abstract

The application discloses a sleeper image hole filling method and system, relates to the technical field of sleeper defect detection, and comprises the following steps: collecting a depth image of a sleeper surface and performing filtering processing; performing threshold segmentation on the depth image after the filtering processing to obtain a hole region; performing inflation processing on the hole region, performing difference operation on the inflated hole region and the initial hole region to obtain a foreground region; and calculating the average value of sampling pixels in the foreground region, filling the hole region step by step from outside to inside, and completing the filling of the hole region. The core step of the application lies in the use of the characteristics of the image foreground connected region, because the inflated hole region can completely guarantee the full fitting with the target region, thereby guaranteeing the information sampling for the subsequent hole filling, and being able to adapt to holes of different types and positions, and finally guaranteeing the accuracy of the sleeper region detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sleeper defect detection, and in particular to a sleeper image hole filling method and system. BACKGROUND

[0002] Sleeper, also known as sleeper, is a structure used to support steel rails in a railway, mainly to maintain the position of the steel rails and transmit the huge pressure borne by the steel rails to the track bed, and is an important component of the railway. In the process of maintaining the railway, the position of the sleeper needs to be located and the sleeper needs to be detected. In the past, manual inspection was mainly relied on, but this detection method is time-consuming, inefficient, has many limiting factors, and is highly subjective, and cannot meet the requirements of faster, more accurate and automatic detection. With the development of machine vision in industrial applications, image processing technology has been gradually applied to sleeper detection.

[0003] However, in the process of collecting images of sleepers, due to the shielding of samples and the limitation of the collection environment, the obtained point cloud data often has holes due to data loss. The sleeper image with holes not only causes the positioning of the sleeper detection area to fail, but also greatly affects the accuracy of the sleeper area detection, resulting in false detection. SUMMARY

[0004] Therefore, the present application provides a sleeper image hole filling method and system to solve the problems in the prior art.

[0005] The first aspect of the present application provides a sleeper image hole filling method, comprising:

[0006] collecting images of the surface of the sleeper by an image collection device to obtain a depth image of the surface of the sleeper;

[0007] filtering the collected depth image in a preset manner;

[0008] performing threshold segmentation on the depth image after filtering to obtain an original hole region, and performing inflation processing on all original hole regions to obtain a corresponding inflation hole region;

[0009] performing difference operation on the inflation hole region and the original hole region to obtain a foreground region corresponding to each original hole region;

[0010] calculating the mean value of the sampling pixels in the foreground region, and assigning the neighborhood position of the original hole region and the foreground region, and updating the foreground region; calculating the mean value of the sampling pixels in the updated foreground region, and assigning the neighborhood position of the remaining region in the original hole region and the updated foreground region, until the filling of the original hole region is completed.

[0011] In a possible implementation of the first aspect, capturing an image of the sleeper surface by an image capture device includes:

[0012] The 2D image of the sleeper surface is collected by a linear array camera and a 2D camera, or the 3D image of the sleeper surface is collected by the linear array camera and a 3D camera.

[0013] In a possible implementation of the first aspect, filtering the acquired depth image in a preset manner includes:

[0014] The collected depth image is filtered using median filtering.

[0015] In a possible implementation of the first aspect, performing expansion processing on all original cavity regions includes:

[0016] Define a convolution kernel C, and the convolution kernel C has an anchor point;

[0017] Convolution processing is performed on the convolution kernel C and the image containing the original hole area, and the maximum value of pixels in the area covered by the convolution kernel C is calculated;

[0018] Assign the maximum pixel value to the pixel position corresponding to the current convolution kernel C anchor point.

[0019] In a possible implementation of the first aspect, calculating the mean of the sampled pixels in the foreground area and assigning the mean to a neighborhood position where the original hole area contacts the foreground area includes:

[0020] Dividing the foreground area into a plurality of equal sub-areas;

[0021] Calculating the sampled pixel value of each sub-region, and generating a pixel heat map of the foreground region based on the sampled pixel values ​​of all sub-regions;

[0022] Analyzing the pixel heat map to obtain an analysis result;

[0023] Based on the analysis results, determining the trend characteristics of the foreground area;

[0024] Determine the midpoint of the original cavity area using a preset method and perform expansion processing to obtain a central area;

[0025] Based on the strike feature, all strike normals passing through the central area are obtained and recorded as first strike normals;

[0026] Counting all sub-regions where one end of all first-direction normals touches the foreground region, which are recorded as first sub-regions, and counting all sub-regions where the other end of all first-direction normals touches the foreground region, which are recorded as second sub-regions;

[0027] Calculate the mean of all sampled pixels in the first sub-area, recorded as the first pixel mean, and the mean of all sampled pixels in the second sub-area, recorded as the second pixel mean;

[0028] Calculate the difference between the first pixel mean and the second pixel mean, and determine whether the difference exceeds a set value. If not, calculate the mean of the sampled pixels in the foreground area, and assign the neighborhood position where the original hole area contacts the foreground area. If so, calculate the mean of the sampled pixels in the foreground area by region, and assign the neighborhood position where the original hole area contacts the foreground area by region.

[0029] In a possible implementation of the first aspect, determining the trend feature of the foreground area based on the analysis result includes:

[0030] determining an initial trend feature based on a color distribution of the pixel heat map;

[0031] Updating the initial trend feature by counting the gradient directions of all pixels in the pixel heat map;

[0032] The updated initial direction feature is modified in combination with the semantic feature contained in the foreground area to obtain the direction feature of the foreground area.

[0033] In a possible implementation of the first aspect, the modifying of the updated initial direction feature in combination with the semantic feature contained in the foreground area includes:

[0034] Determine whether the foreground area contains a defective area. If so, obtain the semantic features corresponding to the defective area, and correct the updated initial direction features based on the semantic features of the defective area; if not, obtain the semantic features of the components in the foreground area, and correct the updated initial direction features based on the semantic features of the components.

[0035] In a possible implementation of the first aspect, the preset method includes:

[0036] Extract all pixel coordinates in the original hole area using an image tool;

[0037] Calculate the weighted average of all pixel coordinates to get the midpoint coordinates;

[0038] Based on the midpoint coordinates, the midpoint of the original hole area is determined.

[0039] In a possible implementation of the first aspect, calculating the mean of the sampled pixels in the foreground area by region, and assigning the neighboring position where the original hole area contacts the foreground area by region includes:

[0040] Obtain a perpendicular line perpendicular to the normal line passing through the midpoint, recorded as a first perpendicular line;

[0041] Based on the difference and the trend feature, performing translation adjustment on the first vertical line along the normal direction of the first trend;

[0042] The adjusted vertical line divides the foreground area into area A and area B;

[0043] The mean value of the sampled pixels in region A is calculated and assigned to a neighborhood position where the original hole region contacts region A, and the mean value of the sampled pixels in region B is calculated and assigned to a neighborhood position where the original hole region contacts region B.

[0044] A second aspect of the present application provides a sleeper image hole filling system, comprising:

[0045] An image acquisition unit is used to acquire an image of the sleeper surface through an image acquisition device to obtain a depth image of the sleeper surface;

[0046] A filtering unit, configured to filter the acquired depth image using a preset method;

[0047] The segmentation and expansion unit is used to perform threshold segmentation on the depth image after filtering to obtain the original hole area, and to expand all the original hole areas to obtain the corresponding expanded hole areas;

[0048] a foreground area unit, configured to perform a difference operation on the expanded hole area and the original hole area to obtain a foreground area corresponding to each original hole area;

[0049] A filling unit is used to calculate the mean value of the sampled pixels in the foreground area and assign it to the neighboring position where the original hole area contacts the foreground area, while updating the foreground area; calculate the mean value of the sampled pixels in the updated foreground area and assign it to the neighboring position where the remaining area in the original hole area contacts the updated foreground area, until the original hole area is filled.

[0050] Its beneficial effects are as follows: the present invention discloses a method and system for filling holes in sleeper images, which obtains hole areas by collecting a depth image of the sleeper surface and filtering it, then performing threshold segmentation on the filtered depth image; dilating the hole area, performing a difference operation between the expanded hole area and the initial hole area to obtain a foreground area; and calculating the mean of the sampled pixels in the foreground area, filling the hole area step by step from the outside to the inside, thereby completing the filling of the hole area. The core step of the present invention is to utilize the characteristics of the connected area of ​​the foreground of the image, because the hole area after expansion can fully guarantee full fitting with the target area, thereby ensuring the subsequent sampling of information for hole filling, and can adapt to holes of different types and positions, ultimately ensuring the accuracy of sleeper area detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0052] Figure 1 This is a flow chart of a method for filling holes in sleeper images provided in an embodiment of the present application;

[0053] Figure 2 This is a schematic diagram of the composition of a sleeper image void filling system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0055] In this application, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0056] Example 1

[0057] In the existing technology, due to the occlusion of samples and the limitations of the collection environment, the point cloud data obtained often form holes due to missing data. The sleeper image with holes will not only cause the failure of positioning the sleeper detection area, but also greatly affect the accuracy of the sleeper area detection, resulting in false detection.

[0058] Therefore, this application provides a method for filling holes in sleeper images, such as Figure 1 Shown, including:

[0059] Capturing the sleeper surface using an image acquisition device to obtain a depth image of the sleeper surface;

[0060] Perform filtering on the acquired depth image using a preset method;

[0061] Perform threshold segmentation on the filtered depth image to obtain the original hole area, and perform expansion processing on all the original hole areas to obtain the corresponding expanded hole areas;

[0062] Performing a difference operation on the expanded hole area and the original hole area to obtain a foreground area corresponding to each original hole area;

[0063] Calculate the mean of the sampled pixels in the foreground area and assign it to the neighboring position where the original hole area contacts the foreground area, and update the foreground area at the same time; calculate the mean of the sampled pixels in the updated foreground area and assign it to the neighboring position where the remaining area in the original hole area contacts the updated foreground area, until the original hole area is filled.

[0064] Among them, this embodiment adopts machine vision technology, and performs 2D / 3D imaging of the sleeper surface through a linear array camera and a 2D / 3D camera to obtain 2D / 3D imaging data of the sleeper surface. The collected image type is a depth image, and the image grayscale value represents the height information of the photographed sleeper. The advantage is that it can make full use of the prior information of each component in the image, remove image noise, and achieve the purpose of subsequent processing.

[0065] Among them, this embodiment uses median filtering to filter the collected depth image. The basic principle of median filtering is to replace the value of a point in a digital image or digital sequence with the median value of each point in a neighborhood of the point, so that the surrounding pixel values ​​are close to the true value, thereby eliminating isolated noise points.

[0066] The filtered depth image is threshold-segmented to obtain multiple hole regions, which are then dilated. This step leverages the characteristics of the image's foreground connected regions to ensure that the dilated hole regions fully fit the target region, thus ensuring subsequent sampling of hole-filling information and adapting to holes of different types and locations.

[0067] The principle behind dilating the hole region is to find the local maximum. The specific process is as follows: a convolution kernel of any shape and size is defined, with a single defined anchor point, typically a square or disk with a reference point. This kernel is called a template or mask. The kernel C is convolved with the image corresponding to the hole region, and the maximum value of the pixels in the area covered by kernel C is calculated. This maximum pixel value is assigned to the pixel corresponding to the anchor point of kernel C. After dilating the hole region, the highlight area gradually increases, expanding the hole region record and ensuring sampling accuracy.

[0068] Among them, the hole areas at various positions are traversed. Since the hole areas are expanded at this time, the difference operation is performed between the hole areas after expansion and the original hole areas. What is retained is the foreground area in the neighborhood of the hole. The mean of the sampled pixels in the foreground area is calculated and assigned to the neighborhood hole position. The above process is repeated until the hole area is completely eliminated and the hole filling purpose is achieved.

[0069] Furthermore, when using the mean of the sampled pixels in the foreground area to fill the neighborhood hole area, if the hole area occurs at the same component position, that is, the pixel values ​​at both ends of the hole area are similar, then using the mean of the sampled pixels in the foreground area to fill the neighborhood hole area can achieve a better filling effect on the hole area; but if the hole area occurs at the connection between different components or at the connection between the component and the sleeper surface, that is, the pixel values ​​at both ends of the hole area are quite different, if the mean of the sampled pixels in all foreground areas is still used to fill the neighborhood hole area, the filling effect is poor and smooth filling of the hole area cannot be achieved. To this end, this embodiment adopts the following method for processing:

[0070] Divide the foreground area into a number of equal sub-areas. The division method can be to calculate the total pixel area of ​​the foreground area as s, and use s / n to calculate the pixel area of ​​each sub-area to complete the division of the multiple sub-areas.

[0071] Calculate the sampled pixel values ​​of each sub-region to generate a pixel heat map of the corresponding foreground region. Analyze the pixel heat map to determine the trend characteristics of the foreground region. It should be noted that the trend characteristics of the foreground region are the spatial extension direction of the high / low pixel value regions, the gradient change direction, or the distribution direction of key features. The trend characteristics can be used to obtain the specific trend normal and offset angle.

[0072] First, the color distribution of the pixel heat map (high / low value area shape, color gradient) is used to preliminarily determine the direction, i.e., the initial direction feature. Then, the dominant arrangement direction of the pixel values ​​is counted using the directional histogram. The processing logic is to divide the image into multiple local areas, calculate the direction of the pixel value gradient in each area, and then summarize it into a global directional histogram. The peak value is the overall direction, and the initial direction feature is updated using this peak value. Finally, the semantic features contained in the foreground area, specifically the component semantic features or defect semantic features, are combined with the analysis of the direction of the component or defect to correct the updated initial direction feature and obtain the final foreground area direction feature.

[0073] A preset method is used to determine the midpoint of the original hole area and perform expansion processing to obtain the central area. The preset method specifically uses an image tool (such as Photoshop) to extract the coordinates of all pixels in the area. , using the formula Calculate the midpoint coordinates ( ), is the weight value of the pixel, and the midpoint of the original hole area is determined by the midpoint coordinate; and the midpoint is expanded into the central area, and the midpoint can be enlarged into a central circle area with a preset radius. The expansion method is not specifically limited in this embodiment;

[0074] Based on the trend characteristics of the foreground area, multiple trend normals of the foreground area are obtained, that is, they represent the direction of the foreground area. The central area actually also belongs to the central area of ​​the foreground area. All trend normals passing through the central area are obtained and recorded as first trend normals. The sub-areas where both ends of the first trend normal contact the foreground area are the areas at the two ends of the foreground area. In this embodiment, they are respectively recorded as the first sub-area and the second sub-area. The mean value of the sampled pixels in the first sub-area is recorded as the first pixel mean, and the mean value of the sampled pixels in the second sub-area is recorded as the second pixel mean.

[0075] If the difference between the first pixel mean and the second pixel mean does not exceed the set value, it indicates that the hole area occurs on the same component or the complete sleeper surface. At this time, the mean value of the sampled pixels in the foreground area is calculated, and the neighboring position where the original hole area and the foreground area touch is assigned to complete the filling of the hole area.

[0076] If the difference between the first pixel mean and the second pixel mean exceeds the set value, it indicates that the hole area occurs at the connection between different components or at the connection between the component and the sleeper surface. In this embodiment, a perpendicular line corresponding to the normal line passing through the midpoint is obtained, which is recorded as the first perpendicular line. According to the difference and the direction characteristics, the first perpendicular line is translated and adjusted, that is, the first perpendicular line is adaptively adjusted toward one end of the foreground area so that the subsequent filling of the hole area can be smoother; the foreground area is divided into area A and area B using the adjusted first perpendicular line, and then the mean of the sampled pixels in area A is calculated to assign the neighborhood position where the original hole area contacts the area A, and the mean of the sampled pixels in area B is calculated to assign the neighborhood position where the original hole area contacts the area B, until all the hole areas are filled.

[0077] Optionally, this embodiment also provides a method different from using the mean value of sampled pixels in the foreground area for filling. Instead, any hole area is selected and searched in the upward, downward, left and right directions until the respective non-hole areas are found and their corresponding pixel values ​​are recorded to obtain value1, value2, value3, and value4. The minimum value is taken to fill the hole neighborhood position, and the process is repeated until the hole area is filled.

[0078] In some embodiments, capturing an image of the sleeper surface by an image capturing device includes:

[0079] The 2D image of the sleeper surface is collected by a linear array camera and a 2D camera, or the 3D image of the sleeper surface is collected by the linear array camera and a 3D camera.

[0080] In some embodiments, filtering the acquired depth image in a preset manner includes:

[0081] The collected depth image is filtered using median filtering.

[0082] In some embodiments, dilating all original cavity regions includes:

[0083] Define a convolution kernel C, and the convolution kernel C has an anchor point;

[0084] Convolution processing is performed on the convolution kernel C and the image containing the original hole area, and the maximum value of pixels in the area covered by the convolution kernel C is calculated;

[0085] Assign the maximum pixel value to the pixel position corresponding to the current convolution kernel C anchor point.

[0086] In some embodiments, calculating the mean of the sampled pixels in the foreground area and assigning the mean value to the neighborhood position where the original hole area contacts the foreground area comprises:

[0087] Dividing the foreground area into a plurality of equal sub-areas;

[0088] Calculating the sampled pixel value of each sub-region, and generating a pixel heat map of the foreground region based on the sampled pixel values ​​of all sub-regions;

[0089] Analyzing the pixel heat map to obtain an analysis result;

[0090] Based on the analysis results, determining the trend characteristics of the foreground area;

[0091] Determine the midpoint of the original cavity area using a preset method and perform expansion processing to obtain a central area;

[0092] Based on the strike feature, all strike normals passing through the central area are obtained and recorded as first strike normals;

[0093] Counting all sub-regions where one end of all first-direction normals touches the foreground region, which are recorded as first sub-regions, and counting all sub-regions where the other end of all first-direction normals touches the foreground region, which are recorded as second sub-regions;

[0094] Calculate the mean of all sampled pixels in the first sub-area, recorded as the first pixel mean, and the mean of all sampled pixels in the second sub-area, recorded as the second pixel mean;

[0095] The difference between the first pixel mean value and the second pixel mean value is calculated, and it is determined whether the difference exceeds a set value. If not, the mean value of the sampling pixels in the foreground region is calculated, and the neighborhood position of the original hollow region contacting the foreground region is assigned. If yes, the mean value of the sampling pixels in the foreground region is calculated regionally, and the neighborhood position of the original hollow region contacting the foreground region is assigned regionally.

[0096] In some embodiments, based on the analysis result, determining the orientation feature of the foreground region comprises:

[0097] Based on the color distribution of the pixel hotspot map, an initial orientation feature is determined.

[0098] The initial orientation feature is updated by counting the gradient direction of all pixels in the pixel hotspot map.

[0099] The updated initial orientation feature is corrected in combination with the semantic feature contained in the foreground region to obtain the orientation feature of the foreground region.

[0100] In some embodiments, correcting the updated initial orientation feature in combination with the semantic feature contained in the foreground region comprises:

[0101] It is determined whether the foreground region contains a defect region. If yes, the semantic feature corresponding to the defect region is obtained, and the updated initial orientation feature is corrected based on the semantic feature of the defect region. If not, the semantic feature of a component in the foreground region is obtained, and the updated initial orientation feature is corrected based on the semantic feature of the component.

[0102] In some embodiments, the preset method comprises:

[0103] All pixel coordinates in the original hollow region are extracted using an image tool.

[0104] The weighted average value of all pixel coordinates is calculated to obtain a midpoint coordinate.

[0105] Based on the midpoint coordinate, the midpoint of the original hollow region is determined.

[0106] In some embodiments, the mean value of the sampling pixels in the foreground region is calculated regionally, and the neighborhood position of the original hollow region contacting the foreground region is assigned regionally.

[0107] A perpendicular line to the orientation normal line passing through the midpoint is obtained, denoted as a first perpendicular line.

[0108] Based on the difference and the orientation feature, the first perpendicular line is adjusted in translation along the direction of the first orientation normal line.

[0109] The adjusted vertical line divides the foreground area into area A and area B;

[0110] The mean value of the sampled pixels in region A is calculated and assigned to a neighborhood position where the original hole region contacts region A, and the mean value of the sampled pixels in region B is calculated and assigned to a neighborhood position where the original hole region contacts region B.

[0111] Example 2

[0112] Based on the sleeper image hole filling method provided in the first embodiment of the present application, the second embodiment of the present application also provides a sleeper image hole filling system, such as Figure 2 As shown, the system includes:

[0113] An image acquisition unit is used to acquire an image of the sleeper surface through an image acquisition device to obtain a depth image of the sleeper surface;

[0114] A filtering unit, configured to filter the acquired depth image using a preset method;

[0115] The segmentation and expansion unit is used to perform threshold segmentation on the depth image after filtering to obtain the original hole area, and to expand all the original hole areas to obtain the corresponding expanded hole areas;

[0116] a foreground area unit, configured to perform a difference operation on the expanded hole area and the original hole area to obtain a foreground area corresponding to each original hole area;

[0117] A filling unit is used to calculate the mean value of the sampled pixels in the foreground area and assign it to the neighboring position where the original hole area contacts the foreground area, while updating the foreground area; calculate the mean value of the sampled pixels in the updated foreground area and assign it to the neighboring position where the remaining area in the original hole area contacts the updated foreground area, until the original hole area is filled.

[0118] The specific principles and execution processes of each unit in the sleeper image hole filling system disclosed in the above-mentioned embodiment 2 of the present application are the same as those of the sleeper image hole filling method disclosed in the above-mentioned embodiment 1 of the present application. Please refer to the corresponding parts of the sleeper image hole filling method disclosed in the above-mentioned embodiment 1 of the present application, and no further details will be given here.

[0119] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computing software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0121] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for filling holes in sleeper images, characterized in that: include: Capturing the sleeper surface using an image acquisition device to obtain a depth image of the sleeper surface; Perform filtering on the acquired depth image using a preset method; Perform threshold segmentation on the filtered depth image to obtain the original hole area, and perform expansion processing on all the original hole areas to obtain the corresponding expanded hole areas; Performing a difference operation on the expanded hole area and the original hole area to obtain a foreground area corresponding to each original hole area; Calculating the mean of the sampled pixels in the foreground area and assigning it to the neighboring position where the original hole area contacts the foreground area, while updating the foreground area; Calculating the mean of the sampled pixels in the updated foreground area and assigning the mean value to the neighboring position where the remaining area in the original hole area contacts the updated foreground area, until the original hole area is completely filled; Calculating the mean of the sampled pixels in the foreground area and assigning the mean value to the neighboring position where the original hole area contacts the foreground area includes: Dividing the foreground area into a plurality of equal sub-areas; Calculating the sampled pixel value of each sub-region, and generating a pixel heat map of the foreground region based on the sampled pixel values ​​of all sub-regions; Analyzing the pixel heat map to obtain an analysis result; Based on the analysis results, determining the trend characteristics of the foreground area; Determine the midpoint of the original cavity area using a preset method and perform expansion processing to obtain a central area; Based on the strike feature, all strike normals passing through the central area are obtained and recorded as first strike normals; Counting all sub-regions where one end of all first-direction normals touches the foreground region, which are recorded as first sub-regions, and counting all sub-regions where the other end of all first-direction normals touches the foreground region, which are recorded as second sub-regions; Calculate the mean of all sampled pixels in the first sub-area, recorded as the first pixel mean, and the mean of all sampled pixels in the second sub-area, recorded as the second pixel mean; Calculating a difference between the first pixel mean and the second pixel mean, and determining whether the difference exceeds a set value; if not, calculating the mean of the sampled pixels in the foreground area and assigning a neighboring position where the original hole area and the foreground area are in contact; if so, calculating the mean of the sampled pixels in the foreground area by region, and assigning a neighboring position where the original hole area and the foreground area are in contact by region; Determining the trend characteristics of the foreground area based on the analysis results includes: determining an initial trend feature based on a color distribution of the pixel heat map; Updating the initial trend feature by counting the gradient directions of all pixels in the pixel heat map; Combining the semantic features contained in the foreground area, the updated initial direction feature is modified to obtain the direction feature of the foreground area; Corrections to the updated initial trend characteristics include: Determine whether the foreground area contains a defective area; if so, obtain semantic features corresponding to the defective area, and modify the updated initial trend features based on the semantic features of the defective area; if not, obtain semantic features of components in the foreground area, and modify the updated initial trend features based on the semantic features of the components; The neighboring positions of the original hole region and the foreground region contacting each other are assigned by regions: Obtain a perpendicular line perpendicular to the normal line passing through the midpoint, recorded as a first perpendicular line; Based on the difference and the trend feature, performing translation adjustment on the first vertical line along the normal direction of the first trend; The adjusted vertical line divides the foreground area into area A and area B; The mean value of the sampled pixels in region A is calculated and assigned to a neighborhood position where the original hole region contacts region A, and the mean value of the sampled pixels in region B is calculated and assigned to a neighborhood position where the original hole region contacts region B.

2. A method for filling holes in sleeper images according to claim 1, characterized in that: The image acquisition of the sleeper surface by the image acquisition device includes: The 2D image of the sleeper surface is collected by a linear array camera and a 2D camera, or the 3D image of the sleeper surface is collected by the linear array camera and a 3D camera.

3. The method for filling holes in sleeper images according to claim 1, characterized in that: The preset method for filtering the acquired depth image includes: The collected depth image is filtered using median filtering.

4. The method for filling holes in sleeper images according to claim 1, characterized in that: Dilation of all original void areas includes: Define a convolution kernel C, and the convolution kernel C has an anchor point; Convolution processing is performed on the convolution kernel C and the image containing the original hole area, and the maximum value of pixels in the area covered by the convolution kernel C is calculated; Assign the maximum pixel value to the pixel position corresponding to the current convolution kernel C anchor point.

5. The method for filling holes in sleeper images according to claim 1, characterized in that: The preset method includes: Extract all pixel coordinates in the original hole area using an image tool; Calculate the weighted average of all pixel coordinates to get the midpoint coordinates; Based on the midpoint coordinates, the midpoint of the original hole area is determined.

6. A system for filling a sleeper image hole, implemented by the sleeper image hole filling method according to claim 1, characterized in that: include: An image acquisition unit is used to acquire an image of the sleeper surface through an image acquisition device to obtain a depth image of the sleeper surface; A filtering unit, configured to filter the acquired depth image using a preset method; The segmentation and expansion unit is used to perform threshold segmentation on the depth image after filtering to obtain the original hole area, and to expand all the original hole areas to obtain the corresponding expanded hole areas; a foreground area unit, configured to perform a difference operation on the expanded hole area and the original hole area to obtain a foreground area corresponding to each original hole area; a filling unit, configured to calculate the mean of the sampled pixels in the foreground area, assign the mean to the neighborhood position where the original hole area contacts the foreground area, and update the foreground area; The mean value of the sampled pixels in the updated foreground area is calculated and assigned to the neighboring position where the remaining area in the original hole area contacts the updated foreground area, until the original hole area is completely filled.

Images