Rail bar segmentation method, device, electronic equipment and storage medium
The rail barricade segmentation method using aerial depth images addresses resource-intensive visible light issues by predicting and segmenting rail fort areas with adjusted thresholds and pixel distributions, enhancing accuracy and efficiency.
Patent Information
- Application Number
- CN202410293698.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-03-14
AI Technical Summary
Existing methods for rail barricade segmentation using visible light images require significant computing resources and large training datasets, making them impractical for determining key point information of rail fortresses effectively.
A rail barricade segmentation method utilizing aerial depth images, which involves predicting the area ratio of target rail fort regions, adjusting segmentation thresholds, and identifying base and pile areas based on non-zero pixel distributions within minimum circumscribed rectangles, utilizing a mapping relationship and mixed Gaussian models to enhance accuracy and reduce computational demands.
The method effectively removes interference and color camouflage, improves accuracy, and reduces computing resources, enabling rapid identification of rail fort components under limited conditions.
Smart Images

Figure CN118053085B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a rail barrier segmentation method, device, electronic equipment and storage medium. Background Art
[0002] Rail forts are a commonly used military facility, usually set up on the beach area convenient for landing along the coast, and used in conjunction with other obstacles such as mines to form an anti-landing obstacle system to block medium or light landing tools. Figure 1 As shown, the rail barrage generally includes two parts: a base 1 and a pile 2. The base 1 is a trapezoidal reinforced concrete structure, and the pile 2 is generally a rail, which is inserted into the base 1 at a 45-degree angle and faces the landing party. Therefore, for the landing party, how to effectively blast the rail barrage is one of the key issues related to whether the landing can be successful.
[0003] At present, one type of existing technology is to perform semantic analysis on visible light images based on deep neural networks to determine the key point information of rail fortresses. This method requires more computing resources and a large amount of training data, which is not conducive to the subsequent application of the key point information of rail fortresses. Summary of the invention
[0004] The present invention provides a rail bar segmentation method, device, electronic equipment and storage medium, which are used to solve the defects in the prior art.
[0005] The present invention provides a rail barricade segmentation method, comprising:
[0006] Obtain the aerial depth image to be processed;
[0007] Based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail fortress area in the aerial depth image, predict the area ratio of the target rail fortress area in the aerial depth image to be processed;
[0008] Based on the current segmentation threshold, the aerial depth image to be processed is binarized and segmented, the current foreground area of the aerial depth image to be processed is determined, and the difference information between the proportion of the area mean of the current foreground area and the proportion of the area of the target railroad fortress area is calculated; if the difference information is outside the preset range, the current segmentation threshold is adjusted, and the aerial depth image to be processed is continuously binarized and segmented until the difference information is within the preset range, or the current segmentation threshold fluctuates within the specified range, and the foreground area with the smallest difference information is determined to be the target railroad fortress area in the aerial depth image to be processed;
[0009] The minimum circumscribed rectangle of the target rail fort region is determined, and the base region and the pile column region in the target rail fort region are identified based on the distribution information of non-zero pixels in the target region enclosed by the minimum circumscribed rectangle in the target rail fort region.
[0010] According to a rail fort segmentation method provided by the present invention, the method of identifying the base area and the pile column area in the target rail fort area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail fort area includes:
[0011] Based on the distribution information, determine the proportion of non-zero pixels in a specified area corresponding to each short side of the minimum circumscribed rectangle;
[0012] The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated area corresponding to each short side.
[0013] According to a rail fort segmentation method provided by the present invention, the base area and the pile area are identified based on the size relationship of the non-zero pixel points in the designated area corresponding to each short side, including:
[0014] The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated areas corresponding to the short sides and the size relationship of the pixel means corresponding to the designated areas in the aerial depth image to be processed.
[0015] According to a rail fort segmentation method provided by the present invention, the base area and pile column area in the target rail fort area are identified based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail fort area, and further includes:
[0016] Based on the distribution information, non-zero pixel points in the target area are sampled, and a mixed Gaussian model is constructed based on the sampling results;
[0017] Based on the parameter characteristics of the mixed Gaussian model, the base area and the pile area are identified.
[0018] According to a rail segmentation method provided by the present invention, the adjusting the current segmentation threshold comprises:
[0019] If the proportion of the average area of the current foreground area is greater than the proportion of the area of the target rail fortress area, the current segmentation threshold is reduced by a specified step size;
[0020] If the proportion of the average area of the current foreground area is smaller than the proportion of the area of the target track barrier area, the current segmentation threshold is increased by the specified step size.
[0021] According to a rail fort segmentation method provided by the present invention, the mapping relationship between the image information of the aerial depth image and the area ratio of the rail fort region in the aerial depth image is used to predict the area ratio of the target rail fort region in the aerial depth image to be processed, including:
[0022] Substituting the target image information of the aerial depth image to be processed into the mapping relationship to obtain the area ratio of the target rail fortress area;
[0023] The target image information includes the aerial photography height corresponding to the aerial photography depth image to be processed and / or the image pixel mean value of the aerial photography depth image to be processed.
[0024] According to a rail segmentation method provided by the present invention, the mapping relationship is constructed based on the following steps:
[0025] Collect aerial depth images containing a single rail barrage at different aerial altitudes;
[0026] Based on the automated segmentation algorithm, determine a single rail fortress area in the aerial depth image, and calculate the area ratio of the single rail fortress area;
[0027] The mapping relationship is constructed based on the image information of the aerial depth image and the area ratio of the single rail fortress area.
[0028] According to a railroad track segmentation method provided by the present invention, the method performs binary segmentation on the aerial depth image to be processed based on the current segmentation threshold, determines the current foreground area of the aerial depth image to be processed, and then includes:
[0029] Based on the regional features or classification model of the current foreground area, the current foreground area is filtered to remove the area in the current foreground area that does not contain the rail barrage.
[0030] The present invention also provides a rail barricade segmentation device, comprising:
[0031] An image acquisition module is used to acquire the aerial depth image to be processed;
[0032] A mapping module, for predicting the area ratio of the target rail bar area in the aerial depth image to be processed based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail bar area in the aerial depth image;
[0033] A segmentation module is used to perform binary segmentation on the aerial depth image to be processed based on the current segmentation threshold, determine the current foreground area of the aerial depth image to be processed, and calculate the difference information between the proportion of the area mean of the current foreground area and the proportion of the area of the target railroad fortress area; if the difference information is outside the preset range, adjust the current segmentation threshold, and continue to perform binary segmentation on the aerial depth image to be processed until the difference information is within the preset range, or the current segmentation threshold fluctuates within a specified range, and determine that the foreground area with the smallest difference information is the target railroad fortress area in the aerial depth image to be processed;
[0034] The identification module is used to determine the minimum circumscribed rectangle of the target rail fort area, and identify the base area and pile column area in the target rail fort area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail fort area.
[0035] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the rail bar segmentation method as described in any one of the above is implemented.
[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the rail bar segmentation method as described in any one of the above is implemented.
[0037] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the rail bar segmentation method as described in any one of the above is implemented.
[0038] The rail fort segmentation method, device, electronic device and storage medium provided by the present invention use aerial depth images, which can effectively remove the interference of irrelevant objects, avoid color camouflage, and have better anti-interference ability compared to visible light images. Moreover, the introduction of the mapping relationship can improve the accuracy of determining the area ratio of the target rail fort region. The method determines the target rail fort region in the aerial depth image to be processed by adjusting the current segmentation threshold, and uses the distribution information of non-zero pixels in the target area surrounded by the minimum circumscribed rectangle of the target rail fort region. It has the advantages of fast calculation speed and less required computing resources. It can quickly determine the base and piles of the rail fort under the conditions of limited computing resources and a small number of aerial depth images, so as to facilitate timely subsequent processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without paying any creative work.
[0040] Figure 1 It is a structural schematic diagram of a rail barrage in the prior art;
[0041] Figure 2 It is a schematic diagram of the flow of the rail barrage segmentation method provided by the present invention;
[0042] Figure 3 It is a schematic diagram of a target rail barricade area in the rail barricade segmentation method provided by the present invention;
[0043] Figure 4 It is a schematic diagram of the minimum circumscribed rectangle of the target rail barricade area in the rail barricade segmentation method provided by the present invention;
[0044] Figure 5 It is a schematic diagram of a process of determining a target rail fort region in an aerial depth image to be processed in the rail fort segmentation method provided by the present invention;
[0045] Figure 6 It is a structural schematic diagram of the rail barricade segmentation device provided by the present invention;
[0046] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] The term "first" or "second" in the specification and claims of the present invention may include one or more of the features explicitly or implicitly. In the description of the invention, unless otherwise specified, "plurality" means two or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the objects connected before and after are in an "or" relationship.
[0049] Figure 2 FIG. 1 is a flow chart of a rail barricade segmentation method provided in an embodiment of the present invention, as shown in FIG. Figure 2As shown, the method includes:
[0050] S21, obtaining an aerial depth image to be processed;
[0051] S22, predicting the area ratio of the target rail fortress area in the aerial depth image to be processed based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail fortress area in the aerial depth image;
[0052] S23, based on the current segmentation threshold, perform binary segmentation on the aerial depth image to be processed, determine the current foreground area of the aerial depth image to be processed, and calculate the difference information between the proportion of the area mean of the current foreground area and the proportion of the area of the target railroad fortress area; if the difference information is outside the preset range, adjust the current segmentation threshold, and continue to perform binary segmentation on the aerial depth image to be processed until the difference information is within the preset range, or the current segmentation threshold fluctuates within the specified range, and determine that the foreground area with the smallest difference information is the target railroad fortress area in the aerial depth image to be processed;
[0053] S24, determining the minimum circumscribed rectangle of the target rail fort region, and identifying the base region and the pile column region in the target rail fort region based on the distribution information of non-zero pixels in the target region enclosed by the minimum circumscribed rectangle in the target rail fort region.
[0054] Specifically, the rail barrier segmentation method provided in the embodiment of the present invention is executed by a rail barrier segmentation device, which can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., which is not specifically limited here.
[0055] First, step S21 is performed to obtain an aerial depth image to be processed. The aerial depth image to be processed may be a depth image acquired by a depth image acquisition device, which may be carried on a drone, an airplane or an aircraft, and may be a depth camera, a laser radar or other sensors.
[0056] The depth image acquisition device can be connected to the rail barrier segmentation device to send the acquired aerial depth image to be processed to the rail barrier segmentation device.
[0057] The aerial depth image to be processed may be a single-channel image, in which the pixel value of each pixel represents the distance value between the photographed position and the depth image acquisition device.
[0058] After the aerial depth image to be processed is acquired, image preprocessing may be performed on the aerial depth image to be processed. The image preprocessing method adopted may include a scaling operation and a denoising operation, etc., to improve the image quality of the aerial depth image to be processed.
[0059] The denoising operation may include Gaussian filtering denoising or truncation denoising. It is understood that the truncation denoising is to re-assign the pixel values outside the effective measurement range of the depth image acquisition device in the aerial depth image to be processed, for example, the pixel value may be assigned to the pixel mean value of the pixel points within a fixed range around the corresponding pixel point.
[0060] After acquiring the aerial depth image to be processed, the target image information of the aerial depth image to be processed can be determined. The target image information may include the aerial height and / or image pixel mean corresponding to the aerial depth image to be processed. The image pixel mean refers to the average value of the pixel values of each pixel point in the aerial depth image to be processed.
[0061] Then, step S22 is executed to determine the area ratio of the target rail fortress area in the aerial depth image to be processed by using the mapping relationship between the image information of the aerial depth image and the area ratio of the rail fortress area in the aerial depth image.
[0062] The mapping relationship is pre-constructed and stored, and is used to characterize the correspondence between the image information of the aerial depth image and the area ratio of the rail fort in the aerial depth image. The aerial depth image can be acquired by a depth image acquisition device at different aerial heights toward the rail fort. The image information of the aerial depth image may include the aerial height corresponding to the aerial depth image and / or the image pixel mean, and the image pixel mean refers to the average value of the pixel values of each pixel point in the aerial depth image.
[0063] The proportion of the rail fort area in the aerial depth image can be obtained by identifying the rail fort area in the aerial depth image and calculating the ratio of the identified rail fort area to the area of the aerial depth image.
[0064] By substituting the target image information of the aerial depth image to be processed into the mapping relationship constructed above, the area ratio of the rail fortress area corresponding to the target image information in the mapping relationship can be obtained, and the area ratio of the rail fortress area is the area ratio of the target rail fortress area.
[0065] After that, step S23 is executed, and the traversal method or the binary method can be used to search for the optimal segmentation threshold. For the current segmentation threshold found, the current segmentation threshold is used to perform binary segmentation on the aerial depth image to be processed, and the current foreground area of the aerial depth image to be processed is determined. That is, the pixel values of all pixels in the aerial depth image to be processed whose pixel values are greater than or equal to the current segmentation threshold are assigned a value of 1 to form the current foreground area, and the pixel values of all pixels in the aerial depth image to be processed whose pixel values are less than the current segmentation threshold are assigned a value of 0 to form the current background area. At this time, the current foreground area is white and the current background area is black.
[0066] You can also assign the pixel values of all pixels in the aerial depth image to be processed whose pixel values are greater than or equal to the current segmentation threshold to 0 to form the current foreground area, and assign the pixel values of all pixels in the aerial depth image to be processed whose pixel values are less than the current segmentation threshold to 1 to form the current background area. At this time, the current foreground area is black and the current background area is white.
[0067] After that, the average area of the current foreground region is calculated. Since the binary segmentation may obtain one current foreground region or multiple disconnected current foreground regions, when the binary segmentation obtains one current foreground region, the area of the current foreground region is directly used as the average area of the current foreground region. When the binary segmentation obtains multiple disconnected current foreground regions, the average area of the multiple disconnected current foreground regions is calculated.
[0068] After that, the proportion of the average area of the current foreground area is calculated, that is, the proportion of the area of the current foreground area in the aerial depth image to be processed, and the difference information between the proportion of the average area of the current foreground area and the proportion of the area of the target rail fortress area is obtained. The difference information may include the size relationship between the proportion of the average area of the current foreground area and the proportion of the area of the target rail fortress area, as well as the absolute value of the difference between the two.
[0069] If the difference information is outside the preset range, that is, the absolute value of the difference is outside the preset range, then adjust the current segmentation threshold, continue to perform binary segmentation on the aerial depth image to be processed, and calculate the difference information between the current foreground area mean ratio and the target track fort area ratio, and judge whether the difference information is within the preset range, until the difference information is within the preset range, or the current segmentation threshold fluctuates within the specified range, then the segmentation threshold with the smallest difference information in the segmentation process can be output, that is, the optimal segmentation threshold, and the foreground area obtained by the segmentation result is the target track fort area in the aerial depth image to be processed. Among them, the target track fort area can include one or more. If Figure 3 As shown, the aerial depth image to be processed includes three target rail fortification areas.
[0070] The adjustment process of the current segmentation threshold is a dynamic process. The initial value of the current segmentation threshold can be set as needed. A random value can be selected as the initial value of the current segmentation threshold, the image pixel mean of the aerial depth image to be processed can be selected as the initial value of the current segmentation threshold, or the initial value of the current segmentation threshold can be obtained based on the shooting height of the aerial depth image to be processed from the ground. In this way, the process of determining the optimal segmentation threshold can be shortened and the efficiency of determining the target track fortress area can be improved.
[0071] It is understandable that both the preset range and the specified range can be set as needed, and the preset range can be a smaller range. In particular, the preset range can only include 0. In this case, the difference information within the preset range means that the proportion of the area mean of the current foreground area is equal to the proportion of the area of the target rail fortress area.
[0072] The specified range may include two specified values, and the current segmentation threshold fluctuates within the specified range, which means that the current segmentation threshold jumps between the two specified values.
[0073] Finally, step S24 is executed to determine the minimum circumscribed rectangle of the target rail fort region, and the base region and the pile column region in the target rail fort region are identified by using the distribution information of the non-zero pixels in the target region enclosed by the minimum circumscribed rectangle in the target rail fort region. The target region enclosed by the minimum circumscribed rectangle may include the target rail fort region and the background region. Since the aerial depth image to be processed has been binarized, the non-zero pixels in the target region are all the target rail fort region. Since the rail fort includes a base and a pile column, and the sizes of the base and the pile column are different, the area of the area in the image is different, specifically, the area of the base is larger than the area of the pile column, so the distribution information of the non-zero pixels in the target region can be used to determine the base region and the pile column region in the target rail fort region.
[0074] A rail fort segmentation method provided in an embodiment of the present invention uses an aerial depth image, which can effectively remove the interference of irrelevant objects, avoid color camouflage, and has better anti-interference ability compared to visible light images. Moreover, the introduction of the mapping relationship can improve the accuracy of determining the area ratio of the target rail fort region. The method determines the target rail fort region in the aerial depth image to be processed by adjusting the current segmentation threshold, and uses the distribution information of non-zero pixels in the target area surrounded by the minimum circumscribed rectangle of the target rail fort region. It has the advantages of fast calculation speed and less required computing resources. It can quickly determine the base and piles of the rail fort under limited computing resources and a small number of aerial depth images, so as to facilitate timely subsequent processing.
[0075] On the basis of the above embodiment, the method of identifying the base area and the pile column area in the target rail bar area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail bar area includes:
[0076] Based on the distribution information, determine the proportion of non-zero pixels in a specified area corresponding to each short side of the minimum circumscribed rectangle;
[0077] The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated area corresponding to each short side.
[0078] Specifically, when identifying the base area and the pile area in the target rail fortress area, the distribution information can be used to first determine the proportion of non-zero pixels in the specified area corresponding to each short side of the minimum bounding rectangle. The minimum bounding rectangle includes two long sides and two short sides, each short side can correspond to a specified area, and the specified areas corresponding to the short sides are equal in size.
[0079] The designated area can be set as needed, for example, the designated area can be set to a rectangle with equal side length and short side length, or the designated area can be set to a preset ratio of less than half the long side length of the minimum circumscribed rectangle and less than the short side length of the minimum circumscribed rectangle, and greater than the short side length. The preset ratio can be determined based on the ratio of the width of the base and the pile of the rail bar. Figure 4 As shown, the designated area corresponding to the left short side is A, the designated area corresponding to the right short side is C, and the area between the two designated areas is B.
[0080] The proportion of non-zero pixels in A refers to the ratio of the area occupied by non-zero pixels in A to the area of A. Figure 4 The non-zero pixel ratio in C refers to the ratio of the area occupied by the non-zero pixels in C to the area of C.
[0081] Thereafter, the base area and the pile area can be identified according to the size relationship of the non-zero pixel ratio in the designated area corresponding to each short side. For example, the designated area with a large non-zero pixel ratio can be determined as the base area, and the designated area with a small non-zero pixel ratio can be determined as the pile area. Here, the identified base area and pile area are used to locate the base and the pile, so the base area may not be the complete base area in the aerial depth image to be processed, and the pile area may not be the complete pile area in the aerial depth image to be processed.
[0082] In an embodiment of the present invention, the base area and the pile area are identified by the size relationship of the non-zero pixel ratio in the designated area corresponding to each short side, which can simplify the recognition process, reduce the amount of data, and ensure the accuracy of the recognition result.
[0083] On the basis of the above embodiment, the identifying of the base area and the pile area based on the size relationship of the non-zero pixel points in the designated area corresponding to each short side includes:
[0084] The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated areas corresponding to the short sides and the size relationship of the pixel means corresponding to the designated areas in the aerial depth image to be processed.
[0085] Specifically, when identifying the base area and pile area in the target rail fort area, the pixel mean corresponding to each specified area in the aerial depth image to be processed can also be introduced. The pixel mean is the average value of the pixel values of all pixel points in the area corresponding to each specified area in the aerial depth image to be processed.
[0086] by Figure 4 For example, the pixel mean corresponding to A and the pixel mean corresponding to C in the aerial depth image to be processed are introduced, and the size relationship between the two pixel means is determined.
[0087] The base area and the column area are determined by combining the size relationship of the non-zero pixel ratio in the specified area and the size relationship of the pixel mean corresponding to each specified area in the aerial depth image to be processed. For example, the specified area with a large non-zero pixel ratio and a large pixel mean can be determined as the base area, and the specified area with a small non-zero pixel ratio and a small pixel mean can be determined as the column area.
[0088] In an embodiment of the present invention, based on the size relationship of the proportion of non-zero pixels in the designated area corresponding to each short side, combined with the size relationship of the pixel means corresponding to each designated area in the aerial depth image to be processed, the base area and the pile column area are identified, which can further improve the accuracy of the recognition result.
[0089] On the basis of the above embodiment, the method of identifying the base area and the pile area in the target rail bar area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail bar area further includes:
[0090] Based on the distribution information, non-zero pixel points in the target area are sampled, and a mixed Gaussian model is constructed based on the sampling results;
[0091] Based on the parameter characteristics of the mixed Gaussian model, the base area and the pile area are identified.
[0092] Specifically, when identifying the base area and pile area in the target rail fortress area, the distribution information can be used to sample the non-zero pixels in the target area. The sampling process can be adopted randomly or according to a preset sampling rate, which is not specifically limited here.
[0093] After sampling, the sampling results can be used to construct a mixed Gaussian model, that is, the sampling results can be used to fit two Gaussian models, and the base area and the column area can be identified by analyzing the parameter characteristics of the two Gaussian models. For example, the parameter characteristics of the two Gaussian models can be an elliptical shape on a two-dimensional plane corresponding to the covariance matrix of the two Gaussian models. If the length of the major axis and the minor axis of the ellipse are approximately equal, then the non-zero pixel point area corresponding to the Gaussian model can be determined to be the base area. If the length of the major axis and the minor axis of the ellipse differ greatly, then the non-zero pixel point area corresponding to the Gaussian model can be determined to be the column area.
[0094] Based on the above embodiment, the adjusting the current segmentation threshold includes:
[0095] If the proportion of the average area of the current foreground area is greater than the proportion of the area of the target rail fortress area, the current segmentation threshold is reduced by a specified step size;
[0096] If the proportion of the average area of the current foreground area is smaller than the proportion of the area of the target track barrier area, the current segmentation threshold is increased by the specified step size.
[0097] Specifically, when adjusting the current segmentation threshold, that is, under the premise that the difference information between the proportion of the area mean of the current foreground area and the area proportion of the target rail fortress area is outside the preset range, if the proportion of the area mean of the current foreground area is greater than the area proportion of the target rail fortress area, the current segmentation threshold is reduced by a specified step size. The specified step size can be set as needed and is not specifically limited here. If the proportion of the area mean of the current foreground area is less than the area proportion of the target rail fortress area, the current segmentation threshold is increased by a specified step size.
[0098] In the embodiment of the present invention, by comparing the proportion of the average area of the current foreground area with the proportion of the area of the target rail fortress area, the current segmentation threshold is increased or decreased by a specified step size, which can simplify the adjustment process.
[0099] On the basis of the above embodiment, the mapping relationship between the image information of the aerial depth image and the area ratio of the rail fortress area in the aerial depth image is used to predict the area ratio of the target rail fortress area in the aerial depth image to be processed, including:
[0100] Substituting the target image information of the aerial depth image to be processed into the mapping relationship to obtain the area ratio of the target rail fortress area;
[0101] The target image information includes the aerial photography height corresponding to the aerial photography depth image to be processed and / or the image pixel mean value of the aerial photography depth image to be processed.
[0102] Specifically, when determining the area ratio of the target rail fort region in the aerial depth image to be processed, the target image information of the aerial depth image to be processed can be substituted into the mapping relationship to obtain the area ratio of the target rail fort region.
[0103] It is understandable that the target image information includes the aerial height corresponding to the aerial depth image to be processed and / or the image pixel mean of the aerial depth image to be processed. Correspondingly, the image information of the aerial depth image in the mapping relationship also includes the aerial height corresponding to the aerial depth image and / or the image pixel mean of the aerial depth image to be processed.
[0104] In the embodiment of the present invention, appropriate information can be selected to determine the target rail bar area ratio, which facilitates the determination of the target rail bar area ratio. Moreover, the introduction of the mapping relationship can improve the accuracy of determining the target rail bar area ratio.
[0105] Based on the above embodiment, the mapping relationship is constructed based on the following steps:
[0106] Collect aerial depth images containing a single rail barrage at different aerial altitudes;
[0107] Based on the automated segmentation algorithm, determine a single rail fortress area in the aerial depth image, and calculate the area ratio of the single rail fortress area;
[0108] The mapping relationship is constructed based on the image information of the aerial depth image and the area ratio of the single rail fortress area.
[0109] Specifically, when constructing the mapping relationship, a depth image acquisition device mounted on a drone, airplane or aircraft may be used to first acquire aerial depth images containing a single rail barrage at different aerial heights.
[0110] Afterwards, an automated segmentation algorithm is used to determine a single railroad barrage area in the aerial depth image. The automated segmentation algorithm can be selected as needed, for example, a semantic segmentation algorithm based on a neural network, or other segmentation algorithms, which are not specifically limited here.
[0111] Thereafter, the area ratio of a single rail bar area may be calculated, that is, the ratio of the area of a single rail bar area to the area of the aerial depth image.
[0112] Function fitting is performed on the image information of the aerial depth image and the area proportion of a single rail fortress to construct a mapping relationship.
[0113] In the embodiment of the present invention, a single railroad fortress area in an aerial depth image is obtained through an automated segmentation algorithm, which can improve the efficiency of constructing the mapping relationship and reduce the time cost of constructing the mapping relationship.
[0114] On the basis of the above embodiment, the process of performing binary segmentation on the aerial depth image to be processed based on the current segmentation threshold value, and determining the current foreground area of the aerial depth image to be processed, then includes:
[0115] Based on the regional features or classification model of the current foreground area, the current foreground area is filtered to remove the area in the current foreground area that does not contain the rail barrage.
[0116] Specifically, after determining the current foreground area of the aerial depth image to be processed, in order to avoid the influence of other objects in the shooting scene except the railroad barricade, the current foreground area needs to be filtered. The filtering process can be achieved through the regional features or classification model of the current foreground area. The regional features may include regional contours and / or morphological features. It can filter out smaller or larger foreground areas that are not railroad barricades based on the contour features and / or morphological features of the railroad barricades, or it can identify areas in the current foreground area that do not contain railroad barricades by constructing a classification model. The classification model can be obtained by training an initial neural network model with depth image samples carrying railroad barricade labels, which is not specifically limited here.
[0117] In the embodiment of the present invention, by filtering the current foreground area, the accuracy of the binary segmentation result can be guaranteed and the amount of data can be reduced.
[0118] like Figure 5 As shown, when determining the target track barrier area in the aerial depth image to be processed, the area ratio r of the target track barrier area in the aerial depth image to be processed can be determined first, and then the initial value of the current segmentation threshold t, the specified step size s, the initial value 0 of the first variable n and the initial value of the second variable T are determined, and the third variable S is set.
[0119] The initial value of the current segmentation threshold t can be obtained based on the image pixel mean of the aerial depth image to be processed, or based on the shooting height of the aerial depth image to be processed from the ground. The initial value of the second variable T can be a random value that is not equal to the initial value of the current segmentation threshold t. The first variable n represents the number of segmentations, and the initial value of the second variable T and the third variable S are both intermediate variables.
[0120] The aerial depth image to be processed is binarized with the initial value of the current segmentation threshold t to determine the current foreground area of the aerial depth image to be processed, and the current foreground area is filtered based on the regional features or classification model of the current foreground area to filter out the area in the current foreground area that does not contain the railroad barrage.
[0121] Calculate the proportion a of the average area of the current foreground area, and then determine whether a is equal to r, or whether t is equal to T.
[0122] If a is equal to r, or t is equal to T, then t is determined to be the optimal segmentation threshold. At this time, the current foreground area is the target track fortress area in the aerial depth image to be processed.
[0123] If a is not equal to r, and t is not equal to T, then when n%2=0, T is assigned the value t.
[0124] Thereafter, it is determined whether a is greater than r. If a is greater than r, S is assigned a value of -s; otherwise, that is, if a is less than r, S is assigned a value of s.
[0125] Thereafter, t is assigned a value of t+S, n is assigned a value of n+1, and t is used to continue to perform binary segmentation on the aerial depth image to be processed.
[0126] like Figure 6 As shown, based on the above embodiment, a rail bar segmentation device is provided in an embodiment of the present invention, including:
[0127] An image acquisition module 61 is used to acquire the aerial depth image to be processed;
[0128] A mapping module 62, configured to predict the area ratio of the target railroad barrage area in the aerial depth image to be processed based on the mapping relationship between the image information of the aerial depth image and the area ratio of the railroad barrage area in the aerial depth image;
[0129] The segmentation module 63 is used to perform binary segmentation on the aerial depth image to be processed based on the current segmentation threshold, determine the current foreground area of the aerial depth image to be processed, and calculate the difference information between the proportion of the area mean of the current foreground area and the proportion of the area of the target railroad fortress area; if the difference information is outside the preset range, adjust the current segmentation threshold, and continue to perform binary segmentation on the aerial depth image to be processed until the difference information is within the preset range, or the current segmentation threshold fluctuates within a specified range, and determine that the foreground area with the smallest difference information is the target railroad fortress area in the aerial depth image to be processed;
[0130] The identification module 64 is used to determine the minimum circumscribed rectangle of the target rail fort area, and identify the base area and pile column area in the target rail fort area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail fort area.
[0131] On the basis of the above-mentioned embodiment, in the rail bar segmentation device provided in the embodiment of the present invention, the identification module is specifically used for:
[0132] Based on the distribution information, determine the proportion of non-zero pixels in a specified area corresponding to each short side of the minimum circumscribed rectangle;
[0133] The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated area corresponding to each short side.
[0134] On the basis of the above-mentioned embodiment, in the rail bar segmentation device provided in the embodiment of the present invention, the identification module is specifically used for:
[0135] The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated areas corresponding to the short sides and the size relationship of the pixel means corresponding to the designated areas in the aerial depth image to be processed.
[0136] On the basis of the above-mentioned embodiment, in the rail bar segmentation device provided in the embodiment of the present invention, the identification module is specifically used for:
[0137] Based on the distribution information, non-zero pixel points in the target area are sampled, and a mixed Gaussian model is constructed based on the sampling results;
[0138] Based on the parameter characteristics of the mixed Gaussian model, the base area and the pile area are identified.
[0139] On the basis of the above-mentioned embodiment, in the rail bar segmentation device provided in the embodiment of the present invention, the segmentation module is specifically used for:
[0140] If the proportion of the average area of the current foreground area is greater than the proportion of the area of the target rail fortress area, the current segmentation threshold is reduced by a specified step size;
[0141] If the proportion of the average area of the current foreground area is smaller than the proportion of the area of the target track barrier area, the current segmentation threshold is increased by the specified step size.
[0142] On the basis of the above-mentioned embodiment, in the rail bar segmentation device provided in the embodiment of the present invention, the mapping module is specifically used for:
[0143] Substituting the target image information of the aerial depth image to be processed into the mapping relationship to obtain the area ratio of the target rail fortress area;
[0144] The target image information includes the aerial photography height corresponding to the aerial photography depth image to be processed and / or the image pixel mean value of the aerial photography depth image to be processed.
[0145] On the basis of the above-mentioned embodiment, the rail bar segmentation device provided in the embodiment of the present invention further includes a relationship building module for:
[0146] Collect aerial depth images containing a single rail barrage at different aerial altitudes;
[0147] Based on the automated segmentation algorithm, determine a single rail fortress area in the aerial depth image, and calculate the area ratio of the single rail fortress area;
[0148] The mapping relationship is constructed based on the image information of the aerial depth image and the area ratio of the single rail fortress area.
[0149] On the basis of the above-mentioned embodiment, the rail bar segmentation device provided in the embodiment of the present invention further includes a filtering module for:
[0150] Based on the regional features or classification model of the current foreground area, the current foreground area is filtered to remove the area in the current foreground area that does not contain the rail barrage.
[0151] Specifically, the functions of each module in the rail barrier segmentation device provided in the embodiment of the present invention correspond one-to-one to the operating procedures of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further details will be given in the embodiment of the present invention.
[0152] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor (Processor) 710, a communication interface (Communications Interface) 720, a memory (Memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the track segmentation method provided in the above embodiments.
[0153] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the rail segmentation method provided in the above embodiments.
[0155] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the rail segmentation method provided in the above embodiments.
[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A rail bar segmentation method, characterized in that: include: Obtain the aerial depth image to be processed; Based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail fortress area in the aerial depth image, predict the area ratio of the target rail fortress area in the aerial depth image to be processed; Based on the current segmentation threshold, the aerial depth image to be processed is binarized and segmented, the current foreground area of the aerial depth image to be processed is determined, and the difference information between the proportion of the area mean of the current foreground area and the proportion of the area of the target rail fortress area is calculated; If the difference information is outside the preset range, adjust the current segmentation threshold, and continue to perform binary segmentation on the aerial depth image to be processed until the difference information is within the preset range, or the current segmentation threshold fluctuates within the specified range, and determine that the foreground area with the smallest difference information in the segmentation process is the target track fortress area in the aerial depth image to be processed; Determine the minimum circumscribed rectangle of the target rail fort region, and identify the base region and the pile column region in the target rail fort region based on the distribution information of non-zero pixels in the target region enclosed by the minimum circumscribed rectangle in the target rail fort region; The method predicts the area ratio of the target rail bar area in the aerial depth image to be processed based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail bar area in the aerial depth image, including: Substituting the target image information of the aerial depth image to be processed into the mapping relationship to obtain the area ratio of the target rail fortress area; Wherein, the target image information includes the aerial photography height corresponding to the aerial photography depth image to be processed and / or the image pixel mean value of the aerial photography depth image to be processed; The mapping relationship is constructed based on the following steps: Collect aerial depth images containing a single rail barrage at different aerial altitudes; Based on the automated segmentation algorithm, determine a single rail fortress area in the aerial depth image, and calculate the area ratio of the single rail fortress area; The mapping relationship is constructed based on the image information of the aerial depth image and the area ratio of the single rail fortress area.
2. The rail bar segmentation method according to claim 1, characterized in that: The identifying the base area and the pile column area in the target rail bar area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail bar area includes: Based on the distribution information, determine the proportion of non-zero pixels in a specified area corresponding to each short side of the minimum circumscribed rectangle; The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated area corresponding to each short side.
3. The rail bar segmentation method according to claim 2, characterized in that: The identifying the base area and the pile area based on the size relationship of the non-zero pixel points in the designated area corresponding to each short side includes: The base area and the pile area are identified based on the size relationship of the proportion of non-zero pixels in the designated areas corresponding to the short sides and the size relationship of the pixel means corresponding to the designated areas in the aerial depth image to be processed.
4. The rail bar segmentation method according to claim 1, characterized in that: The method of identifying the base area and the pile area in the target rail bar area based on the distribution information of non-zero pixels in the target area enclosed by the minimum circumscribed rectangle in the target rail bar area also includes: Based on the distribution information, non-zero pixel points in the target area are sampled, and a mixed Gaussian model is constructed based on the sampling results; Based on the parameter characteristics of the mixed Gaussian model, the base area and the pile area are identified.
5. The rail bar segmentation method according to claim 1, characterized in that: The adjusting the current segmentation threshold comprises: If the proportion of the average area of the current foreground area is greater than the proportion of the area of the target rail fortress area, the current segmentation threshold is reduced by a specified step size; If the proportion of the average area of the current foreground area is smaller than the proportion of the area of the target track barrier area, the current segmentation threshold is increased by the specified step size.
6. The rail bar segmentation method according to any one of claims 1 to 5, characterized in that: The method further comprises: performing binary segmentation on the aerial depth image to be processed based on the current segmentation threshold value, and determining the current foreground area of the aerial depth image to be processed, and then comprising: Based on the regional features or classification model of the current foreground area, the current foreground area is filtered to remove the area in the current foreground area that does not contain the rail barrage.
7. A rail bar dividing device, characterized in that: include: An image acquisition module is used to acquire the aerial depth image to be processed; A mapping module, for predicting the area ratio of the target rail bar area in the aerial depth image to be processed based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail bar area in the aerial depth image; A segmentation module, for performing binary segmentation on the aerial depth image to be processed based on a current segmentation threshold, determining a current foreground area of the aerial depth image to be processed, and calculating difference information between a proportion of an area average of the current foreground area and a proportion of an area of the target rail fortress area; If the difference information is outside the preset range, adjust the current segmentation threshold, and continue to perform binary segmentation on the aerial depth image to be processed until the difference information is within the preset range, or the current segmentation threshold fluctuates within the specified range, and determine that the foreground area with the smallest difference information in the segmentation process is the target track fortress area in the aerial depth image to be processed; An identification module, used to determine the minimum circumscribed rectangle of the target rail fort region, and identify the base region and the pile column region in the target rail fort region based on the distribution information of non-zero pixels in the target region enclosed by the minimum circumscribed rectangle in the target rail fort region; The method predicts the area ratio of the target rail bar area in the aerial depth image to be processed based on the mapping relationship between the image information of the aerial depth image and the area ratio of the rail bar area in the aerial depth image, including: Substituting the target image information of the aerial depth image to be processed into the mapping relationship to obtain the area ratio of the target rail fortress area; Wherein, the target image information includes the aerial photography height corresponding to the aerial photography depth image to be processed and / or the image pixel mean value of the aerial photography depth image to be processed; The mapping relationship is constructed based on the following steps: Collect aerial depth images containing a single rail barrage at different aerial altitudes; Based on the automated segmentation algorithm, determine a single rail fortress area in the aerial depth image, and calculate the area ratio of the single rail fortress area; The mapping relationship is constructed based on the image information of the aerial depth image and the area ratio of the single rail fortress area.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the rail bar segmentation method as described in any one of claims 1-6 is implemented.
