Region segmentation methods, systems, electronic devices and storage media

Abstract

This application discloses a region segmentation method, system, electronic device, and storage medium, relating to the field of image processing technology. The method includes: acquiring an image to be processed from railway video surveillance equipment; if it is determined that the image to be processed includes a railway track region, annotating the railway track region in the image to obtain annotation information; inputting the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain a railway track region segmentation result; wherein the global feature extraction network is a network with multiple global feature extraction layers at different network depths. This method can accurately and comprehensively analyze railway track features, enabling the obtained railway track region segmentation result to accurately characterize railway track features, thereby accurately dividing the railway track region and improving the accuracy of foreign object detection within the railway track region.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a region segmentation method, system, electronic device, and storage medium. Background Technology

[0002] Currently, image segmentation has become a research hotspot in the fields of computer vision and machine learning. In the process of detecting foreign object intrusions or track surface debris in railways, it is necessary to first automatically segment the area where the railway tracks are located from video images. However, segmenting the railway tracks from a large number of video images is extremely time-consuming and labor-intensive. Related technical solutions propose using methods such as thresholding, active contour models, region growing, graph cut models, and machine learning to achieve automatic segmentation of railway track protection zones.

[0003] However, the aforementioned segmentation methods all require manually designing corresponding feature extraction methods based on information such as the size, shape, texture, and quantity of the rails. These extracted features are then used for training to obtain an intelligent segmentation model. Therefore, the quality of the feature extraction method design directly affects the final segmentation performance, resulting in a high technical barrier. Furthermore, manually designed feature extraction methods struggle to accurately and comprehensively describe the rail features and suffer from poor noise resistance and low segmentation accuracy, thus reducing the accuracy of feature extraction. Summary of the Invention

[0004] To address this issue, this application provides a region segmentation method, system, electronic device, and storage medium to solve the problem of how to accurately extract features within the area where the railway tracks are located, thereby improving the accuracy of railway track area identification.

[0005] To achieve the above objectives, the first aspect of this application provides a region segmentation method, the method comprising: acquiring an image to be processed from a railway video surveillance device; when it is determined that the image to be processed includes a railway track region, annotating the railway track region in the image to be processed to obtain annotation information; inputting the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain a railway track region segmentation result; wherein, the global feature extraction network is a network with multiple global feature extraction layers set at different network depths.

[0006] In some optional implementations, if it is determined that the image to be processed includes a railway track area, before annotating the railway track area in the image to be processed and obtaining the annotation information, the method further includes: extracting a first type of pixels in the image to be processed based on a preset pixel threshold, and extracting a second type of pixels in the adjacent images of the image to be processed.

[0007] Calculate the first similarity between the first type of pixels and the preset railway track feature pixels;

[0008] Calculate the second similarity between the second type of pixels and the preset railway track feature pixels;

[0009] If the first similarity and / or the second similarity are determined to be within the range of a preset similarity threshold, the image to be processed is determined to include the railway track area.

[0010] In some optional implementations, the global feature extraction layer is used to process the first type of pixels, the second type of pixels, the first similarity and the second similarity based on normalization parameters to obtain an output feature map;

[0011] In this layer, the input feature map of the global feature extraction layer has the same size as its corresponding output feature map.

[0012] In some alternative implementations, the global feature extraction layer includes: three parallel-connected first convolutional layers, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer sequentially connected to the third convolutional layer;

[0013] The first, second, and third convolutional layers are all used to extract features from the input feature map of the global feature extraction layer, to obtain a first feature extraction map corresponding to the first convolutional layer, a second feature extraction map corresponding to the second convolutional layer, and a third feature extraction map corresponding to the third convolutional layer.

[0014] The input image of the fourth convolutional layer includes the product of the first and second feature extraction maps, and the third feature extraction map;

[0015] The output image of the global feature extraction layer includes the feature map obtained by superimposing the input feature map and the output feature extraction map of the fourth convolutional layer.

[0016] In some alternative implementations, the global feature extraction network includes at least four downsampling layers; and a global feature extraction layer is set above at least two downsampling layers.

[0017] In some optional implementations, the image to be processed and its corresponding annotation information are input into a global feature extraction network for processing to obtain the railway track region segmentation result, including:

[0018] The image to be processed and its corresponding annotation information are input into the first convolutional layer for feature extraction to obtain the first feature map;

[0019] The first feature map is input into the first downsampling layer for processing, and the processing result is input into at least one convolutional layer for feature extraction to obtain the initial output feature map;

[0020] The initial output feature map is input into the first global feature extraction layer for processing. At the same time, the initial output feature map is input into other convolutional layers for secondary processing to obtain intermediate output feature maps.

[0021] The intermediate output feature map is input into the second global feature extraction layer for processing. At the same time, the intermediate output feature map is input into other convolutional layers for further processing to obtain the third output feature map.

[0022] The output feature maps of the first global feature extraction layer, the second global feature extraction layer, and the third output feature map are superimposed and fused to obtain the target feature map.

[0023] The target feature map is input into the logistic regression layer in the global feature extraction network for pixel-by-pixel prediction to obtain the railway track area segmentation result.

[0024] In some optional implementations, before inputting the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain the railway track region segmentation result, the following steps are also included:

[0025] Based on multiple training sample images, the preset feature extraction network is initially trained to obtain the network to be processed.

[0026] The network to be processed is used to segment the test sample image to obtain the test segmentation result. Both the training sample image and the test sample image include the railway track area.

[0027] Based on the test segmentation results and the preset railway area segmentation standard, the loss function value is determined and fed back to the network to be processed for parameter update;

[0028] The updated network to be processed is used to segment the test sample image to obtain the updated test segmentation results.

[0029] If the updated test segmentation results meet the preset railway area segmentation criteria, the updated network to be processed is determined to be a global feature extraction network.

[0030] To achieve the above objectives, a second aspect of this application provides a region segmentation system, comprising: a data acquisition module configured to acquire an image to be processed from a railway video surveillance device; a data annotation module configured to annotate the railway track region in the image to be processed, and obtain annotation information, when it is determined that the image to be processed includes a railway track region; and a segmentation module configured to input the image to be processed and its corresponding annotation information into a global feature extraction network for processing, and obtain a railway track region segmentation result; wherein the global feature extraction network is a network with multiple global feature extraction layers at different network depths.

[0031] To achieve the above objectives, a third aspect of this application provides an electronic device, comprising: one or more processors; and a memory storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the region segmentation methods in the embodiments of this application.

[0032] To achieve the above objectives, a third aspect of this application provides a readable storage medium storing a computer program, which, when executed by a processor, implements any of the region segmentation methods described in the embodiments of this application.

[0033] The region segmentation method, system, electronic device, and storage medium in this application acquire images to be processed from railway video surveillance equipment to facilitate the analysis of the railway line environment. When it is determined that the image to be processed includes a railway track area, the railway track area in the image is labeled to obtain labeling information. Based on this labeling information, the area containing railway tracks in the image can be clearly identified, narrowing the identification range of the railway track area. The image to be processed and its corresponding labeling information are input into a global feature extraction network for processing to obtain the railway track region segmentation result. This global feature extraction network is a network with multiple global feature extraction layers at different network depths, which can accurately and comprehensively analyze railway track features, enabling the obtained railway track region segmentation result to accurately represent railway track features, thereby accurately dividing the railway track area and improving the accuracy of foreign object detection within the railway track area. Attached Figure Description

[0034] The accompanying drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0035] Figure 1 This is a flowchart illustrating a region segmentation method provided in an embodiment of this application.

[0036] Figure 2 This is a schematic diagram of the structure of the global feature extraction layer provided in an embodiment of this application.

[0037] Figure 3 This is a block diagram of a region segmentation system provided in an embodiment of this application.

[0038] Figure 4 This is a schematic diagram of the structure of the global feature extraction network provided in an embodiment of this application.

[0039] Figure 5This is a schematic diagram illustrating the effect of using a global feature extraction network to extract global features from an image, as provided in an embodiment of this application.

[0040] Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] Before introducing the specific implementation methods, let's briefly outline the relevant technical solutions used in previous research on railway track region segmentation. These solutions mainly utilize methods such as thresholding, active contour models, region growing, graph cut models, and machine learning to segment railway track regions in images. Among these, the thresholding method involves first statistically analyzing the intensity of all pixels, setting a preset threshold, and then designating pixels with intensities greater than the preset threshold as foreground pixels and all remaining pixels as background pixels.

[0042] The region growing technique involves manually selecting a seed point and then expanding outwards based on a preset similarity metric, which enables simple and fast region segmentation.

[0043] Active contour models and graph cut optimization segmentation methods are often used together. The principle is to use various machine learning methods (such as k-means clustering, extreme machine learning, etc.) to segment regions.

[0044] Furthermore, deep learning has been proven in recent years to outperform traditional segmentation methods in region segmentation. However, most existing deep learning segmentation models are based on convolutional encoder-decoder frameworks. Due to the inherent limitations of convolutional operations, they can only extract local neighborhood features of the target object and cannot learn to extract feature dependencies over long distances. This means they have a weak ability to represent the global features of the target object, affecting the performance of the trained model. Therefore, designing a global feature extraction mechanism to effectively capture features of the railway defense zone within its neighborhood and over long distances, learning more representative features, and improving the model's segmentation and robustness is a problem that urgently needs to be solved.

[0045] In summary, this application proposes a global feature extraction and segmentation network to address the problem that existing deep learning segmentation methods can only extract local features of the target object.

[0046] The specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of this application. Those skilled in the art will recognize that this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0048] Firstly, embodiments of this application provide a region segmentation method. Figure 1 This is a flowchart illustrating a region segmentation method provided in an embodiment of this application. This method can be applied to a region segmentation system. Figure 1 As shown, this region segmentation method includes, but is not limited to, the following steps:

[0049] Step S101: Obtain the image to be processed from the railway video surveillance equipment.

[0050] Among them, railway video surveillance equipment can be image acquisition devices facing the railway track area to facilitate image acquisition of the railway track area. For example, it can acquire images of the railway track area under different lighting and weather conditions such as daytime, nighttime, sunny days, cloudy days, rainy days, and snowy days, and use any one of the above images as the image to be processed.

[0051] Step S 102: If it is determined that the image to be processed includes a railway track area, the railway track area in the image to be processed is marked to obtain the marking information.

[0052] Step S103: Input the image to be processed and its corresponding annotation information into the global feature extraction network for processing to obtain the railway track area segmentation result.

[0053] Among them, the global feature extraction network is a network with multiple global feature extraction layers at different network depths.

[0054] The global feature extraction layer effectively mines the feature dependencies of railway tracks within both short and long-range spatial ranges, enabling the global feature extraction network to learn more comprehensive railway track features and achieve more accurate segmentation. Using this global feature extraction network to segment railway tracks in images yields higher segmentation accuracy and better robustness. It can accurately capture features within railway tracks of different sizes, shapes, locations, and quantities, demonstrating high reliability.

[0055] In this embodiment, an image to be processed is acquired from railway video surveillance equipment to facilitate analysis of the railway line environment. If the image to be processed is determined to include a railway track area, the railway track area in the image is labeled to obtain labeling information. Based on this labeling information, the area containing the railway track in the image can be clearly identified, narrowing the identification range of the railway track area. The image to be processed and its corresponding labeling information are input into a global feature extraction network for processing to obtain the railway track area segmentation result. This global feature extraction network is a network with multiple global feature extraction layers at different network depths, which can accurately and comprehensively analyze railway track features, enabling the obtained railway track area segmentation result to accurately represent railway track features, thereby accurately dividing the railway track area and improving the accuracy of foreign object detection within the railway track area.

[0056] In some optional embodiments, the division of the railway track area in the image to be processed can be achieved in the following manner: First, multiple sample images (i.e., images including the railway track area) are acquired from the railway video surveillance equipment; then, the multiple sample images are labeled by connecting multiple line segments in a certain order to form a closed polygonal region, thereby obtaining the labeling information.

[0057] Further, multiple sample images and their corresponding representation information are divided into training set data and test set data in a 10:1 ratio. Then, the training set data is input into a preset feature extraction network for training, enabling the preset feature extraction network to automatically learn and extract the features of the railway track region in the image to be processed through backpropagation mechanism according to a set loss function. Upon confirmation that training is complete, the global feature extraction network used in step S103 is obtained.

[0058] Then, the global feature extraction network is used to segment and predict the images to be processed in the test set data to obtain the railway track area segmentation results. That is, other video surveillance images containing railway track areas can be input into the global feature extraction network to automatically obtain the corresponding railway track area segmentation results.

[0059] In this embodiment, the global feature extraction network refers to adding a designed global feature extraction layer between the convolutional and deconvolutional layers at specific locations in the network, based on the pre-defined fully convolutional neural network and according to the rail segmentation task, in order to optimize the features of the extracted rail area and capture the global feature expression of the rail area.

[0060] Unlike other existing deep learning methods that only use a single convolutional encoder-decoder architecture or only add a feature optimization layer to the last layer of the network encoder, the global feature extraction network used in this application adds a global feature extraction layer tailored to the railway track segmentation task at specific locations in both the shallow and deep layers of the network to optimize the representation of railway track features and improve segmentation performance.

[0061] In other words, this application uses a global feature extraction layer capable of global feature extraction to extract features of the railway track region in the image, which can extract features of the railway track region more comprehensively. Furthermore, the global feature extraction layer in this application can add a global feature extraction layer tailored to the railway track segmentation task at specific locations in both the shallow and deep layers of the global feature extraction network, in order to better optimize the learned railway track semantics and abstract features.

[0062] This application embodiment provides another possible implementation, wherein, before performing step S102, in which the image to be processed is determined to include a railway track area, and before obtaining the annotation information by annotating the railway track area in the image to be processed, the following method is further included:

[0063] Based on a preset pixel threshold, first-class pixels are extracted from the image to be processed, and second-class pixels are extracted from neighboring images of the image to be processed; a first similarity between the first-class pixels and preset railway feature pixels is calculated; a second similarity between the second-class pixels and preset railway feature pixels is calculated; if the first similarity and / or the second similarity are determined to be within the range of the preset similarity threshold, the image to be processed is determined to include a railway region.

[0064] The first similarity is used to characterize the degree of similarity between the first type of pixels and the preset railway feature pixels, and the second similarity is used to characterize the degree of similarity between the second type of pixels and the preset railway feature pixels.

[0065] When the first similarity and / or the second similarity are within the preset similarity threshold range, it can be determined that the image to be processed includes the railway track area, thereby obtaining an image to be processed that meets the preset similarity threshold range requirement, so that the railway track area can be queried in the image to be processed in the future, which is convenient for dividing the railway track area.

[0066] In some optional embodiments, the global feature extraction layer is used to process the first type of pixels, the second type of pixels, the first similarity, and the second similarity based on normalization parameters to obtain an output feature map.

[0067] In this process, the input feature map of the global feature extraction layer has the same size as its corresponding output feature map. This ensures that the global feature extraction layer does not change the size of the image when processing the input feature map, so that the output feature map can truly reflect the image features corresponding to the input feature map.

[0068] For example, the output feature map can be obtained by weighting and summing the first type of pixels, the second type of pixels, the first similarity, and the second similarity based on the normalization parameter.

[0069] The normalization parameter can be expressed as: Where, x i Indicates the first type of pixel; x j This represents the second type of pixel, i.e., pixels within the image to be processed that are adjacent to pixel x. i Similar pixels, and all pixels in neighboring images of the image to be processed that are similar to pixel x. i Similar pixels; C(x) represents the normalization factor.

[0070] In some optional embodiments, the global feature extraction layer includes: three parallel-connected first convolutional layers, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer sequentially connected to the third convolutional layer.

[0071] The first, second, and third convolutional layers are used to extract features from the input feature map of the global feature extraction layer, obtaining a first feature extraction map corresponding to the first convolutional layer, a second feature extraction map corresponding to the second convolutional layer, and a third feature extraction map corresponding to the third convolutional layer. The input image of the fourth convolutional layer includes the product of the first and second feature extraction maps and the third feature extraction map. The output feature map of the global feature extraction layer includes the feature map obtained by superimposing the input feature map and the output feature extraction map of the fourth convolutional layer.

[0072] For example, Figure 2 This is a schematic diagram of the global feature extraction layer provided in an embodiment of this application. Figure 2 As shown, the global feature extraction layer includes a first convolutional layer 201, a second convolutional layer 202, a third convolutional layer 203, a fourth convolutional layer 204, two multipliers, and one adder.

[0073] The input feature maps are fed into the first convolutional layer 201 (parameter 1*1), the second convolutional layer 202 (parameter 1*1), and the third convolutional layer 203 (parameter 1*1) for feature extraction, resulting in a first feature extraction map, a second feature extraction map, and a third feature extraction map. Then, a multiplier is used to multiply the first feature extraction map output from the first convolutional layer 201 and the second feature extraction map output from the second convolutional layer 202, obtaining a first product result. This first product result is then multiplied by the third feature extraction map output from the third convolutional layer 203 in the next stage multiplier to obtain a second product result. This second product result is then processed in the fourth convolutional layer 204 to obtain the output feature map of the fourth convolutional layer 204. Further, the output feature extraction map of the fourth convolutional layer 204 and the original input feature map are added together in an adder to obtain the output feature map of the global feature extraction layer.

[0074] The multiplier used to multiply the first feature extraction map output by the first convolutional layer 201 and the second feature extraction map output by the second convolutional layer 202 is implemented using matrix multiplication. The adder used to add the output feature extraction map of the fourth convolutional layer 204 and the original input feature map is implemented using pixel-level addition.

[0075] In some specific implementations, feature fusion of two feature maps can also be achieved by weighted superposition (or direct superposition, or connection, etc.), and this application does not impose any restrictions on this.

[0076] By processing the input feature map through the aforementioned global feature extraction layer, the feature dependencies of the railway tracks in the input feature map can be captured in both local regions and long-distance spatial ranges. This allows for the learning of more comprehensive and accurate railway track features, which helps improve the segmentation performance of the network.

[0077] In some optional embodiments, the global feature extraction network includes at least four downsampling layers; a global feature extraction layer is disposed above at least two downsampling layers.

[0078] By setting a global feature extraction layer above at least two downsampling layers, global features in intermediate feature maps can be extracted through global feature extraction layers at different levels, making the processed image features more complete.

[0079] For example, a first global feature extraction layer can be added before the first downsampling layer; a second global feature extraction layer can be added before the second downsampling layer; a third global feature extraction layer can be added before the third downsampling layer; and a fourth global feature extraction layer can be added before the fourth downsampling layer. Through the processing of multiple global feature extraction layers, the feature map can be further processed to extract global image features before entering the downsampling layer, so as to ensure the integrity of the feature map and improve the accuracy of image feature extraction.

[0080] In some optional embodiments, step S103, which involves inputting the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain the railway track region segmentation result, can be implemented in the following way:

[0081] The image to be processed is input into a first convolutional layer for feature extraction to obtain a first feature map. The first feature map is then input into a first downsampling layer for processing, and the processing result is input into at least one convolutional layer for feature extraction to obtain an initial output feature map. The initial output feature map is then input into a first global feature extraction layer for processing, and simultaneously, the initial output feature map is input into other convolutional layers for secondary processing to obtain an intermediate output feature map. The intermediate output feature map is then input into a second global feature extraction layer for processing, and simultaneously, the intermediate output feature map is input into other convolutional layers for further processing to obtain a third output feature map. The features from the first global feature extraction layer, the second global feature extraction layer, and the third output feature map are superimposed and fused to obtain a target feature map. The target feature map is then input into a logistic regression layer in the global feature extraction network for pixel-by-pixel prediction to obtain the railway track region segmentation result.

[0082] By processing the initial output feature map and the intermediate output feature map through the two global feature extraction layers mentioned above, global image features corresponding to the initial output feature map and the intermediate output feature map can be extracted. Then, the output feature maps of the first global feature extraction layer, the second global feature extraction layer, and the third output feature map are superimposed and fused so that the obtained target feature map can reflect multiple different levels of image features, thereby improving the accuracy of image feature extraction.

[0083] In some optional embodiments, before inputting the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain the railway track region segmentation result, the method further includes: performing initial training on a preset feature extraction network based on multiple training sample images to obtain a network to be processed; using the network to be processed to perform segmentation testing on test sample images to obtain test segmentation results; determining a loss function value based on the test segmentation results and a preset railway track region segmentation standard, and feeding the loss function value back to the network to be processed for parameter updates; using the updated network to be processed to perform segmentation testing on test sample images to obtain updated test segmentation results; and determining the updated network to be processed as a global feature extraction network if the updated test segmentation results meet the preset railway track region segmentation standard.

[0084] Both the training and testing sample images include the railway track region. A preset railway track region segmentation standard serves as a constraint for network training. If the updated test segmentation result meets the preset railway track region segmentation standard, the parameters corresponding to the updated network can be saved, thus obtaining the global feature extraction network. Otherwise, multiple training sample images are selected, and the updated network is iteratively trained until the updated test segmentation result meets the preset railway track region segmentation standard, thereby obtaining an accurate global feature extraction network.

[0085] It should be noted that using the network to be processed to segment the test sample image and obtain the test segmentation result is a process of using the test sample image to predict the segmentation of the network to be processed. This is used to predict whether the network to be processed can accurately segment the railway track area in the test sample image. Furthermore, based on the test segmentation result, the network to be processed is judged to determine whether it needs to be trained again.

[0086] For example, the preset criteria for railway track area segmentation include: the number of training iterations reaches a preset threshold, or the loss function value is within the range of a preset threshold.

[0087] Through the above training, the obtained global feature extraction network can be equipped with the ability to segment the railway defense zone from video surveillance images.

[0088] Figure 3 This is a block diagram illustrating the composition of a region segmentation system provided in an embodiment of this application. Figure 3 As shown, the region segmentation system includes, but is not limited to, the following modules:

[0089] The data acquisition module 301 is configured to acquire images to be processed from railway video surveillance equipment.

[0090] For example, the data acquisition module 301 is used to acquire images to be processed from railway video surveillance equipment. These images contain a railway track area. For example, images are extracted from railway video surveillance equipment imaging towards the railway track area at preset time intervals. The extracted images may include images under different lighting and weather conditions, such as morning, noon, evening, daytime, nighttime, sunny, cloudy, rainy, and snowy conditions.

[0091] The data annotation module 302 is configured to annotate the railway track area in the image to be processed when it is determined that the image to be processed includes a railway track area, and obtain annotation information.

[0092] For example, the data annotation module 502 uses annotation software to annotate the railway track area in the image to be processed. In other words, the railway track area in the image is manually outlined using annotation software, and then the obtained annotation information is saved separately.

[0093] The segmentation module 303 is configured to input the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain the railway track area segmentation result.

[0094] Among them, the global feature extraction network is a network with multiple global feature extraction layers at different network depths.

[0095] It should be noted that the global feature extraction network is a network obtained by training multiple sample images containing railway tracks and the corresponding annotation information of the railway tracks in the sample images.

[0096] For example, a sample image containing a railway track area is used as the input image, and the railway track region segmentation result of the corresponding image sample with railway track region annotation information is used as the network output to train a global feature extraction network. If the test segmentation result of the network output obtained by training meets the preset railway track region segmentation criteria, the network is determined to be a global feature extraction network.

[0097] The rail segmentation module 503 can input a frame of the image to be processed, which is captured from the railway video surveillance equipment, into the global feature extraction network for processing, thereby obtaining the segmentation result of the rail area corresponding to the image to be processed.

[0098] Using a global feature extraction network can fully extract global features of the railway track region in the image to be processed, giving the global feature extraction network better region segmentation performance.

[0099] In this embodiment, a data acquisition module acquires images to be processed from railway video surveillance equipment to facilitate analysis of the railway line environment. A data annotation module, upon determining that the image includes a railway track area, annotates the track area within the image, obtaining annotation information. This annotation information clarifies the location of the track within the image, narrowing the identification range. A segmentation module inputs the image and its corresponding annotation information into a global feature extraction network for processing, obtaining the track area segmentation result. This global feature extraction network, with multiple global feature extraction layers at different depths, accurately and comprehensively analyzes track features, accurately characterizes track features, and thus accurately delineates the track area, improving the accuracy of foreign object detection within the track area.

[0100] Figure 4 This is a schematic diagram of the structure of the global feature extraction network provided in an embodiment of this application. Figure 4 As shown, this global feature extraction network includes the following layers:

[0101] The following layers are connected in sequence: first convolutional layer, first downsampling layer, second convolutional layer, third convolutional layer, first global feature extraction layer; second downsampling layer, fourth convolutional layer, fifth convolutional layer, second global feature extraction layer; third downsampling layer, sixth convolutional layer, seventh convolutional layer, third global feature extraction layer; fourth downsampling layer, eighth convolutional layer, ninth convolutional layer, fourth global feature extraction layer, fifth global feature extraction layer;

[0102] The tenth convolutional layer, the first deconvolutional layer, and the eleventh convolutional layer;

[0103] The twelfth convolutional layer, the second deconvolutional layer, and the thirteenth convolutional layer;

[0104] Fourteenth convolutional layer, third deconvolutional layer, fifteenth convolutional layer;

[0105] The sixteenth convolutional layer, the fourth deconvolutional layer, and the seventeenth convolutional layer;

[0106] Fifth deconvolutional layer, eighteenth convolutional layer, nineteenth convolutional layer;

[0107] The twentieth convolutional layer, the sixth deconvolutional layer, and the twenty-first convolutional layer;

[0108] The twenty-second convolutional layer, the seventh deconvolutional layer, and the twenty-third convolutional layer.

[0109] The input feature map of the fourteenth convolutional layer includes the output feature map of the thirteenth convolutional layer and the output feature map of the second global feature extraction layer.

[0110] The input feature map of the sixteenth feature fusion layer includes the output feature map of the fifteenth convolutional layer and the output feature map of the first global feature extraction layer.

[0111] The input feature map of the fifth deconvolutional layer includes the output feature map of the seventeenth convolutional layer, the output feature map of the twenty-first convolutional layer, and the output feature map of the twenty-third convolutional layer.

[0112] The input feature map of the second downsampling layer includes the output feature map of the first downsampling layer and the output feature map of the third convolutional layer;

[0113] The input feature map of the third downsampling layer includes the output feature map of the second downsampling layer and the output feature map of the fifth convolutional layer;

[0114] The input feature map of the fourth downsampling layer includes the output feature map of the third downsampling layer and the output feature map of the seventh convolutional layer;

[0115] The input feature map of the fourth global feature extraction layer includes the output feature map of the fourth downsampling layer and the output feature map of the ninth convolutional layer;

[0116] The input feature map of the twelfth convolutional layer includes the output feature map of the fifth global feature extraction layer and the output feature map of the eleventh convolutional layer;

[0117] The input feature map of the fourteenth convolutional layer includes the output feature map of the eleventh convolutional layer and the output feature map of the thirteenth convolutional layer;

[0118] The input feature map of the fifth deconvolutional layer includes the output feature map of the fifteenth convolutional layer and the output feature map of the seventeenth convolutional layer.

[0119] Specifically, the image processing at each level includes:

[0120] The input image containing the railway track area is fed into the first convolutional layer (parameter is 4*4) for feature extraction to obtain the first feature map.

[0121] The first feature map is input into the first downsampling layer (with parameters of 3*3) and scaled to obtain the second feature map, thereby reducing the number of parameters required for network training.

[0122] The second feature map is input into the second convolutional layer (with parameters of 3*3) for further feature extraction to obtain the third feature map.

[0123] The third feature map is input into the third convolutional layer (with parameters of 3*3) for further feature extraction to obtain the fourth feature map.

[0124] The fourth and second feature maps are input into the second downsampling layer (with parameters of 3*3) for further feature extraction to obtain the fifth feature map.

[0125] The fifth feature map is input into the fourth convolutional layer (with parameters of 3*3) for further feature extraction to obtain the sixth feature map.

[0126] The 6th feature map is input into the 5th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 7th feature map.

[0127] The 7th and 5th feature maps are input into the third downsampling layer (with parameters of 3*3) for further feature extraction to obtain the 8th feature map.

[0128] The 8th feature map is input into the 6th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 9th feature map.

[0129] The 9th feature map is input into the 7th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 10th feature map.

[0130] The 10th and 8th feature maps are input into the fourth downsampling layer (with parameters of 3*3) for further feature extraction to obtain the 11th feature map.

[0131] The 11th feature map is input into the 8th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 12th feature map.

[0132] The 12th feature map is input into the 9th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 13th feature map.

[0133] The 13th and 11th feature maps are input into the fourth global feature layer for further feature extraction to obtain the 14th feature map.

[0134] The 14th feature map is input into the fifth global feature layer for further feature extraction to obtain the 15th feature map.

[0135] The 15th feature map is input into the 10th convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 16th feature map.

[0136] The 16th feature map is input into the first deconvolution layer (with parameters of 3*3) for bilinear interpolation to recover the learned features and obtain the 17th feature map.

[0137] The 17th feature map is input into the 11th convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 18th feature map.

[0138] The 18th and 15th feature maps are input into the 12th convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 19th feature map.

[0139] The 19th feature map is input into the second deconvolution layer (with parameters of 3*3) for bilinear interpolation to recover the learned features and obtain the 20th feature map.

[0140] The 20th feature map is input into the 13th convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 21st feature map.

[0141] The 21st feature map, the 18th feature map, the 15th feature map, and the feature map output by the second global feature extraction layer (i.e., the feature map obtained by inputting the 7th feature map into the second global feature extraction layer for feature extraction) are input into the fourteenth convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 22nd feature map.

[0142] The 22nd feature map is input into the third deconvolution layer (with parameters of 3*3) for bilinear interpolation to recover the learned features and obtain the 23rd feature map.

[0143] The 23rd feature map is input into the 15th convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 24th feature map.

[0144] The 24th feature map, the feature map output by the first global feature extraction layer (i.e., the feature map obtained by inputting the 4th feature map into the first global feature extraction layer for feature extraction), and the input feature map of the fourteenth convolutional layer (i.e., the 21st feature map, the 18th feature map, the 15th feature map, and the feature map output by the second global feature extraction layer) are input into the sixteenth convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 25th feature map.

[0145] The 25th feature map is input into the fourth deconvolution layer (with parameters of 3*3) for bilinear interpolation to recover the learned features and obtain the 26th feature map.

[0146] The 26th feature map is input into the 17th convolutional layer (with parameters of 1*1) for further feature extraction to obtain the 27th feature map.

[0147] Before processing the 27th feature map, the following operations are required:

[0148] The 10th feature map is input into the third global feature extraction layer for further feature extraction, resulting in the 28th feature map. Then, the 28th feature map is input into the 22nd convolutional layer (with parameters of 1*1) for further feature extraction, resulting in the 29th feature map. The 29th feature map is then input into the 7th deconvolutional layer (with parameters of 10*10) for bilinear interpolation to recover the learned features, resulting in the 30th feature map. Furthermore, the 30th feature map is input into the 23rd convolutional layer (with parameters of 1*1) for further feature extraction, resulting in the 31st feature map.

[0149] The 13th feature map is input into the 20th convolutional layer (parameter 1*1) for further feature extraction to obtain the 32nd feature map; the 32nd feature map is input into the 6th deconvolutional layer (parameter 18*18) for bilinear interpolation to recover the learned features to obtain the 33rd feature map; the 33rd feature map is input into the 21st convolutional layer (parameter 1*1) for further feature extraction to obtain the 34th feature map.

[0150] After the above operations, the 27th feature map, the 31st feature map, the 34th feature map, and the input feature map of the 16th convolutional layer are superimposed, and the superimposed feature maps are fused to obtain the input feature map of the fifth deconvolutional layer (i.e., the 35th feature map).

[0151] It should be noted that the 35th feature map is a fusion feature map that can reflect relatively abstract semantic information and has local features of the railway track, and a feature map from a shallow layer that has more detailed information and has global features of the railway track. This can make up for the feature loss caused by multiple convolution and downsampling operations, thus giving the 35th feature map richer image features.

[0152] Furthermore, the 35th feature map is input into the fifth deconvolution layer (with parameters of 4*4) for bilinear interpolation to recover the learned features and obtain the 36th feature map.

[0153] The 36th feature map is input into the 18th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 37th feature map.

[0154] The 37th feature map is input into the 19th convolutional layer (with parameters of 3*3) for further feature extraction to obtain the 38th feature map.

[0155] In some specific implementations, the 38th feature map can also be input into a logistic regression layer (e.g., softmax) for pixel-by-pixel prediction to obtain the final output image, which includes the railway track region segmentation results.

[0156] In this embodiment, multiple global feature extraction layers are embedded into network layers of different depths within the global feature extraction network. This enables more accurate extraction of features specific to the railway track region from the image, improving the accuracy and robustness of feature extraction. Furthermore, by performing pixel-based weighted summation on the feature maps processed at different levels, the resulting output feature map reflects detailed information in the image, thereby accurately determining whether the output image includes the railway track region and improving the accuracy of railway track region segmentation.

[0157] Figure 5This is a schematic diagram illustrating the effect of using a global feature extraction network to extract global features from an image, as provided in an embodiment of this application. For example... Figure 5 As shown, pixel x i Characterization Figure 5 The pixels of the railway track area in the first image on the left, pixel x j Characterization Figure 5 The pixels of the railroad track area in all images except the first image on the left (i.e., the images adjacent to the first image on the left).

[0158] The image to be processed is composed of global feature extraction layers of different depths in a global feature extraction network (e.g., including pixels x). i The image) and its neighboring images (e.g., including pixel x) j Global feature extraction is performed on multiple images to obtain the feature of pixel x. i Image features of the corresponding region (i.e., the railway track area).

[0159] Where, pixel x i The output response y i It is a weighted summation of all x. j The response obtained from the output response at a pixel can be expressed by formula (1) as the output response y. i .

[0160]

[0161] Where, x i Indicates the pixels in the image to be processed that need to be predicted; x j Indicates the area within the image to be processed relative to pixel x. i Similar pixels, and all pixels in neighboring images of the image to be processed that are similar to pixel x. i Similar pixels; y i This represents the output response (i.e., pixel x). i The region to which it belongs, such as pixel x i Belongs to the railway track pixel, or pixel x i (Including background pixels, etc.); This represents all pixels j in the image.

[0162] C(x) represents the factor that normalizes the output response. For example,

[0163] g() represents the function representation of the image to be processed (i.e., the input image) at pixel j; for example, g(x j ) = W g x j Among them, W g This represents the weight matrix corresponding to pixel j when it is in the first convolutional layer.

[0164] f() represents the calculation of pixel x i and pixel x j The similarity between them is a function, for example, the exponential similarity calculation function is used to represent the function, as shown in formula (2).

[0165]

[0166] Where, θ(x) i ) = W θ x i , W θ and These represent the weight matrices for the two parameters that need to be learned. For example, This represents the weight matrix corresponding to the third convolutional layer. θ(x) i ) T Represents θ(x) i The transpose of ).

[0167] Through the above operations, it is possible to extract the rail features within a local area, and also to capture pixel x. i Feature representation over a long spatial range. Further, the rail features extracted from different network depths are fused, and then the fused features are input into deconvolutional and softmax layers for pixel-by-pixel prediction, thereby obtaining the final output image, which includes the rail region segmentation results.

[0168] In some alternative implementations, each global feature extraction layer can also obtain formula (3) by mapping and encapsulating formula (1).

[0169] z i =W Z y i +x i (3)

[0170] Among them, y i This represents the output response of formula (1), and y i It can be used as input to the global feature extraction layer; W z This represents a 1x1 weight matrix.

[0171] “x i " indicates a residual connection, z" ii This represents the output of the global feature extraction layer.

[0172] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problem proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0173] This application also provides electronic devices and computer-readable storage media, all of which can be used to implement any of the region segmentation methods in this application. The corresponding technical solutions and descriptions are the same as those described in the method section and will not be repeated here.

[0174] Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application. (For example...) Figure 6 As shown, this application embodiment provides an electronic device 600, which includes: at least one processor 601; at least one memory 602; and one or more I / O interfaces 603 connected between the processor 601 and the memory 602; wherein the memory 602 stores one or more computer programs that can be executed by at least one processor 601, and the one or more computer programs are executed by at least one processor 601 to enable at least one processor 601 to perform the above-described region segmentation method.

[0175] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program implements the aforementioned region segmentation method when executed by a processor / processor core. The computer-readable storage medium can be volatile or non-volatile.

[0176] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described region segmentation method.

[0177] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0178] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0179] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0180] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0181] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0182] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0183] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0184] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0186] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this application as set forth by the appended claims.

Claims

1. A region segmentation method, characterized in that, The method includes: Obtain images to be processed from railway video surveillance equipment; If it is determined that the image to be processed includes a railway track area, the railway track area in the image to be processed is marked to obtain marking information; The image to be processed and its corresponding annotation information are input into a global feature extraction network for processing to obtain the railway track region segmentation result. The global feature extraction network is a network with multiple global feature extraction layers at different network depths. The global feature extraction network comprises the following layers: a first convolutional layer, a first downsampling layer, a second convolutional layer, a third convolutional layer, and a first global feature extraction layer, connected sequentially; a second downsampling layer, a fourth convolutional layer, a fifth convolutional layer, and a second global feature extraction layer; a third downsampling layer, a sixth convolutional layer, a seventh convolutional layer, and a third global feature extraction layer; a fourth downsampling layer, an eighth convolutional layer, a ninth convolutional layer, a fourth global feature extraction layer, and a fifth global feature extraction layer; a tenth convolutional layer, a first deconvolutional layer, and an eleventh convolutional layer; a twelfth convolutional layer, a second deconvolutional layer, and a thirteenth convolutional layer. Convolutional layers; the fourteenth convolutional layer, the third deconvolutional layer, the fifteenth convolutional layer; the sixteenth convolutional layer, the fourth deconvolutional layer, the seventeenth convolutional layer; the fifth deconvolutional layer, the eighteenth convolutional layer, the nineteenth convolutional layer; the twentieth convolutional layer, the sixth deconvolutional layer, the twenty-first convolutional layer; the twenty-second convolutional layer, the seventh deconvolutional layer, and the twenty-third convolutional layer; wherein, the inputs of the first global feature extraction layer and the second downsampling layer include: the output feature map of the first downsampling layer and the output feature map of the third convolutional layer; the inputs of the second global feature extraction layer and the third downsampling layer include: the second downsampling layer... The inputs to the third global feature extraction layer and the fourth downsampling layer include the output feature maps of the third downsampling layer and the seventh convolutional layer; the inputs to the fourth global feature extraction layer and the twentieth convolutional layer include the output feature maps of the fourth downsampling layer and the ninth convolutional layer; the input to the tenth convolutional layer is the output feature map of the fifth global feature extraction layer; the input to the twelfth convolutional layer includes the output feature maps of the fifth global feature extraction layer and the eleventh convolutional layer; the input to the fourteenth convolutional layer includes the output feature maps of the thirteenth, second, and eleventh convolutional layers; the input to the sixteenth convolutional layer includes the output feature maps of the fifteenth, first, and thirteenth convolutional layers; the input to the twenty-second convolutional layer is the output feature map of the third global feature extraction layer; the input to the fifth deconvolutional layer includes the output feature maps of the seventeenth, twenty-first, fifteenth, and twenty-third convolutional layers.

2. The method according to claim 1, characterized in that, Before determining that the image to be processed includes a railway track area, and before obtaining the annotation information, the method further includes: Based on a preset pixel threshold, first-type pixels are extracted from the image to be processed, and second-type pixels are extracted from neighboring images of the image to be processed. Calculate the first similarity between the first type of pixels and the preset railway track feature pixels; Calculate the second similarity between the second type of pixels and the preset railway track feature pixels; If the first similarity and / or the second similarity are determined to be within a preset similarity threshold range, the image to be processed is determined to include the railway track area.

3. The method according to claim 2, characterized in that, The global feature extraction layer is used to process the first type of pixels, the second type of pixels, the first similarity and the second similarity based on normalization parameters to obtain an output feature map; The input feature map of the global feature extraction layer has the same size as its corresponding output feature map.

4. The method according to claim 3, characterized in that, The global feature extraction layer includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer that are sequentially connected to the third convolutional layer of the global feature extraction layer; The first convolutional layer, the second convolutional layer, and the third convolutional layer of the global feature extraction layer are all used to extract features from the input feature map of the global feature extraction layer, thereby obtaining a first feature extraction map corresponding to the first convolutional layer of the global feature extraction layer, a second feature extraction map corresponding to the second convolutional layer of the global feature extraction layer, and a third feature extraction map corresponding to the third convolutional layer of the global feature extraction layer. The input image of the fourth convolutional layer of the global feature extraction layer includes the product of the first feature extraction map and the second feature extraction map, as well as the third feature extraction map; The output image of the global feature extraction layer includes a feature map resulting from the superposition of the input feature map of the global feature extraction layer and the output feature extraction map of the fourth convolutional layer of the global feature extraction layer.

5. The method according to claim 3, characterized in that, Before inputting the image to be processed and its corresponding annotation information into the global feature extraction network for processing to obtain the railway track region segmentation result, the process further includes: Based on multiple training sample images, the preset feature extraction network is initially trained to obtain the network to be processed. The network to be processed is used to segment the test sample image to obtain the test segmentation result, wherein both the training sample image and the test sample image include the railway track area; Based on the test segmentation results and the preset railway area segmentation standard, the loss function value is determined, and the loss function value is fed back to the network to be processed for parameter update; The updated network to be processed is used to perform segmentation tests on the test sample images to obtain updated test segmentation results. If the updated test segmentation result is determined to meet the preset railway region segmentation standard, the updated network to be processed is determined to be the global feature extraction network.

6. A region segmentation system, comprising: The data acquisition module is configured to acquire images to be processed from railway video surveillance equipment; The data annotation module is configured to annotate the railway track area in the image to be processed when it is determined that the image to be processed includes a railway track area, and obtain annotation information. The segmentation module is configured to input the image to be processed and its corresponding annotation information into a global feature extraction network for processing to obtain the railway track area segmentation result; The global feature extraction network is a network with multiple global feature extraction layers at different network depths. The global feature extraction network comprises the following layers: a first convolutional layer, a first downsampling layer, a second convolutional layer, a third convolutional layer, and a first global feature extraction layer, connected sequentially; a second downsampling layer, a fourth convolutional layer, a fifth convolutional layer, and a second global feature extraction layer; a third downsampling layer, a sixth convolutional layer, a seventh convolutional layer, and a third global feature extraction layer; a fourth downsampling layer, an eighth convolutional layer, a ninth convolutional layer, a fourth global feature extraction layer, and a fifth global feature extraction layer; a tenth convolutional layer, a first deconvolutional layer, and an eleventh convolutional layer; a twelfth convolutional layer, a second deconvolutional layer, and a thirteenth convolutional layer. Convolutional layers; the fourteenth convolutional layer, the third deconvolutional layer, the fifteenth convolutional layer; the sixteenth convolutional layer, the fourth deconvolutional layer, the seventeenth convolutional layer; the fifth deconvolutional layer, the eighteenth convolutional layer, the nineteenth convolutional layer; the twentieth convolutional layer, the sixth deconvolutional layer, the twenty-first convolutional layer; the twenty-second convolutional layer, the seventh deconvolutional layer, and the twenty-third convolutional layer; wherein, the inputs of the first global feature extraction layer and the second downsampling layer include: the output feature map of the first downsampling layer and the output feature map of the third convolutional layer; the inputs of the second global feature extraction layer and the third downsampling layer include: the second downsampling layer... The inputs to the third global feature extraction layer and the fourth downsampling layer include the output feature maps of the third downsampling layer and the seventh convolutional layer; the inputs to the fourth global feature extraction layer and the twentieth convolutional layer include the output feature maps of the fourth downsampling layer and the ninth convolutional layer; the input to the tenth convolutional layer is the output feature map of the fifth global feature extraction layer; the input to the twelfth convolutional layer includes the output feature maps of the fifth global feature extraction layer and the eleventh convolutional layer; the input to the fourteenth convolutional layer includes the output feature maps of the thirteenth, second, and eleventh convolutional layers; the input to the sixteenth convolutional layer includes the output feature maps of the fifteenth, first, and thirteenth convolutional layers; the input to the twenty-second convolutional layer is the output feature map of the third global feature extraction layer; the input to the fifth deconvolutional layer includes the output feature maps of the seventeenth, twenty-first, fifteenth, and twenty-third convolutional layers.

7. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the region segmentation method as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the region segmentation method as described in any one of claims 1 to 5.

Images