A real-time semantic segmentation system integrating edge feature details

Abstract

The present invention relates to a real-time semantic segmentation system that integrates edge feature details. An edge branch module is added to a dual-branch segmentation network, so that the final result of the semantic segmentation is more excellent in edge and detail performance. By setting a shallow feature sharing module and using a shallow feature sharing method, the number of model parameters is further reduced, so that the overall system has a faster reasoning speed, which greatly facilitates the deployment of the model. At the same time, by setting a semantic context attention fusion module and a segmentation result attention fusion module, context information can be better taken into account when fusing the features of the segmented image, so that the model has higher accuracy and better performance.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a real-time semantic segmentation system integrating edge feature details. Background Art

[0002] Semantic segmentation is one of the important tasks in computer vision, and its goal is to classify each pixel in an image so that the computer can understand objects and scenes. The existing semantic segmentation models have serious defects both in the model itself and in actual deployment.

[0003] From the perspective of the semantic segmentation model of the existing technology, there are defects such as blurred edges and lost details in the prediction results. Although multi-scale feature fusion technology is used in some models (such as pyramid pooling module or multi-scale feature fusion module) to overcome these two defects, these models are more effective in processing large object images, but they are still unable to cope with the edges of small objects and complex backgrounds, especially in complex scenes. There are defects of misclassification and missed classification, so that such models can only partially solve the problem of blurred edges, and the processing of fine edges is still not ideal.

[0004] From the actual deployment of the semantic segmentation model of the existing technology, there is a technical defect of excessive computational complexity. For example, the training and reasoning process of the second version of the bilateral segmentation network (Bilateral Segmentation Network Version 2, referred to as BiseNet V2) in the existing technology requires certain computing resources. In some scenarios that are sensitive to resource consumption, further optimization is still needed to reduce resource usage. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to further improve the semantic segmentation accuracy and reduce the cost based on the existing technology. In order to overcome the defects of the above existing technology, the present invention provides a real-time semantic segmentation system integrating edge feature details.

[0006] The present invention provides a real-time semantic segmentation system for integrating edge feature details, comprising an input end and an output end. In addition, the real-time semantic segmentation system further comprises:

[0007] A shallow feature sharing module extracts shallow features from the image under test by multiple downsampling to obtain a shallow feature image;

[0008] A context branching module performs several downsampling operations on the shallow feature image to complete semantic information retention, and obtains context feature maps corresponding to each downsampling operation;

[0009] A semantic context fusion module obtains a detail branch feature map by fusing the shallow feature image with the context feature maps corresponding to each downsampling operation;

[0010] The edge branch module extracts edge information from the context feature maps corresponding to each downsampling operation obtained by the context branch module in sequence, and obtains an edge branch feature map by fusion;

[0011] The segmentation result attention fusion module performs segmentation fusion on the context feature map, the detail branch feature map and the edge branch feature map obtained by the last downsampling operation of the context branch module to obtain the semantic segmentation result of the image under test; wherein,

[0012] The shallow feature sharing module communicates with the input end, the segmentation result attention fusion module communicates with the output end, and the context branch module communicates with the semantic context fusion module and the edge branch module respectively;

[0013] The shallow feature sharing module, the context branch module, and the segmentation result attention fusion module form a first communication network branch;

[0014] The shallow feature sharing module, the semantic context fusion module, and the segmentation result attention fusion module form a second communication network branch;

[0015] The shallow feature sharing module, the edge branch module, and the segmentation result attention fusion module form a third communication network branch.

[0016] Compared with the prior art, the real-time semantic segmentation system integrating edge feature details disclosed in the present invention has at least the following beneficial effects:

[0017] First, on the basis of the dual-branch semantic segmentation network represented by the existing two communication network branches, an edge branch module is innovatively added to form a third communication network branch. The edge branch module is specifically used to capture and process the edge features of the image, so that the final semantic segmentation result is better in edge and detail performance, solving the problems of edge blur and detail loss in the existing technology.

[0018] Secondly, by setting up a shallow feature sharing module, the context branch module, the semantic context fusion module and the edge branch module share information about the shallow feature image. This design adopts a simpler design concept and further reduces the number of model parameters through the shallow feature sharing method, which can simplify the design and improve the reasoning speed and reduce costs. At the same time, this design greatly reduces the computational complexity, thereby significantly improving the reasoning speed of the system and improving the deployment efficiency of the system or model in practical applications.

[0019] Finally, by setting up the semantic context attention fusion module and the segmentation result attention fusion module, the technical effect of introducing the attention fusion module to improve accuracy is achieved, thereby better taking into account context information in the feature fusion process. Moreover, the introduction of these two modules enables the system to more accurately fuse features at different levels when processing semantic information, significantly improving segmentation accuracy and overall performance.

[0020] In a possible implementation, the shallow feature sharing module is a communication network chain structure formed by a plurality of first feature extractors connected in series, the first feature extractor at the head end communicates with the input end, and the first feature extractor at the end communicates with the context branch module, the semantic context fusion module, and the edge branch module respectively;

[0021] This solution can achieve the technical effect of three communication network branches sharing the same feature map by designing a downsampling strategy, and has the following advantages:

[0022] By setting up multiple feature extractors, the same features can be avoided from being repeatedly calculated in different communication network branches. This makes the system more efficient when processing input images, because shallow features only need to be calculated once and can be used by multiple communication network branches.

[0023] By sharing shallow features in this way, the number of parameters in the network can be reduced. This is because different communication network branches no longer need their own independent shallow feature extractors, but share a common feature extractor, thereby reducing the overall number of parameters in the system.

[0024] The shallow feature sharing formed by this scheme reduces redundant calculations and improves the system's reasoning speed, which is particularly important in real-time applications such as real-time semantic segmentation and real-time object detection.

[0025] The shallow feature sharing formed by this scheme can also ensure the consistency of features, because all communication network branches share the same shallow features, which helps to better integrate information at different levels in the subsequent feature fusion process.

[0026] In a possible implementation, the context branch module is a communication network chain structure formed by a plurality of second feature extractors connected in series, the second feature extractor at the head end communicates with the first feature extractor at the end, the second feature extractor at the end communicates with the segmentation result attention fusion module, and all the second feature extractors communicate with the semantic context fusion module and the edge branch module respectively;

[0027] The context branch module corresponding to this solution realizes fast downsampling by connecting several second feature extractors in series, so that the context branch module has an increasingly larger receptive field and can retain richer semantic information.

[0028] In a possible implementation, the step of obtaining the detail branch feature map by the semantic context fusion module is:

[0029] A1: When the context branch module performs the first downsampling operation to obtain the context feature map, the shallow feature image is used as the current image:

[0030] A2: fusing the current image with the context feature map obtained by the current downsampling operation of the context branch module to obtain a fused image;

[0031] A3: Determine whether the downsampling operation performed by the context branch module is completed;

[0032] If yes, the fused image is used as the detail branch feature map;

[0033] If not, the fused image is used as the current image, and the context feature map obtained by the context branch module in the next downsampling operation is used as the context feature map obtained by the context branch module in the current downsampling operation, and the step A2 is executed again.

[0034] In a possible implementation, the semantic context fusion module includes a linear interpolation unit, a first product unit, a first classification unit, a first fusion unit, and a judgment unit that form a communication network chain structure;

[0035] The linear interpolation unit communicates with the first feature extractor and each of the second feature extractors located at the end, respectively, and the judgment unit communicates with the linear interpolation unit and the segmentation result attention fusion module, respectively;

[0036] In operation:

[0037] The linear interpolation unit obtains an interpolation image of the same scale as the current image by linearly interpolating the context feature map obtained by the context branch module up to the current downsampling operation, and transmits the interpolation image to the first product unit;

[0038] The first product unit obtains a first product image by performing a matrix Hadamard product operation on the interpolation image and the current image, and transmits the first product image to the first classification unit;

[0039] The first classification unit obtains a first classification feature image by performing sigmoid function activation on the first product image, and transmits the first classification feature image to the first fusion unit;

[0040] The first fusion unit obtains a current fusion image based on the first classified feature image, and transmits the fusion image to the judgment unit;

[0041] The judging unit judges whether the second feature extractor at the end performs a downsampling operation;

[0042] If yes, the judgment unit uses the fused image as the detail branch feature map and transmits it to the segmentation result attention fusion module;

[0043] If not, the judging unit takes the fused image as the current image and transmits it to the linear interpolation unit;

[0044] This scheme can ensure the fusion of information with context features, thereby achieving better segmentation results.

[0045] In a possible implementation manner, the step of the first fusion unit obtaining the current fusion image and transmitting the image to the judgment unit is as follows:

[0046] B1: obtaining a first sub-image by subtracting the first classified feature image from the unit matrix;

[0047] B2: Obtaining a second product image by performing a matrix Hadamard product operation on the interpolation image and the first sub-image;

[0048] B3: Obtaining a third product image by performing a matrix Hadamard product operation on the first classified feature image and the current image;

[0049] B4: Obtaining the current fused image by adding the second product image to the third product image, and transmitting the fused image to the judgment unit.

[0050] In a possible implementation, the segmentation result attention fusion module includes a second classification unit, a second product unit, a third product unit and a second fusion unit;

[0051] The second classification unit includes an activator, a first output terminal and a second output terminal;

[0052] The activator communicates with the edge branch module, the second product unit communicates with the first output end and the second feature extractor at the end respectively, the third product unit communicates with the second output end and the judgment unit respectively, and the second fusion unit communicates with the second product unit and the third product unit respectively;

[0053] The activator obtains a second classification feature image by performing sigmoid function activation on the edge branch feature image, and transmits the second classification feature image to the first output end and the second output end respectively;

[0054] The first output terminal obtains a second sub-image by subtracting the second classified feature image from the unit matrix, and transmits the second sub-image to the second product unit;

[0055] The second product unit obtains a fourth product image by performing a matrix Hadamard product operation on the second sub-image and the context feature map obtained by the last downsampling operation of the context branch module, and transmits the fourth product image to the second fusion unit;

[0056] The second output terminal transmits the second classified feature image to the third product unit;

[0057] The third product unit obtains a fifth product image by performing a matrix Hadamard product operation on the second classification feature image and the detail branch feature image, and transmits the fifth product image to the second fusion unit;

[0058] The second fusion unit obtains a semantic segmentation result of the image under test by adding the fourth product image to the fifth product image;

[0059] Since the edge branch can predict the edge of the target very well, the second classified feature image obtained after activating the sigmoid function through the activator contains a lot of edge information, which is reflected in the numerical value that the probability of being considered as the edge is higher, and the value is larger and closer to 1, while the probability of the edge that is not the edge is smaller, and the value is smaller and closer to 0. The Hadamard product of the second classified feature image and the semantic branch result highlights the edges and other details, while the Hadamard product of the second sub-image and the context feature map reflects strong semantic information, thereby achieving the technical effect of accurate semantic segmentation.

[0060] In a possible implementation, the real-time semantic segmentation system further includes a segmentation head, which communicates with the second fusion unit and the output end respectively; this solution ensures the complete implementation of semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the structure of a real-time semantic segmentation system integrating edge feature details disclosed in an embodiment of the present application;

[0062] Figure 2 This is a flowchart of the operation of the semantic context fusion module disclosed in the embodiment of the present application;

[0063] Figure 3 This is a flowchart corresponding to step A2 when the semantic context fusion module disclosed in the embodiment of the present application is in operation;

[0064] Figure 4 This is a schematic diagram of the structure of the segmentation result attention fusion module disclosed in the embodiment of the present application;

[0065] Figure 5 This is a flow chart of the operation of the segmentation result attention fusion module disclosed in the embodiment of the present application;

[0066] Figure 6 The semantic segmentation results of the real-time semantic segmentation system integrating edge feature details disclosed in the embodiment of the present application for the white-background photo images of pet cats and pet dogs;

[0067] Figure 7 The semantic segmentation result of a photo image of a pet cat and a pet dog in a scene performed by the real-time semantic segmentation system integrating edge feature details disclosed in an embodiment of the present application;

[0068] Figure 8 The real-time semantic segmentation system integrating edge feature details disclosed in the embodiment of the present application performs semantic segmentation on a photo image of a pet cat and a pet dog in another scene.

[0069] Description of reference numerals:

[0070] 1. Input end, 2. Output end, 3. Shallow feature sharing module, 31. First feature extractor, 4. Context branch module, 41. Second feature extractor, 5. Semantic context fusion module, 51. Linear interpolation unit, 52. First product unit, 53. First classification unit, 54. First fusion unit, 55. Judgment unit, 6. Edge branch module, 7. Segmentation result attention fusion module, 71. Second classification unit, 711. Activator, 712. First output end, 713. Second output end, 72. Second product unit, 73. Third product unit, 74. Second fusion unit, 8. Segmentation head. DETAILED DESCRIPTION

[0071] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present application, and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0072] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the term "communication network link structure" refers to a network structure or a network link structure formed by multiple communication elements, units or modules involved through communication connections, and communication or communication connection refers to the transmission of information between the first feature and the second feature. This information transmission can be either unidirectional or bidirectional, and the communication connection can be achieved by electrical connection of wires, radio connection, electrical connection of electromagnetic media (such as semiconductors), communication achieved by channels, etc.

[0073] The present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] See also Figure 1 to Figure 5 As shown, the embodiment of the present application discloses a real-time semantic segmentation system integrating edge feature details, see Figure 1 As shown, the real-time semantic segmentation system includes an input terminal 1, an output terminal 2, a shallow feature sharing module 3, a context branch module 4, a semantic context fusion module 5, an edge branch module 6, a segmentation result attention fusion module 7 and a segmentation head 8. Among them, the shallow feature sharing module 3 communicates with the input terminal 1, the segmentation result attention fusion module 7 communicates with the output terminal 2, and the context branch module 4 communicates with the semantic context fusion module 5 and the edge branch module 6 respectively; the shallow feature sharing module 3, the context branch module 4, and the segmentation result attention fusion module 7 form a first communication network branch; the shallow feature sharing module 3, the semantic context fusion module 5, and the segmentation result attention fusion module 7 form a second communication network branch; the shallow feature sharing module 3, the edge branch module 6, and the segmentation result attention fusion module 7 form a third communication network branch.

[0075] Please continue to see Figure 1 As shown, in this embodiment, the shallow feature sharing module 3 is configured to extract shallow features from the image under test in a multiple downsampling manner to obtain a shallow feature image. Specifically, the shallow feature sharing module 3 is a communication network chain structure formed by a plurality of first feature extractors 31 connected in series, each first feature extractor 31 completes a downsampling operation, the first feature extractor 31 at the head end communicates with the input end 1, and the first feature extractor 31 at the end communicates with the context branch module 4, the semantic context fusion module 5, and the edge branch module 6 respectively. Furthermore, the first feature extractor 31 performs a downsampling, which can achieve the technical effect that three communication network branches share the same feature map at the same time, thereby simplifying the design and improving the reasoning speed.

[0076] Please continue to see Figure 1As shown, in this embodiment, the context branch module 4 is configured to perform several downsampling operations on the shallow feature image to complete the semantic information retention, and obtain the context feature map corresponding to each downsampling operation. Specifically, the context branch module 4 is a communication network chain structure formed by a plurality of second feature extractors 41 connected in series. The second feature extractor 41 at the head end communicates with the first feature extractor 31 at the end, and the second feature extractor 41 at the end communicates with the segmentation result attention fusion module 7. All the second feature extractors 41 communicate with the semantic context fusion module 5 and the edge branch module 6 respectively. Each second feature extractor 41 completes one downsampling to obtain the corresponding context feature map.

[0077] Please continue to see Figure 1 As shown, in this embodiment, the semantic context fusion module 5 includes a linear interpolation unit 51, a first product unit 52, a first classification unit 53, a first fusion unit 54 and a judgment unit 55 forming a communication network chain structure. Among them, the linear interpolation unit 51 communicates with the first feature extractor 31 and each second feature extractor 41 located at the end, respectively, and the judgment unit 55 communicates with the linear interpolation unit 51 and the segmentation result attention fusion module 7 respectively.

[0078] See also Figure 2 As shown, in this embodiment, the semantic context fusion module 5 operates as follows:

[0079] A1: When the context branch module 4 performs the first downsampling operation to obtain the context feature map, that is, when the second feature extractor 41 at the head end performs the downsampling operation to obtain the context feature map and is acquired by the linear interpolation unit 51, the linear interpolation unit 51 uses the shallow feature image as the current image, and the linear interpolation unit 51 uses the context feature map obtained by the context branch module 4 performing the first downsampling operation as the context feature map obtained by the context branch module 4 performing the current downsampling operation;

[0080] A2: Perform information fusion on the current image and the context feature map obtained by the current downsampling operation performed by the context branch module 4 (that is, the context feature map obtained by the current downsampling operation performed by the second feature extractor 41) to obtain the current fused image.

[0081] See also Figure 3 As shown, the specific steps of this step are as follows:

[0082] A21: the linear interpolation unit 51 obtains an interpolation image of the same scale as the current image by linearly interpolating the context feature map obtained by the context branch module 4 up to the current downsampling operation, and transmits the interpolation image to the first product unit 52;

[0083] A22: The first product unit 52 obtains a first product image by performing a matrix Hadamard product operation on the interpolation image and the current image, and transmits the first product image to the first classification unit 53;

[0084] A23: The first classification unit 53 obtains a first classification feature image by performing sigmoid function activation on the first product image, and transmits the first classification feature image to the first fusion unit 54;

[0085] A24: The first fusion unit 54 obtains a first sub-image by subtracting the first classified feature image from the unit matrix;

[0086] A25: The first fusion unit 54 obtains a second product image by performing a matrix Hadamard product operation on the interpolation image and the first sub-image;

[0087] A26: The first fusion unit 54 obtains a third product image by performing a matrix Hadamard product operation on the first classified feature image and the current image;

[0088] A27: The first fusion unit 54 obtains the current fusion image by adding the second product image to the third product image, and transmits the result to the judgment unit 55.

[0089] A3: Determine whether the downsampling operation performed by the context branch module 4 is completed, that is, the determination unit 55 determines whether the second feature extractor 41 at the end performs the downsampling operation;

[0090] If so, the judgment unit 55 uses the fused image as the detail branch feature map and transmits it to the segmentation result attention fusion module 7; if not, the judgment unit 55 uses the fused image as the current image and transmits it to the linear interpolation unit 51, and the linear interpolation unit 51 uses the context feature map obtained by the context branch module 4 for the next downsampling operation (that is, the context feature map obtained by the next downsampling operation of the second feature extractor 41) as the context feature map obtained by the context branch module 4 for the current downsampling operation, and returns to execute step A21.

[0091] Please continue to see Figure 1 As shown, in this embodiment, the edge branch module 6 is configured to extract edge information from the context feature maps corresponding to each downsampling operation obtained by the context branch module 4 (i.e., the context feature maps obtained by each second feature extractor 41 performing downsampling operations) in sequence, and obtain the edge branch feature map by fusion. The edge branch module 6 can extract high-frequency features to predict the boundary area.

[0092] See also Figure 1 , Figure 4 and Figure 5As shown, in this embodiment, the segmentation result attention fusion module 7 is configured to segment and fuse the context feature map, detail branch feature map and edge branch feature map obtained by the last downsampling operation of the context branch module 4 to obtain the semantic segmentation result of the image under test.

[0093] For details, see Figure 1 and Figure 4 As shown, in this embodiment, the segmentation result attention fusion module 7 includes a second classification unit 71, a second product unit 72, a third product unit 73 and a second fusion unit 74. The second classification unit 71 includes an activator 711, a first output terminal 712 and a second output terminal 713. Among them, the activator 711 communicates with the edge branch module 6, the second product unit 72 communicates with the first output terminal 712 and the second feature extractor 41 at the end, the third product unit 73 communicates with the second output terminal 713 and the judgment unit 55, and the second fusion unit 74 communicates with the second product unit 72 and the third product unit 73. The segmentation head 8 communicates with the second fusion unit 74 and the output terminal 2 respectively.

[0094] See also Figure 5 As shown, in this embodiment, the operation mode of the segmentation result attention fusion module 7 is as follows:

[0095] The activator 711 obtains a second classified feature image by performing sigmoid function activation on the edge branch feature image, and transmits the second classified feature image to the first output terminal 712 and the second output terminal 713 respectively;

[0096] The first output terminal 712 obtains the second sub-image by subtracting the second classified feature image from the unit matrix, and transmits the second sub-image to the second product unit 72;

[0097] The second product unit 72 obtains a fourth product image by performing a matrix Hadamard product operation on the second sub-image and the context feature map obtained by the last downsampling operation of the context branch module 4, and transmits the fourth product image to the second fusion unit 74;

[0098] The second output terminal 713 transmits the second classified feature image to the third product unit 73;

[0099] The third product unit 73 obtains a fifth product image by performing a matrix Hadamard product operation on the second classification feature image and the detail branch feature image, and transmits the fifth product image to the second fusion unit 74;

[0100] The second fusion unit 74 obtains the semantic segmentation result of the measured image by adding the fourth product image and the fifth product image.

[0101] The real-time semantic segmentation system for integrating edge feature details disclosed in the embodiment of the present application innovatively adds an edge branch module on the basis of the dual-branch segmentation network of the prior art, so that the final result of the semantic segmentation is more excellent in terms of edge and detail performance. By setting a shallow feature sharing module 3, the method of shallow feature sharing is used to further reduce the number of parameters of the model, so that the overall system has a faster reasoning speed, which greatly facilitates the deployment of the model. At the same time, by setting a semantic context attention fusion module 5 and a segmentation result attention fusion module 7, context information can be better taken into account during feature fusion, so that the model has higher accuracy and better performance.

[0102] Figure 6 The semantic segmentation effect of the real-time semantic segmentation system integrating edge feature details disclosed in the embodiment of the present application on white-background photo images of pet cats and pet dogs is demonstrated. Figure 6 (a) in the figure is the original image of this photo. Figure 6 (b) is the segmentation result of this photo by the real-time semantic segmentation system of this embodiment. Figure 6 (c) shows the result of applying a simple mask to this photo.

[0103] Figure 7 The real-time semantic segmentation system of the embodiment of the present application shows the semantic segmentation effect of a photo image of a pet cat and a pet dog in a scene. Figure 7 (a) in the figure is the original image of this photo. Figure 7 (b) is the segmentation result of this photo by the real-time semantic segmentation system of this embodiment. Figure 7 (c) shows the result of applying a simple mask to this photo.

[0104] Figure 8 The real-time semantic segmentation system of the embodiment of the present application shows the semantic segmentation effect of a photo image of a pet cat and a pet dog in another scene. Figure 8 (a) in the figure is the original image of this photo. Figure 8 (b) is the segmentation result of this photo by the real-time semantic segmentation system of this embodiment. Figure 8 (c) shows the result of applying a simple mask to this photo.

[0105] By observation Figure 6 , Figure 7 and Figure 8It is not difficult to find that the semantic segmentation system of this embodiment has a good effect on edge detection and can distinguish the edges of the target well, thus overcoming the problem of inaccurate boundary segmentation in the prior art. At the same time, experimental analysis shows that with the cooperation of the shallow feature sharing module 3, the semantic context fusion module 5, and the segmentation result attention fusion module 7, the speed of the corresponding algorithm of the real-time semantic segmentation system is increased by about 30% compared with BiseNet V2.

[0106] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A real-time semantic segmentation system integrating edge feature details, comprising an input end (1) and an output end (2), characterized in that: The real-time semantic segmentation system also includes: A shallow feature sharing module (3) extracts shallow features from the image under test by multiple downsampling to obtain a shallow feature image; A context branch module (4) performs several downsampling operations on the shallow feature image to complete semantic information retention, and obtains context feature maps corresponding to each downsampling operation; A semantic context fusion module (5) is used to obtain a detail branch feature map by fusing the shallow feature image with the context feature maps corresponding to each downsampling operation; An edge branch module (6) extracts edge information from the context feature maps corresponding to each downsampling operation obtained by the context branch module (4) in sequence, and obtains an edge branch feature map by fusion; The segmentation result attention fusion module (7) performs segmentation fusion on the context feature map, the detail branch feature map and the edge branch feature map obtained by the last downsampling operation of the context branch module (4) to obtain a semantic segmentation result of the image under test; wherein: The shallow feature sharing module (3) communicates with the input end (1), the segmentation result attention fusion module (7) communicates with the output end (2), and the context branch module (4) communicates with the semantic context fusion module (5) and the edge branch module (6) respectively; The shallow feature sharing module (3), the context branch module (4), and the segmentation result attention fusion module (7) form a first communication network branch; The shallow feature sharing module (3), the semantic context fusion module (5), and the segmentation result attention fusion module (7) form a second communication network branch; The shallow feature sharing module (3), the edge branch module (6), and the segmentation result attention fusion module (7) form a third communication network branch; The shallow feature sharing module (3) is a communication network chain structure formed by a plurality of first feature extractors (31) connected in series, wherein the first feature extractor (31) at the head end communicates with the input end (1), and the first feature extractor (31) at the end end communicates with the context branch module (4), the semantic context fusion module (5), and the edge branch module (6) respectively; The context branch module (4) is a communication network chain structure formed by a plurality of second feature extractors (41) connected in series, wherein the second feature extractor (41) at the head end communicates with the first feature extractor (31) at the end end, the second feature extractor (41) at the end end communicates with the segmentation result attention fusion module (7), and all the second feature extractors (41) communicate with the semantic context fusion module (5) and the edge branch module (6) respectively; The semantic context fusion module (5) comprises a linear interpolation unit (51) for forming a communication network chain structure, a first product unit (52), a first classification unit (53), a first fusion unit (54) and a judgment unit (55); The linear interpolation unit (51) communicates with the first feature extractor (31) and each of the second feature extractors (41) located at the end, respectively; and the judgment unit (55) communicates with the linear interpolation unit (51) and the segmentation result attention fusion module (7), respectively; The segmentation result attention fusion module (7) comprises a second classification unit (71), a second product unit (72), a third product unit (73) and a second fusion unit (74); The second classification unit (71) comprises an activator (711), a first output end (712) and a second output end (713); The activator (711) communicates with the edge branch module (6), the second product unit (72) communicates with the first output end (712) and the second feature extractor (41) located at the end respectively, the third product unit (73) communicates with the second output end (713) and the judgment unit (55) respectively, and the second fusion unit (74) communicates with the second product unit (72) and the third product unit (73) respectively.

2. The real-time semantic segmentation system integrating edge feature details according to claim 1, characterized in that: The steps of obtaining the detail branch feature map by the semantic context fusion module (5) are as follows: A1: When the context branch module (4) performs the first downsampling operation to obtain the context feature map, the shallow feature image is used as the current image: A2: fusing the current image with the context feature map obtained by the context branch module (4) in the current downsampling operation to obtain a fused image; A3: Determine whether the downsampling operation performed by the context branch module (4) is completed; If yes, the fused image is used as the detail branch feature map; If not, the fused image is used as the current image, and the context feature map obtained by the context branch module (4) in the next downsampling operation is used as the context feature map obtained by the context branch module (4) in the current downsampling operation, and the step A2 is executed again.

3. The real-time semantic segmentation system integrating edge feature details according to claim 2, characterized in that: The semantic context fusion module (5) is configured to: The linear interpolation unit (51) obtains an interpolation image of the same scale as the current image by linearly interpolating the context feature map obtained by the context branch module (4) up to the current downsampling operation, and transmits the interpolation image to the first product unit (52); The first product unit (52) obtains a first product image by performing a matrix Hadamard product operation on the interpolation image and the current image, and transmits the first product image to the first classification unit (53); The first classification unit (53) obtains a first classification feature image by performing sigmoid function activation on the first product image, and transmits the first classification feature image to the first fusion unit (54); The first fusion unit (54) obtains a current fusion image based on the first classified feature image, and transmits the image to the judgment unit (55); The judging unit (55) judges whether the second feature extractor (41) located at the end performs a downsampling operation; If so, the judgment unit (55) uses the fused image as the detail branch feature map and transmits it to the segmentation result attention fusion module (7); If not, the judgment unit (55) uses the current fused image as the current image and transmits it to the linear interpolation unit (51).

4. The real-time semantic segmentation system integrating edge feature details according to claim 3, characterized in that: The first fusion unit (54) obtains the current fusion image and transmits it to the judgment unit (55) in the following steps: B1: obtaining a first sub-image by subtracting the first classified feature image from the unit matrix; B2: Obtaining a second product image by performing a matrix Hadamard product operation on the interpolation image and the first sub-image; B3: Obtaining a third product image by performing a matrix Hadamard product operation on the first classified feature image and the current image; B4: Obtaining the current fused image by adding the second product image to the third product image, and transmitting the fused image to the judgment unit (55).

5. The real-time semantic segmentation system integrating edge feature details according to claim 4, characterized in that: The segmentation result attention fusion module (7) is set as: The activator (711) obtains a second classification feature image by performing sigmoid function activation on the edge branch feature image, and transmits the second classification feature image to the first output end (712) and the second output end (713) respectively; The first output end (712) obtains a second sub-image by subtracting the second classified feature image from the unit matrix, and transmits the second sub-image to the second product unit (72); The second product unit (72) obtains a fourth product image by performing a matrix Hadamard product operation on the second sub-image and the context feature map obtained by the last downsampling operation of the context branch module (4), and transmits the fourth product image to the second fusion unit (74); The second output terminal (713) transmits the second classified feature image to the third product unit (73); The third product unit (73) obtains a fifth product image by performing a matrix Hadamard product operation on the second classification feature image and the detail branch feature image, and transmits the fifth product image to the second fusion unit (74); The second fusion unit (74) obtains the semantic segmentation result of the image under test by adding the fourth product image and the fifth product image.

6. The real-time semantic segmentation system integrating edge feature details according to claim 5, characterized in that: The real-time semantic segmentation system further comprises a segmentation head (8), wherein the segmentation head (8) communicates with the second fusion unit (74) and the output end (2) respectively.

Images