Method and system for referential image segmentation based on edge enhancement and bottleneck vectors

Abstract

The application discloses an edge enhancement and bottleneck vector-based referential image segmentation method and system, relates to the technical field of referential image segmentation, and comprises the following steps: training an incremental cross-modal bottleneck fusion network based on a plurality of sample images and text descriptions of each sample image; in the incremental cross-modal bottleneck fusion network, embedding and representing the text description of a preset sample image through a text encoder; in the process of multi-stage visual extraction of the preset sample image by a visual encoder, introducing the embedding and representation result into the visual features at each visual extraction stage by using a cross-modal attention fusion module; in the process of multi-stage decoding of all visual features by a decoder, generating local edge enhanced features at multiple decoding stages by using a local edge refinement module; and the incremental cross-modal bottleneck fusion network trained can accurately perform referential image segmentation on a target image.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to an image segmentation method and system based on edge enhancement and bottleneck vectors. Background Technology

[0002] In recent years, with breakthroughs in deep learning and intelligent sensing technologies, research on refined image analysis has become a hot topic. The demand for practical applications of this technology in fields such as medical image diagnosis, autonomous driving, industrial quality inspection, and remote sensing ground object analysis continues to rise.

[0003] Referencing image segmentation (RIS) is a key task in computer vision. Its core objective is to achieve pixel-level region segmentation and accurate region classification by integrating multimodal features, contextual semantics, and object boundary-sensitive information. This technique must effectively distinguish foreground objects from complex backgrounds while preserving the original spatial structure of the image, accurately locating and segmenting semantically specific entities (such as biological organs, road obstacles, industrial defects, or ground cover) from complex scene images with pixel-level precision. These segmentation results contain both the specific contours of the objects and rich semantic hierarchical information. Therefore, referring image segmentation, as a crucial visual processing module, provides important structured data support for subsequent research in object recognition, 3D reconstruction, scene semantic understanding, and intelligent decision-making.

[0004] Based on different model architectures, existing algebraic image segmentation techniques are mainly divided into:

[0005] 1) Two-stage segmentation technique: In the first stage, the bounding box of the target is located based on the text description. After obtaining the bounding box of the target, the target within the box is further segmented. However, this method is too cumbersome to implement, and the segmentation performance is heavily dependent on whether the bounding box can accurately select the target.

[0006] 2) End-to-end segmentation techniques can directly learn features from the original input data and output segmentation masks without requiring manual feature extractor design, thus significantly improving efficiency. Early methods employed a CNN-LSTM architecture, using Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTMs) to extract features from two modalities respectively. The resulting feature vectors from the two modalities were then aligned using a multimodal feature alignment module to achieve semantic consistency. After feature alignment, the aligned features were finally used by an existing semantic segmentation network to predict the final segmentation result. However, the inherent limitations of CNNs and LSTMs make it difficult for this architecture to handle long-range interactions between semantic entities within a modality, hindering accurate segmentation results. Furthermore, the multimodal feature fusion strategy in end-to-end models plays a crucial role in the accuracy of the results. Traditional methods such as concatenating and multiplying the deepest multimodal features are insufficient for finely segmenting all targets when dealing with multi-scale, complex boundary-based referential descriptions. Attention-based network frameworks can selectively fuse effective information between modalities to improve segmentation results, but current public datasets lack corresponding attention annotation information, making it impossible to guarantee the learning of correct attention weights. Transformer-based fusion methods can achieve long-range interactions between semantic entities within a modality, but they cannot flexibly handle a large number of complex segmentation scenarios and are prone to introducing a large amount of irrelevant information, resulting in suboptimal performance and making it difficult for current technologies to achieve ideal detection results. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the shortcomings of existing technologies. Specifically, it provides a method and system for image segmentation based on edge enhancement and bottleneck vectors, as detailed below:

[0008] 1) In a first aspect, the present invention provides a method for representing image segmentation based on edge enhancement and bottleneck vectors, the specific technical solution of which is as follows:

[0009] Based on multiple sample images and their text descriptions, a progressive cross-modal bottleneck fusion network is trained to obtain the trained network. This network comprises a text encoder, a visual encoder, a cross-modal attention fusion module, a local edge refinement module, and a decoder. The text encoder embeds the text descriptions of the preset sample images to obtain the embedding representation. During the multi-stage visual extraction of the preset sample images by the visual encoder, the cross-modal attention fusion module incorporates the embedding representation into the visual features of each extraction stage, resulting in fused visual features for each stage. During the multi-stage decoding of all visual features by the decoder, the local edge refinement module generates locally enhanced features for multiple decoding stages. These enhanced features are used as inputs for the next decoding stage until the output of the final decoder is obtained. Based on the output of the final decoder, a target segmentation map corresponding to the preset sample image is generated. The preset sample image can be any sample image.

[0010] The target image and its text description are input into a trained progressive cross-modal bottleneck fusion network to obtain the segmentation result of the reference image corresponding to the target image.

[0011] The beneficial effects of the reference image segmentation method based on edge enhancement and bottleneck vectors provided by this invention are as follows: Compared with the cumbersome process and dependence on bounding boxes faced by two-stage segmentation techniques, the progressive cross-modal bottleneck fusion network adopts a more efficient training and operation method. It embeds text descriptions into a text encoder and performs multi-stage visual extraction of the image using a visual encoder. Furthermore, it introduces the embedded representation results into the visual features of each visual extraction stage, achieving a progressive fusion of text and visual information. This avoids the cumbersome step of locating bounding boxes before segmentation and reduces the performance degradation caused by inaccurate bounding box localization. Moreover, the cross-modal attention fusion module in the progressive cross-modal bottleneck fusion network can selectively fuse effective information between modalities at multiple stages, and gradually generate local edge-enhanced features during the decoding process through the local edge refinement module. Thus, when facing multi-scale, complex boundary representations, it can more accurately segment the target, meet the needs of complex segmentation scenarios, improve the accuracy and precision of representation image segmentation, overcome the shortcomings of existing technologies that easily introduce irrelevant information when dealing with complex scenarios, resulting in suboptimal performance, and provide a better solution for representation image segmentation tasks. It is expected to promote the development of this field towards a more efficient and accurate direction.

[0012] Based on the above scheme, the reference image segmentation method based on edge enhancement and bottleneck vector of the present invention can be further improved as follows.

[0013] Furthermore, the embedding representation results include lexical-level embeddings, and the visual encoder includes N sequentially arranged network layers. During the multi-stage visual extraction process of the preset sample image by the visual encoder, the cross-modal attention fusion module incorporates the embedding representation results into the visual features of each visual extraction stage, obtaining the fused visual features of each visual extraction stage, including:

[0014] After obtaining the visual features of the first visual extraction stage through the first network layer, the first cross-modal attention fusion module fuses the word-level embeddings and the visual features of the first visual extraction stage based on attention and bottleneck vector mechanisms to obtain the first guiding feature. Contextual information features are extracted from the first guiding feature based on a gating mechanism, and then fused with the visual features of the first visual extraction stage to obtain the fused visual features of the first visual extraction stage. The fused visual features of the first visual extraction stage are used as input to the second network layer to obtain the visual features of the second visual extraction stage. The second cross-modal attention fusion module then fuses the word-level embeddings and the visual features of the second visual extraction stage based on an attention mechanism until the fused visual features of each visual extraction stage are obtained.

[0015] The beneficial effects of adopting the above-mentioned further scheme are as follows: By introducing a cross-modal attention fusion module into each network layer of the visual encoder, a progressive fusion of textual lexical embeddings and visual features is achieved. In the first network layer, the cross-modal attention fusion module effectively fuses lexical embeddings and visual features based on attention and bottleneck vector mechanisms to generate the first guiding feature. Subsequently, a gating mechanism extracts contextual information features from the guiding feature and adds it to the original visual features to obtain the fused visual feature, which serves as the input to the next network layer. This layer-by-layer fusion approach allows textual information to progressively guide and optimize the extraction process of visual features, avoiding the information loss or overload problems caused by one-time fusion in traditional methods. By introducing textual guidance at each stage, the sensitivity of visual features to semantic information is enhanced, improving the accuracy and effectiveness of cross-modal feature fusion, thereby improving the accuracy and robustness of referential image segmentation.

[0016] Furthermore, the features for local edge enhancement generated through the local edge refinement module for multiple decoding stages include: features for local edge enhancement in each decoding stage or features for local edge enhancement in some decoding stages;

[0017] When the local edge refinement module generates local edge enhancement features for multiple decoding stages, including features from some decoding stages, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features from each decoding stage are used as input for the next decoding stage until the output of the final decoder is obtained, including:

[0018] The fused visual features from the Nth visual extraction stage and the guiding features generated by the Nth cross-modal attention fusion module are input into the first decoder. The output of the first decoder and the guiding features generated by the (N-1)th cross-modal attention fusion module are input into the second decoder. This process continues until the nth decoder is reached. Then, the output of the nth decoder and the visual features from the Nnth visual extraction stage are input into the first local edge refinement module to generate the first local edge enhancement feature. The first local edge enhancement feature and the guiding features generated by the Nnth cross-modal attention fusion module are input into the (n+1)th decoder. This process continues until the output of the Nth decoder is obtained. The Nth decoder is the last decoder.

[0019] The beneficial effects of adopting the above-mentioned further scheme are as follows: By introducing local edge refinement modules in multiple decoding stages, the edge features of the target can be progressively enhanced, making the boundaries of the segmentation results clearer and more accurate, reducing blurring and omissions. Using the locally enhanced edge features as input to subsequent decoding stages enables progressive feature optimization, gradually improving segmentation quality and ensuring that the final output segmentation map has higher detail and accuracy. Combined with the guided features generated by the cross-modal attention fusion module, textual information and visual features can be better fused, improving the segmentation ability for complex scenes and multi-scale targets, and enhancing the model's adaptability to different target shapes and boundaries. By performing local edge enhancement in some decoding stages, the accumulation and amplification of erroneous information throughout the decoding process is avoided, improving the model's robustness and stability.

[0020] Furthermore, the local edge refinement module is used to: perform bi-branch convolution operations on the received visual features to generate two intermediate features; process one of the intermediate features and the output of the received decoder based on the residual space attention mechanism to obtain a first processing result; process the other intermediate feature and the first processing result based on the residual space attention mechanism and the residual channel attention mechanism to obtain a second processing result; determine edge information based on the first and second processing results; and adaptively fuse the edge information with the two intermediate features to generate features with enhanced local edges.

[0021] The beneficial effects of adopting the above-mentioned further scheme are as follows: This local edge refinement module generates two intermediate features through bi-branch convolution operations, and then extracts and fuses edge information based on the residual space attention mechanism. This makes the segmentation boundaries clearer and more detailed, effectively capturing complex shapes and improving segmentation accuracy. Simultaneously, the adaptive fusion mechanism automatically adjusts the fusion degree according to the importance of edge information, retaining more effective information and enhancing the robustness and stability of segmentation. Furthermore, this module can highlight the boundary between the target and the background, reducing blurring or adhesion problems, which is beneficial for accurate judgment and analysis in post-processing.

[0022] Furthermore, it also includes: using cross-entropy loss for supervision during the training of the progressive cross-modal bottleneck fusion network.

[0023] The beneficial effects of adopting the above-mentioned further scheme are as follows: Cross-entropy loss provides an effective supervision signal, helping the network learn the alignment relationship between text and visual features, thereby improving the accuracy of target segmentation. By minimizing cross-entropy loss, the network can better optimize the classification of target and background regions, improving the quality of segmentation results. The cross-entropy loss function provides a stable gradient during training, which helps the network converge stably and avoids fluctuations and overfitting problems during training. Through the supervision of cross-entropy loss, the model can learn more discriminative feature representations, thereby improving its generalization ability in different scenarios and enabling it to perform well on unseen data.

[0024] 2) In a second aspect, the present invention also provides a referential image segmentation system based on edge enhancement and bottleneck vectors, the specific technical solution of which is as follows:

[0025] Includes a model training module and a referential image segmentation module;

[0026] The model training module is used to: train a progressive cross-modal bottleneck fusion network based on multiple sample images and the text description of each sample image, resulting in a trained progressive cross-modal bottleneck fusion network. This network includes a text encoder, a visual encoder, a cross-modal attention fusion module, a local edge refinement module, and a decoder. The text encoder embeds the text description of the preset sample images to obtain the embedding representation. During the multi-stage visual extraction of the preset sample images by the visual encoder, the cross-modal attention fusion module incorporates the embedding representation into the visual features of each visual extraction stage, resulting in fused visual features for each stage. During the multi-stage decoding of all visual features by the decoder, the local edge refinement module generates local edge enhancement features for multiple decoding stages. These features are used as inputs for the next decoding stage until the output of the final decoder is obtained. Based on the output of the final decoder, a target segmentation map corresponding to the preset sample image is generated. The preset sample image can be any sample image.

[0027] The referential image segmentation module is used to input the target image and its text description into a trained progressive cross-modal bottleneck fusion network to obtain the referential image segmentation result corresponding to the target image.

[0028] Based on the above scheme, the reference image segmentation system based on edge enhancement and bottleneck vector of the present invention can be further improved as follows.

[0029] Furthermore, the embedding representation results include lexical embeddings, and the visual encoder includes N sequentially arranged network layers. After obtaining the visual features of the first visual extraction stage through the first network layer, the first cross-modal attention fusion module fuses the lexical embeddings and the visual features of the first visual extraction stage based on attention and bottleneck vector mechanisms to obtain the first guiding feature. Contextual information features are extracted from the first guiding feature based on a gating mechanism, and the contextual information features in the first guiding feature are fused with the visual features of the first visual extraction stage to obtain the fused visual features of the first visual extraction stage. The fused visual features of the first visual extraction stage are used as the input of the second network layer, and the visual features of the second visual extraction stage are obtained through the second network layer. The second cross-modal attention fusion module fuses the lexical embeddings and the visual features of the second visual extraction stage based on an attention mechanism until the fused visual features of each visual extraction stage are obtained.

[0030] Furthermore, the features for local edge enhancement generated by the local edge refinement module for multiple decoding stages include: features for local edge enhancement in each decoding stage or features for local edge enhancement in some decoding stages. When the features for local edge enhancement generated by the local edge refinement module for multiple decoding stages include features for local edge enhancement in some decoding stages, the fused visual features of the Nth visual extraction stage and the guiding features generated by the Nth cross-modal attention fusion module are input into the first decoder. The output of the first decoder and the guiding features generated by the (N-1)th cross-modal attention fusion module are input into the second decoder. This process continues until the nth decoder is reached. Then, the output of the nth decoder and the visual features of the Nnth visual extraction stage are input into the first local edge refinement module to generate the first local edge enhancement feature. The first local edge enhancement feature and the guiding features generated by the Nnth cross-modal attention fusion module are input into the (n+1)th decoder. This process continues until the output of the Nth decoder is obtained. The Nth decoder is the last decoder.

[0031] Furthermore, the local edge refinement module is used to: perform bi-branch convolution operations on the received visual features to generate two intermediate features; process one of the intermediate features and the output of the received decoder based on the residual space attention mechanism to obtain a first processing result; process the other intermediate feature and the first processing result based on the residual space attention mechanism and the residual channel attention mechanism to obtain a second processing result; determine edge information based on the first processing result and the second processing result; and adaptively fuse the edge information with the two intermediate features to generate features with enhanced local edges.

[0032] Furthermore, the model training module is also used to supervise the training of the progressive cross-modal bottleneck fusion network using cross-entropy loss.

[0033] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the electronic device to implement any of the above-mentioned reference image segmentation methods based on edge enhancement and bottleneck vectors.

[0034] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned referential image segmentation methods based on edge enhancement and bottleneck vectors.

[0035] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below:

[0037] Figure 1 This is a flowchart illustrating a method for representing image segmentation based on edge enhancement and bottleneck vectors according to an embodiment of the present invention.

[0038] Figure 2 A schematic diagram of the framework for a progressive cross-modal bottleneck fusion network;

[0039] Figure 3 This is a schematic diagram of the framework of a cross-modal attention fusion module based on bottleneck vectors;

[0040] Figure 4 A schematic diagram of the framework for the local edge refinement module;

[0041] Figure 5 An introductory diagram for international standard datasets;

[0042] Figure 6 A diagram illustrating the evaluation metrics used to assess performance;

[0043] Figure 7 An introductory diagram for the comparison model;

[0044] Figure 8 This is a comparison chart of the detection accuracy of the model of this invention;

[0045] Figure 9 This is a schematic diagram of the structure of a denotation image segmentation system based on edge enhancement and bottleneck vector according to an embodiment of the present invention;

[0046] Figure 10 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0047] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0048] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0049] like Figure 1 As shown, an embodiment of the present invention provides a method for representing image segmentation based on edge enhancement and bottleneck vectors, comprising the following steps:

[0050] S1. Based on multiple sample images and the text description of each sample image, a progressive cross-modal bottleneck fusion network is trained to obtain a trained progressive cross-modal bottleneck fusion network. The progressive cross-modal bottleneck fusion network includes a text encoder, a visual encoder, a cross-modal attention fusion module, a local edge refinement module, and a decoder. The text encoder embeds the text description of the preset sample images to obtain the embedding representation result. During the multi-stage visual extraction of the preset sample images by the visual encoder, the cross-modal attention fusion module introduces the embedding representation result into the visual features of each visual extraction stage to obtain the fused visual features of each visual extraction stage. During the multi-stage decoding of all visual features by the decoder, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features of each decoding stage are used as the input of the next decoding stage until the output of the last decoder is obtained. The target segmentation map corresponding to the preset sample image is generated based on the output of the last decoder. The preset sample image can be any sample image.

[0051] The text encoder can be Word2Vec, BERT, or GloVe, and the visual encoder can be CNN, ResNet, or ViT. Alternatively, a text encoder and a visual encoder can be selected based on the actual situation.

[0052] S2. Input the target image and its text description into the trained progressive cross-modal bottleneck fusion network to obtain the image segmentation result corresponding to the target image.

[0053] The target image can be from the fields of intelligent medical diagnosis, autonomous driving, security monitoring and public safety, or augmented reality (AR) and interaction design. Images from other fields can also be selected as target images depending on the specific circumstances.

[0054] 1) In the field of intelligent medical diagnosis, the target image is a medical examination image, such as CT images, MRI images, and ultrasound images. These images can clearly show the structure and lesions of internal organs and tissues, serving as important evidence for doctors' diagnosis and treatment. The text description could be: "3cm nodule in the lower lobe of the left lung." This text description accurately points out the location and general characteristics of the lesion, providing a clear segmentation target for the trained progressive cross-modal bottleneck fusion network. The image segmentation result refers to the precise segmentation and marking of the corresponding lesion region in the medical examination image. For example, accurately delineating the boundary of the 3cm nodule region in the lower lobe of the left lung on a CT image, distinguishing it from other normal tissues, and generating an image or data representation with a lesion region mask. This image segmentation result can assist doctors in more accurately locating and determining the extent of the lesion, thus providing a more intuitive and accurate reference for surgical diagnosis, helping doctors develop more reasonable surgical plans, improving the success rate and safety of surgery, and also facilitating further analysis and research of the lesion, such as determining the nature of the lesion and assessing the degree of disease progression.

[0055] 2) In the field of autonomous driving, the target image is a real-time image of the road scene captured by onboard cameras and other devices during vehicle operation, including various traffic elements such as lanes, vehicles, pedestrians, and obstacles. The text description is "a motorcycle approaching from the right lane." This text description enables the trained progressive cross-modal bottleneck fusion network to focus on identifying and segmenting specific high-risk targets to cope with complex traffic conditions. The image segmentation result refers to the real-time and accurate segmentation of the motorcycle approaching from the right lane in the road scene image, obtaining the motorcycle's region mask or bounding box, distinguishing it from other vehicles, lane background, etc., forming a clear segmentation result that can be used as input for subsequent decision-making and control modules. This image segmentation result provides crucial target information for the autonomous vehicle's decision-making system, enabling it to promptly identify high-risk targets and take corresponding avoidance actions, such as changing lanes in advance or slowing down, thereby effectively improving the safety of autonomous driving, reducing traffic accidents, and ensuring the safety of passengers and other road users.

[0056] 3) In the field of security monitoring and public safety, the target image refers to video frame images captured by security monitoring cameras. These images cover various monitoring scenarios, such as public places, areas around important facilities, and major traffic arteries, and can reflect the dynamics of people and objects within the monitored area in real time. The text description, "a person wearing a black coat," provides the target person features to be found and segmented for the trained progressive cross-modal bottleneck fusion network, enabling rapid location among numerous monitoring images. The image segmentation result refers to accurately segmenting the region of the person wearing a black coat within the monitoring video frame, separating it from other irrelevant personnel and background environment, resulting in a segmentation result containing only the region of the person wearing a black coat. This can be a binary mask image or a target region bounding box with semantic annotations. This image segmentation result helps security personnel quickly and accurately locate target objects from massive amounts of monitoring video, thereby enabling trajectory tracking and improving the efficiency and accuracy of security monitoring.

[0057] 4) In the field of augmented reality and interaction design, the target image is a real-world scene image acquired through a camera or other image acquisition device. This can be an indoor environment, outdoor landscape, or any other scene requiring augmented reality operations. The text description is a text instruction involving a specific object in the scene, such as "separate the sofa from the image." This text description clearly identifies the target object to be segmented for subsequent virtual element fusion or manipulation. The image segmentation result refers to accurately segmenting the sofa in the real-world scene image, obtaining its shape and boundary mask, thus distinguishing the sofa from other background elements such as walls, floors, and other furniture. This results in a segmentation result containing only the sofa area, which can be presented as a sofa image with a transparent background or a set of data describing the sofa's outline. This image segmentation result provides the foundation for virtual furniture replacement and other operations in augmented reality applications. By accurately segmenting the area where the sofa is located, a virtual furniture model can be precisely placed in that area, achieving seamless integration with the real scene. This provides users with a more realistic and natural augmented reality experience, enhancing the fun and practicality of interaction design and meeting users' needs in home decoration, scene arrangement, and other aspects.

[0058] Based on different practical needs, data was designed and collected, covering diverse types such as medical images, autonomous driving scenarios, security monitoring footage, and augmented reality images. Corresponding text descriptions were also matched, enabling the progressive cross-modal bottleneck fusion network to accurately learn the features and association patterns of different types of targets. Ultimately, the network, customized for the needs of various fields, was trained to efficiently perform image segmentation, providing accurate, stable, and efficient data support for everything from lesion localization in medical diagnosis and real-time target segmentation in autonomous driving to target tracking in security monitoring and object replacement in augmented reality.

[0059] In this application, we will use "BERT as the text encoder and ResNet as the visual encoder" as an example. Then: Figure 2 As shown, the text description of the preset sample image is embedded using a text encoder to obtain the embedding representation result, including:

[0060] The text descriptions of the input preset sample images are embedded using the standard text encoder BERT to obtain the embedding representation results. The embedding representation results include: word-level embeddings L in sequence form and sentence-level embeddings S, which can be represented by the following formula:

[0061] [L, S] = Bert(text) Bert(·) represents the text encoder Bert, and text represents the text description of the preset sample image.

[0062] For example, Figure 2 In the sample image, the text description can be "A boy playing skateboard" or in Chinese, depending on the actual situation.

[0063] Optionally, in the above technical solution, the visual encoder includes N sequentially arranged network layers; wherein the number of N can be 2, 3, 4, or 5, etc., and can be set according to the actual situation, and the scales of the visual features output by the N network layers are all different. When the visual encoder is a ResNet, the network layers of the visual encoder ResNet are residual networks.

[0064] In the process of multi-stage visual extraction of preset sample images by the visual encoder, the cross-modal attention fusion module incorporates the embedded representation results into the visual features of each visual extraction stage to obtain the fused visual features of each visual extraction stage, including:

[0065] After obtaining the visual features of the first visual extraction stage through the first network layer, the first cross-modal attention fusion module fuses the word-level embeddings and the visual features of the first visual extraction stage based on the attention mechanism (also known as the pixel-word attention mechanism) and the bottleneck vector mechanism to obtain the first guiding feature. The contextual information features in the first guiding feature are extracted based on the gating mechanism, and the contextual information features in the first guiding feature are fused with the visual features of the first visual extraction stage to obtain the fused visual features of the first visual extraction stage. The fused visual features of the first visual extraction stage are used as the input of the second network layer, and the visual features of the second visual extraction stage are obtained through the second network layer. The second cross-modal attention fusion module fuses the word-level embeddings and the visual features of the second visual extraction stage based on the attention mechanism until the fused visual features of each visual extraction stage are obtained.

[0066] Taking N=4 as an example, and combining Figure 2 The process of obtaining the fused visual features at each visual extraction stage is described as follows:

[0067] 1) The process of obtaining the fused visual features in the first visual extraction stage is as follows:

[0068] The preset sample image is input into the first network layer Encoder-1 of the visual encoder to obtain the visual features output by the first network layer Encoder-1 (i.e., the visual features of the first visual extraction stage), denoted as V1. The first cross-modal attention fusion module CMAFM-BV-1 fuses the word-level embedding L and the visual features V1 of the first visual extraction stage based on the attention mechanism and the bottleneck vector mechanism to obtain the first guiding feature F1. Based on the gating mechanism, the context information features in the first guiding feature F1 are fused with the visual features V1 of the first visual extraction stage to obtain the fused visual features E1 of the first visual extraction stage.

[0069] 2) The process of obtaining the fused visual features in the second visual extraction stage is as follows:

[0070] The fused visual feature E1 from the first visual extraction stage is used as the input to the second network layer Encoder-2. The second visual feature (i.e., the visual feature from the second visual extraction stage) is obtained through the second network layer Encoder-2 and denoted as V2. The second cross-modal attention fusion module CMAFM-BV-2, based on the attention mechanism and bottleneck vector mechanism, fuses the word-level embedding L and the visual feature V2 from the second visual extraction stage to obtain the second guiding feature F2. Based on the gating mechanism, the contextual information features in the second guiding feature F2 are fused with the visual feature V2 from the second visual extraction stage to obtain the fused visual feature E2 from the second visual extraction stage.

[0071] 3) The process of obtaining the fused visual features in the third visual extraction stage is as follows:

[0072] The fused visual feature E2 from the second visual extraction stage is used as the input to the third network layer Encoder-3. The third visual feature (i.e., the visual feature from the third visual extraction stage) is obtained through the third network layer Encoder-3 and denoted as V3. The third cross-modal attention fusion module CMAFM-BV-3, based on the attention mechanism and bottleneck vector mechanism, fuses the word-level embedding L and the visual feature V3 from the third visual extraction stage to obtain the third guiding feature F3. Based on the gating mechanism, the contextual information features in the third guiding feature F3 are fused with the visual feature V3 from the third visual extraction stage to obtain the fused visual feature E3 from the third visual extraction stage.

[0073] 4) The process of obtaining the fused visual features in the fourth visual extraction stage is as follows:

[0074] The fused visual feature E3 from the third visual extraction stage is used as the input to the fourth network layer Encoder-4. The fourth visual feature (i.e., the visual feature of the fourth visual extraction stage) is obtained through the fourth network layer Encoder-4 and denoted as V4. The fourth cross-modal attention fusion module CMAFM-BV-4 fuses the word-level embedding L and the visual feature V4 from the fourth visual extraction stage based on the attention mechanism and the bottleneck vector mechanism to obtain the fourth guiding feature F4. Based on the gating mechanism, the contextual information features in the fourth guiding feature F4 are fused with the visual feature V4 from the fourth visual extraction stage to obtain the fused visual feature E4 of the fourth visual extraction stage.

[0075] The preset sample image size is 480×480, the visual feature V1 size in the first visual extraction stage is 120×120, the visual feature V2 size in the second visual extraction stage is 60×60, the visual feature V3 size in the third visual extraction stage is 30×30, and the visual feature V4 size in the fourth visual extraction stage is 15×15.

[0076] The process of obtaining V1, V2, V3, and V4 described above can be represented as follows:

[0077] V i =ResNet i (E i-1 )

[0078] Among them, ResNet i (·) represents the i-th network layer of the visual encoder ResNet, E i-1 E0 represents the (i-1)th fused visual feature, where i = 1, 2, 3, 4. It should be noted that when i = 1, E0 is a preset sample image.

[0079] Specifically, the i-th cross-modal attention fusion module, based on attention and bottleneck vector mechanisms, performs word-level embedding L and visual features V from the i-th visual extraction stage. i By performing fusion, the i-th guiding feature F is obtained. i , combined Figure 3 The specific implementation process is described below:

[0080] ① The input visual features V are processed through the first convolutional layer Conv1. i Perform a 1×1 convolution operation to obtain the result of the first convolution operation, and flatten the result of the first convolution operation from spatial form into a sequence form to obtain the feature V. iqSpecifically, it can be expressed by the following formula:

[0081] V iq =flatten(ω) iq (V i ))

[0082] Where, ω iq (V i This indicates that the input visual features V are processed through the first convolutional layer Conv1. i Perform a 1×1 convolution operation, that is, process the input visual features V through the first convolutional layer Conv1. i Perform a linear mapping operation, flatten(ω) iq (V i )) represents: ω iq (V i ) represents the result of the first convolution operation, flatten(ω) iq (V i )) means: flatten the result of the first convolution operation from spatial form into a sequence form.

[0083] ② Using a second convolutional layer (Conv2) and a third convolutional layer (Conv3) with different kernel parameters (such as weights and biases), perform 1×1 convolution operations on the word-level embedding L to obtain the feature L. ik and feature L iv Specifically, it can be expressed by the following formula:

[0084] L ik =ω ik (L)

[0085] L iv =ω iv (L)

[0086] Where, ω ik (L) indicates that a 1×1 convolution operation is performed on the lexical embedding L through the second convolutional layer Conv2. In other words, a linear mapping operation is performed on the input lexical embedding L through the second convolutional layer Conv2, ω iv (L) indicates that the lexical embedding L is convolved by a 1×1 convolution operation through the third convolutional layer Conv3. In other words, the input lexical embedding L is linearly mapped through the third convolutional layer Conv3.

[0087] ③ For feature L ik and feature V iq Perform matrix multiplication to obtain the features T is the transpose symbol, C i Representation: Feature L ik Channel number or feature L ivThe number of channels, it should be noted, feature L ik Channel number and feature L iv The number of channels is equal, and the feature G″ is calculated using the following formula. i :

[0088]

[0089] Indicates: Features Enter the softmax function.

[0090] ④ Feature G″ i The sequence is restored to its spatial form, and a 1×1 convolution operation is performed through the fourth convolutional layer Conv4 to obtain the feature G. i Specifically, it can be expressed by the following formula:

[0091] in, Indicates: feature G″ i Recovering spatial form from sequence form Indicates: Through the fourth convolutional layer Conv4 Perform a 1×1 convolution operation, that is, pass the fourth convolutional layer Conv4 to... Perform a linear mapping operation.

[0092] ⑤ Utilize the fifth convolutional layer Conv5 to process the input visual features V i Perform a 1×1 convolution operation to obtain ω inv V i , will ω inv V i and the i-th bottleneck vector neck i Perform feature concatenation to obtain cat[ω] inv V i , neck i ], will feature G i and the i-th bottleneck vector neck i Perform feature concatenation to obtain cat[G] i ,neck i ];

[0093] ⑥ Set cat[ω inv V i , neck i The input is fed into the first Transformer layer, Transformer1, which contains a self-attention mechanism and a multilayer perceptron, to obtain [F]. iv , neck iv The formula is expressed as:

[0094] [F iv, neck iv ] = transformer1(cat[ω inv V i , neck i ])

[0095] The above formula can be broken down as follows:

[0096] [F′ iv ,neck′ iv ] = transformer_selfA1(cat[ω inv V i , neck i ])

[0097] [F iv , neck iv ] = transformer_MLP1([F′ iv ,neck′ iv ])

[0098] Where transformer_selfA1() represents the self-attention mechanism of the first Transformer layer, Transformer1, [F′ iv ,neck′ iv [F] represents the intermediate features calculated by the self-attention mechanism of the first Transformer layer (Transformer1), where transformer_MLP1() represents the multilayer perceptron of the first Transformer layer (Transformer1). iv , neck iv [This represents the final output feature obtained by mapping through the multilayer perceptron of the first Transformer layer, Transformer1.] transformer_selfA1() and transformer_MLP1() together constitute a complete first Transformer layer, Transformer1.

[0099] cat[G i , neck i The input is fed into the second Transformer layer, Transformer2, which contains a self-attention mechanism and a multilayer perceptron, to obtain [F]. ig , neck ig The formula is expressed as:

[0100] [F ig , neck ig ] = transformer2(cat[G i , necki ])

[0101] The above formula can be broken down as follows:

[0102] [F′ ig ,neck′ ig ] = transformer_selfA2(cat[G i ,neck i ])

[0103] [F ig ,neck ig ] = transformer_MLP2([F′ ig ,neck′ ig ])

[0104] Where transformer_selfA2() represents the self-attention mechanism of the second Transformer layer, Transformer2, [F′ ig ,neck′ ig [F] represents the intermediate features computed through the self-attention mechanism of the second Transformer layer, Transformer2. `transformer_MLP2()` represents the multilayer perceptron of the second Transformer layer, Transformer2. ig , neck ig [This represents the final output feature obtained after mapping through the multilayer perceptron of the second Transformer layer, Transformer2. transformer_selfA2() and transformer_MLP2() together constitute a complete second Transformer layer, Transformer2.]

[0105] The initial bottleneck vector neck1 is calculated using sentence-level embedding S and first-layer visual encoding V1, and can be expressed by the following formula:

[0106] neck1=AP(ω v (V1)ω s (S))

[0107] Where, ω v and ω s It represents a linear mapping function with different parameters, and AP(·) represents an average pooling network.

[0108] ⑦ Calculate the neck iv and neck ig The mean, as the neck i+1 In the next phase, we will use the neck.i+1 Feature concatenation can be represented by the following formula:

[0109] neck i+1 =mean(neck iv , neck ig )

[0110] Wherein, mean(neck) iv , neck ig ) indicates the calculation of neck iv and neck ig The mean.

[0111] For feature F iv and feature F ig Perform pixel-level multiplication to obtain the i-th guiding feature F. i Specifically, it can be expressed by the following formula:

[0112] F i =F iv ⊙F ig

[0113] Using bottleneck vectors for selective feature fusion at each visual encoding stage can filter out redundant information and improve model performance.

[0114] Based on the gating mechanism, the contextual information features in the i-th guiding feature are fused with the visual features in the i-th visual extraction stage to obtain the fused visual features in the i-th visual extraction stage. The specific implementation process is as follows:

[0115] Extracting F using two learnable linear layers i Contextual information feature F″ i Then, the mined contextual information features (i.e., contextual information in feature form) are integrated into the visual feature V. i Specifically, it can be expressed by the following formula:

[0116] F″ i =γ i (F i )

[0117]

[0118] Where, γ i (·) represents a simple network consisting of two learnable linear layers, using ReLU and Tanh as activation functions, respectively. Indicates: For F″ i and F″ i Perform a dot product.

[0119] Optionally, in the above technical solution, the features of local edge enhancement generated by the local edge refinement module for multiple decoding stages include: features of local edge enhancement for each decoding stage or features of local edge enhancement for some decoding stages;

[0120] When the local edge refinement module generates local edge enhancement features for multiple decoding stages, including features from some decoding stages, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features from each decoding stage are used as input for the next decoding stage until the output of the final decoder is obtained, including:

[0121] The fused visual features from the Nth visual extraction stage and the guiding features generated by the Nth cross-modal attention fusion module are input into the first decoder. The output of the first decoder and the guiding features generated by the (N-1)th cross-modal attention fusion module are input into the second decoder. This process continues until the nth decoder is reached. Then, the output of the nth decoder and the visual features from the Nnth visual extraction stage are input into the first local edge refinement module to generate the first local edge enhancement feature. The first local edge enhancement feature and the guiding features generated by the Nnth cross-modal attention fusion module are input into the (n+1)th decoder. This process continues until the output of the Nth decoder is obtained. The Nth decoder is the last decoder.

[0122] Taking "n=2" as an example, the process of obtaining the last decoder is described as follows:

[0123] ① Input the fused visual features F4 from the fourth visual extraction stage and the guiding features F4 generated by the fourth cross-modal attention fusion module CMAFM-BV-4 into the first decoder Decoder1 to obtain the output V′1 of the first decoder Decoder1.

[0124] ② Input the output V′1 of the first decoder Decoder1 and the guiding feature F3 generated by the third cross-modal attention fusion module CMAFM-BV-3 into the second decoder Decoder2 to obtain the output V′2 of the second decoder Decoder2.

[0125] ③ Input the output V′2 of the second decoder Decoder2 and the visual feature V2 of the second visual extraction stage into the first local edge refinement module LERM-1 to generate the first local edge enhancement feature V″1. Input the first local edge enhancement feature V″1 and the guiding feature F2 generated by the second cross-modal attention fusion module CMAFM-BV-2 into the third decoder Decoder3 to obtain the output V′3 of the third decoder Decoder3.

[0126] ④ Input the output V′3 of the third decoder Decoder3 and the visual feature V1 of the first visual extraction stage into the second local edge refinement module LERM-2 to generate the second local edge enhancement feature V″2. Input the second local edge enhancement feature V″2 and the guiding feature F1 generated by the first cross-modal attention fusion module CMAFM-BV-1 into the fourth decoder Decoder4 to obtain the output V′4 of the fourth decoder Decoder4. The output V′4 of the fourth decoder Decoder4 is the output of the last decoder.

[0127] Then, based on the output V′4 of the fourth decoder, a target segmentation map corresponding to the preset sample image is generated. Specifically, the output V′4 of the fourth decoder is subjected to a 1×1 convolution to make V′4 a single-channel feature map. Then, the single-channel feature map is upsampled, and the upsampled result is input into the sigmoid function to calculate the target segmentation map corresponding to the preset sample image. The size of the target segmentation map is 480×480.

[0128] The decoder processes the two received features as follows:

[0129] like Figure 2 As shown, the decoder concatenates the two received features into a concatenated vector. This concatenated vector is then processed sequentially through the sixth convolutional layer (Conv6), the first batch of normalization layers (BN1), the first activation layer, the seventh convolutional layer (Conv7), the second batch of normalization layers (BN2), and the second activation layer to obtain the decoder's output. The sixth convolutional layer (Conv6) and the seventh convolutional layer (Conv7) are used to perform 3×3 convolution operations. The first and second activation layers both use the ReLU activation function.

[0130] It should be noted that the two features received by the first decoder (Decoder1) are: the fused visual feature E4 from the fourth visual extraction stage and the guiding feature F4 generated by the fourth cross-modal attention fusion module CMAFM-BV-4; the two features received by the second decoder (Decoder2) are: the output V′1 of the first decoder (Decoder1) and the guiding feature F3 generated by the third cross-modal attention fusion module CMAFM-BV-3; the two features received by the third decoder (Decoder3) are: the feature V″1 of the first local edge enhancement and the guiding feature F2 generated by the second cross-modal attention fusion module CMAFM-BV-2; and the two features received by the fourth decoder (Decoder4) are: the feature V″2 of the second local edge enhancement and the guiding feature F1 generated by the first cross-modal attention fusion module CMAFM-BV-1.

[0131] Optionally, in the above technical solution, the local edge refinement module is used to: perform bi-branch convolution operation on the received visual features to generate two intermediate features; process one of the intermediate features and the output of the received decoder based on the residual space attention mechanism to obtain a first processing result; process the other intermediate feature and the first processing result based on the residual space attention mechanism and the residual channel attention mechanism to obtain a second processing result; determine edge information based on the first processing result and the second processing result; and adaptively fuse the edge information with the two intermediate features to generate features with enhanced local edges.

[0132] Combination Figure 4 The following explanation is provided:

[0133] ① Regarding the received visual features V i Perform a two-branch convolution operation to generate the first intermediate feature V. a Second intermediate feature V b V a =Conv8(V i V b =Conv9(V i ), where Conv8(V i This indicates that the visual feature V is processed through the eighth convolutional layer, Conv8. i Perform a 3×3 convolution operation, Conv2(V i This indicates that V is processed through the ninth convolutional layer, Conv9. i Perform a 3×3 convolution operation.

[0134] ② Based on the residual space attention mechanism, focus on one of the intermediate features (e.g., the second intermediate feature V). bThe first processing result is obtained by processing the received decoder output and the first processing result, specifically:

[0135] By employing a residual space attention mechanism, the received decoder output V′ is used to analyze the second intermediate feature V. b By providing guidance and enhancing structural dependencies, features can be obtained. Among them, f SA (·) represents the residual space attention mechanism, and V′ represents the received output of the decoder.

[0136] Among them, the features are obtained The specific implementation process is as follows:

[0137] After upsampling the received decoder output V′, it is then processed by the first residual space attention module SA1. The processing result of the first residual space attention module SA1 is then compared with the second intermediate feature V. b Perform matrix multiplication to obtain the features feature This is the first processing result.

[0138] ③ Based on the residual space attention mechanism and the residual channel attention mechanism, another intermediate feature (the first intermediate feature V) is applied. a The first processing result is processed to obtain the second processing result, which includes: features. feature feature and characteristics Specifically:

[0139] Calculate residual channel attention and residual spatial attention separately to enhance discriminative feature representation:

[0140]

[0141]

[0142] Among them, f CA (·) indicates the residual channel attention mechanism.

[0143] Among them, features The specific acquisition process is as follows:

[0144] The first intermediate feature V is processed by the first residual channel attention module CA1. a The processing result of the first residual channel attention module CA1 is combined with the first intermediate feature V. a Perform pixel-level multiplication to obtain the first pixel-level multiplication result, and then combine the first pixel-level multiplication result with the first intermediate feature V. a Perform pixel-level multiplication to obtain features.

[0145] Among them, features The specific acquisition process is as follows:

[0146] Features are processed by the second residual channel attention module CA2. The processing results of the second residual channel attention module CA2 are combined with the features. Perform pixel-level multiplication to obtain a second pixel-level multiplication result, and then combine the second pixel-level multiplication result with the feature. Perform pixel-level addition to obtain features

[0147] in, The specific acquisition process is as follows:

[0148] Features are processed through the second residual space attention module SA2. The processing is performed to obtain the processing result of the second residual spatial attention module SA2. The processing result of the second residual spatial attention module SA2 and the features are then combined. Perform matrix multiplication, and then multiply the result of the matrix multiplication with the features. Add them together to obtain the features.

[0149] in, The specific acquisition process is as follows:

[0150] Features are processed through the third residual space attention module SA3. The processing is performed to obtain the processing result of the third residual space attention module SA3. The processing result of the third residual space attention module SA3 and the features are then combined. Perform matrix multiplication, and then multiply the result of the matrix multiplication with the features. Add them together to obtain the features.

[0151] ④ From the perspective of visual edge features, the edge regions of objects exhibit relatively drastic changes, with a large difference between max pooling and average pooling values ​​within a small 2×2 pixel area. In contrast, non-edge regions are relatively smooth, with smaller differences between max pooling and average pooling values ​​within the same 2×2 pixel area. Therefore, for features... Perform average pooling to obtain minimum pooled features, and then refine the features. Max pooling is performed to obtain the max pooled features. The min pooling features are then subtracted from the max pooled features to obtain the differential sensing features. The L2 norm of the differential sensing features in the channel dimension is calculated and denoted as V. c Specifically, it can be expressed by the following formula:

[0152]

[0153] Where MP(·) represents max pooling, AP(·) represents max pooling, and AP(·) represents average pooling. Represents the minimum pooling feature. Representation: Calculate the L2 norm of the differential sensing feature in the channel dimension.

[0154] ⑤ V c ,feature and characteristics The data is stitched together, and a 1×1 convolution operation is performed using the tenth convolutional layer Conv10, combined with the received visual features V. i This forms an edge-aware saliency map, enabling the capture of clear edge information, specifically represented by the following formula:

[0155]

[0156] in, Indicates V c ,feature and characteristics To splice, Indicates: Using the tenth convolutional layer Conv10 to... Perform a 1×1 convolution operation (linear transformation).

[0157] In this module, the visual feature received by the first local edge refinement module LERM-1 is V2, and the output of the decoder received is V′2, which is the output of the second decoder Decoder2. At this time, V″ is the output of the first local edge refinement module LERM-1, that is, the feature V″1 of the first local edge enhancement. The visual feature received by the second local edge refinement module LERM-2 is V1, and the output of the decoder received is V′3, which is the output of the third decoder Decoder3. At this time, V″ is the output of the second local edge refinement module LERM-2, that is, the feature V″2 of the second local edge enhancement.

[0158] Optionally, in the above technical solution, cross-entropy loss is used for supervision during the training of the progressive cross-modal bottleneck fusion network. That is, cross-entropy loss is applied for supervision in all four decoding stages throughout the end-to-end training process to achieve comprehensive guidance for feature learning. Specifically, the output V′ of each decoder is first passed through a 1×1 convolutional layer (with a kernel size of 1) to transform it into a single-channel mask feature map, then upsampled to match the resolution of the original ground truth label G. Finally, the upsampled result is input into the sigmoid function to calculate the pred. iThen, the cross-entropy loss is calculated. The average of the losses from the four decoder layers is taken to obtain the total loss L:

[0159]

[0160] Among them, pred i = sigmoid(upsampling(conv(V′))), where conv(V′) represents performing a 1×1 convolution operation on the decoder output V′, upsampling(conv(V′)) represents upsampling conv(V′), and sigmoid represents the sigmoid function.

[0161] Cross-entropy loss provides effective supervision, helping the network learn the alignment between text and visual features, thereby improving the accuracy of target segmentation. By minimizing cross-entropy loss, the network can better optimize the classification of target and background regions, improving the quality of segmentation results. The cross-entropy loss function provides a stable gradient during training, which helps the network converge stably and avoids fluctuations and overfitting during training. Through the supervision of cross-entropy loss, the model can learn more discriminative feature representations, thereby improving its generalization ability in different scenarios and enabling it to perform well on unseen data.

[0162] based on Figures 5 to 9 This paper describes the performance of the trained progressive cross-modal bottleneck fusion network in image segmentation, specifically using... Figure 5 The international standard dataset (RefCOCO dataset) shown is relevant to the progressive cross-modal bottleneck fusion network of this application. Figure 7 The comparison models shown (including DMN and RRN, etc.) are trained using... Figure 6 The oIoU evaluation metric shown compares the reference image segmentation performance of the trained progressive cross-modal bottleneck fusion network with that of the trained contrast model. The comparison results are as follows: Figure 8 As shown, the progressive cross-modal bottleneck fusion network trained in this application has good performance in the segmentation of reference images.

[0163] This invention enables accurate segmentation of matching targets within an image based on a given linguistic description. To this end, the invention first constructs a progressive cross-modal feature fusion network framework, primarily utilizing the Encoder-101 image encoder and the BERT text encoder to build a cross-modal feature pyramid. This enhances cross-modal semantic understanding and utilization through staged, multi-scale fusion of multimodal information, while simultaneously enabling the model to handle multi-scale referential targets. Secondly, the invention proposes a Cross-Modal Attention Fusion Module Based on Bottleneck Vector (CMAFM-BV), which leverages the bottleneck vector to promote efficient cross-modal information interaction during the encoding stage while minimizing redundancy. Finally, the invention proposes a novel Local Edge Refinement Module (LERM), which refines the segmentation boundaries by introducing rich edge information from shallow visual features during the decoding stage, thereby improving object integrity and boundary accuracy. Specifically:

[0164] A progressive cross-modal feature fusion network framework is constructed to enhance the understanding and utilization of cross-modal semantics by fusing multimodal information at multiple scales in stages. A cross-modal attention fusion module based on bottleneck vectors is constructed to guide visual feature encoding with important clues in text information at each visual encoding stage. A local edge refinement module is constructed to introduce edge difference perception in the visual decoding stage and refine the edges of the target object using the feature map output by the shallow visual encoder to obtain the final object segmentation map.

[0165] Furthermore, the specific steps for extracting visual features from a given input image using a multi-scale encoder include: embedding the given language description using the classic BERT text encoder to obtain word embeddings and sentence-level embeddings; extracting features from the input image using a four-stage Encoder-101 network to obtain visual features at different scales; performing multi-scale visual feature fusion in each visual encoding stage to retain feature representations of the potential target at different scales; and progressively introducing language guidance in each visual encoding stage to fully understand and utilize information between different modalities.

[0166] Furthermore, a specially designed bottleneck vector-based cross-modal attention fusion module is used to effectively mine key referential cues in language descriptions, selectively compressing key information within a modality for cross-modal interaction, and avoiding redundant information that could lead to suboptimal model performance. Specific steps include:

[0167] For the visual features output by the visual encoder layer and the word embedding features output by the text encoder layer, cross-attention calculation is performed using the visual features as the query and the word embedding features as the key and value, respectively, to perform preliminary cross-modal feature fusion. Using the visual features output by the first-layer visual encoder and the sentence-level embedding representation output by the text encoder, pixel-level multiplication and average pooling are used to obtain the initial bottleneck vector. At each visual encoding stage, the bottleneck vector of this stage is concatenated with the text features obtained from the cross-attention calculation of the visual features and word embeddings, respectively. Then, feature transformation is performed through a Transformer layer to filter redundant information. The two features processed by the Transformer layer are then vector-splittered. Pixel-level multiplication is performed on the visual feature part for integration into the next layer's visual encoding. Pixel-level mean calculation is performed on the bottleneck vector part, which serves as the bottleneck vector for the next layer.

[0168] Furthermore, by constructing a local edge refinement module, the specific steps for accurately capturing the boundary of the target in the visual decoding process include: inputting the shallow visual encoding into a dual-branch convolution to generate two intermediate features; applying residual spatial attention and residual channel attention mechanisms to the two intermediate features respectively to guide cross-modal semantic alignment and enhance structural dependency and discriminative feature representation; generating an edge-aware saliency map by calculating the difference between the max-pooling feature and the average-pooling feature in the local region of the dual-branch features; adaptively fusing the dual-branch features with the edge saliency map and fusing them with the original shallow visual encoding features to obtain edge-enhanced features, which serve as the input to the lower-layer visual decoder.

[0169] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0170] 1) The progressive cross-modal feature fusion network framework constructed in this invention builds a cross-modal feature pyramid through the Encoder-101 image encoder and the Bert text encoder, and performs multi-modal feature fusion in stages and at multiple scales to enhance cross-modal semantic understanding and utilization. At the same time, it enables the model to process multi-scale referential targets and achieve accurate localization and fine segmentation of referential targets.

[0171] 2) This invention proposes a cross-modal attention fusion module based on bottleneck vectors, which uses an improved cross-attention mechanism to mine key referential clues, guide visual feature encoding, and effectively reduce the segmentation performance degradation caused by redundant intra-modal information.

[0172] 3) The local edge refinement module proposed in this invention innovatively designs an edge difference perception scheme, which effectively utilizes the rich detail information in the feature map output by the shallow visual encoder to adaptively refine the segmentation boundary of the target.

[0173] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0174] like Figure 9 As shown, an embodiment of the present invention provides a referential image segmentation system 200 based on edge enhancement and bottleneck vectors, which includes a model training module 201 and a referential image segmentation module 202.

[0175] The model training module 201 is used to: train a progressive cross-modal bottleneck fusion network based on multiple sample images and the text description of each sample image, to obtain a trained progressive cross-modal bottleneck fusion network. The progressive cross-modal bottleneck fusion network includes a text encoder, a visual encoder, a cross-modal attention fusion module, a local edge refinement module, and a decoder. The text encoder embeds the text description of the preset sample images to obtain the embedding representation result. During the multi-stage visual extraction of the preset sample images by the visual encoder, the cross-modal attention fusion module introduces the embedding representation result into the visual features of each visual extraction stage to obtain the fused visual features of each visual extraction stage. During the multi-stage decoding of all visual features by the decoder, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features of each decoding stage are used as the input of the next decoding stage until the output of the last decoder is obtained. The output of the last decoder is used to generate the target segmentation map corresponding to the preset sample image. The preset sample image can be any sample image.

[0176] The referential image segmentation module 202 is used to: input the target image and its text description into the trained progressive cross-modal bottleneck fusion network to obtain the referential image segmentation result corresponding to the target image.

[0177] Optionally, in the above technical solution, the embedding representation result includes lexical embedding, and the visual encoder includes N sequentially arranged network layers. After obtaining the visual features of the first visual extraction stage through the first network layer, the first cross-modal attention fusion module fuses the lexical embedding and the visual features of the first visual extraction stage based on the attention mechanism and the bottleneck vector mechanism to obtain the first guiding feature. The contextual information features in the first guiding feature are extracted based on the gating mechanism, and the contextual information features in the first guiding feature are fused with the visual features of the first visual extraction stage to obtain the fused visual features of the first visual extraction stage. The fused visual features of the first visual extraction stage are used as the input of the second network layer, and the visual features of the second visual extraction stage are obtained through the second network layer. The second cross-modal attention fusion module fuses the lexical embedding and the visual features of the second visual extraction stage based on the attention mechanism until the fused visual features of each visual extraction stage are obtained.

[0178] Optionally, in the above technical solution, the features for local edge enhancement generated by the local edge refinement module for multiple decoding stages include: features for local edge enhancement in each decoding stage or features for local edge enhancement in some decoding stages; when the features for local edge enhancement generated by the local edge refinement module for multiple decoding stages include features for local edge enhancement in some decoding stages, the fused visual features of the Nth visual extraction stage and the guiding features generated by the Nth cross-modal attention fusion module are input into the first decoder, the output of the first decoder and the guiding features generated by the (N-1)th cross-modal attention fusion module are input into the second decoder, until the nth decoder is reached, the output of the nth decoder and the visual features of the Nnth visual extraction stage are input into the first local edge refinement module to generate the first local edge enhancement feature, the first local edge enhancement feature and the guiding features generated by the Nnth cross-modal attention fusion module are input into the (n+1)th decoder, until the output of the Nth decoder is obtained, where the Nth decoder is the last decoder.

[0179] Optionally, in the above technical solution,

[0180] The local edge refinement module is used to: perform bi-branch convolution operations on the received visual features to generate two intermediate features; process one of the intermediate features and the output of the received decoder based on the residual space attention mechanism to obtain a first processing result; process the other intermediate feature and the first processing result based on the residual space attention mechanism and the residual channel attention mechanism to obtain a second processing result; determine edge information based on the first and second processing results; and adaptively fuse the edge information with the two intermediate features to generate features with enhanced local edges.

[0181] Optionally, in the above technical solution, the model training module 201 is further used for:

[0182] During the training of the progressive cross-modal bottleneck fusion network, cross-entropy loss is used for supervision.

[0183] It should be noted that the beneficial effects of the image segmentation system 200 based on edge enhancement and bottleneck vectors provided in the above embodiments are the same as those of the image segmentation method based on edge enhancement and bottleneck vectors described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0184] The image segmentation system based on edge enhancement and bottleneck vectors of the present invention can be a computer program (including program code) running on a computer device. For example, the image segmentation system based on edge enhancement and bottleneck vectors of the present invention is an application software that can be used to execute the corresponding steps in the image segmentation method based on edge enhancement and bottleneck vectors of the present invention.

[0185] In some embodiments, the image segmentation system based on edge enhancement and bottleneck vectors of the present invention can be implemented in a combination of hardware and software. As an example, the image segmentation system based on edge enhancement and bottleneck vectors of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the image segmentation method based on edge enhancement and bottleneck vectors of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0186] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the aforementioned image segmentation methods based on edge enhancement and bottleneck vectors. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the image segmentation method based on edge enhancement and bottleneck vectors shown in any embodiment of the present invention by calling the computer program.

[0187] In one alternative embodiment, an electronic device is provided, such as Figure 10 As shown, Figure 10 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0188] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0189] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0190] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0191] The memory 4003 stores the application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0192] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0193] It should be noted that, Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0194] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the aforementioned reference image segmentation methods based on edge enhancement and bottleneck vectors.

[0195] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0196] In an exemplary embodiment, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the aforementioned reference image segmentation methods based on edge enhancement and bottleneck vectors.

[0197] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0198] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0199] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EEPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0200] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0201] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0202] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0203] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for representing image segmentation based on edge enhancement and bottleneck vectors, characterized in that, include: A progressive cross-modal bottleneck fusion network is trained based on multiple sample images and the text description of each sample image to obtain a trained progressive cross-modal bottleneck fusion network. The progressive cross-modal bottleneck fusion network includes a text encoder, a visual encoder, a cross-modal attention fusion module, a local edge refinement module, and a decoder. The text encoder embeds the text description of the preset sample images to obtain an embedding representation result. During the multi-stage visual extraction of the preset sample images by the visual encoder, the cross-modal attention fusion module incorporates the embedding representation result into the visual features of each visual extraction stage to obtain fused visual features for each visual extraction stage. During the multi-stage decoding of all visual features by the decoder, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features of each decoding stage are used as input for the next decoding stage until the output of the last decoder is obtained. Based on the output of the last decoder, a target segmentation map corresponding to the preset sample image is generated, where the preset sample image is any sample image. The target image and its text description are input into the trained progressive cross-modal bottleneck fusion network to obtain the image segmentation result corresponding to the target image. The embedding representation results include lexical-level embeddings, and the visual encoder includes N sequentially arranged network layers; During the multi-stage visual extraction process of the preset sample image by the visual encoder, the cross-modal attention fusion module incorporates embedded representation results into the visual features of each visual extraction stage to obtain the fused visual features of each visual extraction stage, including: After obtaining the visual features of the first visual extraction stage through the first network layer, the first cross-modal attention fusion module fuses the word-level embeddings and the visual features of the first visual extraction stage based on the attention mechanism and bottleneck vector mechanism to obtain the first guiding feature. The contextual information features in the first guiding feature are extracted based on the gating mechanism, and the contextual information features in the first guiding feature are fused with the visual features of the first visual extraction stage to obtain the fused visual features of the first visual extraction stage. The fused visual features of the first visual extraction stage are used as the input of the second network layer, and the visual features of the second visual extraction stage are obtained through the second network layer. The second cross-modal attention fusion module fuses the word-level embeddings and the visual features of the second visual extraction stage based on the attention mechanism and bottleneck vector mechanism until the fused visual features of each visual extraction stage are obtained. The specific steps of the cross-modal attention fusion module include: for the visual features output by the visual encoder layer and the word embedding features output by the text encoder layer, cross-attention calculation is performed using the visual features as the query and the word embedding features as the key and value to perform preliminary cross-modal feature fusion; using the visual features output by the first-layer visual encoder and the sentence-level embedding representation output by the text encoder, pixel-level multiplication and average pooling are used to obtain the initial bottleneck vector; in each visual encoding stage, the bottleneck vector of this stage is concatenated with the text features obtained by cross-attention calculation of the visual features and word embeddings, and then transformed by the Transformer layer to filter redundant information; the two features processed by the Transformer layer are vector-splittered, and pixel-level multiplication is performed on the visual feature part for integration into the next layer of visual encoding; for the bottleneck vector part, pixel-level mean calculation is performed as the bottleneck vector of the next layer. The local edge refinement module is used to: perform bi-branch convolution operations on the received visual features to generate two intermediate features; process one of the intermediate features and the output of the received decoder based on the residual space attention mechanism to obtain a first processing result; process the other intermediate feature and the first processing result based on the residual space attention mechanism and the residual channel attention mechanism to obtain a second processing result; determine edge information based on the first processing result and the second processing result; and adaptively fuse the edge information with the two intermediate features to generate features with enhanced local edges.

2. The method for image segmentation based on edge enhancement and bottleneck vectors according to claim 1, characterized in that, The local edge enhancement features generated by the local edge refinement module for multiple decoding stages include: local edge enhancement features for each decoding stage or local edge enhancement features for some decoding stages; When the local edge refinement module generates local edge enhancement features for multiple decoding stages, including features from some decoding stages, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features from each decoding stage are used as input for the next decoding stage until the output of the final decoder is obtained, including: The fused visual features from the Nth visual extraction stage and the guiding features generated by the Nth cross-modal attention fusion module are input into the first decoder. The output of the first decoder and the guiding features generated by the (N-1)th cross-modal attention fusion module are input into the second decoder. This process continues until the nth decoder is reached. Then, the output of the nth decoder and the visual features from the Nnth visual extraction stage are input into the first local edge refinement module to generate the first local edge enhancement feature. The first local edge enhancement feature and the guiding features generated by the Nnth cross-modal attention fusion module are input into the (n+1)th decoder. This process continues until the output of the Nth decoder is obtained. The Nth decoder is the last decoder.

3. The method for image segmentation based on edge enhancement and bottleneck vectors according to any one of claims 1 to 2, characterized in that, Also includes: During the training of the progressive cross-modal bottleneck fusion network, cross-entropy loss is used for supervision.

4. A referential image segmentation system based on edge enhancement and bottleneck vectors, characterized in that, Includes a model training module and a referential image segmentation module; The model training module is used to: train a progressive cross-modal bottleneck fusion network based on multiple sample images and the text description of each sample image, to obtain a trained progressive cross-modal bottleneck fusion network. The progressive cross-modal bottleneck fusion network includes a text encoder, a visual encoder, a cross-modal attention fusion module, a local edge refinement module, and a decoder. The text encoder embeds the text description of the preset sample images to obtain an embedding representation result. During the multi-stage visual extraction of the preset sample images by the visual encoder, the cross-modal attention fusion module incorporates the embedding representation result into the visual features of each visual extraction stage to obtain fused visual features for each visual extraction stage. During the multi-stage decoding of all visual features by the decoder, the local edge refinement module generates local edge enhancement features for multiple decoding stages. The local edge enhancement features of each decoding stage are used as input for the next decoding stage until the output of the last decoder is obtained. Based on the output of the last decoder, a target segmentation map corresponding to the preset sample image is generated, where the preset sample image is any sample image. The referential image segmentation module is used to: input the target image and the text description of the target image into the trained progressive cross-modal bottleneck fusion network to obtain the referential image segmentation result corresponding to the target image; The embedding representation result includes lexical embeddings, and the visual encoder includes N sequentially arranged network layers. After obtaining the visual features of the first visual extraction stage through the first network layer, the first cross-modal attention fusion module fuses the lexical embeddings and the visual features of the first visual extraction stage based on attention and bottleneck vector mechanisms to obtain the first guiding feature. The contextual information features in the first guiding feature are extracted based on a gating mechanism, and the contextual information features in the first guiding feature are fused with the visual features of the first visual extraction stage to obtain the fused visual features of the first visual extraction stage. The fused visual features of the first visual extraction stage are used as the input of the second network layer, and the visual features of the second visual extraction stage are obtained through the second network layer. The second cross-modal attention fusion module fuses the lexical embeddings and the visual features of the second visual extraction stage based on attention and bottleneck vector mechanisms until the fused visual features of each visual extraction stage are obtained. The specific steps of the cross-modal attention fusion module include: for the visual features output by the visual encoder layer and the word embedding features output by the text encoder layer, cross-attention calculation is performed using the visual features as the query and the word embedding features as the key and value to perform preliminary cross-modal feature fusion; using the visual features output by the first-layer visual encoder and the sentence-level embedding representation output by the text encoder, pixel-level multiplication and average pooling are used to obtain the initial bottleneck vector; in each visual encoding stage, the bottleneck vector of this stage is concatenated with the text features obtained by cross-attention calculation of the visual features and word embeddings, and then transformed by the Transformer layer to filter redundant information; the two features processed by the Transformer layer are vector-splittered, and pixel-level multiplication is performed on the visual feature part for integration into the next layer of visual encoding; for the bottleneck vector part, pixel-level mean calculation is performed as the bottleneck vector of the next layer. The local edge refinement module is used to: perform bi-branch convolution operations on the received visual features to generate two intermediate features; process one of the intermediate features and the output of the received decoder based on the residual space attention mechanism to obtain a first processing result; process the other intermediate feature and the first processing result based on the residual space attention mechanism and the residual channel attention mechanism to obtain a second processing result; determine edge information based on the first processing result and the second processing result; and adaptively fuse the edge information with the two intermediate features to generate features with enhanced local edges.

5. The image segmentation system based on edge enhancement and bottleneck vectors according to claim 4, characterized in that, The local edge enhancement features generated by the local edge refinement module for multiple decoding stages include: local edge enhancement features for each decoding stage or local edge enhancement features for some decoding stages. When the local edge refinement module generates local edge enhancement features for multiple decoding stages, including local edge enhancement features for some decoding stages, the fused visual features of the Nth visual extraction stage and the guiding features generated by the Nth cross-modal attention fusion module are input into the first decoder. The output of the first decoder and the guiding features generated by the (N-1)th cross-modal attention fusion module are input into the second decoder. This process continues until the nth decoder is reached. Then, the output of the nth decoder and the visual features of the Nnth visual extraction stage are input into the first local edge refinement module to generate the first local edge enhancement feature. The first local edge enhancement feature and the guiding features generated by the Nnth cross-modal attention fusion module are input into the (n+1)th decoder. This process continues until the output of the Nth decoder is obtained. The Nth decoder is the last decoder.

6. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the reference image segmentation method based on edge enhancement and bottleneck vector as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the reference image segmentation method based on edge enhancement and bottleneck vectors as described in any one of claims 1 to 3.

Images