A method, system and related equipment for predicting structure in ultrasound images

Abstract

This application relates to the field of image analysis technology, and in particular to a method, system, and related equipment for predicting the structure of ultrasound images. The method includes: acquiring an ultrasound image; dividing the ultrasound image into column images according to a preset division rule, wherein the column images include continuous small segments; inputting the column images into a Transformer model and generating structure tokens through the Transformer model; and performing structure prediction on the ultrasound image based on the structure tokens through prediction branches. This application, through the column-by-column autoregressive mechanism of the structure tokens, can simultaneously perform unified modeling of region type and skin layer boundary, realizing information sharing and continuous transmission between columns.

Description

Technical Field

[0001] This application relates to the field of image analysis technology, and in particular to a method, system and related equipment for predicting the structure of ultrasound images. Background Technology

[0002] Ultrasound imaging is a non-invasive medical imaging technique widely used for observing the structures of skin, blood vessels, muscles, and internal organs. Unlike optical imaging or MRI, ultrasound images are characterized by high noise, low contrast, and blurred tissue boundaries, making direct tissue or structural segmentation challenging. In medical diagnosis and research, it is often necessary to annotate structures in ultrasound images at both the region and boundary levels. Region type: includes different tissue layers (such as epidermis, dermis, subcutaneous tissue) or fillers (such as cysts, adipose tissue), and is usually closed or irregular in shape; Layer type: Indicates the boundary or transition zone between layers, such as the transition line between the epidermis and the dermis, used to reflect changes in tissue layers.

[0003] Traditional ultrasound image analysis methods mainly rely on human experience or image processing algorithms: Edge detection (such as the Canny operator and the Sobel operator) is used to find interlayer boundaries, but it is susceptible to noise. Thresholding segmentation and region growing can extract tissue regions, but their effectiveness is limited for structures with complex shapes and blurred boundaries. Manual annotation is time-consuming and highly subjective, making it difficult to extend to large-scale data.

[0004] In recent years, deep learning methods, especially convolutional neural networks (CNNs), have made significant progress in ultrasound image segmentation: U-Net achieves end-to-end pixel-level segmentation through an encoder-decoder structure and is widely used in medical image (including ultrasound) analysis. Traditional CNNs are limited in their ability to model interlayer continuity by the receptive field of the convolution, and are prone to producing holes, breaks, or discontinuous predictions.

[0005] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0006] The main objective of this application is to provide a method, system, and related equipment for predicting the structure of ultrasound images, in order to solve the aforementioned problems.

[0007] To achieve the above objectives, one aspect of this application proposes a method for predicting structures in ultrasound images, the method comprising: Acquire ultrasound images and divide them into column images according to a preset division rule. Each column image consists of continuous small segments. The column image is input into the Transformer model, and a structure token is generated by the Transformer model. Based on the structure token, structural prediction is performed on the ultrasound image through a prediction branch.

[0008] In some embodiments, inputting the column image into a Transformer model and generating a structure token using the Transformer model specifically includes: The column images are divided into a starting column image and subsequent column images. The starting column image is the column image at the beginning of each column image, and the subsequent column images are the column images other than the starting column image. Feature mapping is performed on consecutive small segments of the starting column image, and a structure token is generated by combining it with position encoding; For any two adjacent subsequent column images, the current column image employs a self-attention mechanism to extract the current column features; The structure token of the previous column image is used as a query value or key value through a cross-attention mechanism and fused with the features of the current column to generate the structure token of the current column.

[0009] In some embodiments, the step of predicting the structure of an ultrasound image based on the structure token via a prediction branch includes: The structure token is input into the MLP classification head of the prediction branch, and a non-linear transformation is performed through the hidden layer of the MLP to generate the raw score for each class. The raw scores are input into the output layer of the MLP, and the SoftMax function of the output layer transforms the raw scores into probability distributions for each region classification. The category with the highest probability is selected as the region type label corresponding to the structural token. The region type label includes the epidermis, dermis, subcutaneous tissue, fascia layer, and interlayer transition zone.

[0010] In some embodiments, the step of performing structural prediction on ultrasound images based on the structural token via prediction branches further includes: The structure token is input into the MLP regression head of the prediction branch, and a nonlinear transformation is performed through the hidden layer of the MLP to generate the boundary score. The boundary scores are input into the output layer of the MLP, and the boundary scores are normalized by the Sigmoid function of the output layer to generate predicted coordinates. The predicted coordinates are mapped to pixel-level coordinates of the ultrasound image, and a hierarchical type label corresponding to the structure token is generated based on the pixel-level coordinates.

[0011] In some embodiments, after performing structural prediction on the ultrasound image based on the structural token via a prediction branch, the method further includes: The column image is stitched together segment by segment to generate a complete structural representation; According to the spatial order of each column of the ultrasound image, all the reconstructed column images are sorted sequentially to generate a complete two-dimensional image structure representation.

[0012] In some embodiments, the expression for the region type label is: ; ; in For region type labels, The activation function for the MLP classification head. For structure tokens, The loss function for the classification task, These are the actual category labels.

[0013] In some embodiments, the expression for the hierarchical type label is: ; ; in For hierarchical type tags, The activation function for the MLP regression head. For structure tokens, The loss function for the regression task, For the prediction of the current column image Axis coordinates The current column image for the true skin layer Axis coordinates.

[0014] To achieve the above objectives, another aspect of this application proposes an ultrasound image structure prediction system, which is applied to the aforementioned ultrasound image structure prediction method. The system includes: The image processing module is used to acquire preprocessed ultrasound images, divide the ultrasound images into column images according to preset division rules, and define the column images as a starting column image and subsequent column images, wherein the column images include continuous small segments; The structure token construction module is used to input the starting column image and subsequent column images into the Transformer model and generate a structure token through the Transformer model. The structure prediction module is used to predict the structure of ultrasound images based on the structure token and through prediction branches.

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0016] To achieve the above objectives, another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a method, system, and related equipment for predicting the structure of ultrasound images. First, through the column-by-column autoregression mechanism of structural tokens, this invention can simultaneously model the region type (such as elliptical or irregular tissue regions, fillers) and the skin layer boundary (such as the epidermal-dermal transition line) in a unified manner, realizing information sharing and continuous transmission between columns, thereby effectively avoiding the problems of holes, breaks, and structural misalignment that are prone to occur in traditional convolutional networks. Attached Figure Description

[0018] Figure 1 This is a flowchart of the ultrasound image structure prediction method provided in the embodiments of this application; Figure 2 for Figure 1 The flowchart of step S102 in the document; Figure 3 for Figure 1 Flowchart of region type prediction in step S103; Figure 4 for Figure 1 Flowchart of hierarchical type prediction in step S103; Figure 5 This is a block diagram of an ultrasound image structure prediction system provided in an embodiment of this application; Figure 6 This is a schematic diagram of the ultrasound image structure prediction method provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0021] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0024] A structure token is an abstract data unit used in deep learning models to represent and model the structured information of an image. Essentially, it is a vector in a high-dimensional feature space that encodes structured semantic information of a specific region or type of image.

[0025] Negative samples are data samples used in machine learning model training to represent patterns, categories, or relationships that the model should not learn or should reject. They are the opposite of positive samples and together constitute the complete training signal in supervised learning.

[0026] Cross-Attention is a variant of attention mechanism that allows a query sequence to interact with another key-value sequence, enabling each element in the query sequence to focus on and incorporate relevant information from the other sequence.

[0027] U-Net is an encoder-decoder architecture specifically designed for biomedical image segmentation. It is named for its U-shaped structure and combines the multi-scale features of the encoder with the upsampled features of the decoder through skip connections.

[0028] Attention U-Net is an enhanced version of the U-Net architecture. It introduces attention gates into the skip connections, enabling the model to adaptively focus on important regions in the image while suppressing irrelevant or noisy regions, thereby improving the accuracy and efficiency of medical image segmentation.

[0029] In related technologies, structural prediction using ultrasound images has the following drawbacks: 1. Region type and hierarchical type cannot be modeled in a unified manner. Existing methods typically only handle region segmentation or boundary extraction separately, lacking a unified structural representation.

[0030] Disadvantages: It cannot accurately predict closed regions (ellipses, irregular shapes) and layered boundaries (epidermal-dermal transition lines) at the same time, which can easily lead to inconsistencies in predictions.

[0031] 2. Sensitive to noise and modal characteristics Spot noise is often present in ultrasound images. This is random noise caused by echo interference and tissue scattering.

[0032] Disadvantages include: unstable prediction of regional boundaries and hierarchical structures, poor model robustness, and results significantly affected by local noise.

[0033] 3. Modeling the continuity of local receptive fields Traditional convolutional networks (such as U-Net and Attention U-Net) mainly rely on local convolution operations, resulting in a limited receptive field.

[0034] Disadvantages: In ultrasound images, voids, breaks, or discontinuities are easily observed at interlayer boundaries or across structures, making it impossible to guarantee the prediction of continuous structures.

[0035] 4. Difficulty in ensuring consistency across columns In the image width direction (column direction), traditional methods and existing deep learning methods (such as the U-Net series) lack an effective mechanism to maintain the continuity of predictions between columns. Specifically, the U-Net network mainly optimizes global pixel-level accuracy. When most pixels are predicted correctly, the overall score is high, the loss function value is reduced, and the model may converge prematurely, making it difficult to further optimize the local structure.

[0036] The drawbacks are as follows: even if multiple columns are predicted relatively accurately, the structure of the entire image may still be discontinuous or misaligned, especially in region type prediction. Because large areas are predicted correctly, the overall loss value is low, making it difficult for the model to fine-tune key boundaries, thus affecting the accuracy of interlayer boundaries and local structures.

[0037] In view of this, the present application provides a method, system and related equipment for predicting the structure of ultrasound images.

[0038] Please see Figure 1 and Figure 6 As shown, one embodiment of this application proposes a method for predicting structures in ultrasound images. Figure 1 An optional method flowchart provided for this application includes, but is not limited to, steps S101-103, specifically: S101: Acquire ultrasound images and divide the ultrasound images into column images according to preset division rules, wherein the column images include continuous small segments; S102: Input the column image into the Transformer model and generate a structure token through the Transformer model; S103: Based on the structure token, perform structure prediction on the ultrasound image through prediction branches.

[0039] Specifically, the division of the column image in step S101 includes: First, acquire ultrasound images and perform necessary preprocessing on them, including denoising, normalization, and enhancement, to meet the model input requirements. Specifically, an adaptive filter is applied to perform spatial domain filtering on the image, a frequency domain filter is applied to perform transform domain filtering on the image, and a machine learning-based denoising model is applied to predict and remove noise from the image.

[0040] The processed image is divided into several image columns, and each column serves as the input unit of the Transformer model for column-by-column structure prediction.

[0041] Specifically, image normalization operations include: Map the pixel value range of the ultrasound image to the target dynamic range, and calculate the global statistics or statistics based on the region of interest of the ultrasound image pixel values. Based on the aforementioned statistics, grayscale mapping is performed using linear stretching, histogram equalization, or histogram matching methods, and the spatial dimensions of the image are standardized and scaled.

[0042] Specifically, image enhancement includes: A contrast-limited adaptive histogram equalization method is used to enhance local contrast. Homomorphic filtering is applied to simultaneously compress the brightness range and enhance contrast in the frequency domain. An algorithm based on Retinex theory is applied for illumination correction and detail enhancement.

[0043] Specifically, the structural prediction in step S103 includes regional type prediction and hierarchical type prediction.

[0044] Please see Figure 2 As shown, step 102 includes, but is not limited to, steps S201-S204, specifically as follows: S201: Divide the column image into a starting column image and subsequent column images, wherein the starting column image is the column image at the beginning position among all the column images, and the subsequent column images are the column images among all the column images except the starting column image; S202: Perform feature mapping on consecutive small segments of the starting column image and combine it with position encoding to generate a structure token; S203: For any two adjacent subsequent column images, the current column image uses a self-attention mechanism to extract the current column features; S204: The structure token of the previous column image is used as a query value or key value through a cross-attention mechanism and fused with the current column feature to generate the structure token of the current column.

[0045] In some embodiments, the generation of the structure token in step S202 includes: Acquire the ultrasound image to be processed; divide the ultrasound image into at least one column image in the column direction, specifically, divide the ultrasound image into several consecutive, equal-width column images along the horizontal direction; The number, width, or position of the column images are dynamically determined based on the content characteristics of the ultrasound images.

[0046] For each column image, it is sequentially divided into multiple consecutive image blocks along its axial or vertical direction, and each image block contains a fixed number of consecutive small segments; Each of the continuous fields is input into a projection model, that is, a linear layer is used to linearly transform the pixel values ​​of the continuous small segments to generate the feature vector; Alternatively, a small multilayer perceptron (MLP) can be used to perform a nonlinear transformation on the pixel values ​​of the consecutive small segments to generate the feature vector.

[0047] For each feature vector, a position encoding vector is generated or obtained by combining its corresponding position information. The position encoding vector is used to represent the sequential position of the corresponding image block in its respective image column. The final visual token is generated by element-wise summing or concatenating the position encoding vector with the feature vector.

[0048] Through steps S101-103 and S201-204, each column of the input skin image (a one-dimensional vertical pixel vector) is divided into several continuous segments according to preset rules. Each continuous segment consists of several consecutive pixels, and its length can be set according to a fixed pixel interval. Each continuous segment is mapped to the token space through a linear layer or a small MLP as a corresponding structural token, thereby forming a token sequence representing the fine-grained structure of the column. Each structural token can represent a part of the region type (such as an elliptical, circular, or irregularly shaped tissue region or filler) or layer type (such as the transition line between the epidermis and dermis) in the column.

[0049] Based on steps S201-S204, the Transformer model can refer to the structural information of adjacent columns for prediction. This mechanism, which combines self-attention and cross-attention, achieves effective transfer of information between columns and structural continuity modeling. The column-by-column autoregressive prediction mechanism ensures the natural continuity of the entire image in the column direction, significantly reducing holes, breaks, and structural misalignments, while improving the accuracy of region type and hierarchical type prediction.

[0050] This invention can simultaneously model both region types (such as elliptical or irregular tissue regions, fillers) and skin layer boundaries (such as epidermal-dermal transition lines), enabling inter-column information sharing and continuous transmission, thereby effectively avoiding the problems of holes, breaks, and structural misalignments that are prone to occur in traditional convolutional networks.

[0051] Specifically, the structure token in step S202 is used to represent information about the region type (such as elliptical, circular, or irregularly shaped tissue regions or fillers) and hierarchical boundaries (such as transition lines between the epidermis and dermis) in the column. At the same time, the structure token generated in the starting column also provides a basic structural reference for the prediction of structure tokens in subsequent column images and forms the starting point for the structural analysis of the entire column.

[0052] The Transformer model's self-attention and cross-column attention mechanisms can integrate neighboring column information, effectively mitigating the interference of high-intensity noise, especially speckle noise, in ultrasound images on prediction. Even if local columns are affected by noise, the model can still maintain the stability and continuity of the overall structure through neighboring column information.

[0053] Please see Figure 3 As shown, in some embodiments, step S103 includes, but is not limited to, steps S301-S303, specifically: S301: Input the structure token into the MLP classification head of the prediction branch and perform a nonlinear transformation through the hidden layer of the MLP to generate the raw score for each classification. S302: Input the raw scores into the output layer of the MLP, and use the SoftMax function of the output layer to transform the raw scores into the probability distribution of each region classification; S303: Select the category with the highest probability as the region type label corresponding to the structure token. The region type label includes epidermis, dermis, subcutaneous tissue, fascia layer and interlayer transition zone.

[0054] Please see Figure 4 As shown, in some embodiments, step S103 includes, but is not limited to, steps S401-S403, specifically: S401: Input the structure token into the MLP regression head of the prediction branch and perform a nonlinear transformation through the hidden layer of the MLP to generate the boundary score; S402: Input the boundary score into the output layer of the MLP, normalize the boundary score through the Sigmoid function of the output layer, and generate the predicted coordinates; S403: Map the predicted coordinates to pixel-level coordinates of the ultrasound image, and generate a hierarchical type label corresponding to the structure token based on the pixel-level coordinates.

[0055] As can be seen from steps S301-303 and S401-403, after the entire column of Structure Tokens has been predicted by the Transformer, the model can perform classification and regression prediction for each Structure Token: the classification task is used to determine the region type or line type, and the regression task is used to accurately locate the layer boundary or region boundary. In this application, two branches are used for region type prediction or hierarchical regression prediction, specifically: Region type: MLP-SoftMax is used as the classification head for structural prediction. The categories can include (epidermis, dermis, subcutaneous tissue, fascia layer, and interlayer transition areas, etc.). Layer type: MLP-Sigmoid is used as the regression head to calculate layer coordinates, such as the coordinates of the boundary between the epidermis and dermis. Through joint modeling, the consistency between the predicted closed regions and interlayer boundaries is ensured, achieving structurally complete and continuous image reconstruction.

[0056] In some embodiments, the expression for the region type label in step S303 is: ; ; in For region type labels, The activation function for the MLP classification head. For structure tokens, The loss function for the classification task, These are the actual category labels.

[0057] In some embodiments, the expression for the hierarchical type label in step S403 is: ; ; in For hierarchical type tags, The activation function for the MLP regression head. For structure tokens, The loss function for the regression task, For the prediction of the current column image Axis coordinates The current column image for the true skin layer Axis coordinates.

[0058] In some embodiments, after steps S301-303 and S401-403, the method further includes: The column image is stitched together segment by segment to generate a complete structural representation; According to the spatial order of each column of the ultrasound image, all the reconstructed column images are sorted sequentially to generate a complete two-dimensional image structure representation.

[0059] Specifically, after the Structure Token prediction for all columns is completed, all segments within a column are first stitched together to restore the complete structural representation of that column. Then, the reconstructed columns are arranged sequentially according to the spatial order of each column in the original image to form a complete two-dimensional image structural representation, including region type and hierarchical boundaries. The final output structural map not only ensures prediction accuracy but also maintains visual aesthetics and scientific reliability.

[0060] In some embodiments, during the training of the Transformer model, a portion of the previous column's erroneous Structure Token (Negative sample) is intentionally input into the model as reference information for cross-attention. When fusing the erroneous preceding column information, the model needs to learn how to correct potential errors through self-attention and cross-attention mechanisms to avoid the phenomenon of "incorrect initial column prediction leading to completely incorrect subsequent column predictions." This mechanism improves the model's robustness to ultrasound image noise and anomaly prediction, making structure prediction more stable and reliable.

[0061] This invention incorporates an anti-interference strategy during training. By introducing negative samples from the preceding column, the model can correct potential errors, preventing incorrect predictions in the initial column from causing all subsequent columns to be incorrect. This mechanism ensures accurate prediction of key boundaries and local details, improving the structural integrity and visual aesthetics of the entire image.

[0062] In some embodiments, to ensure the continuity of predictions between columns and avoid gaps, breaks, jumps, or structural misalignments, a continuity constraint loss is introduced to ensure that the Structure Tokens in adjacent columns maintain consistency in position and type, thereby reducing gaps, breaks, or structural misalignments. The specific continuity constraint loss function is as follows: ; in For continuous constraint loss, Let x be the predicted coordinate representation of the x-th column. This represents the predicted coordinates of the (x+1)th column.

[0063] Please see Figure 5 As shown, to achieve the above objectives, another aspect of this application proposes an ultrasound image structure prediction system, the system comprising: The image processing module is used to acquire ultrasound images and divide the ultrasound images into column images according to a preset division rule, wherein the column images include continuous small segments; The structure token construction module is used to input the column image into the Transformer model and generate structure tokens through the Transformer model. The structure prediction module is used to perform structure prediction on the ultrasound image based on the structure token and through prediction branches.

[0064] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0065] To achieve the above objectives, another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0066] This application relates to the field of information technology. The ultrasound image structure prediction method provided in the embodiments of this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited thereto; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application implementing the XXX method, but is not limited to the above forms.

[0067] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0068] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0069] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0070] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0071] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0073] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0076] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0077] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for predicting structures in ultrasound images, characterized in that, The method includes: Acquire ultrasound images and divide them into column images according to a preset division rule. Each column image consists of continuous small segments. The column image is input into the Transformer model, and a structure token is generated by the Transformer model. Based on the structure token, structural prediction is performed on the ultrasound image through a prediction branch.

2. The ultrasound image structure prediction method according to claim 1, characterized in that, The step of inputting the column image into the Transformer model and generating a structure token through the Transformer model specifically includes: The column images are divided into a starting column image and subsequent column images. The starting column image is the column image at the beginning of each column image, and the subsequent column images are the column images other than the starting column image. Feature mapping is performed on consecutive small segments of the starting column image, and a structure token is generated by combining it with position encoding; For any two adjacent subsequent column images, the current column image employs a self-attention mechanism to extract the current column features; The structure token of the previous column image is used as a query value or key value through a cross-attention mechanism and fused with the features of the current column to generate the structure token of the current column.

3. The ultrasound image structure prediction method according to claim 2, characterized in that, The step of predicting the structure of an ultrasound image based on the structure token, through a prediction branch, includes: The structure token is input into the MLP classification head of the prediction branch, and a nonlinear transformation is performed through the hidden layer of the MLP to generate the raw score for each class. The raw scores are input into the output layer of the MLP, and the SoftMax function of the output layer transforms the raw scores into probability distributions for each region classification. The category with the highest probability is selected as the region type label corresponding to the structural token. The region type label includes the epidermis, dermis, subcutaneous tissue, fascia layer, and interlayer transition zone.

4. The ultrasound image structure prediction method according to claim 1, characterized in that, The method of predicting the structure of ultrasound images based on the structure token and through prediction branches also includes: The structure token is input into the MLP regression head of the prediction branch, and a nonlinear transformation is performed through the hidden layer of the MLP to generate the boundary score. The boundary scores are input into the output layer of the MLP, and the boundary scores are normalized by the Sigmoid function of the output layer to generate predicted coordinates. The predicted coordinates are mapped to pixel-level coordinates of the ultrasound image, and a hierarchical type label corresponding to the structure token is generated based on the pixel-level coordinates.

5. The ultrasound image structure prediction method according to claim 1, characterized in that, After performing structural prediction on the ultrasound image based on the structural token and through prediction branches, the process further includes: The column image is stitched together segment by segment to generate a complete structural representation; According to the spatial order of each column of the ultrasound image, all the reconstructed column images are sorted sequentially to generate a complete two-dimensional image structure representation.

6. The ultrasound image structure prediction method according to claim 3, characterized in that, The expression for the region type label is: ； ； in For region type labels, The activation function for the MLP classification head. For structure tokens, The loss function for the classification task, These are the actual category labels.

7. The ultrasound image structure prediction method according to claim 4, characterized in that, The expression for the hierarchical type label is: ； ； in For hierarchical type tags, The activation function for the MLP regression head. For structure tokens, The loss function for the regression task, For the prediction of the current column image Axis coordinates The current column image for the true skin layer Axis coordinates.

8. An ultrasound image structure prediction system, wherein the system is applied to the ultrasound image structure prediction method according to any one of claims 1-7, characterized in that, The system includes: The image processing module is used to acquire ultrasound images and divide the ultrasound images into column images according to a preset division rule, wherein the column images include continuous small segments; The structure token construction module is used to input the column image into the Transformer model and generate structure tokens through the Transformer model. The structure prediction module is used to perform structure prediction on the ultrasound image based on the structure token and through prediction branches.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images