Bone metastasis lesion positioning method and device, electronic equipment and storage medium
Patent Information
- Application Number
- CN202410561311.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-08
- Publication Date
- 2026-10-13
- Estimated Expiration
- 2044-05-08
AI Technical Summary
同时，全身骨扫描影像还有分辨率较低、病灶前景小的特征，这些将为影像解读造成困难
[0042]本申请实施例方法通过获取骨扫描影像，提取骨扫描影像的特征，并定位骨转移病灶位置，得到原始特征序列，原始特征序列包括骨转移病灶位置信息，然后对原始特征序列进行不同尺度间的特征融合，得到融合特征信息，最终根据融合特征信息定位骨转移病灶。实现了骨转移病灶的自动识别和定位，大大节省了人力和定位时间，并保证了定位结果的一致性。
Smart Images
Figure CN118552481B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of data processing technology, specifically relating to a method, device, electronic device, and storage medium for locating bone metastases. Background Technology
[0002] Breast cancer is a major global health challenge, becoming a leading health problem for women due to its high incidence and mortality rates. Breast cancer is characterized by high heterogeneity and a high rate of metastasis and recurrence, with bone being one of the most common sites of metastasis in breast cancer patients. Screening for bone metastases in breast cancer patients is crucial for early detection and timely treatment. Whole-body bone scanning is the most commonly used method for assessing bone metastases in breast cancer. This technique is highly sensitive, capable of whole-body examination, easy to standardize, and inexpensive, and is therefore widely used in clinical practice and trial guidelines.
[0003] From whole-body bone scan images, breast cancer bone metastases are often multiple, meaning multiple lesions (sometimes densely distributed) frequently appear on the same image. Common sites of bone metastasis include the spine, sternum, and ribs. Furthermore, whole-body bone scan images are characterized by low resolution and small foreground areas of lesions, which complicates image interpretation. Localization on whole-body bone scan images is a labor-intensive task, especially time-consuming for inexperienced physicians, and the consistency of localization results is often poor. Summary of the Invention
[0004] The purpose of this application is to provide a method, device, electronic device, and storage medium for locating bone metastases to save time and improve the consistency of the location results.
[0005] According to a first aspect of the embodiments of this application, a method for locating bone metastatic lesions is provided, the method including:
[0006] Obtain bone scan images;
[0007] Features were extracted from bone scan images to obtain the original feature sequence;
[0008] Auxiliary features of bone scan images were extracted and the location of bone metastases was determined to obtain location information of bone metastases.
[0009] The original feature sequence, auxiliary features, and bone metastasis lesion location information were fused at different scales to obtain fused feature information.
[0010] Bone metastatic lesions can be located based on fusion feature information.
[0011] In some optional embodiments of this application, the auxiliary features include: shallow texture features and high-level semantic features; the auxiliary features for extracting bone scan images include:
[0012] By extracting shallow texture features from bone scan images and gradually expanding the receptive field, high-level semantic features are obtained.
[0013] In some optional embodiments of this application, locating the site of bone metastases includes:
[0014] Calculate the binary information entropy of bone scan images.
[0015] In some optional embodiments of this application, the binary information entropy is calculated using the following formula:
[0016] H(X) = -plogp - (1-p)log(1-p)
[0017] Where p is the probability that a pixel in a bone scan image is predicted as a bone metastasis, and H(X) is the binary information entropy of the pixel.
[0018] In some optional embodiments of this application, feature fusion at different scales is performed on the original feature sequence, auxiliary features, and bone metastasis lesion location information to obtain fused feature information, including:
[0019] The original feature sequence is concatenated and fused with auxiliary features to obtain the initial mixed feature;
[0020] Extract the correlation of the initial mixed features;
[0021] Fusion feature information is obtained by fusing the initial mixed features based on correlation and the location information of bone metastases.
[0022] In some optional embodiments of this application, the correlation of the initial mixed features is extracted, including:
[0023] Downsample the initial mixed features;
[0024] Capture local features and details of low-level feature sequences;
[0025] Capture the contextual relationships of high-level feature sequences.
[0026] In some optional embodiments of this application, locating bone metastatic lesions based on fusion feature information includes:
[0027] Predefine prior boxes on the feature map grid points of the fused feature information to cover the shape and size of the target;
[0028] A set of prediction vectors is generated based on each feature map grid point and its associated prior box to measure the quality of the prediction box and filter out background predictions.
[0029] Non-maximum suppression is applied to the predicted vector to obtain the location and confidence level of bone metastases.
[0030] According to a second aspect of the embodiments of this application, a bone metastasis lesion localization device is provided, the device comprising:
[0031] The acquisition module is used to acquire bone scan images;
[0032] The feature extraction module is used to extract features from bone scan images to obtain the original feature sequence;
[0033] The auxiliary localization module is used to extract auxiliary features from bone scan images and locate the location of bone metastases to obtain bone metastasis location information.
[0034] The fusion module is used to fuse the original feature sequence, auxiliary features, and bone metastasis lesion location information at different scales to obtain fused feature information.
[0035] The localization module is used to locate bone metastases based on fusion feature information.
[0036] According to a third aspect of the embodiments of this application, an electronic device is provided, which may include:
[0037] processor;
[0038] Memory used to store processor-executable instructions;
[0039] The processor is configured to execute instructions to implement the bone metastasis lesion localization method as shown in any embodiment of the first aspect.
[0040] According to a fourth aspect of the embodiments of this application, a storage medium is provided, wherein when the instructions in the storage medium are executed by a processor of an information processing device or a server, the information processing device or server implements the bone metastasis lesion localization method as shown in any embodiment of the first aspect.
[0041] The above-mentioned technical solution of this application has the following beneficial technical effects:
[0042] The method described in this application acquires bone scan images, extracts features from the bone scan images, and locates the bone metastases to obtain an original feature sequence. This original feature sequence includes information about the location of the bone metastases. Then, the original feature sequence is fused across different scales to obtain fused feature information. Finally, the bone metastases are located based on this fused feature information. This method achieves automatic identification and location of bone metastases, significantly saving manpower and location time, and ensuring the consistency of the location results. Attached Figure Description
[0043] Figure 1 This is a flowchart illustrating the bone metastasis lesion localization method in an exemplary embodiment of this application;
[0044] Figure 2 This is an architectural diagram of a bone metastasis lesion localization system in an exemplary embodiment of this application;
[0045] Figure 3 This is a schematic diagram of the structure of the bone metastasis lesion localization device in an exemplary embodiment of this application;
[0046] Figure 4 This is a schematic diagram of the structure of an electronic device in an exemplary embodiment of this application;
[0047] Figure 5 This is a schematic diagram of the hardware structure of an electronic device in an exemplary embodiment of this application. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this application. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.
[0049] The accompanying drawings illustrate layer structure diagrams according to embodiments of this application. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.
[0050] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0051] In the description of this application, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0052] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.
[0053] The bone metastasis lesion localization method, device, electronic device, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.
[0054] like Figure 1As shown, in a first aspect of this application, a method for locating bone metastatic lesions is provided, the method including:
[0055] S110: Acquire bone scan images;
[0056] S120: Extract features from bone scan images to obtain the original feature sequence;
[0057] S130: Extract auxiliary features from bone scan images and locate the location of bone metastases to obtain bone metastasis location information;
[0058] S140: Perform feature fusion at different scales on the original feature sequence, auxiliary features, and bone metastasis lesion location information to obtain fused feature information;
[0059] S150: Locate bone metastases based on fusion feature information.
[0060] The method in this embodiment enables automatic identification and localization of bone metastases, greatly saving manpower and localization time, and ensuring the consistency of localization results.
[0061] In the specific implementation process, a bone metastasis lesion detection system can be used. This system may include a feature extraction module, an auxiliary localization module, a feature fusion module, and a detection head module, such as... Figure 2 As shown, the image is simultaneously input into the feature extraction module and the auxiliary localization module for processing. The output of the feature extraction module is used as the main feature sequence and then fed into the feature fusion module. During the processing of this module, the features extracted by the auxiliary localization module are gradually fused together. Finally, the mixed features are input into the improved detection head module to obtain the target bounding box output.
[0062] This embodiment addresses the challenges of low resolution, numerous lesions, and difficulty in feature extraction in whole-body bone scan images. It incorporates an auxiliary localization module into the deep learning network to mine lesion location information; introduces a self-attention mechanism into the feature fusion process to enhance the system's global understanding of the images; and sets up a small target detection head for the system to improve the detection rate of small lesions.
[0063] The weights of the auxiliary localization module were obtained and fixed through training on pixel-level annotations of bone metastases. Specifically, a batch of whole-body bone scan images (510 images in JPEG format) from breast cancer patients with bone metastases were collected, covering various lesion morphologies, sizes, locations, and lesion conditions of different bone structures. The contours of the bone metastases were manually delineated by a radiologist with 10 years of experience using ITK-SNAP, generating corresponding binary mask images (pixel-level annotations). The images, after being uniformly sized and normalized, were divided into training and testing sets at a 4:1 ratio.
[0064] The auxiliary localization module consists of an encoder and a decoder. The encoder uses a ResNet_50 pre-trained model to extract multi-scale features from the input image through a series of convolutional and pooling layers. The decoder restores the spatial resolution of the feature map through upsampling operations and concatenates the features of the corresponding layer of the encoder with the features of the current layer of the decoder, fusing shallow detail information with deep semantic information. During training, this module reads in the patient's whole-body bone scan image, performs forward propagation to output a binary segmentation prediction map of the lesion location and shape, and compares it with the correct binary mask annotation to obtain the focal loss. Based on the gradient of the loss function, the Adam optimizer is used for backpropagation to adjust the model weights.
[0065] To address the limited number of annotations, the weights of the pre-trained model are utilized. First, 50 rounds of training are performed to freeze the parameters of the backbone module (encoder feature extraction network) and update the parameters of the decoder part of the module with a batch size of 4. This allows the module to focus on learning high-level features more directly related to lesion segmentation. Then, 50 rounds of training are performed on the entire module with a batch size of 2 to update all parameters, stabilizing the module's performance. Compared to object detection networks, this module can extract features from images with finer granularity. However, due to the difficulty and limited quantity of pixel-level annotations, the module's generalization ability is limited; therefore, only auxiliary features are provided here. After individual training, the module's parameters will not change with the overall training of the object detection system. For the input bone scan image, the module will obtain two types of output: one is auxiliary features (composed of three sets of feature sequences) that simultaneously contain high-level semantic features and shallow texture features; the second is information entropy providing lesion location information.
[0066] This module consists of an encoder and a decoder block. The encoder includes convolution and max pooling operations to initially extract features and gradually expand the receptive field. The decoder performs upsampling while fusing shallow features collected from the encoder. To utilize both low-level and high-level features simultaneously, three sets of feature sequences obtained during the upsampling phase of the decoder block are selected as the first output of the auxiliary localization module. To facilitate feature fusion, this embodiment selects three sets of feature sequences with dimensions similar to the feature map size of the feature receiving point. These three sets of feature sequences contain both low-level texture features and high-level global features, which will improve the difficulty of effective feature extraction caused by the low resolution of bone scan images.
[0067] In some embodiments, the auxiliary features include: shallow texture features and high-level semantic features; the auxiliary features extracted from bone scan images include:
[0068] By extracting shallow texture features from bone scan images and gradually expanding the receptive field, high-level semantic features are obtained.
[0069] In some embodiments, locating the site of bone metastases includes:
[0070] Calculate the binary information entropy of bone scan images.
[0071] In some embodiments, the binary information entropy is calculated using the following formula:
[0072] H(X) = -plogp - (1-p)log(1-p)
[0073] Where p is the probability that a pixel in a bone scan image is predicted as a bone metastasis, and H(X) is the binary information entropy of the pixel.
[0074] The formula in this embodiment is used to further mine the features of bone scan images, providing the system with accurate lesion localization information. Binary information entropy can effectively delineate the boundaries of the lesions. These features, including lesion location, will be the second output of the auxiliary localization module and will also be provided to the feature fusion module for fusion and utilization.
[0075] In some embodiments, feature fusion at different scales is performed on the original feature sequence, auxiliary features, and bone metastasis lesion location information to obtain fused feature information, including:
[0076] The original feature sequence is concatenated and fused with auxiliary features to obtain the initial mixed feature;
[0077] Extract the correlation of the initial mixed features;
[0078] Fusion feature information is obtained by fusing the initial mixed features based on correlation and the location information of bone metastases.
[0079] Specifically, taking three sets as an example, after obtaining three sets of feature sequences at different scales extracted by the system's own feature extraction module and auxiliary localization module, the neck module of the target detection network YOLOv5 can be used to perform feature fusion and global feature correlation extraction.
[0080] First, the three sets of multi-level feature sequences obtained by the auxiliary positioning module are received at the positions corresponding to the feature dimensions and then fused in a concatenated manner in the channel dimension.
[0081] Secondly, a self-attention mechanism is introduced using three ST_Encoder (Swin Transformer Encoder, sliding window self-attention model encoder) blocks to improve the model's awareness of global feature correlations;
[0082] Third, add a T_Decoder (Transformer Decoder, self-attention model decoder) block to receive and fuse the location features extracted from the auxiliary localization module to obtain more lesion location information.
[0083] In this embodiment, after the original feature sequence is input into the feature fusion module, it will be upsampled. During this process, the size of the feature sequence changes (becomes larger). When the size is the same as that of the auxiliary features (one of the three groups), the auxiliary features are fused into the original feature sequence by splicing, resulting in the initial mixed features.
[0084] After the correlation extraction is completed, the feature sequence will be downsampled step by step to adapt to the size requirements of the feature sequence for different target detection heads. The initial mixed features generated in this process first pass through the T_Decoder module to fuse the bone metastasis lesion location information (i.e., binary information entropy) to obtain fused feature information.
[0085] In some embodiments, extracting the correlation of the initial mixed features includes:
[0086] Downsample the initial mixed features;
[0087] Capture local features and details of low-level feature sequences;
[0088] Capture the contextual relationships of high-level feature sequences.
[0089] The correlation is extracted after the auxiliary features are assembled and before the bone metastasis lesion localization information is fused. This correlation can help the model more effectively aggregate information across regions and levels, thereby enhancing its ability to determine the presence of bone metastases and to accurately locate bounding boxes.
[0090] Because breast cancer bone metastases exhibit certain distributional characteristics, such as being mostly located in the spine, sternum, and ribs, and patients often have multiple metastatic lesions, fully extracting the relationships between foreground and background, and between foreground and other foreground elements, is essential for grasping the overall picture.
[0091] Three self-attention encoder blocks (ST_Encoder) can be added at different levels during the downsampling stage. As the downsampling stage deepens, the receptive field gradually increases, and these three encoder blocks can capture objects and their relationships at different scales and complexities. Lower-level encoder blocks mainly focus on local features and details, while higher-level blocks tend to capture broader and more complex contextual information. In this way, the self-attention encoder blocks at different levels jointly construct multi-level, multi-scale associations between objects, making better use of the global information contained in the extracted features.
[0092] Specifically, a self-attention mechanism incorporating sliding window operations can be used to quickly extract relevant information between pixels. After layer normalization (LN) and window partitioning (dividing the entire feature map into multiple uniformly sized windows for simultaneous processing to improve image processing efficiency),
[0093] ST_Encoder first performs in-window self-attention (Window-based MSA, W-MSA), which means that for the feature values of each pixel in the window, a linear transformation (1x1 convolutional layer) is used to generate corresponding key, value, and query vectors respectively. Then, the dot product of the query vector and all key vectors is calculated, and the attention weights are obtained by normalization through the softmax function. Finally, these weights are multiplied element-wise with the corresponding value vector in the window and summed to obtain the attention aggregation result of each pixel in the current window, thus obtaining the semantic relationship between different pixels in the same window.
[0094] Then, non-linear changes in word vectors are achieved through an MLP layer.
[0095] The module then uses window sliding (SW-MSA) to translate the window horizontally or vertically, giving pixels that were not originally in the same window the opportunity to interact directly within the newly defined window. This establishes cross-window information exchange and gradually reveals the relationships between objects in the image. Repeating the above steps completes global feature extraction.
[0096] To fuse features contained in the information entropy obtained from the auxiliary localization module, a self-attention model decoder (T_Decoder) can be added to perform the localization information fusion step after the main features have undergone global information extraction and before the final processing step in the input detection head module. Similar to the encoder, the self-attention model decoder also transforms the input feature map into key, value, and query vectors. Here, to fuse the location features contained in the binary information entropy, the query vector of the original main features can be discarded, and the binary information entropy can be used as a new query vector. This is combined with the key and value vectors of the original main features to complete the self-attention calculation (Multi-Head Attention, MHA). By calculating the dot product of the query vector and all key vectors, attention weights containing lesion location features can be obtained, realizing the fusion between the system's main features and the features extracted by the auxiliary localization module. The lesion location features contained in the information entropy can guide the system to focus on sites where lesions are more likely to occur.
[0097] In some embodiments, locating bone metastatic lesions based on fusion feature information includes:
[0098] Predefine prior boxes on the feature map grid points of the fused feature information to cover the shape and size of the target;
[0099] A set of prediction vectors is generated based on each feature map grid point and its associated prior box to measure the quality of the prediction box and filter out background predictions.
[0100] Non-maximum suppression is applied to the predicted vector to obtain the location and confidence level of bone metastases.
[0101] Since the area of a single bone metastasis lesion in a whole-body bone scan is usually small, the target bounding boxes predicted by the detection head in the original YOLOv5 network are relatively large, making them unsuitable for bone lesion detection tasks. To improve the ability to detect small lesions, this embodiment adds a small target detection head, which is used to process larger-dimensional feature images containing more details, thereby improving the overall detection performance of the system.
[0102] The small target detection head is mainly responsible for predicting the location (predicted bounding box coordinates) and confidence level of lesions from the feature map. Specifically, the detection head first predefines a set of prior boxes on the grid points of the input feature map. These prior boxes have different aspect ratios and sizes, aiming to cover the shapes and sizes of common targets.
[0103] During forward propagation, for each grid point and its associated prior box, the small object detection head generates a set of prediction vectors. Each prediction vector contains: class probability (the probability that each prior box belongs to a bone metastasis lesion or the foreground), bounding box coordinate adjustment value (the offset relative to the preset prior box position), and object confidence (the prediction of whether the prior box contains a lesion and the intersection-over-union ratio of the predicted box with the real object, used to measure the quality of the predicted box and filter out background predictions).
[0104] During the inference phase, the output of the small target detection head undergoes post-processing steps such as non-maximum suppression (NMS) to remove redundant predictions, retain the best prediction box for each lesion, and finally output the location and confidence of the lesion in the bone scan image.
[0105] Hybrid feature sequences can be input into different small target detection heads for processing based on their scale. The larger the scale of the feature sequence, the more comprehensively the detailed texture features of the original image are preserved, and the easier it is for the small target detection head to predict the location of lesions. A larger-scale feature transmission channel can be added, allowing the feature sequence with the largest scale and most shallow texture features to be input into a newly configured small target detection head, thereby improving the detection rate of smaller lesions.
[0106] The above-described embodiments demonstrate the ability to mine lesion location features and other texture and semantic features, addressing the difficulty in extracting effective features due to the low resolution of bone scan images. The introduction of a self-attention mechanism into the feature fusion process allows for correlation analysis of the extracted features, enhancing the system's global understanding of the images and thus adapting to the situation of multiple lesions in breast cancer bone metastases. The small target detection head can handle large-scale features containing more details, improving the detection rate of small lesions in patients with early-stage bone metastases.
[0107] It should be noted that the bone metastasis lesion localization method provided in this application embodiment can be executed by a bone metastasis lesion localization device, or a control module in the bone metastasis lesion localization device for executing the bone metastasis lesion localization method. This application embodiment uses the bone metastasis lesion localization device executing the bone metastasis lesion localization method as an example to illustrate the bone metastasis lesion localization device provided in this application embodiment.
[0108] like Figure 3 As shown, in a second aspect of the embodiments of this application, a bone metastasis lesion localization device is provided, the device may include:
[0109] Acquisition module 310 is used to acquire bone scan images;
[0110] Feature extraction module 320 is used to extract features from bone scan images to obtain the original feature sequence;
[0111] The auxiliary positioning module 330 is used to extract auxiliary features from bone scan images and locate the location of bone metastatic lesions.
[0112] The fusion module 340 is used to perform feature fusion at different scales on the original features, auxiliary features and bone metastasis lesion location information to obtain fused feature information.
[0113] The positioning module 350 is used to locate bone metastatic lesions based on fusion feature information.
[0114] The bone metastasis lesion localization device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.
[0115] The bone metastasis lesion localization device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.
[0116] The bone metastasis lesion localization device provided in this application embodiment can achieve… Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.
[0117] Optionally, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a program or instructions stored in the memory 402 and executable on the processor 401. When the program or instructions are executed by the processor 401, they implement the various processes of the above-described bone metastasis lesion localization method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.
[0118] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0119] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.
[0120] The electronic device 500 includes, but is not limited to, components such as: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.
[0121] Those skilled in the art will understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.
[0122] It should be understood that, in this embodiment, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 507 includes a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here. The memory 509 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understandable that the aforementioned modem processor may not be integrated into processor 510.
[0123] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described bone metastasis lesion localization method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0124] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0125] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described bone metastasis lesion localization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0126] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0129] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for locating bone metastatic lesions, characterized in that, include: Obtain bone scan images; Features are extracted from the bone scan images to obtain the original feature sequence; Auxiliary features of the bone scan images are extracted and the location of bone metastases is located to obtain bone metastasis location information; The original feature sequence, the auxiliary features, and the location information of the bone metastases are fused at different scales to obtain fused feature information. Bone metastatic lesions are located based on the fusion feature information; The auxiliary features include: shallow texture features and high-level semantic features; the auxiliary features for extracting the bone scan image include: Extracting superficial texture features from the bone scan images and gradually expanding the receptive field yields high-level semantic features; locating the bone metastasis lesion includes: Calculate the binary information entropy of the bone scan image; The process of fusing features at different scales from the original feature sequence, the auxiliary features, and the location information of the bone metastases to obtain fused feature information includes: The original feature sequence is concatenated and fused with the auxiliary features to obtain the initial mixed feature; The correlation of the initial mixed features is extracted using the neck module of the YOLOv5 object detection network. The YOLOv5 object detection network introduces a self-attention mechanism using a sliding window self-attention model encoder and uses a self-attention model decoder to receive and fuse the position features extracted from the auxiliary localization module. The correlation is extracted after the auxiliary features are stitched together and before the location information of the bone metastasis lesions is fused. Based on the correlation, the initial mixed features are fused with the location information of the bone metastases to obtain fused feature information.
2. The method for locating bone metastases according to claim 1, characterized in that, The binary information entropy is calculated using the following formula: ； in, p The probability that a pixel in the bone scan image is predicted to be a bone metastasis lesion. H ( X ) represents the binary information entropy of the pixel.
3. The method for locating bone metastases according to claim 1, characterized in that, The extraction of the correlation of the initial mixed features includes: The initial mixed features are downsampled; Capture local features and details of low-level feature sequences; Capture the contextual relationships of high-level feature sequences.
4. The method for locating bone metastases according to claim 1, characterized in that, The step of locating bone metastatic lesions based on the fusion feature information includes: Prior bounding boxes are predefined on the feature map grid points of the fused feature information to cover the shape and size of the target; Based on each feature map grid point and its associated prior box, a set of prediction vectors is generated to measure the quality of the prediction box and filter out background predictions. The predicted vector is subjected to nonmaximum suppression to obtain the location and confidence level of bone metastases.
5. A device for locating bone metastases, characterized in that, include: The acquisition module is used to acquire bone scan images; The feature extraction module is used to extract features from the bone scan image to obtain the original feature sequence; An auxiliary localization module is used to extract auxiliary features from the bone scan image and locate the bone metastasis lesion, obtaining bone metastasis lesion location information. The auxiliary features include: superficial texture features and high-level semantic features. Specifically, the auxiliary localization module is used to extract the superficial texture features from the bone scan image and gradually expand the receptive field to obtain high-level semantic features. The auxiliary localization module is also used to calculate the binary information entropy of the bone scan image. A fusion module is used to perform feature fusion at different scales on the original feature sequence, the auxiliary features, and the bone metastasis lesion location information to obtain fused feature information. Specifically, the fusion module is used to... The original feature sequence is concatenated and fused with the auxiliary features to obtain an initial mixed feature. The correlation of the initial mixed feature is extracted using the neck module of the YOLOv5 target detection network. The YOLOv5 target detection network uses a sliding window self-attention model encoder to introduce a self-attention mechanism, and a self-attention model decoder to receive and fuse the positional features extracted from the auxiliary localization module. The correlation is extracted after concatenating the auxiliary features and before fusing the bone metastasis location information. Based on the correlation, the initial mixed feature and the bone metastasis location information are fused to obtain fused feature information. The positioning module is used to locate bone metastatic lesions based on the fusion feature information.
6. An electronic device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the bone metastasis localization method as described in any one of claims 1-4.
7. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the bone metastasis lesion localization method as described in any one of claims 1-4.