Image processing method and system, electronic equipment, storage medium and computer program product

Abstract

The invention discloses an image processing method and system, electronic equipment, a storage medium and a computer program product, and relates to the technical field of artificial intelligence and image processing. The method comprises the steps that multi-scale feature extraction is carried out on a to-be-processed image of a target part, a first feature map and a second feature map are obtained, and the resolution ratio of the second feature map is lower than that of the first feature map; performing feature interaction calculation based on the first feature map and the second feature map to obtain fused feature data; and performing perceptual mapping on the fused feature data to obtain a target segmentation result and a target classification result, the target segmentation result being used for identifying a target pixel region in the to-be-processed image, and the target classification result being used for representing an attribute category corresponding to the target pixel region. The technical problem that the segmentation accuracy and the classification accuracy are low due to the scheme of independently processing the segmentation task and the classification task in the related technology is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and image processing, in particular to an image processing method and system, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] With the development of artificial intelligence technology, deep learning networks are increasingly widely applied to solve image processing tasks. In particular, in some scenarios, it is often necessary to perform region of interest segmentation based on images (i.e., a segmentation task) and object classification based on images (i.e., a classification task). In related technologies, the segmentation task and the classification task corresponding to the same image are mainly processed through independent processes, which makes it difficult to fully exploit image features in the image processing or analysis process, thereby limiting the improvement of the accuracy of the segmentation result and the classification result.

[0003] At present, no effective solution has been proposed to solve the above problems. SUMMARY

[0004] Embodiments of the present application provide an image processing method, system, electronic device, storage medium and computer program product to at least solve the technical problem of low segmentation accuracy and classification accuracy caused by the scheme of independently processing the segmentation task and the classification task in related technologies.

[0005] According to an aspect of an embodiment of the present application, an image processing method is provided, including: performing multi-scale feature extraction on a target part of a to-be-processed image to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; performing feature interaction calculation based on the first feature map and the second feature map to obtain fusion feature data; and performing perceptual mapping on the fusion feature data to obtain a target segmentation result and a target classification result, wherein the target segmentation result is used to identify a target pixel region in the to-be-processed image, and the target classification result is used to represent an attribute category corresponding to the target pixel region.

[0006] According to another aspect of an embodiment of the present application, a computer-aided diagnosis method for breast cancer is also provided, including: performing multi-scale feature extraction on a computed tomography image of a human chest to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; performing feature interaction calculation based on the first feature map and the second feature map to obtain fusion feature data; and performing perceptual mapping on the fusion feature data to obtain a target segmentation result and a target classification result, wherein the target segmentation result is used to identify a breast lesion region in the computed tomography image, and the target classification result is used to represent a benign or malignant category corresponding to the breast lesion region.

[0007] According to another aspect of the embodiments of the present application, an image processing method is also provided, including: obtaining an image processing request through a first application programming interface, wherein the request data carried in the image processing request includes a to-be-processed image of a target part; and returning an image processing response through a second application programming interface, wherein the response data carried in the image processing response includes a target segmentation result and a target classification result, which are generated according to the image processing method of any one of the above.

[0008] According to another aspect of the embodiments of the present application, an image processing method is also provided, including: obtaining a current input image processing conversation request, wherein the request data carried in the image processing conversation request includes a to-be-processed image of a target part; in response to the image processing conversation request, returning an image processing conversation reply, wherein the information carried in the image processing conversation reply includes a target segmentation result and a target classification result, which are generated according to the image processing method of any one of the above; and presenting the target segmentation result and the target classification result in a graphical user interface.

[0009] According to another aspect of the embodiments of the present application, an image processing method is also provided, including: in response to an input instruction acting on an operation interface, displaying a to-be-processed image of a target part on the operation interface; and in response to a processing instruction acting on the operation interface, displaying a target segmentation result and a target classification result on the operation interface, wherein the target segmentation result and the target classification result are generated according to the image processing method of any one of the above.

[0010] According to another aspect of the embodiments of the present application, an image processing system is also provided, including: a client configured to send a target topic; and a server connected with the client, configured to perform multi-scale feature extraction on a to-be-processed image of a target part to obtain a first feature map and a second feature map, perform feature interaction calculation based on the first feature map and the second feature map to obtain fusion feature data, and perform perception mapping on the fusion feature data to obtain a target segmentation result and a target classification result, wherein the resolution of the second feature map is lower than that of the first feature map, the target segmentation result is used to identify a target pixel region in the to-be-processed image, and the target classification result is used to represent an attribute category corresponding to the target pixel region; and the client is further configured to output the target segmentation result and the target classification result.

[0011] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory storing an executable program; and a processor configured to run the program, wherein the program, when running, performs the image processing method of any one of the above or the computer-aided diagnosis method of breast cancer.

[0012] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which includes a stored executable program, wherein the executable program controls the device where the computer readable storage medium is located to perform the image processing method or the computer-aided diagnosis method of breast cancer of any one of the above when the executable program is executed.

[0013] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes a computer program, and the computer program implements the image processing method or the computer-aided diagnosis method of breast cancer of any one of the above when executed by a processor.

[0014] In the embodiments of the present application, multi-scale feature extraction is performed on the to-be-processed image of the target part to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; feature interaction calculation is performed based on the first feature map and the second feature map to obtain fusion feature data; and perception mapping is performed on the fusion feature data to obtain a target segmentation result and a target classification result, wherein the target segmentation result is used to identify a target pixel region in the to-be-processed image, and the target classification result is used to represent an attribute category corresponding to the target pixel region. Thus, the embodiments of the present application achieve the purpose of jointly processing the segmentation task and the classification task based on feature interaction calculation of feature maps with different resolutions, thereby realizing the technical effect of cooperatively improving the accuracy of the segmentation result and the classification result, and further solving the technical problem of low segmentation accuracy and classification accuracy caused by the scheme of independently processing the segmentation task and the classification task in the related art.

[0015] It is easy to note that the general description above and the detailed description below are only for exemplifying and explaining the present application, and do not constitute a limitation on the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is an application scenario schematic diagram of an image processing method according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of an image processing method according to an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of an optional image preprocessing process according to an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of an optional image main processing process according to an embodiment of the present application;

[0021] Figure 5 This is a flowchart of a computer-aided diagnosis method for breast cancer according to an embodiment of this application;

[0022] Figure 6 This is a flowchart of another image processing method according to an embodiment of this application;

[0023] Figure 7 This is a flowchart of another image processing method according to an embodiment of this application;

[0024] Figure 8 This is a flowchart of another image processing method according to an embodiment of this application;

[0025] Figure 9 This is a structural block diagram of an image processing apparatus according to an embodiment of this application;

[0026] Figure 10 This is a structural block diagram of a computer-aided diagnostic device for breast cancer according to an embodiment of this application;

[0027] Figure 11 This is a structural block diagram of another image processing apparatus according to an embodiment of this application;

[0028] Figure 12 This is a structural block diagram of another image processing apparatus according to an embodiment of this application;

[0029] Figure 13 This is a structural block diagram of another image processing apparatus according to an embodiment of this application;

[0030] Figure 14 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0034] The technical solution provided in this application is mainly implemented using a deep learning model. Deep learning models can be widely applied in fields such as Natural Language Processing (NLP), computer vision, and speech processing. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as to NLP tasks such as text-based sentiment classification, text summarization, and machine translation. Therefore, the main application scenarios of this application include, but are not limited to, digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0035] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows.

[0036] Lesion detection: In three-dimensional medical images, segmentation techniques are used to identify and locate potential tumors or abnormal areas within a target site (such as the breast) to assist doctors in preliminary screening.

[0037] Malignancy classification: The process of qualitatively analyzing the health status of a target area (such as the breast) based on medical images. Specifically, it involves determining whether a tumor exists in the target area based on medical images, and distinguishing between benign and malignant tumors (e.g., malignant, benign, and normal).

[0038] Non-contrast chest computed tomography (NCCT) is a computed tomography technique commonly used in chest examinations. It does not require contrast agents and can naturally capture chest structures, including the breast, with lower cost and radiation exposure.

[0039] Mamba-3D is a 3D vision modeling architecture based on a selective state-space model. It has linear time complexity, is suitable for processing long sequences of 3D data (such as 3D images), and can effectively capture global and local features.

[0040] Dual-resolution mechanism: This refers to a resolution mechanism in model design where the network structure includes two paths corresponding to different resolutions. The high-resolution path focuses on capturing detailed features of the image and is suitable for voxel-level segmentation, while the low-resolution path focuses on global contextual features and is suitable for global classification. In the dual-resolution mechanism, information between the high-resolution and low-resolution paths is coordinated through a fusion mechanism.

[0041] Tandem Input mechanism: This refers to an input mechanism in model design that simultaneously inputs symmetrical features from both sides of the target area (such as the left and right breast features of the chest) into the model's computation unit. This allows the computation unit to analyze the potential asymmetry between the two sides of the features, thereby enhancing the accuracy of segmentation and classification.

[0042] According to an embodiment of this application, an image processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, steps shown or described may be executed in a different order than that shown here.

[0043] The technical solutions provided in this application can employ deep learning models with relatively large parameter scales, such as large models containing billions or even more model parameters. Here, "large model" is merely an example; this application does not limit the number of model parameters supported by the deep learning model used, aiming to meet actual needs. The deep learning models involved in this application can be artificial intelligence-based language models (LanguageMode, LM) or multimodal models (Multimodal Model, MM).

[0044] Considering the limited computing resources of mobile terminals, the methods described above in this application embodiment can be applied to, for example... Figure 1 The application scenarios shown are not limited to these. In, for example... Figure 1 In the application scenario shown, the deep learning model is deployed on server 10. Server 10 can connect to one or more client devices 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client devices 20 can interact with users through a graphical user interface to invoke the deep learning model, thereby implementing the method provided in this embodiment.

[0045] In this embodiment, the system consisting of a client device and a server can perform the following steps: the client device uploads the image of the target region to be processed to the server; the server performs multi-scale feature extraction on the image of the target region to be processed, obtaining a first feature map and a second feature map; based on the first and second feature maps, feature interaction calculation is performed to obtain fused feature data; perceptual mapping is performed on the fused feature data to obtain target segmentation results and target classification results. Further, the server returns the target segmentation results and target classification results to the client. The resolution of the second feature map is lower than that of the first feature map. The target segmentation result is used to identify the target pixel region in the image to be processed, and the target classification result is used to characterize the attribute category corresponding to the target pixel region.

[0046] It should be noted that with the rapid development of high-performance computing units, the methods provided in this application embodiment can also be applied to model all-in-one machines in other application scenarios. In one optional embodiment, the model all-in-one machine has multiple built-in models, and users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the model all-in-one machine can then directly call the adjusted model to execute the methods provided in this application embodiment. In another optional embodiment, the large model all-in-one machine has a pre-trained model built-in, and the high-performance computing unit built into the model all-in-one machine can then directly call that model to execute the methods provided in this application embodiment.

[0047] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.

[0048] Under the aforementioned operating environment, this application provides the following: Figure 2 The image processing method shown. Figure 2 This is a flowchart of an image processing method according to an embodiment of this application. For example... Figure 2 As shown, the method may include the following steps S202 to S206.

[0049] Step S202: Perform multi-scale feature extraction on the image to be processed of the target area to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map.

[0050] The target area mentioned above can be the image acquisition location to be analyzed in the application scenario. In particular, the target area can be a partial anatomical region on a living organism (such as the human body). The image to be processed mentioned above can be an image obtained by acquiring images of the target area. For example, the image to be processed can be a scanned image of the target area. Specifically, the image to be processed can be a pre-processed image, that is, the region of interest is cropped from the initial image corresponding to the target area, and the cropping result is standardized and / or resized to obtain the image to be processed. For example, in the application scenario of breast lesion detection, the target area can be the human chest. The region of interest is cropped from the scanned image of the human chest to obtain the image corresponding to the breast tissue region, which is the image to be processed.

[0051] In the process of multi-scale feature extraction of the image to be processed, different scales are used to analyze and extract image features, capturing multi-level information in the image to be processed, resulting in first and second feature maps with different resolutions. In the embodiments of this application, the first feature map can also be referred to as a high-resolution feature map, and the second feature map can also be referred to as a low-resolution feature map. For example, in deep learning scenarios, multi-scale feature extraction can be achieved by using receptive fields of different sizes, different numbers of downsampling layers, or by using a pyramid network structure, thereby obtaining first and second feature maps with different resolutions.

[0052] The aforementioned first feature map possesses high spatial resolution, enabling it to capture subtle details in the image and facilitating voxel-level segmentation tasks. In embodiments of this application, the aforementioned first feature map may include a detailed feature description of the target region.

[0053] The aforementioned second feature map has a lower spatial resolution, making it more suitable for understanding and capturing the global features of the target region, which is helpful for handling global classification tasks. In the embodiments of this application, the aforementioned second feature map focuses on including macroscopic feature descriptions of the target region.

[0054] It should be noted that the specific resolution values ​​corresponding to the first and second feature maps mentioned above can be set according to the needs of the application scenario. This application embodiment does not limit the setting of specific resolution values.

[0055] Through step S202 above, this embodiment of the application performs a multi-scale feature extraction process on the image to be processed, generating high-resolution feature maps and low-resolution feature maps in parallel. Compared with high-resolution feature maps, low-resolution feature maps sacrifice some spatial detail, but are suitable for capturing long-range dependency information, which is beneficial for processing classification tasks based on global features. The first and second feature maps described above jointly characterize the features of the target region from different levels, providing a comprehensive information foundation for subsequent segmentation and classification tasks.

[0056] Step S204: Perform feature interaction calculation based on the first feature map and the second feature map to obtain fused feature data.

[0057] In this embodiment, an interactive fusion mechanism is used to exchange features between the first and second feature maps, ensuring that the analysis paths corresponding to the first and second feature maps can share relevant information, thereby improving the overall comprehensive analysis capability of the technical solution. The fused feature data may include global context features corresponding to the low-resolution analysis path and local pixel-level features corresponding to the high-resolution analysis path.

[0058] It should be noted that in application scenarios, a deep learning model can be built based on the dual-resolution mechanism and the concatenated input mechanism, and this deep learning model can be used to realize the above-mentioned feature interaction calculation.

[0059] In step S204, this embodiment of the application implements feature interaction calculation, which interactively aligns the feature information obtained in the low-resolution path with the feature information obtained in the high-resolution path, ensuring information sharing (such as soft parameter sharing) between subsequent segmentation and classification tasks, achieving higher quality feature representation, and avoiding information conflicts between segmentation and classification tasks.

[0060] Step S206: Perform perceptual mapping on the fused feature data to obtain target segmentation results and target classification results. The target segmentation results are used to identify target pixel regions in the image to be processed, and the target classification results are used to characterize the attribute categories corresponding to the target pixel regions.

[0061] In the embodiments of this application, the aforementioned perceptual mapping refers to a process of converting fused feature data into an interpretable and operable output form, which may further include the generation process of target segmentation results and the generation process of target classification results. In application scenarios, the final network layer of a deep learning model (e.g., set as a convolutional layer or a fully connected layer) can be used to convert the fused feature data obtained from feature interaction calculations into specific segmentation and classification results.

[0062] The target segmentation results described above can identify voxel regions in the image to be processed that are related to a preset abnormality (such as a breast lesion). In other words, the target segmentation results are used to characterize the specific location of the breast lesion. It is easy to understand that when the image to be processed is a three-dimensional image, the target pixel region can be a target voxel region. The target segmentation results described above can be a three-dimensional segmentation map corresponding to the target voxel region.

[0063] The above target classification results can provide classification information corresponding to preset abnormalities (such as breast lesions). For example, it can determine whether a breast region contains a tumor (i.e., the target voxel region). If a tumor is found in the breast region, it can further determine whether the tumor is malignant, benign, or normal tissue, thus obtaining the above target classification results. The above classification results can be binary calculation results corresponding to multiple preset categories.

[0064] It is noteworthy that existing technologies typically handle segmentation and classification tasks independently, making it difficult to effectively and accurately detect the segmentation regions and classification results corresponding to preset anomalies from the image to be processed. In contrast, the embodiments of this application employ a feature interaction calculation process based on a dual-resolution mechanism, performing parallel processing and analysis on high-resolution and low-resolution feature maps to jointly solve the segmentation and classification tasks, thereby simultaneously improving the accuracy of target segmentation and target classification results.

[0065] Through steps S202 to S206 above, this embodiment of the application performs multi-scale feature extraction on the image to be processed of the target region to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; feature interaction calculation is performed based on the first feature map and the second feature map to obtain fused feature data; perceptual mapping is performed on the fused feature data to obtain target segmentation results and target classification results, wherein the target segmentation results are used to identify the target pixel region in the image to be processed, and the target classification results are used to characterize the attribute category corresponding to the target pixel region. This achieves the purpose of jointly processing the segmentation task and the classification task based on feature interaction calculation of feature maps with different resolutions, thereby achieving the technical effect of synergistically improving the accuracy of the segmentation result and the classification result, and thus solving the technical problem that the scheme of independently processing the segmentation task and the classification task in the related technology leads to low segmentation accuracy and classification accuracy.

[0066] The image processing method provided in this application can be used to achieve region-of-interest segmentation and target object classification based on image processing in specific fields. For example, in the medical field, based on medical images of human body parts, organs, or tissues, the image processing method described above performs feature interaction calculations on feature maps of different resolutions to jointly analyze and obtain lesion region segmentation results and lesion classification results. Based on the image processing method described above, intelligent screening software for target lesions (such as breast cancer) can be designed and integrated into the hospital's disease detection system to automatically analyze patients' chest CT images. In addition, the lesion region segmentation results and lesion classification results obtained using the image processing method described above can also assist in generating intelligent physical examination reports in physical examination scenarios, providing preliminary lesion screening results for physical examination users. The image processing method described above can also be integrated into a mobile medical image-assisted diagnostic system, and the generated lesion region segmentation results and lesion classification results can provide reference information for doctors' diagnoses.

[0067] For example, the aforementioned specific field can also be the industrial field, where product defect areas are segmented and faulty parts are located based on images of industrial products (such as photos, surveillance video frames, etc.), and further product defect categories and part fault categories are determined.

[0068] For example, the aforementioned specific field could also be the field of autonomous driving, where road obstacles are segmented based on vehicle perception images (such as radar images and dashcam images), and the category of the road obstacle is further determined.

[0069] In an optional embodiment, the image to be processed is a three-dimensional scanned image. In step S202, multi-scale feature extraction is performed on the scanned image to obtain a first feature map and a second feature map, including the following method steps:

[0070] Step S221: Based on the first embedding dimension, perform embedding calculation on the 3D scanned image to obtain the first feature map;

[0071] Step S222: Based on the second embedding dimension, the first feature map is downsampled to obtain the second feature map.

[0072] In the above optional embodiments, the three-dimensional scan image can be three-dimensional image data obtained from a non-contrast chest CT (NCCT) scan. This three-dimensional scan image contains detailed structural information of the region to be analyzed in the target area (such as the breast region of the human chest).

[0073] The aforementioned first embedding dimension refers to the preset dimensionality used to encode image information during the embedding process of a 3D scanned image. This embedding process can be implemented through embedding computation, mapping each voxel of the 3D scanned image to a higher-dimensional feature space to obtain a first feature map. This embedding computation not only preserves the original information of the 3D scanned image but also provides richer feature representations for subsequent deep learning analysis by increasing the dimensionality of the feature representation.

[0074] The second embedding dimension is used to further process the first feature map. Based on this second embedding dimension, the size and detail complexity of the first feature map are reduced through downsampling and embedding computation, resulting in a second feature map, i.e., a feature map with lower resolution. The downsampling process may include pooling and / or averaging on the first feature map to maintain or enhance the global structure and contextual information of the feature map while reducing its resolution. By using downsampling based on the second embedding dimension, more macroscopic image features can be captured in the second feature map to support subsequent classification tasks, while reducing computational costs and improving the system's performance in classifying long sequence data.

[0075] Through the above steps S221 and S222, this embodiment of the application obtains first feature maps and second feature maps of different resolutions through multi-scale feature extraction, thereby extracting global feature representations while maintaining local image details, thus achieving a balance between segmentation and classification tasks, ensuring accurate positioning and classification of target pixel regions.

[0076] In an optional embodiment, in step S204, feature interaction calculation is performed based on the first feature map and the second feature map to obtain fused feature data, including the following method steps:

[0077] Step S241: Use the target model to perform dual-path feature interaction calculation on the first feature map and the second feature map to obtain fused feature data. The target model is used to model feature sequences at different scales for the first feature map and the second feature map respectively.

[0078] The target model described above can be a pre-trained deep learning model for processing 3D image data. Specifically, the target model can include two processing paths to achieve dual-path feature interaction computation on the first and second feature maps. That is, the target model simultaneously possesses the ability to process detailed features in the higher-resolution first feature map and global features in the lower-resolution second feature map, enabling it to simultaneously focus on both the overall structure and local details of the image.

[0079] In the aforementioned dual-path feature interaction computation process, the two processing paths can interact. For example, when the two processing paths are used to process the first feature map and the second feature map respectively, they can exchange and align information. The high-resolution processing path used to process the first feature map can collect local detail information, while the low-resolution processing path used to process the second feature map can collect overall structural information. The high-resolution processing path can supplement its information using the generalized overall structural information from the low-resolution processing path, thereby enhancing its segmentation performance. Conversely, the low-resolution processing path can supplement its information using the precise local detail information from the high-resolution processing path, thereby enhancing its classification performance.

[0080] The aforementioned fused feature data can be the output of the target model after dual-path feature interaction calculation. Fusion feature data combines the advantages of global and local features, reflecting the detail, features, and distribution of the target pixel region in the image. It contains not only detailed information helpful for accurate segmentation but also contextual information supporting precise classification. Through fused feature data, the system can further synchronize and accurately process segmentation and classification tasks in subsequent processes.

[0081] It should be noted that the computational units in the aforementioned target model can employ a dual-path 3D Mamba network architecture, thereby performing feature sequence modeling on the feature map in each processing path to support accurate feature analysis within that path. In some application scenarios, the aforementioned target model is designed as a dual-resolution path 3D Mamba architecture model (Dual-Res Tandem Mamba-3D, abbreviated as DRT-M3D), meaning that this target model is constructed by comprehensively utilizing a 3D Mamba network, a dual-resolution mechanism, and a concatenated input mechanism.

[0082] In application scenarios, when processing the first and second feature maps of a target region using a target model, firstly, independent feature sequence modeling is performed on the first feature map (feature map of the high-resolution path) and the second feature map (feature map of the low-resolution path). For the first feature map, the target model performs local feature analysis using 3D Mamba network blocks on the high-resolution processing path. For the second feature map, the target model performs global dependency modeling using 3D Mamba network blocks on the low-resolution processing path. Further, the target model performs feature interaction computation, combining the local feature details of the first feature map with the global context information of the second feature map through downsampling and upsampling processes to generate fused feature data.

[0083] Through step S241 above, this embodiment of the application achieves dual-path feature interaction calculation through the target model, effectively extracting complementary information from the first and second feature maps to generate fused feature data. This fusion mechanism not only enhances the target model's sensitivity to subtle voxel-level changes in image data, thereby improving the target model's accurate segmentation performance for target pixel regions, but also improves the target model's accurate classification performance for target pixel regions by considering the global perspective embodied in the second feature map. Specifically, in the field of medical technology, by generating fused feature data, this embodiment of the application achieves efficient segmentation and accurate classification of breast lesions in a breast lesion analysis scenario based on chest NCCT, providing reference information for clinical diagnosis.

[0084] In an optional embodiment, the target model includes a first path and a second path, and the fused feature data includes global fused features and local fused features. In step S241, the target model is used to perform dual-path feature interaction calculation on the first feature map and the second feature map to obtain the fused feature data, including the following method steps:

[0085] Step S242: Pixel-level feature sequence modeling and calculation are performed on the first feature map using the first path to obtain spatial detail features;

[0086] Step S243: Utilize the second path to perform global-scale contextual feature sequence modeling and calculation on the second feature map and spatial detail features to obtain global fusion features;

[0087] Step S244: Perform interactive fusion processing based on spatial detail features and global fusion features to obtain local fusion features.

[0088] In the aforementioned target model, the first path is used to process the higher-resolution first feature map and is referred to as the high-resolution processing path, while the second path is used to process the lower-resolution second feature map and is referred to as the low-resolution processing path. These first and second paths are used to capture image feature information at different viewpoints and scales and to enhance each other through feature interaction during feature analysis.

[0089] The target model described above is a deep learning architecture designed with a dual-path approach, comprising a first path and a second path, to extract and fuse image information from different dimensions. This dual-path processing allows the target model to simultaneously focus on both local details and global context of the image. The fused feature data obtained from the first and second paths includes both global and local fused features, thereby enhancing the final analysis results.

[0090] The first path described above is used to perform pixel-level (or voxel-level for 3D image data) feature sequence modeling calculations on the first feature map, aiming to capture fine-grained information of the image, i.e., spatial detail features. The first feature map typically retains a high resolution, enabling the target model to analyze each pixel in the image one by one and learn microscopic features such as texture and shape.

[0091] The second path described above is used to perform global-scale contextual feature sequence modeling and calculation on the second feature map to generate global fusion features. The second feature map is obtained by downsampling the first feature map and has a lower resolution, which is beneficial for the target model to capture the macroscopic structure and long-distance dependencies of the image. Through global-scale modeling, the second path can extract information such as the distribution and layout of objects in the image.

[0092] The aforementioned interactive fusion processing refers to integrating the spatial detail features obtained from the first path with the global fusion features obtained from the second path to generate local fusion features. This integration process typically includes two aspects: firstly, the spatial detail features of the first path are upsampled to align with the global fusion features of the second path, thereby incorporating global information at the local scale; secondly, the global fusion features of the second path are also downsampled to the same spatial resolution as the spatial detail features, so as to combine coarse-grained global information with fine-grained local information, ultimately obtaining a feature representation that contains both local details and global context, i.e., local fusion features.

[0093] Through steps S242 to S244 above, based on the dual-path architecture and interactive fusion processing mechanism of the target model, the embodiments of this application can effectively extract multi-level features from images, focusing on both pixel-level subtle differences and the overall macroscopic structure of the image, thereby providing a more comprehensive and accurate understanding in various image analysis tasks. The above technical solution ensures that the target model can simultaneously utilize high-resolution details and low-resolution contextual information when processing complex images, avoiding information fragmentation and enhancing the accuracy of the target model.

[0094] In an optional embodiment, the target region includes: a first side region and a second side region that are symmetrical to each other. The first feature map includes a first feature corresponding to the first side region and a second feature corresponding to the second side region. In step S242, the first feature map is modeled and calculated using a first path at the pixel level to obtain spatial detail features, including the following method steps:

[0095] Step S2421: Integrate and map the first feature and the second feature into a pseudo-three-dimensional space to obtain the first integrated data, wherein the first feature and the second feature are spatially symmetrically distributed in the pseudo-three-dimensional space;

[0096] Step S2422: The first integrated data is transformed to obtain the first concatenated input data, wherein the first feature and the second feature in the first concatenated input data are concatenated and spliced ​​into a one-dimensional feature sequence.

[0097] Step S2423: Use the network units set on the first path to perform feature sequence modeling and analysis on the first serial input data, so as to compare the symmetric correlation between the first feature and the second feature and obtain the first serial output data;

[0098] Step S2424: Perform three-dimensional reconstruction processing on the first serial output data to obtain spatial detail features, wherein the spatial detail features include first detail features corresponding to the first side part and second detail features corresponding to the second side part.

[0099] The target areas mentioned above include spatially paired and intrinsically related first and second lateral areas. First and second lateral areas typically occupy similar positions in an image and possess similar structural properties, but in specific instances, they may exhibit unique differences, such as subtle variations in the shape or position of the object, or differences in local details. For example, the target area might be the human chest, including the symmetrical left and right mammary glands. The left and right mammary glands have similar structures, but when a lesion occurs on one or both sides of the mammary glands, asymmetry may appear between the left and right mammary glands.

[0100] The first and second features are integrated and mapped to a pseudo-3D space to generate the first integrated data. The pseudo-3D space is a data organization method that places the originally independent first and second features in the same multi-dimensional coordinate system. In this space, the first and second features can exhibit a spatially symmetrical distribution according to specific rules. Specifically, the aforementioned multi-dimensional coordinate system can also be organized as a four-dimensional tensor space. This technical solution not only preserves the integrity of the first and second features but also introduces the relative positional information between the first and second side parts, helping the target model understand the potential connections between the first and second side parts.

[0101] Furthermore, the first integrated data undergoes transformation processing to generate the first concatenated input data. This transformation process concatenates the first feature and the second feature into a one-dimensional feature sequence. This data transformation method simplifies the input format, enabling the target model to process the feature sequence with linear computational complexity, maintaining efficiency even when handling large-scale image datasets.

[0102] Furthermore, the network units set on the first path are used to perform feature sequence modeling and analysis on the first concatenated input data, aiming to compare the symmetric correlation between the first feature and the second feature. The network units may adopt a state-space modeling architecture based on a three-dimensional Mamba network, which can dynamically adjust the state update speed and allocate computational resources according to the importance of the input features. By comparing the correlation between the first feature corresponding to the first side and the second feature corresponding to the second side, the target model can learn the symmetric pattern that should be presented under normal circumstances, as well as the important information that may be implied when this symmetric pattern is abnormal.

[0103] Furthermore, the first concatenated output data undergoes 3D reconstruction processing to restore the original spatial layout of the image, obtaining spatial detail features. This 3D reconstruction process, as the reverse operation of the aforementioned data transformation process, reconstructs the one-dimensional feature sequence into a 3D feature map. The 3D feature map used to characterize spatial detail features includes: first detail features corresponding to the first side portion and second detail features corresponding to the second side portion. The 3D reconstruction process not only restores the spatial location of image features but also helps the target model to more intuitively understand and utilize image features in subsequent task processing. For example, in pixel-level tasks (such as target pixel region segmentation), the features of the 3D layout can guide the localization and recognition of specific objects.

[0104] Through steps S2421 to S2424 described above, the target model in this embodiment can fully utilize the mutual symmetry between the first and second side parts, and through integration, analysis, and reconstruction, obtain feature representations that contain both local details and understand the global context. The above technical solution is particularly suitable for situations requiring comparison and analysis across spatial dimensions, such as evaluating the integrity and symmetry of objects or searching for asymmetric features. Furthermore, by performing feature sequence modeling and analysis in different dimensions, the target model can effectively adapt to various input sizes while maintaining computational efficiency, providing a foundation for processing various image recognition and understanding tasks.

[0105] In an optional embodiment, in step S243, the second path is used to perform global-scale context feature sequence modeling calculation on the second feature map and spatial detail features to obtain global fusion features, including the following method steps:

[0106] Step S2431: Based on the resolution and channel dimension corresponding to the second feature map, the spatial detail features are downsampled to obtain the first sampling result;

[0107] Step S2432: Perform pixel overlay calculation on the second feature map and the first sampling result to obtain the first overlay result;

[0108] Step S2433: Use the second path to perform global-scale context feature sequence modeling and calculation on the first superposition result to obtain global fusion features.

[0109] The resolution and channel dimension of the second feature map mentioned above refer to the size and information capacity of the image features used in the low-resolution path. Resolution determines the size of the feature map, while channel dimension defines the number of features carried by each pixel. Based on the resolution and channel dimension, the original spatial detail features are downsampled to match the specifications of the second feature map, generating the first sampling result. The downsampling process typically involves reducing the size of the feature map and decreasing information redundancy, allowing the model to focus on more abstract features, making it suitable for tasks requiring a global perspective.

[0110] Furthermore, pixel overlay calculations are performed, combining the second feature map and the first sampling result to generate a first overlay result. Pixel overlay calculations can be viewed as a feature fusion technique. It creates a new feature map that integrates local details and global background information by directly adding the feature values ​​of corresponding pixels in the two feature maps, or by merging these features through a weighted sum function. This approach allows the model to retain the detailed features transmitted through high-resolution paths while also utilizing the macroscopic structural features extracted from low-resolution paths, laying the foundation for subsequent analysis.

[0111] Furthermore, a second path is used to perform global-scale contextual feature sequence modeling computation on the first overlay result to extract global fusion features. The second path is typically used to process lower-resolution feature maps and can effectively capture long-distance dependencies between image elements, which helps in understanding and classifying the global properties of the image. Through global-scale contextual feature sequence modeling computation, the second path can identify the distribution of objects in the feature map, the connections between image regions, and potential patterns; this information is synthesized into global fusion features. Global fusion features not only contain macroscopic structural information of the image but also incorporate key information extracted from local detail features, enabling the target model to handle high-level tasks (such as segmentation or classification tasks).

[0112] Through steps S2431 to S2433, this embodiment of the application, via a second-path processing flow, can effectively extract feature representations from images that contain both local details and global context. The global fusion feature combines the high-precision details of the first feature map with the broad background of the second feature map, forming a highly integrated information carrier. This enables the model to perform more complex and in-depth analyses, offering significant advantages whether identifying specific objects in an image or performing high-level tasks such as scene interpretation or category prediction. The multi-scale, multi-path feature processing method in the above technical solution enhances the model's flexibility and effectiveness, enabling it to cope with diverse image analysis needs.

[0113] In an optional embodiment, the second feature map includes a third feature corresponding to the first side portion and a fourth feature corresponding to the second side portion, and the first superposition result includes a first-side superposition feature corresponding to the first side portion and a second-side superposition feature corresponding to the second side portion. In step S2432, pixel superposition calculation is performed on the second feature map and the first sampling result to obtain the first superposition result, including the following method steps:

[0114] Step S2434: Perform pixel superposition calculation on the third feature and the first side sampling feature in the first sampling result to obtain the first side superposition feature, wherein the first side sampling feature is obtained by sampling the first detail feature;

[0115] Step S2435: Perform pixel overlay calculation on the fourth feature and the second side sampling feature in the first sampling result to obtain the second side overlay feature, wherein the second side sampling feature is obtained by sampling the second detail feature.

[0116] The third feature corresponding to the first side and the fourth feature corresponding to the second side refer to the feature representations of the two symmetrical parts extracted from the second feature map in the low-resolution path (second path). The third and fourth features have undergone preliminary processing in the high-resolution path, but in the low-resolution path, the third and fourth features are further refined to capture more abstract global patterns and structural information.

[0117] Furthermore, these low-resolution features are combined with the downsampled first sampling result through pixel overlay calculation to generate first-side overlay features and second-side overlay features for the first and second side regions, respectively. In this process, the first-side sampling features and second-side sampling features are obtained by dimensionality reduction of the spatial detail features (first detail features and second detail features) obtained from the high-resolution path to ensure that they can be aligned in size with the features (third feature and fourth feature) in the low-resolution path.

[0118] Pixel overlay computation can be viewed as a feature fusion mechanism that combines feature values ​​from different paths directly at the pixel level, thereby enabling the free flow of information across different resolutions and dimensions. The pixel overlay computation typically involves weighted summation or direct addition of feature values ​​at corresponding positions in two feature maps. The resulting overlay feature retains high-resolution details and low-resolution contextual information, providing a more comprehensive and deeper understanding of the image.

[0119] The generated first-side overlay features and second-side overlay features correspond to the comprehensive feature representations of the first and second side regions in the image, respectively. These features not only include detailed features of the corresponding side regions but also incorporate structural and pattern information from a global perspective. This technical solution provides the target model with richer and more multi-dimensional input, helping it to consider a wider range of contexts during analysis and improving the accuracy of recognition and classification.

[0120] Through steps S2434 to S2435, this embodiment effectively integrates features from different resolution paths to generate overlay features that include both detailed information and global patterns. The first-side overlay features and the second-side overlay features can serve as important inputs for subsequent high-level analysis (such as handling classification or segmentation tasks). Combining local and global perspectives, the first-side and second-side overlay features provide the model with a more powerful and comprehensive image understanding capability, thereby improving performance in various image processing tasks. The aforementioned feature fusion method fully utilizes the multi-layered information of the image, improving the model's adaptability and analytical capabilities for complex scenes.

[0121] In an optional embodiment, in step S2433, the first overlay result is modeled and calculated using a global-scale contextual feature sequence using the second path to obtain global fusion features, including the following method steps:

[0122] Step S2436: Integrate and map the first side superimposed features and the second side superimposed features into a pseudo-3D space to obtain the second integrated data, wherein the first side superimposed features and the second side superimposed features are spatially symmetrically distributed in the pseudo-3D space.

[0123] Step S2437: The second integrated data is converted to obtain the second serial input data;

[0124] Step S2438: Use the network units set on the second path to perform feature sequence modeling and analysis on the second serial input data to obtain the second serial output data;

[0125] Step S2439: Perform three-dimensional reconstruction processing on the second serial output data to obtain global fusion features.

[0126] The aforementioned pseudo-3D space is a special framework for organizing and representing data. It integrates originally independent features (such as first-side overlay features and second-side overlay features) into a shared coordinate system to facilitate cross-feature interaction and symmetry analysis. The symmetrical distribution of first-side overlay features and second-side overlay features in the pseudo-3D space helps the target model understand the potential relationships between features, especially when features come from relatively opposite regions in the image. It can more effectively capture the differences and similarities between first-side overlay features and second-side overlay features, thereby enhancing the target model's understanding of the overall image structure.

[0127] The second integrated data includes superimposed features extracted from the first and second side regions, integrated in a specific distribution pattern in pseudo-3D space. The step of integrating and mapping the first and second side superimposed features to pseudo-3D space ensures that the target model can simultaneously consider the symmetry and complementarity of the features on both sides when processing features, providing a unified feature representation for subsequent segmentation and classification tasks.

[0128] Furthermore, the second integrated data is transformed into second concatenated input data. This typically involves data reorganization and preprocessing so that the target model can receive and process the information optimally. The aforementioned transformation processes may include feature encoding, data format adjustments, or sequence reassembly, thereby ensuring that network units on the second path can effectively perform feature sequence modeling and analysis.

[0129] Furthermore, the network units set on the second path are used to perform feature sequence modeling analysis on the second concatenated input data. Here, the network units may refer to specific functions or modules, such as Mamba-3D blocks. The network units can perform efficient and selective state-space operations on the sequence data, capture long-range dependencies, and help the target model understand the global structure and contextual information of the image, thereby generating the second concatenated output data.

[0130] Furthermore, the second concatenated output data undergoes 3D reconstruction processing to restore it to the original 3D layout of the image, thereby obtaining global fusion features. This 3D reconstruction process, as the reverse operation of the aforementioned data transformation process, reconstructs the one-dimensional feature sequence into a 3D feature map. The 3D reconstruction process may include upsampling, interpolation, or backprojection to ensure that the output features spatially correspond to the input image. The global fusion features integrate global information from the second path and local details from the first path, providing a comprehensive understanding of the image content.

[0131] Through steps S2436 to S2439, the target model in this embodiment can effectively extract and fuse features at different paths and scales, focusing on both detailed features and understanding global features to ensure a deep and comprehensive understanding of the image. The multi-path processing and data conversion flow in the above technical solution provides the model with powerful information processing capabilities, enabling it to adapt to and optimize various image analysis tasks.

[0132] In an optional embodiment, in step S244, interactive fusion processing is performed based on spatial detail features and global fusion features to obtain local fusion features, including the following method steps:

[0133] Step S2441: Based on the resolution and channel dimension corresponding to the first feature map, the global fusion feature is upsampled to obtain the second sampling result;

[0134] Step S2442: Pixel overlay calculation is performed based on spatial detail features and second sampling results to obtain local fusion features.

[0135] In the above optional embodiments, the previously obtained global fusion features are upsampled based on the resolution and channel dimension of the first feature map to obtain the second sampling result. The first feature map has a higher resolution and describes the fine information of the image, while the global fusion features are obtained based on the low-resolution path (the second feature map) and describe the global context information of the image. The upsampling is achieved by increasing the spatial size of the features, such as by using techniques like nearest neighbor interpolation, bilinear interpolation, or transposed convolution, to enlarge the spatial resolution of the global fusion features to the same level as the first feature map. Based on this, feature fusion between the global fusion features and the first feature map is achieved, that is, the global information learned by the low-resolution path is fused into the high-resolution path that retains the fine information, thereby improving the target model's sensitivity to image details and its ability to understand the global context.

[0136] Furthermore, the spatial detail features (i.e., the first feature map) and the second sampling result (the upsampled global fusion feature) are pixel-wise superimposed to generate local fusion features. This pixel superimposition calculation can be viewed as a feature fusion mechanism, typically accomplished through pixel-wise addition or multiplication. The process can be seen as the direct superposition of two feature maps at corresponding pixel positions, effectively integrating information from different scales. Through pixel superimposition, the target model can introduce deeper semantic information while preserving the original details, resulting in local fusion features that can understand and capture both local image details and the global context. This feature fusion method helps the target model handle various image understanding tasks (such as segmentation and classification), helping it better balance local and global information and improve the accuracy and robustness of segmentation and classification tasks.

[0137] Through steps S2441 to S2442 above, in this embodiment of the application, upsampling processing and pixel overlay calculation enable the target model to seamlessly combine global and local features. The generated local fusion features not only preserve image details but also understand the global background of the image, providing data support for subsequent high-level analysis.

[0138] In an optional embodiment, in step S206, perceptual mapping is performed on the fused feature data to obtain target segmentation results and target classification results, including the following method steps:

[0139] Step S261: Use the first perceptron to perform perceptual pooling on the local fusion features in the fusion feature data to obtain the target segmentation result;

[0140] Step S262: Use the second perceptron to perform perceptual analysis on the global fusion features in the fusion feature data to obtain the target classification result.

[0141] The first and second perceptrons mentioned above refer to neural network components used to process different types of features. The first perceptron is used to process local fusion features, generating target segmentation results by performing perceptual pooling. The perceptual pooling process involves detailed identification and boundary delineation of various parts of the image. By intelligently aggregating information from local fusion features, perceptual pooling can distinguish and locate various objects or regions in the image, thereby producing accurate target segmentation results. The location and shape of specific objects in the image can be labeled in this target segmentation result.

[0142] The second perceptron described above is used to analyze global fusion features and obtain target classification results through perceptual analysis. During this perceptual analysis process, the content of the image is understood from a global perspective, thereby identifying and classifying objects in the image and determining their categories. The second perceptron integrates information from the global fusion features for inference analysis to obtain the target classification result. The target classification result can determine the type of objects in the image.

[0143] Through steps S261 to S262 described above, the first perceptron in this embodiment is used to process the segmentation task, and the second perceptron is used to process the classification task. By utilizing local fusion features and global fusion features, the first and second perceptrons can deeply understand the image content from different levels, providing detailed target segmentation results and accurate target classification results. The above technical solution improves the efficiency and accuracy of image analysis. In this way, the target model can understand the overall context of the image while maintaining sensitivity to details, thereby achieving joint and synchronous processing of segmentation and classification tasks, providing more comprehensive and accurate image processing services for application scenarios.

[0144] In one exemplary application scenario, specifically in the medical field of breast lesion detection, the following is provided: Figure 3 This illustrates an image preprocessing procedure based on NCCT. For example... Figure 3 As shown, a segmentation network is used to segment the region of interest in chest NCCT to obtain the image to be processed of the target area, which is the breast region.

[0145] Furthermore, providing such Figure 4 This illustrates a main image processing procedure. For example... Figure 4 As shown, the image to be processed is further segmented into a 3D image of the right breast and a 3D image of the left breast. Low-resolution features are obtained through embedding calculations on the 3D image of the right breast. Figure 1 and high-resolution features Figure 1 Low-resolution features were obtained through embedding calculations on the three-dimensional image of the left breast. Figure 2 and high-resolution features Figure 2Furthermore, a target model is used to analyze low-resolution features. Figure 1 High-resolution features Figure 1 Low-resolution features Figure 2 and high-resolution features Figure 2 Feature interaction processing is performed to obtain low-resolution fusion feature 1, high-low resolution fusion feature 1, low-resolution fusion feature 2, and high-low resolution fusion feature 2. Further, perceptual analysis is performed using a perceptron to obtain the classification result of the right breast, the segmentation result of the right breast, the classification result of the left breast, and the segmentation result of the left breast.

[0146] In one optional embodiment, the image processing method further includes the following method steps:

[0147] Step S208: Fine-tune the model parameters of the initial model using the training dataset to obtain the target model. The training samples in the training dataset include: training images of the target region, the real segmentation mask corresponding to the training images, and the real classification label.

[0148] The training dataset described above is a collection containing a large number of training samples. Each training sample includes: a training image of the target region, the corresponding ground truth segmentation mask, and a ground truth classification label. The training images provide the input data that the model needs to learn from, the ground truth segmentation mask guides the model to learn how to accurately segment specific parts of the image, and the ground truth classification label helps the model learn how to correctly classify these segmented regions.

[0149] The initial model mentioned above refers to a model that has been pre-trained or randomly initialized before training begins. The initial model is the starting point for model training and includes the model architecture and initially set parameters. Through fine-tuning the model parameters, the initial model is iteratively optimized using the training dataset. During training, the model can adjust its internal parameters based on feedback from training samples to better fit the data. This fine-tuning process typically involves forward propagation to generate predictions, calculating the difference between the predictions and the true labels (e.g., the loss function), and then updating the model parameters through backpropagation to improve the model's performance on the training set.

[0150] The target model described above is the model after fine-tuning. This target model is the final version capable of effectively performing the specific task. The target model has learned how to accurately segment the target pixel regions corresponding to the target parts from the image to be processed, and how to classify these target pixel regions. By utilizing the segmentation masks and classification labels in the training dataset, the target model can not only accurately identify and highlight the target pixel regions in the image, but also make correct classification decisions based on these target pixel regions.

[0151] Through step S208 described above, this embodiment of the application uses a training dataset containing training images, ground truth segmentation masks, and ground truth classification labels to fine-tune the parameters of the initial model, thereby obtaining a target model capable of accurately segmenting and classifying the image to be processed for the target region. This parameter fine-tuning process is a training procedure in machine learning projects, designed to allow the model to learn from data and improve its performance on specific tasks. Through fine-tuning the model parameters, the target model can more accurately understand and process the target region, achieving higher accuracy in segmentation and classification tasks for the target region.

[0152] In an optional embodiment, in step S208, the initial model is fine-tuned using the training dataset to obtain the target model, including the following method steps:

[0153] Step S281: The initial model is used to analyze and predict the training images to obtain the predicted segmentation results and the predicted classification results;

[0154] Step S282: Calculate the segmentation task loss using the first loss function, the predicted segmentation result, and the real segmentation mask. The first loss function is constructed based on the region overlap between the predicted segmentation result and the real segmentation mask.

[0155] Step S283: Calculate the classification task loss using the second loss function, the predicted classification result, and the true classification label. The second loss function is constructed based on the cross-entropy loss between the predicted classification result and the true classification label.

[0156] Step S284: Perform a weighted summation of the segmentation task loss and the classification task loss to obtain the target loss;

[0157] Step S285: Fine-tune the model parameters of the initial model based on the target loss to obtain the target model.

[0158] In the above optional embodiments, an initial model is used to analyze and predict a set of training images to obtain predicted segmentation results and predicted classification results. The predicted segmentation results are the model's identification and region division of target areas in the training images, while the predicted classification results are the model's classification of these segmented regions. The predicted segmentation results and predicted classification results are the output of the model's understanding and analysis of the input training images based on the current parameters.

[0159] Furthermore, to measure the difference between the predicted segmentation result and the true segmentation mask, the first loss function is calculated based on the region overlap between the predicted segmentation result and the true segmentation mask. Commonly used metrics, such as the Dice coefficient, can intuitively reflect the degree of matching between the model's predicted segmentation region and the actual region. On the other hand, the second loss function is used to calculate the classification task loss. The second loss function is usually constructed based on cross-entropy loss and is used to evaluate the difference between the model's predicted classification result and the true classification label, reflecting the accuracy of the model's category judgment.

[0160] Furthermore, the target loss is calculated by weighted summation of the segmentation task loss and the classification task loss. The weights in this weighted summation are determined by the relative importance of the segmentation and classification tasks in the application scenario. Additionally, the weights can be adjusted by considering the potential resource allocation of the model across different tasks. This target loss is the overall loss function that needs to be minimized in model optimization, combining the model's performance on both the segmentation and classification tasks.

[0161] Furthermore, the model fine-tunes its parameters based on feedback from the target loss. This process typically involves backpropagation, which calculates the gradient of the target loss with respect to the model parameters and then adjusts these parameters according to a preset learning rate to reduce the target loss. Model parameter fine-tuning is a crucial part of the machine learning training cycle, enabling the model to learn from the data and progressively improve its predictive capabilities in segmentation and classification tasks until an optimized target model is obtained.

[0162] Through steps S281 to S285 described above, this embodiment of the application can train a target model capable of simultaneously performing accurate segmentation and correct classification when processing images. The target model can accurately understand complex image scenes and achieve precise task processing, such as breast lesion segmentation and classification in the medical field, obstacle recognition and classification in autonomous vehicles, and product defect detection and type determination in industrial production. Fine-tuning of the model parameters ensures that the target model can adjust its internal behavioral strategies according to the needs of the actual scenario, improving the model's performance in specific tasks.

[0163] In the aforementioned operating environment, this application also provides, as follows: Figure 5 This illustrates a computer-aided diagnostic method for breast cancer. Figure 5 This is a flowchart of a computer-aided diagnosis method for breast cancer according to an embodiment of this application, such as... Figure 5 As shown, the method includes:

[0164] Step S501: Multi-scale feature extraction is performed on the computed tomography image of the human chest to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map.

[0165] Step S502: Perform feature interaction calculation based on the first feature map and the second feature map to obtain fused feature data;

[0166] Step S503: Perform perceptual mapping on the fused feature data to obtain target segmentation results and target classification results. The target segmentation results are used to identify breast lesion areas in the computed tomography scan images, and the target classification results are used to characterize the benign or malignant category of the breast lesion areas.

[0167] This application implements a processing scheme for medical images based on the aforementioned image processing method, which is particularly suitable for computer-aided diagnosis of breast diseases. By processing computed tomography images of the human chest, it achieves precise localization of breast lesions and assessment of malignancy risk, providing reference information for clinicians.

[0168] In medical settings, while conventional CT scans are widely used for lung examinations, the breast region they cover is often overlooked due to low tissue contrast. This application's embodiment extracts multi-scale features from NCCT images and jointly optimizes lesion segmentation and classification tasks through a dual-path feature interaction mechanism.

[0169] The segmentation results described above can identify the three-dimensional spatial distribution of breast lesions, assisting clinicians in accurately locating suspicious areas such as microcalcification clusters and masses. These classification results can quantify the malignancy probability of breast lesions, helping clinicians to provide early warning of high-risk lesions and significantly reducing the probability of missed diagnoses.

[0170] The computer-aided diagnostic method for breast cancer in the above medical scenarios can overcome the limitations of traditional independent models (which handle lesion segmentation and lesion classification tasks independently), improve diagnostic sensitivity and reduce misdiagnosis, and is especially suitable for public health scenarios that combine lung cancer screening and breast health monitoring.

[0171] In an optional embodiment, in step S502, feature interaction calculation is performed based on the first feature map and the second feature map to obtain fused feature data, including the following method steps:

[0172] Step S521: Use the target model to perform dual-path feature interaction calculation on the first feature map and the second feature map to obtain fused feature data. The target model is used to model feature sequences at different scales for the first feature map and the second feature map respectively.

[0173] The target model described above is a dual-resolution path 3D Mamba network model trained using pre-set training data in the medical field, which can demonstrate high performance in breast lesion segmentation and classification tasks based on medical images.

[0174] Through steps S501 to S503, this embodiment of the application performs multi-scale feature extraction on computed tomography (CT) images of the human chest to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map. Feature interaction calculation is performed based on the first and second feature maps to obtain fused feature data. Perceptual mapping is then performed on the fused feature data to obtain target segmentation results and target classification results. The target segmentation results are used to identify breast lesion regions in the CT scan image, and the target classification results are used to characterize the benign or malignant category corresponding to the breast lesion regions. Therefore, this embodiment of the application is particularly suitable for breast lesion screening scenarios in the medical field, achieving the goal of jointly processing segmentation and classification tasks based on feature interaction calculations of feature maps with different resolutions. This achieves the technical effect of synergistically improving the accuracy of segmentation and classification results, thereby solving the technical problem in related technologies where independent processing of segmentation and classification tasks leads to low segmentation and classification accuracy.

[0175] It should be noted that the preferred embodiments of steps S501 to S503 described above can be found in the foregoing descriptions, and will not be repeated here.

[0176] In the aforementioned operating environment, this application also provides, as follows: Figure 6 Another image processing method is shown. Figure 6 This is a flowchart of another image processing method according to an embodiment of this application, such as... Figure 6 As shown, the image processing method includes:

[0177] Step S601: Obtain an image processing request through the first application programming interface, wherein the request data carried in the image processing request includes: the image to be processed of the target area;

[0178] Step S602: Return the image processing response through the second application programming interface, wherein the response data carried in the image processing response includes: target segmentation result and target classification result;

[0179] The above target segmentation results and the above target classification results are generated according to any one of the above image processing methods.

[0180] The image processing method described in this application embodiment can run on a cloud server to provide image processing cloud services to clients. The client sends an image processing request by calling a first application programming interface (API). After obtaining the image processing request through the first API, the cloud server generates target segmentation and target classification results according to the image processing method, and further returns the target segmentation and target classification results to the client through a second API.

[0181] The first and second application programming interfaces (APIs) mentioned above can be the same or different APIs. In an optional embodiment, the interface parameters in the first and second APIs may include, but are not limited to: a global interface identifier, an interface signing key, an interface timestamp, an interface request identifier, and a system call credential identifier. The first API can use either a get request or a submit request as its interface request method to obtain a file processing request. The second API can use a lightweight data exchange format (such as JavaScript Object Notation, or JSON for short) to return a file processing response.

[0182] Through steps S601 to S602 described above, this embodiment of the application obtains an image processing request through a first application programming interface (API), wherein the request data carried in the image processing request includes: an image of the target region to be processed; and returns an image processing response through a second API, wherein the response data carried in the image processing response includes: target segmentation results and target classification results; the target segmentation results and the target classification results are generated according to any one of the above image processing methods. Therefore, this embodiment of the application achieves the goal of jointly processing segmentation and classification tasks based on feature interaction computation of feature maps at different resolutions, thereby achieving the technical effect of synergistically improving the accuracy of segmentation and classification results, and thus solving the technical problem in related technologies where schemes that independently process segmentation and classification tasks result in low segmentation and classification accuracy.

[0183] It should be noted that the preferred embodiments of steps S601 to S602 described above can be found in the foregoing description, and will not be repeated here.

[0184] In the aforementioned operating environment, this application also provides, as follows: Figure 7 This is another image processing method shown. Figure 7 This is a flowchart of another image processing method according to an embodiment of this application, such as... Figure 7 As shown, the image processing method includes:

[0185] Step S701: Obtain the currently input image processing dialogue request, wherein the request data carried in the image processing dialogue request includes: the image to be processed of the target area;

[0186] Step S702: In response to the image processing dialogue request, return the image processing dialogue response, wherein the information carried in the image processing dialogue response includes: target segmentation result and target classification result;

[0187] Step S703: Present the target segmentation results and target classification results in the graphical user interface;

[0188] The above target segmentation results and the above target classification results are generated according to any one of the above image processing methods.

[0189] Based on the above method steps, a visualization scheme for image processing functions is provided. The terminal device provides a graphical user interface (GUI), which displays at least an image processing scene. The GUI's display content also includes input components (such as text input boxes, voice input controls, etc.) and output components (such as text display windows, image display windows, audio playback windows, etc.). The user inputs an image processing dialog request through the input components to specify the target area of ​​the image to be processed in the image processing task. After detecting the user's input, target segmentation results and target classification results are obtained according to the aforementioned image processing method. Furthermore, the target segmentation results and target classification results are displayed through the output components within the GUI.

[0190] Through steps S701 to S703 described above, this embodiment of the application obtains the currently input image processing dialogue request, wherein the request data carried in the image processing dialogue request includes: the image to be processed of the target region; in response to the image processing dialogue request, an image processing dialogue reply is returned, wherein the information carried in the image processing dialogue reply includes: target segmentation result and target classification result; the target segmentation result and target classification result are presented in the graphical user interface; the target segmentation result and the target classification result are generated according to any one of the above image processing methods. Therefore, this embodiment of the application achieves the goal of jointly processing segmentation and classification tasks based on feature interaction calculations of feature maps at different resolutions, thereby achieving the technical effect of synergistically improving the accuracy of segmentation and classification results, and thus solving the technical problem in related technologies where schemes that independently process segmentation and classification tasks result in low segmentation and classification accuracy.

[0191] It should be noted that the preferred embodiments of steps S701 to S703 described above can be found in the foregoing descriptions, and will not be repeated here.

[0192] In the aforementioned operating environment, this application also provides, as follows: Figure 8 This is another image processing method shown. Figure 8 This is a flowchart of another image processing method according to an embodiment of this application, such as... Figure 8 As shown, the image processing method includes:

[0193] Step S801: In response to the input command applied to the operation interface, display the image of the target area to be processed on the operation interface;

[0194] Step S802: In response to the processing instructions applied to the operation interface, display the target segmentation result and the target classification result on the operation interface;

[0195] The target segmentation result and the target classification result are generated according to any of the above image processing methods.

[0196] In this embodiment, a visual image processing scheme can be implemented using any of the aforementioned image processing methods, facilitating human-computer interaction. The user inputs the image of the target region to be processed by triggering input commands on the operation interface. Further, the user triggers processing commands on the operation interface, instructing the system to process and analyze the image according to the aforementioned image processing methods, obtaining target segmentation and target classification results. Further, the system displays the target segmentation and target classification results on the operation interface for the user.

[0197] Through steps S801 to S802 described above, this embodiment of the application responds to input commands applied to the operation interface, displays the image to be processed of the target region on the operation interface, and responds to processing commands applied to the operation interface, displays the target segmentation result and the target classification result on the operation interface; wherein, the target segmentation result and the target classification result are generated according to any of the above image processing methods. Therefore, this embodiment of the application achieves the goal of jointly processing segmentation and classification tasks based on feature interaction calculations of feature maps at different resolutions, thereby achieving the technical effect of synergistically improving the accuracy of segmentation and classification results, and thus solving the technical problem in related technologies where schemes that independently process segmentation and classification tasks result in low segmentation and classification accuracy.

[0198] It should be noted that the preferred embodiments of steps S801 to S802 described above can be found in the foregoing description, and will not be repeated here.

[0199] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0200] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM), random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0201] According to an embodiment of this application, an apparatus embodiment for implementing the above-described image processing method is also provided. Figure 9 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of this application, as shown below. Figure 9 As shown, the device includes: an extraction module 901, used to perform multi-scale feature extraction on the image to be processed of the target region to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; a calculation module 902, used to perform feature interaction calculation based on the first feature map and the second feature map to obtain fused feature data; and a processing module 903, used to perform perceptual mapping on the fused feature data to obtain a target segmentation result and a target classification result, wherein the target segmentation result is used to identify the target pixel region in the image to be processed, and the target classification result is used to characterize the attribute category corresponding to the target pixel region.

[0202] Optionally, the image to be processed is a three-dimensional scanned image, and the extraction module 901 is further configured to: perform embedding calculation on the three-dimensional scanned image based on a first embedding dimension to obtain a first feature map; and perform downsampling processing on the first feature map based on a second embedding dimension to obtain a second feature map.

[0203] Optionally, the above-mentioned calculation module 902 is further used to: perform dual-path feature interaction calculation on the first feature map and the second feature map using the target model to obtain fused feature data, wherein the target model is used to model feature sequences at different scales for the first feature map and the second feature map respectively.

[0204] Optionally, the target model includes a first path and a second path, and the fused feature data includes global fused features and local fused features. The above-mentioned calculation module 902 is further used to: use the first path to perform pixel-level feature sequence modeling calculation on the first feature map to obtain spatial detail features; use the second path to perform global-scale context feature sequence modeling calculation on the second feature map and spatial detail features to obtain global fused features; and perform interactive fusion processing based on spatial detail features and global fused features to obtain local fused features.

[0205] Optionally, the target region includes: a first side region and a second side region that are symmetrical to each other. The first feature map includes a first feature corresponding to the first side region and a second feature corresponding to the second side region. The calculation module 902 is further configured to: integrate and map the first feature and the second feature to a pseudo-three-dimensional space to obtain first integrated data, wherein the first feature and the second feature are spatially symmetrically distributed in the pseudo-three-dimensional space; perform transformation processing on the first integrated data to obtain first concatenated input data, wherein the first feature and the second feature in the first concatenated input data are concatenated and spliced ​​into a one-dimensional feature sequence; perform feature sequence modeling and analysis on the first concatenated input data using network units set on the first path to compare the symmetrical correlation between the first feature and the second feature to obtain first concatenated output data; and perform three-dimensional reconstruction processing on the first concatenated output data to obtain spatial detail features, wherein the spatial detail features include a first detail feature corresponding to the first side region and a second detail feature corresponding to the second side region.

[0206] Optionally, the above-mentioned calculation module 902 is further configured to: perform downsampling processing on the spatial detail features based on the resolution and channel dimension corresponding to the second feature map to obtain a first sampling result; perform pixel overlay calculation on the second feature map and the first sampling result to obtain a first overlay result; and use the second path to perform global-scale context feature sequence modeling calculation on the first overlay result to obtain a global fusion feature.

[0207] Optionally, the second feature map includes a third feature corresponding to the first side portion and a fourth feature corresponding to the second side portion, and the first superposition result includes a first-side superposition feature corresponding to the first side portion and a second-side superposition feature corresponding to the second side portion. The calculation module 902 is further configured to: perform pixel superposition calculation on the third feature and the first-side sampling feature in the first sampling result to obtain a first-side superposition feature, wherein the first-side sampling feature is obtained by sampling the first detail feature; and perform pixel superposition calculation on the fourth feature and the second-side sampling feature in the first sampling result to obtain a second-side superposition feature, wherein the second-side sampling feature is obtained by sampling the second detail feature.

[0208] Optionally, the above-mentioned calculation module 902 is further configured to: integrate and map the first side superimposed features and the second side superimposed features to a pseudo-three-dimensional space to obtain second integrated data, wherein the first side superimposed features and the second side superimposed features are spatially symmetrically distributed in the pseudo-three-dimensional space; perform transformation processing on the second integrated data to obtain second serial input data; use network units set on the second path to perform feature sequence modeling and analysis on the second serial input data to obtain second serial output data; and perform three-dimensional reconstruction processing on the second serial output data to obtain global fusion features.

[0209] Optionally, the above-mentioned calculation module 902 is further configured to: perform upsampling processing on the global fusion features based on the resolution and channel dimension corresponding to the first feature map to obtain a second sampling result; and perform pixel overlay calculation based on the spatial detail features and the second sampling result to obtain local fusion features.

[0210] Optionally, the processing module 903 is further configured to: perform perceptual pooling processing on the local fusion features in the fusion feature data using the first perceptron to obtain the target segmentation result; and perform perceptual analysis on the global fusion features in the fusion feature data using the second perceptron to obtain the target classification result.

[0211] Optionally, in addition to all the modules mentioned above, the image processing device also includes: a training module (not shown in the figure), used to fine-tune the model parameters of the initial model using a training dataset to obtain the target model, wherein the training samples in the training dataset include: training images of the target region, the real segmentation mask corresponding to the training images, and the real classification label.

[0212] Optionally, the training module is further configured to: analyze and predict training images using the initial model to obtain predicted segmentation and classification results; calculate the segmentation task loss using a first loss function, the predicted segmentation results, and the true segmentation mask, wherein the first loss function is constructed based on the region overlap between the predicted segmentation results and the true segmentation mask; calculate the classification task loss using a second loss function, the predicted classification results, and the true classification labels, wherein the second loss function is constructed based on the cross-entropy loss between the predicted classification results and the true classification labels; perform a weighted summation of the segmentation task loss and the classification task loss to obtain the target loss; and fine-tune the model parameters of the initial model based on the target loss to obtain the target model.

[0213] It should be noted that the extraction module 901, calculation module 902 and processing module 903 mentioned above correspond to steps S202 to S206 in the embodiments. The three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiments.

[0214] According to an embodiment of this application, an apparatus embodiment for implementing the above-described computer-aided diagnosis method for breast cancer is also provided. Figure 10 This is a schematic diagram of the structure of a computer-aided diagnostic device for breast cancer according to an embodiment of this application, as shown below. Figure 10 As shown, the device includes: an extraction module 1001, used to perform multi-scale feature extraction on computed tomography images of the human chest to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; a calculation module 1002, used to perform feature interaction calculation based on the first feature map and the second feature map to obtain fused feature data; and a processing module 1003, used to perform perceptual mapping on the fused feature data to obtain target segmentation results and target classification results, wherein the target segmentation results are used to identify breast lesion regions in the computed tomography images, and the target classification results are used to characterize the benign or malignant category corresponding to the breast lesion regions.

[0215] It should be noted that the extraction module 1001, calculation module 1002 and processing module 1003 mentioned above correspond to steps S501 to S503 in the embodiments. The three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiments.

[0216] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described image processing method is also provided. Figure 11 This is a schematic diagram of the structure of another image processing apparatus according to an embodiment of this application, such as... Figure 11 As shown, the device includes: an acquisition module 1101, used to acquire an image processing request through a first application programming interface, wherein the request data carried in the image processing request includes: an image of the target region to be processed; and a return module 1102, used to return an image processing response through a second application programming interface, wherein the response data carried in the image processing response includes: target segmentation results and target classification results, wherein the target segmentation results and target classification results are generated according to any one of the above image processing methods.

[0217] It should be noted that the above-mentioned acquisition module 1101 and return module 1102 correspond to steps S601 to S602 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0218] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described image processing method is also provided. Figure 12 This is a schematic diagram of the structure of another image processing apparatus according to an embodiment of this application, such as... Figure 12As shown, the device includes: an acquisition module 1201, used to acquire a currently input image processing dialogue request, wherein the request data carried in the image processing dialogue request includes: an image of the target region to be processed; a return module 1202, used to respond to the image processing dialogue request and return an image processing dialogue response, wherein the information carried in the image processing dialogue response includes: target segmentation results and target classification results, the target segmentation results and target classification results being generated according to any one of the above image processing methods; and a display module 1203, used to present the target segmentation results and target classification results within a graphical user interface.

[0219] It should be noted that the above-mentioned acquisition module 1201, return module 1202 and display module 1203 correspond to steps S701 to S703 in the embodiment. The three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0220] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described image processing method is also provided. Figure 13 This is a schematic diagram of the structure of another image processing apparatus according to an embodiment of this application, such as... Figure 13 As shown, the device includes: a first response module 1301, used to respond to input commands applied to the operation interface and display the image to be processed of the target part on the operation interface; a second response module 1302, used to respond to processing commands applied to the operation interface and display the target segmentation result and the target classification result on the operation interface; wherein the target segmentation result and the target classification result are generated according to any of the above image processing methods.

[0221] It should be noted here that the first response module 1301 and the second response module 1302 mentioned above correspond to steps S801 to S802 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0222] It should be noted that the above-mentioned modules or units may be hardware or software components stored in memory and processed by one or more processors. The above-mentioned modules may also be part of a device and run in a computer terminal.

[0223] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0224] It should be noted that the preferred embodiments involved in the above embodiments of this application are the same as the solutions, application scenarios and implementation processes provided in the above embodiments, but are not limited to the solutions provided in the above embodiments.

[0225] Embodiments of this application can provide an image processing system, including: a client for sending a target topic; a server connected to the client for performing multi-scale feature extraction on the image to be processed of the target region to obtain a first feature map and a second feature map, performing feature interaction calculation based on the first and second feature maps to obtain fused feature data, and performing perceptual mapping on the fused feature data to obtain a target segmentation result and a target classification result, wherein the resolution of the second feature map is lower than that of the first feature map, the target segmentation result is used to identify the target pixel region in the image to be processed, and the target classification result is used to characterize the attribute category corresponding to the target pixel region; the client is also used to output the target segmentation result and the target classification result.

[0226] Embodiments of this application may provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the image processing method of any one of the foregoing methods or the computer-aided diagnosis method for breast cancer described above.

[0227] Figure 14 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 14 As shown, the electronic device 140 may include one or more (only one is shown in the figure) processors 142, memory 144, memory controller, and peripheral interfaces.

[0228] The aforementioned electronic device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, personal computers (PCs), all-in-one model machines, etc., and the electronic device may have the model described in the above embodiments of this application pre-installed.

[0229] Specifically, this electronic device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this electronic device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this electronic device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this electronic device can also create applications based on models, providing Application Programming Interface (API) invocation capabilities. Models can be invoked into the created applications through the API interface, and application management tools are provided to manage and monitor the applications.

[0230] Furthermore, the electronic device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing rich training resources to help users learn and master artificial intelligence (AI) technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0231] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method, image processing device, or computer-aided diagnosis method or device for breast cancer described in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the image processing method or computer-aided diagnosis method for breast cancer described in the embodiments. The memory may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0232] The processor can invoke the executable program stored in the memory through the transmission device to execute any of the above-described image processing methods or computer-aided diagnosis methods for breast cancer in the above embodiments.

[0233] Those skilled in the art will understand that, Figure 14 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, and mobile internet devices (MIDs) and other terminal devices. Figure 14 This does not limit the structure of the aforementioned electronic device. For example, electronic device 140 may also include components that are more... Figure 14 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 14 The different configurations shown.

[0234] Those skilled in the art will understand that all or part of the steps in the various image processing methods or computer-aided diagnosis methods for breast cancer in the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0235] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the image processing method or the computer-aided diagnosis method for breast cancer described above.

[0236] Optionally, in this embodiment, the storage medium may be located in an electronic device.

[0237] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute any of the above-described image processing methods or computer-aided diagnosis methods for breast cancer.

[0238] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the image processing method or the computer-aided diagnosis method for breast cancer provided in the above embodiments.

[0239] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the image processing method or the computer-aided diagnosis method for breast cancer provided in the above embodiments.

[0240] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the above-described computer program is executed by a processor, it implements the image processing method or the computer-aided diagnosis method for breast cancer provided in the above embodiments.

[0241] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0242] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0243] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0244] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0245] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the image processing method or the computer-aided diagnosis method for breast cancer described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, ROM, RAM, portable hard drives, magnetic disks, or optical disks.

[0246] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Multi-scale feature extraction is performed on the image to be processed of the target region to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map; Based on the first feature map and the second feature map, feature interaction calculation is performed to obtain fused feature data; Perceptual mapping is performed on the fused feature data to obtain target segmentation results and target classification results. The target segmentation results are used to identify target pixel regions in the image to be processed, and the target classification results are used to characterize the attribute category corresponding to the target pixel regions.

2. The image processing method according to claim 1, characterized in that, The image to be processed is a three-dimensional scanned image. Multi-scale feature extraction is performed on the scanned image to obtain the first feature map and the second feature map, including: Based on the first embedding dimension, the embedding calculation is performed on the three-dimensional scanned image to obtain the first feature map; The second feature map is obtained by downsampling the first feature map based on the second embedding dimension.

3. The image processing method according to claim 1, characterized in that, Based on the first feature map and the second feature map, feature interaction calculations are performed to obtain the fused feature data, including: The target model is used to perform dual-path feature interaction calculation on the first feature map and the second feature map to obtain the fused feature data. The target model is used to model feature sequences at different scales for the first feature map and the second feature map respectively.

4. The image processing method according to claim 3, characterized in that, The target model includes a first path and a second path, and the fused feature data includes global fused features and local fused features. The target model is used to perform dual-path feature interaction calculations on the first feature map and the second feature map to obtain the fused feature data, which includes: The first path is used to perform pixel-level feature sequence modeling and calculation on the first feature map to obtain spatial detail features; The second path is used to perform global-scale context feature sequence modeling and calculation on the second feature map and the spatial detail features to obtain the global fusion feature; The local fusion features are obtained by performing interactive fusion processing based on the spatial detail features and the global fusion features.

5. The image processing method according to claim 4, characterized in that, The target region includes two symmetrical sides: a first side and a second side. The first feature map includes a first feature corresponding to the first side and a second feature corresponding to the second side. The first feature map is modeled and calculated using the first path at the pixel level to obtain the spatial detail features, which include: The first feature and the second feature are integrated and mapped to a pseudo-three-dimensional space to obtain the first integrated data, wherein the first feature and the second feature are spatially symmetrically distributed in the pseudo-three-dimensional space. The first integrated data is transformed to obtain the first concatenated input data, wherein the first feature and the second feature in the first concatenated input data are concatenated and spliced ​​into a one-dimensional feature sequence. The network units set on the first path are used to perform feature sequence modeling and analysis on the first serial input data to compare the symmetric correlation between the first feature and the second feature, and obtain the first serial output data. The first serial output data is subjected to three-dimensional reconstruction processing to obtain the spatial detail features, wherein the spatial detail features include a first detail feature corresponding to the first side part and a second detail feature corresponding to the second side part.

6. The image processing method according to claim 5, characterized in that, Using the second path, global-scale contextual feature sequence modeling is performed on the second feature map and the spatial detail features to obtain the global fusion features, which include: Based on the resolution and channel dimension corresponding to the second feature map, the spatial detail features are downsampled to obtain the first sampling result; Pixel overlay calculations are performed on the second feature map and the first sampling result to obtain a first overlay result; The global fusion feature is obtained by using the second path to perform global-scale context feature sequence modeling and calculation on the first superposition result.

7. The image processing method according to claim 6, characterized in that, The second feature map includes a third feature corresponding to the first side portion and a fourth feature corresponding to the second side portion. The first superposition result includes a first side superposition feature corresponding to the first side portion and a second side superposition feature corresponding to the second side portion. Pixel superposition calculation is performed on the second feature map and the first sampling result to obtain the first superposition result, which includes: The first side-sampled feature is obtained by pixel superposition calculation of the third feature and the first side-sampled feature in the first sampling result, wherein the first side-sampled feature is obtained by sampling the first detail feature; The second side-sampled feature is obtained by pixel superposition calculation of the fourth feature and the second side-sampled feature in the first sampling result, wherein the second side-sampled feature is obtained by sampling the second detail feature.

8. The image processing method according to claim 7, characterized in that, The global fusion features are obtained by using the second path to perform global-scale context feature sequence modeling and calculation on the first superposition result, including: The first side superimposed feature and the second side superimposed feature are integrated and mapped to a pseudo-3D space to obtain the second integrated data, wherein the first side superimposed feature and the second side superimposed feature are spatially symmetrically distributed in the pseudo-3D space. The second integrated data is transformed to obtain the second serial input data; The network units set on the second path are used to perform feature sequence modeling and analysis on the second concatenated input data to obtain the second concatenated output data. The second serial output data is subjected to three-dimensional reconstruction processing to obtain the global fusion feature.

9. The image processing method according to claim 4, characterized in that, Based on the spatial detail features and the global fusion features, interactive fusion processing is performed to obtain the local fusion features, including: Based on the resolution and channel dimension corresponding to the first feature map, the global fusion feature is upsampled to obtain the second sampling result; The local fusion feature is obtained by performing pixel overlay calculation based on the spatial detail features and the second sampling result.

10. The image processing method according to claim 3, characterized in that, The method further includes: The initial model is fine-tuned using a training dataset to obtain the target model. The training samples in the training dataset include: training images of the target region, the ground truth segmentation mask and the ground truth classification label corresponding to the training images.

11. A computer-aided diagnostic method for breast cancer, characterized in that, include: Multi-scale feature extraction is performed on computed tomography images of the human chest to obtain a first feature map and a second feature map, wherein the resolution of the second feature map is lower than that of the first feature map. Based on the first feature map and the second feature map, feature interaction calculation is performed to obtain fused feature data; Perceptual mapping is performed on the fused feature data to obtain target segmentation results and target classification results. The target segmentation results are used to identify breast lesion regions in the computed tomography image, and the target classification results are used to characterize the benign or malignant category of the breast lesion regions.

12. An image processing method, characterized in that, include: Obtain the currently input image processing dialogue request, wherein the request data carried in the image processing dialogue request includes: the image to be processed of the target area; In response to the image processing dialogue request, an image processing dialogue response is returned, wherein the information carried in the image processing dialogue response includes: target segmentation result and target classification result, the target segmentation result and the target classification result being generated according to the image processing method according to any one of claims 1 to 10; The target segmentation results and target classification results are presented within a graphical user interface.

13. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the image processing method according to any one of claims 1 to 10 or the computer-aided diagnosis method for breast cancer according to claim 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the image processing method of any one of claims 1 to 10 or the computer-aided diagnosis method for breast cancer of claim 11.

15. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image processing method of any one of claims 1 to 10 or the computer-aided diagnosis method for breast cancer of claim 11.

Images