Bladder lesion detection method, device, equipment, storage medium and program product

Abstract

The application provides a bladder lesion detection method, device, equipment, storage medium and program product, and relates to the technical field of computers. The method identifies the filling state of the bladder in each frame of bladder ultrasound image, provides important prior information for subsequent lesion detection, detects the bladder in different filling states through a detection network model, dynamically adjusts the detection network model through the identified filling state, enables the detection network model to be more focused on fine detection of the lesion, adapts to the image feature changes of the bladder in different filling states, and can effectively improve the accuracy of lesion detection and reduce the probability of false detection or missed detection.

Description

Methods, devices, equipment, storage media, and program products for detecting bladder lesions Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, device, storage medium, and program product for detecting bladder lesions. Background Technology

[0002] With the development of electronic imaging, medical imaging, and artificial intelligence technologies, the ability to visualize internal human tissues and organs through media (such as X-rays, electromagnetic fields, ultrasound, endoscopes, etc.) has become an important means for physicians to diagnose and treat diseases. Therefore, in order to reduce the burden on physicians, lower the difficulty of disease diagnosis, and improve the lesion detection rate, the automatic detection of lesions in images and the real-time display of detection results to inform physicians is of great significance.

[0003] Current methods for detecting lesions in bladder ultrasound imaging mainly rely on traditional image processing techniques (such as edge detection and threshold segmentation). These methods are easily affected by noise in complex backgrounds, leading to false positives or false negatives. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, storage medium, and program product for detecting bladder lesions, so as to improve the problem that existing detection methods are prone to false detection or missed detection.

[0005] In a first aspect, embodiments of this application provide a method for detecting bladder lesions, the method comprising:

[0006] Acquire multiple frames of bladder ultrasound images;

[0007] Identify the target bladder filling status in each frame of bladder ultrasound image;

[0008] The corresponding detection network model is determined based on the target filling state, and bladder lesions are detected through the determined detection network model to obtain the detection results. Each filling state corresponds to a detection network model.

[0009] In the above implementation process, by identifying the bladder filling state in each frame of bladder ultrasound image, important prior information is provided for subsequent lesion detection. A detection network model is used to detect bladders in different filling states. The detection network model is dynamically adjusted based on the identified filling state, so that the detection network model can focus more on the fine detection of lesions and adapt to the changes in imaging characteristics of the bladder in different filling states. This can effectively improve the accuracy of lesion detection and reduce the probability of false detection or missed detection.

[0010] Optionally, determining the corresponding detection network model based on the target filling state includes:

[0011] The current filling status of the bladder is determined based on the target filling status of the bladder in multiple frames of bladder ultrasound images;

[0012] The corresponding detection network model is determined based on the current filling state.

[0013] In the above implementation process, by combining multiple frames of images to determine the current filling state, the problem of inaccurate classification results caused by noise and other factors affecting a single frame image can be reduced.

[0014] Optionally, determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images includes:

[0015] If the target bladder filling state is the same in a first set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to be the target filling state.

[0016] In the above implementation process, by identifying whether there are targets with the same filling state that have reached a set number, the filling state is determined to be stable, which can improve the accuracy and stability of identification.

[0017] Optionally, determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images includes:

[0018] If the target filling state of the bladder is different in a second set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to be the target filling state with the most occurrences.

[0019] In the above implementation process, by analyzing multiple consecutive frames of images, false detections and missed detections caused by misjudgment of a single frame image can be reduced.

[0020] Optionally, determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images includes:

[0021] If the target filling state of the bladder differs in a third set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to include various different target filling states.

[0022] In the above implementation process, if most of the target filling states are different, it indicates that the filling state is changing. At this time, a detection network model corresponding to multiple filling states can be used for detection, which can improve the accuracy and stability of the detection results.

[0023] Optionally, identifying the target bladder filling state in each frame of bladder ultrasound image includes:

[0024] The confidence scores of the bladder in each frame of bladder ultrasound image identified by the classification network model are obtained. The bladder filling states include non-filling, standard filling and overfilling.

[0025] The state with the highest confidence level is determined as the target state of fullness.

[0026] In the above implementation process, the target filling state is determined by the confidence level, which can determine the bladder filling state in each frame of the image, thus facilitating the accurate selection of the detection network model.

[0027] Optionally, determining the corresponding detection network model based on the target filling state includes:

[0028] The current filling status of the bladder is determined based on the target filling status of the bladder and the corresponding confidence level in multiple frames of bladder ultrasound images.

[0029] The corresponding detection network model is determined based on the current filling state.

[0030] In the above implementation process, combining confidence level to determine the current filling state can improve the accuracy and stability of recognition.

[0031] Optionally, identifying the target bladder filling state in each frame of bladder ultrasound image includes:

[0032] The target filling state of the bladder in each frame of bladder ultrasound image is identified by a classification network model. The filling state includes unfilled state, standard filling state and overfilled state.

[0033] The classification network model is the EfficientNet-B4 classification network, and / or the detection network model is the YOLOv9 detection network. This combination of the EfficientNet-B4 classification network and the YOLOv9 detection network enables accurate classification of bladder filling states and accurate detection of bladder lesions.

[0034] Optionally, during the training of the EfficientNet-B4 classification network, training is performed by freezing the shallow feature extractors and fine-tuning the network parameters of the deep networks. By freezing the parameters of the shallow feature extractors (i.e., not updating the weights of these network layers), the pre-trained model's ability to extract basic features is preserved, avoiding a weakening of the generalization ability to these basic features due to overtraining. Fine-tuning the parameters of the deep networks allows the high-level features of the network to better adapt to the specific requirements of medical images, improving the classification ability for bladder fullness.

[0035] Optionally, the anchor size of the YOLOv9 detection network corresponding to the standard filled state is the standard size, the anchor size of the YOLOv9 detection network corresponding to the unfilled state is reduced by a set ratio compared to the standard size, and the anchor size of the YOLOv9 detection network corresponding to the overfilled state is enlarged by a set ratio compared to the standard size.

[0036] In the above implementation process, when the bladder is not full, the lesions may be relatively small due to the more contracted bladder shape. Reducing the YOLOv9 anchor size by 50% allows for more precise prediction of the boundaries of small lesions. When the bladder is overfilled, the lesions may be relatively large due to the more expanded bladder shape. Enlarging the YOLOv9 anchor size by 50% allows for better fitting of the lesion size, thereby improving detection accuracy.

[0037] Secondly, embodiments of this application provide a bladder lesion detection device, the device comprising:

[0038] The image acquisition module is used to acquire multiple frames of bladder ultrasound images;

[0039] The state detection module is used to identify the target bladder filling state in each frame of bladder ultrasound image;

[0040] The lesion detection module is used to determine the corresponding detection network model based on the target filling state, and to detect bladder lesions through the determined detection network model to obtain detection results. Each filling state corresponds to a detection network model.

[0041] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method provided in the first aspect above are performed.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0043] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the steps of the method provided in the first aspect above.

[0044] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 is a flowchart of a bladder lesion detection method provided in an embodiment of this application;

[0047] Figure 2 is a flowchart illustrating a specific example of a bladder lesion detection method provided in an embodiment of this application.

[0048] Figure 3 is a structural block diagram of a bladder lesion detection device provided in an embodiment of this application;

[0049] Figure 4 is a schematic diagram of the structure of an electronic device for performing a bladder lesion detection method according to an embodiment of this application. Detailed Implementation

[0050] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0051] It should be noted that the terms "system" and "network" in the embodiments of this invention can be used interchangeably. "Multiple" refers to two or more; therefore, in the embodiments of this invention, "multiple" can also be understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0052] It should also be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0053] This application provides a method for detecting bladder lesions. This method identifies the bladder filling state in each frame of bladder ultrasound image, providing important prior information for subsequent lesion detection. A detection network model is used to detect bladders in different filling states. The detection network model is dynamically adjusted based on the identified filling state, enabling the detection network model to focus more on the fine detection of lesions and adapt to the changes in image characteristics of the bladder in different filling states. This can effectively improve the accuracy of lesion detection and reduce the probability of false detection or missed detection.

[0054] Please refer to Figure 1, which is a flowchart of a bladder lesion detection method provided in an embodiment of this application. The method includes the following steps:

[0055] Step S110: Acquire multiple frames of bladder ultrasound images.

[0056] Multi-frame bladder ultrasound images can be video frame images obtained through ultrasound equipment. For example, the method of this application is executed by electronic equipment (such as terminals, servers, etc.). The electronic equipment can receive video frame images acquired by the ultrasound equipment. When the ultrasound equipment acquires images, it acquires a video stream, such as a video stream of 30 frames per second. Here, multi-frame bladder ultrasound images refer to video frame images, which can refer to video frame images acquired per second. That is, this solution can perform lesion detection based on video frame images acquired per second, or it can acquire video frame images over a period of time for lesion detection.

[0057] In some implementations, the original video frame images acquired by the ultrasound equipment may have a relatively high resolution. To reduce computational load, the electronic device can perform adaptive resolution adjustment after receiving the original bladder ultrasound image. For example, if the resolution of the original bladder ultrasound image exceeds a set value (e.g., 720p, the specific value can be set according to actual needs), the original bladder ultrasound image can be reduced in size. For instance, an edge-preserving downsampling algorithm can be used to reduce the size of the original bladder ultrasound image to 640*640. Then, the processed bladder ultrasound image is used for subsequent detection. Of course, the specific reduction size can be flexibly set according to actual needs, which can both preserve the key details of the image and reduce the computational load.

[0058] Among them, edge-preserving downsampling is an algorithm that reduces image resolution while preserving image edge information as much as possible. Edge information can provide more key features for subsequent bladder lesion detection, thereby improving the accuracy of subsequent detection results. Reducing image resolution can reduce computational load, while using edge-preserving downsampling to reduce image resolution can simultaneously preserve key details and edge information in the image.

[0059] Step S120: Identify the target bladder filling status in each frame of bladder ultrasound image.

[0060] The filling state can include an unfilled state, a standard filling state, and an overfilled state. Understandably, the target filling state can refer to one of these three states. Specifically, when identifying the target filling state of the bladder in an ultrasound image, the obtained information can be either a category of filling state (i.e., the three aforementioned states) or a value representing the filling state (i.e., the value of each state, referred to as confidence level in subsequent embodiments). The identification can be implemented using appropriate image processing algorithms or machine learning models.

[0061] In some implementations, the bladder filling state is determined based on the bladder's morphology, which can be determined by combining bladder wall thickness, volume, and morphological characteristics. The volume is calculated using the formula V = 0.5 * major diameter * transverse diameter * anteroposterior diameter. When classifying bladders, a bladder wall thickness exceeding 3 mm and a volume less than 100 ml is defined as an unfilled state; a volume between 100-400 ml and a bladder wall thickness of 1-3 mm is defined as a standard filled state; and a volume above 400 ml and a bladder wall thinning to less than 1 mm is defined as an overfilled state.

[0062] Understandably, in practical applications, the filling state can be further distinguished according to actual needs, and is not limited to the three filling states provided in this solution.

[0063] In some implementations, a classification network model can be used to identify the target bladder filling state in each frame of a bladder ultrasound image. For example, if the input is a series of video frames that are sequentially connected in time, these multiple bladder ultrasound images can be input into the classification network model in the order they are received. The classification network model can then sequentially identify the bladder filling state in each input frame. The role of the classification network model is to extract key features from the input bladder ultrasound images and classify them based on these key features, i.e., classifying them as unfilled, standard-filled, or overfilled.

[0064] Step S130: Determine the corresponding detection network model based on the target filling state, and detect bladder lesions through the determined detection network model to obtain detection results.

[0065] After determining the target bladder filling state in each frame of bladder ultrasound image, a corresponding detection network model can be determined based on the target filling state. Each filling state corresponds to a separate detection network model; for example, the unfilled state corresponds to detection network model 1, the standard filling state to detection network model 2, and the overfilled state to detection network model 3. This can be understood as each filling state having its own detection network model, or as multiple filling states corresponding to a single basic detection network model. However, each filling state corresponds to a specific set of network parameters, which can be adjusted based on the filling state. For instance, if the target filling state is determined to be unfilled, the parameters of the basic detection network model can be adjusted to match the parameters corresponding to the unfilled state.

[0066] In some implementations, after determining the target bladder filling state in each frame of bladder ultrasound image, the frame of bladder ultrasound image can be input into the corresponding detection network model for detection. For example, if the target bladder filling state in a certain frame of bladder ultrasound image is determined to be an unfilled state, the frame of image can be further input into detection network model 1. Detection network model 1 can then perform bladder lesion detection on the frame of image and output the detection results. The detection results may include information such as the size, location, shape, and classification of the bladder lesions. The lesion classification can be divided into benign and malignant categories.

[0067] The detection network model can extract image features from bladder ultrasound images and detect bladder lesions based on these features, thereby obtaining detection results. The classification network model can identify the bladder filling state in each frame of bladder ultrasound images. After determining the corresponding detection network model, the bladder lesions in each frame of bladder ultrasound images can be detected using the corresponding detection network model, and the detection results can be obtained.

[0068] To facilitate user viewing, the detection results can be output and displayed so that users can directly view the detection results of bladder lesions in each frame of bladder ultrasound images.

[0069] In the above implementation process, the classification network model identifies the bladder filling state in each frame of bladder ultrasound image, providing important prior information for subsequent lesion detection. The detection network model is used to detect bladders in different filling states. The detection network model is dynamically adjusted based on the identified filling state, enabling the detection network model to focus more on the fine detection of lesions and adapt to the changes in imaging characteristics of the bladder in different filling states. This can effectively improve the accuracy of lesion detection and reduce the probability of false detection or missed detection.

[0070] Based on the above embodiments, when the above classification network model identifies the bladder filling state in each frame of bladder ultrasound image, a single frame image may be affected by noise, occlusion or other interference factors, resulting in inaccurate classification results. Therefore, the current filling state can be determined based on the target filling state of the bladder in multiple frames of bladder ultrasound images, and then the corresponding detection network model can be determined based on the current filling state. This can reduce the influence of these accidental factors and improve the stability of classification.

[0071] The multiple bladder ultrasound images mentioned here can refer to the multiple bladder ultrasound images acquired in step S110 above, or a portion of those images. In this implementation, the current filling state of the bladder can be determined by combining the target filling state of the bladder in the multiple bladder ultrasound images.

[0072] In some implementations, an image buffer can be provided in the electronic device to store multiple frames of received bladder ultrasound images, such as 15 frames. The electronic device can use 15 frames as a window for image recognition and detection. The electronic device can store 15 frames of bladder ultrasound images in the image buffer in chronological order and detect the bladder filling state in each frame of bladder ultrasound images in chronological order. Then, the number of bladder ultrasound images can be set as a unit for recognition, such as 5 frames. The classification network model can first identify the target bladder filling state in 5 frames of bladder ultrasound images. In this case, 5 target filling states can be obtained. Then, the current filling state can be determined based on these 5 target filling states. For example, if the current filling state is determined to be a non-filling state, then the detection network model is determined to be the detection network model 1 corresponding to the non-filling state; if the current filling state is determined to be a standard filling state, then the detection network model is determined to be the detection network model 2 corresponding to the standard filling state.

[0073] In the above implementation process, by combining multiple frames of images to determine the current filling state, the problem of inaccurate classification results caused by noise and other factors affecting a single frame image can be reduced.

[0074] The following section details several methods for determining the current bladder filling status by combining the target filling status of the bladder in multiple frames of bladder ultrasound images.

[0075] (1) If the target filling state of the bladder is the same in a first set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to be the target filling state.

[0076] For example, if the initial set quantity is 5 frames (the specific value can be flexibly set according to actual needs), the target filling state of the bladder can be detected in 5 consecutive frames of bladder ultrasound images. If the target filling state of the bladder is the same in these 5 frames of bladder ultrasound images, such as all being in a non-filling state, the system can determine that the current filling state is a non-filling state, and then determine the detection network model 1 corresponding to the non-filling state. In this case, these 5 frames of bladder ultrasound images can be input into detection network model 1 for detection, thus obtaining the detection results of bladder lesions in these 5 frames of bladder ultrasound images.

[0077] In the above implementation process, by identifying whether there are targets with the same filling state that have reached a set number, the filling state is determined to be stable, which can improve the accuracy and stability of identification.

[0078] (2) If the target filling state of the bladder is different in the consecutive frames of bladder ultrasound images that reach the second set number, then the current filling state is determined to be the target filling state with the most numbers.

[0079] The second set number here can be the same as or different from the first set number mentioned above. For example, if the second set number is also 5 frames, it is determined whether there are differences in the target filling state of the bladder in 5 consecutive frames of bladder ultrasound images. If there are differences, the target filling state with the most occurrences can be determined from these 5 target filling states. For example, if 4 of these 5 target filling states are non-filling states and 1 is a standard filling state, then the current filling state is determined to be a non-filling state. Then the detection network model corresponding to the non-filling state can be determined, i.e., detection network model 1.

[0080] The classification network model can record the classification results in the image buffer after identifying the target filling state of the bladder in each frame of bladder ultrasound image. When the electronic device determines that the target filling state of the bladder is the same or different in a set number of consecutive frames of bladder ultrasound images, it can determine the current filling state and then determine the detection network model corresponding to the current filling state.

[0081] Continuing the example above, if the target filling state of the bladder is the same in the preceding 5 consecutive frames (e.g., frames 1-5) of bladder ultrasound images, after inputting these 5 frames of bladder ultrasound images into detection network model 1 for detection, the classification network model can continue to detect the target filling state of the bladder in the subsequent 5 frames (i.e., frames 6-10) of bladder ultrasound images. Then, the filling state and the corresponding detection network model can be determined according to the above logic. For example, if 3 of the target filling states of the bladder in the subsequent 5 frames of bladder ultrasound images are standard filling states and 2 are overfilling states, then the filling state can be locked as a standard filling state. At this time, the detection network model can be determined as detection network model 2 corresponding to the standard filling state. Then, these 5 frames of bladder ultrasound images can be input into detection network model 2 respectively for lesion detection to obtain the detection results.

[0082] In some other implementations, the second set number may differ from the first set number. For example, the electronic device may first determine whether there are any target filling states that are the same as the first set number, such as whether there are any consecutive 5 frames of images with the same target filling state of the bladder. If they are different, the current filling state is not determined at this time, but the determination can continue. For example, if the second set number is 15 frames, the target filling states of the bladder in the next 10 frames of bladder ultrasound images can be detected to obtain 15 target filling states. The filling state with the most occurrences can then be determined from these 15 target filling states. For example, if the target filling state with the most occurrences is the standard filling state, then the current filling state is determined to be the standard filling state. Of course, if there are 7 non-filling states, 7 standard filling states, and 1 overfilling state, then the current filling state can be determined to include both non-filling states and standard filling states. In this case, the determined detection network model includes the network detection models corresponding to the non-filling states and the standard filling states.

[0083] When performing lesion detection, each of the 15 frames of bladder ultrasound images can be input into the corresponding detection network model for detection. If there are multiple detection network models, each frame of bladder ultrasound image will be input into multiple detection network models for detection, and the detection results of multiple detection network models can be combined in the end.

[0084] In the above implementation process, by analyzing multiple consecutive frames of images, false detections and missed detections caused by misjudgment of a single frame image can be reduced.

[0085] (3) If the target filling state of the bladder is different in the continuous frame ultrasound images of the third set number of frames, then the current filling state is determined to include various different target filling states.

[0086] The third set quantity can be the same as or different from the second set quantity. Continuing the above example, taking a third set quantity of 5 frames as an example, if the target filling state of the bladder is the same in 5 consecutive frames of bladder ultrasound images, the current filling state is determined according to the above method (1). Otherwise, the current filling state can be determined according to method (2) or method (3). If determined by method (3), assuming that the target filling state of the bladder in 5 frames of bladder ultrasound images includes the standard filling state and the overfilling state, the current filling state is determined to include the standard filling state and the overfilling state. If the target filling state of the bladder in 5 frames of bladder ultrasound images includes the unfilled state, the standard filling state and the overfilling state, the current filling state is determined to include the unfilled state, the standard filling state and the overfilling state. The detection network model determined subsequently is also the detection network model corresponding to various filling states.

[0087] In the above implementation process, if most of the target filling states are different, it indicates that the filling state is changing. At this time, a detection network model corresponding to multiple filling states can be used for detection, which can improve the accuracy and stability of the detection results.

[0088] Based on the above embodiments, in the method of identifying the filling state, the classification network model can also output the confidence level of each filling state. Then, the final target filling state can be determined according to the confidence level. That is, the confidence level of each filling state of the bladder in each frame of bladder ultrasound image identified by the classification network model can be obtained, and then the filling state with the highest confidence level can be determined as the target filling state of the bladder in that frame of bladder ultrasound image.

[0089] The confidence level represents the probability of each filling state. For example, in a frame of bladder ultrasound image, the confidence levels of the three filling states of the bladder are: unfilled (0.7), standard filled (0.2), and overfilled (0.1). In this case, the confidence level of the unfilled state is the highest, so the target filling state of the bladder in that frame of bladder ultrasound image can be determined as the unfilled state. The target filling state of the bladder in each frame of bladder ultrasound image can be determined in this way.

[0090] In the above implementation process, the target filling state is determined by the confidence level, which can determine the bladder filling state in each frame of the image, thus facilitating the accurate selection of the detection network model.

[0091] Based on the above embodiments, in order to further improve the accuracy of classification and the accuracy of determining the detection network model, when determining the current filling state, the current filling state can be determined according to the target filling state of the bladder in multiple frames of bladder ultrasound images and the corresponding confidence level, and then the corresponding detection network model can be determined according to the current filling state.

[0092] The following section details the method of combining confidence levels for judgment.

[0093] (4) If the target filling state of the bladder is the same in the continuous frame ultrasound images of the fourth set number of frames and the corresponding confidence scores all exceed the set threshold, then the current filling state is determined to be the target filling state.

[0094] For example, if the target bladder filling state is the same in 5 frames of bladder ultrasound images, such as all being unfilled, and the confidence level of each unfilled state exceeds a set threshold (e.g., 0.85, the specific value can be flexibly set according to actual needs), then the current filling state can be determined as unfilled. However, if the target bladder filling state is the same in 5 frames of bladder ultrasound images, such as all being unfilled, but more than half of the unfilled states have a confidence level exceeding the set threshold, such as 3 unfilled states having a confidence level exceeding the set threshold, then the current filling state can also be determined as unfilled.

[0095] If the confidence level of a certain number of unfilled states does not exceed a set threshold (for example, only two unfilled states have a confidence level exceeding the threshold), then the classification network model can continue to identify the confidence level of the target filling state of the bladder in the next five frames of bladder ultrasound images. This yields ten target filling states, which are then combined with their corresponding confidence levels for judgment. For instance, if all ten target filling states are the same (e.g., still unfilled), but more than five unfilled states have a confidence level exceeding the set threshold, then the current filling state is determined to be unfilled. Otherwise, the system continues to detect the target filling states and confidence levels of the bladder in other frames of bladder ultrasound images.

[0096] Of course, if the target filling state is different in the first 5 frames or the next 10 frames, the current filling state can be determined according to the following methods (2) or (5) or (6), such as selecting the target filling state with the most quantity as the current filling state, or selecting the target filling state with a confidence level exceeding the set threshold as the current filling state.

[0097] (5) If the target filling state of the bladder is different in the fifth set number of consecutive frame bladder ultrasound images, the target filling state with a confidence level exceeding the set threshold is statistically analyzed, and the current filling state is determined as the target filling state.

[0098] For example, if five bladder ultrasound images show different target bladder filling states, such as an unfilled state and a standard filled state, the filling states with confidence levels exceeding a set threshold (e.g., 0.85) can be counted. If the confidence level of an unfilled state exceeds 0.85, while the confidence level of a standard filled state does not exceed 0.85, then the current filling state is determined to be an unfilled state. If the confidence levels of both unfilled and standard filled states exceed 0.85, then the current filling state includes both unfilled and standard filled states.

[0099] (6) If the target filling state of the bladder is different in the sixth set number of consecutive frame bladder ultrasound images, and more than a preset number of them have a confidence level less than the set threshold, then the current filling state is determined to include various different target filling states.

[0100] For example, if different target bladder filling states are identified in 15 frames of bladder ultrasound images, the number of target filling states with a confidence level less than a set threshold (e.g., 0.85) is counted. If the number exceeds a set number (which can be half of a sixth set number, or set according to actual needs, such as 10), it indicates that the bladder filling state is changing, so the filling state cannot be definitively determined. In this case, the various target filling states among these 15 target filling states can be directly used as the current filling state. For example, if the 15 target filling states include an unfilled state and a standard filled state, then the current filling state includes the unfilled state and the standard filled state. If the 15 target filling states include an unfilled state, a standard filled state, and an overfilled state, then the current filling state includes the unfilled state, the standard filled state, and the overfilled state.

[0101] Understandably, the fourth, fifth, and sixth preset quantities mentioned above can be flexibly set according to actual needs. In specific applications, the electronic device can use 15 frames as a window for filling status recognition and detection. When the filling status is the same, the preset quantity can be set to 5; when the filling status is different, the preset quantity can be set to 15. Thus, the first and fourth preset quantities mentioned above can be 5, and the remaining preset quantities can be 15. That is to say, if the filling status is the same, fewer frames of bladder ultrasound images are needed to determine the current filling status; if the filling status is different, more frames of bladder ultrasound images are needed to determine the current filling status, thereby reducing the influence of noise and improving the stability and accuracy of recognition.

[0102] In the above implementation process, combining confidence level to determine the current filling state can improve the accuracy and stability of recognition.

[0103] Based on the above embodiments, when identifying the filling state, if a classification network model is used to identify the target filling state of the bladder in each frame of bladder ultrasound image, in some embodiments, the classification network model can be an EfficientNet-B4 classification network, and / or the detection network model can be a YOLOv9 detection network.

[0104] The EfficientNet-B4 classification network, through its depthwise separable convolution and compound scaling techniques, can efficiently extract multi-scale features and achieve high classification accuracy.

[0105] The YOLOv9 detection network boasts high detection accuracy while significantly reducing parameters and computational load. This solution allows both the classification and detection network models to be deployed and run in real-time on the device.

[0106] In some implementations, the classification network model in this scheme can also be other models, such as the ResNet model, the DenseNet model, or other versions of the EfficientNet system, and the detection network model can also be other models, such as the Faster R-CNN model, the RetinaNet model, or other versions of the YOLO system.

[0107] In the above implementation process, the scheme of combining the EfficientNet-B4 classification network and the YOLOv9 detection network can achieve accurate classification of the filling state and accurate detection of bladder lesions.

[0108] Based on the above embodiments, the classification network model and detection network model are obtained by training a large amount of data in advance. During the training process, in order to ensure that the classification network model can learn the general image feature extraction capability, the training of the EfficientNet-B4 classification network can be carried out by freezing the shallow feature extractor and fine-tuning the network parameters of the deep network.

[0109] In this context, shallow feature extractors refer to the first few layers of the EfficientNet-B4 classification network, such as the first convolutional layer and the first-stage MBConv block, which are mainly responsible for extracting basic image features, such as edges and textures. Deep networks refer to the later layers of the EfficientNet-B4 classification network, which are mainly responsible for extracting high-level image features, such as semantic information. Deep networks may include subsequent stages such as MBConv blocks, global average pooling layers, and fully connected layers.

[0110] Furthermore, when training the EfficientNet-B4 classification network, optimizations such as multi-scale feature enhancement and anatomical prior guidance can be introduced. For example, a dual-path attention mechanism can be embedded in the MBConv block of the EfficientNet-B4 classification network (such as the first MBConv block in stage 5-7 of EfficientNet-B4). One path is the spatial path, which uses dilated convolution to capture large-scale morphological features, and the other is the detail path, which uses 1*1 convolution to extract local texture. Finally, the dual-path features are fused through learnable parameters. When the bladder changes from emptying (wall thickness > 3 mm) to overfilling (wall thickness < 1 mm), the wall folds change from complex to smooth. Dilated convolution expands the receptive field to 10*10 mm. 2 It can fully capture the extensible deformation in the later stages of filling, while 1*1 convolution focuses on 3*3mm. 2 Micro-folds remaining in the area.

[0111] Alternatively, an anatomical constraint module can be inserted after the first downsampling layer of the EfficientNet-B4 classification network. This module is a differentiable preprocessing layer used to encode medical prior knowledge into the feature space. Since bladder wall thickness was collected during data acquisition, the bladder wall thickness distribution map can be converted into a feature mask using the differentiable rendering technique of this module, and then spatially weighted and fused with the convolutional feature map after the first downsampling.

[0112] During training, a large number of images can be pre-collected. For example, during data acquisition, a high-frequency ultrasound probe can be used to obtain dynamic image sequences of the transverse and longitudinal sections of the bladder. Each subject can set several time monitoring points for image acquisition, such as the time after urination (empty state), the natural filling period (gradual detection every 50ml), and the volume saturation period (>=400ml, overfilling). Furthermore, a water load test can be used to control the filling efficiency (500ml / 20 minutes uniform intake) and continuous volume monitoring can be performed simultaneously to obtain images that record the entire cycle from emptying to filling. These images are then input into the classification network model and the detection network model for training.

[0113] The classification network model and the detection network model can be trained separately and independently, or they can be trained together; there are no specific restrictions.

[0114] After collecting a large number of bladder ultrasound images, to expand the training sample, operations such as rotation, flipping, and noise injection can be used to preprocess the images. Furthermore, affine transformation simulation techniques can be introduced. By designing adjustable scaling matrices (scaling factors Δs∈[-0.3,0.3]) and displacement parameters (Δx,Δy∈[-0.1W,0.1H]), the images can be preprocessed to reproduce the dynamic process of bladder expansion and contraction in image space. Simultaneously, a non-rigid deformation field can be generated using a thin-plate spline algorithm to simulate the local stretching effect of the bladder wall during overfilling and the mucosal wrinkling characteristics during emptying, enabling the model to learn invariant features in continuous changes in organ morphology.

[0115] In the above implementation process, during the training of the EfficientNet-B4 classification network, the parameters of the shallow feature extractors can be frozen, i.e., the weights of these network layers are not updated. This preserves the pre-trained model's ability to extract basic features and avoids weakening the generalization ability of these basic features due to overtraining. By fine-tuning the parameters of the deep network, the high-level features of the network can be better adapted to the specific requirements of medical images, improving the classification ability of bladder fullness status.

[0116] Based on the above embodiments, in order to enable the detection network model to adapt to the detection of lesions of different sizes, the anchor size of the YOLOv9 detection network corresponding to different filling states can be set differently. For example, the anchor size of the YOLOv9 detection network corresponding to the standard filling state is the standard size, the anchor size of the YOLOv9 detection network corresponding to the non-filling state is reduced by a set ratio compared to the standard size, and the anchor size of the YOLOv9 detection network corresponding to the overfilling state is enlarged by a set ratio compared to the standard size.

[0117] In the YOLOv9 detection network, Anchor refers to an anchor box, which is a predefined set of bounding boxes used to represent the possible positions and dimensions of target objects of different sizes and shapes in an image. Each Anchor box corresponds to a predicted bounding box, and the model can predict the bounding box of the actual target object by adjusting the size and position of these Anchor boxes.

[0118] Here, the standard size can be understood as the original anchor size of the YOLOv9 detection network. In the unfilled state, the standard size is reduced by a set ratio, such as by 50%, to obtain the anchor size of the YOLOv9 detection network in the unfilled state. In the overfilled state, the standard size is enlarged by a set ratio, such as by 50%, to obtain the anchor size of the YOLOv9 detection network in the overfilled state. Of course, this set ratio can be flexibly adjusted according to actual needs.

[0119] When detecting small lesions, these lesions are relatively small compared to the overall image, while the standard size of traditional anchors may be large. This can make it difficult for the model to accurately match and locate small lesions, leading to missed or false detections. In a dedisplaced bladder state, lesions may appear relatively small due to the more contracted bladder shape. Reducing the anchor size of YOLOv9 by 50% allows for more precise prediction of the boundaries of small lesions. Therefore, for detection network models in a dedisplaced state, reducing the anchor size can accurately capture the features and location of these small lesions because the reduced anchor box better fits the size range of the small lesions, better fitting these tiny targets on the feature map, thus improving the sensitivity of small lesion detection.

[0120] Conversely, when detecting large lesions, these lesions are relatively large compared to the overall effect, while the standard size of traditional anchors may be relatively small. This can make it difficult for the model to accurately match and locate large lesions. In a state of bladder overdistension, lesions may appear relatively large due to the expanded bladder shape. Increasing the anchor size of YOLOv9 by 50% can better fit the size of the lesions, thereby improving detection accuracy.

[0121] During the training of the YOLOv9 detection network, since a separate YOLOv9 detection network was set up for each of the three filling states, in order to reduce the training load, these three YOLOv9 detection networks can achieve knowledge transfer by sharing the first 10 layers of the backbone network. At the same time, the auxiliary task of filling state prediction can be incorporated into the loss function, that is, the YOLOv9 detection network can also identify the filling state, which can enhance the model's sensitivity to morphological features.

[0122] The following section uses a specific example to illustrate the detection method of this scheme, as shown in Figure 2.

[0123] The electronic device first acquires multiple frames of bladder ultrasound images. These images are then standardized for resolution adjustment. The adjusted images are used to identify the filling state through a classification network model. Initially, in units of 5 frames, it checks if the filling state is the same for 5 consecutive frames. If so, it enters a high-confidence mode, where the corresponding detection network model is directly determined based on the current filling state, and lesion detection is then performed. If not, the detection is expanded to 15 frames. It checks if more than 10 of these 15 frames have a filling state confidence level below 0.85. If not, the filling states of these 15 frames are voted on, and then it enters a high-confidence mode, where the corresponding detection network model is determined based on the current filling state, and lesion detection is then performed. If so, it enters a transitional mode, where multiple detection network models are used for mixed detection. The final detection result is obtained through weighted summation.

[0124] For example, in transitional mode, if the filling states corresponding to 15 frames of images are determined to include the three filling states mentioned above, then the detection network model also includes the three models mentioned above: detection network model 1 for the non-filling state, detection network model 2 for the standard filling state, and detection network model 3 for the overfilling state. During detection, a frame of bladder ultrasound image can be input into these three detection network models respectively to obtain three detection results. Then, these three detection results can be weighted and summed. The weights can be determined based on the confidence levels of the three filling states. For example, the classification detection network model can predict the confidence level of the three filling states corresponding to each frame of image. Then, the confidence levels of the three filling states in 15 frames of images can be averaged to obtain the final confidence levels of the three filling states. For example, the final non-filling state: standard filling state: overfilling state = 0.7:0.2:0.1, and the weights are the confidence levels.

[0125] For example, if detection network model 1 outputs detection result 1 with a weight of 0.7, detection network model 2 outputs detection result 2 with a weight of 0.2, and detection network model 3 outputs detection result 3 with a weight of 0.1, then the final detection result is calculated as: Detection result 1 * 0.7 + Detection result 2 * 0.2 + Detection result 3 * 0.1. Each detection result can include the location, size, shape, and classification of the lesion. During weighting, the detection data corresponding to overlapping lesions can be summed using weighted summation. The classification in the detection results can be represented using classification probabilities, facilitating weighting. This method can be used to integrate detection results when using hybrid models, and the final output can be displayed to the user.

[0126] Please refer to Figure 3, which is a structural block diagram of a bladder lesion detection device provided in an embodiment of this application. This device can be a module, program segment, or code on an electronic device. It should be understood that this device corresponds to the method embodiment of Figure 1 above and is capable of performing the various steps involved in the method embodiment of Figure 1. The specific functions of this device can be found in the description above. To avoid repetition, detailed descriptions are appropriately omitted here.

[0127] Optionally, the device 200 includes:

[0128] Image acquisition module 210 is used to acquire multiple frames of bladder ultrasound images;

[0129] The state detection module 220 is used to identify the target filling state of the bladder in each frame of bladder ultrasound image;

[0130] The lesion detection module 230 is used to determine the corresponding detection network model according to the target filling state, and to detect bladder lesions through the determined detection network model to obtain detection results. Each filling state corresponds to a detection network model.

[0131] Optionally, the lesion detection module 230 is used to determine the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images; and to determine the corresponding detection network model based on the current filling state.

[0132] Optionally, the lesion detection module 230 is used to determine the current filling state as the target filling state if the target filling state of the bladder is the same in a first set number of consecutive frames of bladder ultrasound images.

[0133] Optionally, the lesion detection module 230 is used to determine the current filling state as the target filling state with the highest number of different target filling states in a series of consecutive frames of bladder ultrasound images if the target filling states of the bladder are different in a second set number of consecutive frames.

[0134] Optionally, the lesion detection module 230 is used to determine that the current filling state includes various different target filling states if the target filling states of the bladder are different in a third set number of consecutive frames of bladder ultrasound images.

[0135] Optionally, the state detection module 220 is used to obtain the confidence level of various filling states of the bladder in each frame of bladder ultrasound image identified by the classification network model, wherein the filling state includes non-filling state, standard filling state and overfilling state; and to determine the filling state with the highest confidence level as the target filling state.

[0136] Optionally, the lesion detection module 230 is used to determine the current filling state of the bladder based on the target filling state of the bladder and the corresponding confidence level in multiple frames of bladder ultrasound images; and to determine the corresponding detection network model based on the current filling state.

[0137] Optionally, the state detection module 220 is used to identify the target filling state of the bladder in each frame of bladder ultrasound image through a classification network model, wherein the filling state includes an unfilled state, a standard filling state, and an overfilled state.

[0138] Wherein, the classification network model is the EfficientNet-B4 classification network, and / or, the detection network model is the YOLOv9 detection network.

[0139] Optionally, during the training of the EfficientNet-B4 classification network, the shallow feature extractor is frozen and the network parameters of the deep network are fine-tuned.

[0140] Optionally, the anchor size of the YOLOv9 detection network corresponding to the standard filled state is the standard size, the anchor size of the YOLOv9 detection network corresponding to the unfilled state is reduced by a set ratio compared to the standard size, and the anchor size of the YOLOv9 detection network corresponding to the overfilled state is enlarged by a set ratio compared to the standard size.

[0141] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0142] Please refer to Figure 4, which is a schematic diagram of an electronic device for performing a bladder lesion detection method according to an embodiment of this application. The electronic device may include: at least one processor 310, such as a CPU, at least one communication interface 320, at least one memory 330, and at least one communication bus 340. The communication bus 340 is used to establish communication between these components. In this embodiment, the communication interface 320 is used for signaling or data communication with other node devices. The memory 330 may be a high-speed RAM or a non-volatile memory, such as at least one disk storage device. Optionally, the memory 330 may also be at least one storage device located remotely from the aforementioned processor. The memory 330 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 310, the electronic device performs the method process shown in Figure 1.

[0143] It is understood that the structure shown in Figure 4 is merely illustrative, and the electronic device may include more or fewer components than shown in Figure 4, or have a different configuration than shown in Figure 4. The components shown in Figure 4 may be implemented using hardware, software, or a combination thereof.

[0144] This application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it performs the method process executed by the electronic device in the method embodiment shown in FIG1.

[0145] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as including:

[0146] Acquire multiple frames of bladder ultrasound images;

[0147] Identify the target bladder filling status in each frame of bladder ultrasound image;

[0148] The corresponding detection network model is determined based on the target filling state, and bladder lesions are detected through the determined detection network model to obtain the detection results. Each filling state corresponds to a detection network model.

[0149] In summary, the embodiments of this application provide a method, apparatus, device, storage medium, and program product for bladder lesion detection. By identifying the bladder filling state in each frame of bladder ultrasound image, important prior information is provided for subsequent lesion detection. A detection network model is used to detect bladders in different filling states. The detection network model is dynamically adjusted based on the identified filling state, enabling the detection network model to focus more on the precise detection of lesions and adapt to the changes in image characteristics of the bladder in different filling states. This can effectively improve the accuracy of lesion detection and reduce the probability of false detection or missed detection.

[0150] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0151] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0153] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0154] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting bladder lesions, characterized in that, The method includes: acquiring multiple frames of bladder ultrasound images; identifying the target filling state of the bladder in each frame of bladder ultrasound images; determining the corresponding detection network model based on the target filling state, and detecting bladder lesions through the determined detection network model to obtain detection results, wherein each filling state corresponds to a detection network model; wherein identifying the target filling state of the bladder in each frame of bladder ultrasound images includes: identifying the target filling state of the bladder in each frame of bladder ultrasound images through a classification network model, wherein the filling state includes an unfilled state, a standard filling state, and an overfilled state; wherein the classification network model is an EfficientNet-B4 classification network, and / or, the detection network model is a YOLOv9 detection network; the Anc of the YOLOv9 detection network corresponding to the standard filling state. The anchor size is the standard size. The anchor size of the YOLOv9 detection network corresponding to the unfilled state is reduced by a set ratio compared to the standard size, and the anchor size of the YOLOv9 detection network corresponding to the overfilled state is increased by a set ratio compared to the standard size. The MBConv block in the EfficientNet-B4 classification network embeds a dual-path attention mechanism, which includes a spatial path and a detail path. The spatial path is implemented using dilated convolution, and the detail path is implemented using 1*1 convolution. An anatomical constraint module is set after the first downsampling layer in the EfficientNet-B4 classification network. This anatomical constraint module is implemented through a differentiable preprocessing layer and is used to encode medical prior knowledge into the feature space.

2. The method according to claim 1, characterized in that, The step of determining the corresponding detection network model based on the target filling state includes: determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images; and determining the corresponding detection network model based on the current filling state.

3. The method according to claim 2, characterized in that, The step of determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images includes: if the target filling state of the bladder is the same in a first set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to be the target filling state.

4. The method according to claim 2, characterized in that, The step of determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images includes: if the target filling state of the bladder is different in a second set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to be the target filling state with the most occurrences.

5. The method according to claim 2, characterized in that, The step of determining the current filling state of the bladder based on the target filling state of the bladder in multiple frames of bladder ultrasound images includes: if the target filling state of the bladder is different in a third set number of consecutive frames of bladder ultrasound images, then the current filling state is determined to include various different target filling states.

6. The method according to claim 1, characterized in that, The identification of the target bladder filling state in each frame of bladder ultrasound image includes: obtaining the confidence level of various filling states of the bladder in each frame of bladder ultrasound image identified by a classification network model, wherein the filling state includes an unfilled state, a standard filling state, and an overfilled state; and determining the filling state with the highest confidence level as the target filling state.

7. The method according to claim 6, characterized in that, The step of determining the corresponding detection network model based on the target filling state includes: determining the current filling state based on the target filling state of the bladder and the corresponding confidence level in multiple frames of bladder ultrasound images; and determining the corresponding detection network model based on the current filling state.

8. The method according to claim 1, characterized in that, During the training of the EfficientNet-B4 classification network, the shallow feature extractor is frozen and the network parameters of the deep network are fine-tuned.

9. A device for detecting bladder lesions, characterized in that, The device includes: an image acquisition module for acquiring multiple frames of bladder ultrasound images; a state detection module for identifying the target filling state of the bladder in each frame of the bladder ultrasound image; and a lesion detection module for determining a corresponding detection network model based on the target filling state, and detecting bladder lesions using the determined detection network model to obtain detection results, wherein each filling state corresponds to a detection network model; specifically, the state detection module is used to identify the target filling state of the bladder in each frame of the bladder ultrasound image using a classification network model, wherein the filling state includes an unfilled state, a standard filling state, and an overfilled state; wherein the classification network model is an EfficientNet-B4 classification network, and / or, the detection network model is a YOLOv9 detection network; the YOLOv9 detection network corresponding to the standard filling state... The anchor size of the testing network is a standard size. The anchor size of the YOLOv9 detection network corresponding to the unfilled state is reduced by a set ratio compared to the standard size, and the anchor size of the YOLOv9 detection network corresponding to the overfilled state is enlarged by a set ratio compared to the standard size. The MBConv block in the EfficientNet-B4 classification network embeds a dual-path attention mechanism, which includes a spatial path and a detail path. The spatial path is implemented using dilated convolution, and the detail path is implemented using 1*1 convolution. An anatomical constraint module is set after the first downsampling layer in the EfficientNet-B4 classification network. This anatomical constraint module is implemented through a differentiable preprocessing layer and is used to encode medical prior knowledge into the feature space.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-8.

Images