Bladder focus detection method, device, equipment, storage medium and program product

Abstract

The invention provides a bladder focus detection method, device and equipment, a storage medium and a program product, and relates to the technical field of computers. According to the method, important prior information is provided for subsequent focus detection by identifying the filling state of the bladder in each frame of bladder ultrasonic image, the bladder in different filling states is detected through a detection network model, the detection network model is dynamically adjusted through the identified filling state, and the detection accuracy is improved. Therefore, the detection network model can focus on the fine detection of the focus, adapts to the image feature change of the bladder in different filling states, can effectively improve the accuracy of focus detection, and reduces the probability of false detection or missing detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly, to a method, device, equipment, storage medium and program product for detecting bladder lesions. Background Art

[0002] With the development of electronic imaging, medical imaging and artificial intelligence technologies, by means of media (such as X-rays, electromagnetic fields, ultrasonic waves, endoscopes, etc.), the internal tissues and organs of the human body are presented in the form of images, thus facilitating the evaluation of the human health status has become an important means for physicians to diagnose and treat diseases. Therefore, in order to reduce the burden on physicians and the difficulty of disease diagnosis, and improve the detection rate of lesions, it is of great significance to automatically detect lesions in images and display the detection results in real time to prompt physicians.

[0003] Currently, the methods for detecting lesions in bladder ultrasound images mainly rely on traditional image processing techniques (such as edge detection, threshold segmentation, etc.). These methods are easily affected by noise interference in complex backgrounds, resulting in false detection or missed detection. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a method, device, equipment, storage medium and program product for detecting bladder lesions, so as to improve the problem of easy false detection or missed detection in the existing detection methods.

[0005] In a first aspect, the embodiments of this application provide a method for detecting bladder lesions, the method includes:

[0006] Obtain multiple frames of bladder ultrasound images;

[0007] Identify the target filling state of the bladder in each frame of bladder ultrasound image;

[0008] Determine the corresponding detection network model according to the target filling state, and detect bladder lesions through the determined detection network model to obtain a detection result, where each filling state corresponds to a detection network model.

[0009] In the above implementation process, by identifying the filling state of the bladder in each frame of bladder ultrasound image, important prior information is provided for subsequent lesion detection. By using the detection network model to detect the bladder in different filling states, and dynamically adjusting the detection network model according to the identified filling state, the detection network model can focus more on the fine detection of lesions, adapt to the changes in the image features of the bladder in different filling states, effectively improve the accuracy of lesion detection, and reduce the probability of false detection or missed detection.

[0010] Optionally, the determining the corresponding detection network model according to the target filling state includes:

[0011] Determine the current filling state according to the target filling state of the bladder in multiple-frame bladder ultrasound images;

[0012] Determine the corresponding detection network model according to the current filling state.

[0013] In the above implementation process, by combining multiple frames of images to determine the current filling state, the problem that the classification result is inaccurate due to the influence of factors such as noise on a single frame of image can be reduced.

[0014] Optionally, the determining the current filling state according to the target filling state of the bladder in multiple-frame bladder ultrasound images includes:

[0015] If the target filling states of the bladder in the first set number of consecutive frames of bladder ultrasound images are the same, determine the current filling state as the target filling state.

[0016] In the above implementation process, by identifying whether there is a set number of the same target filling states, if so, it is determined that the filling state is stable, which can improve the accuracy and stability of the recognition.

[0017] Optionally, the determining the current filling state according to the target filling state of the bladder in multiple-frame bladder ultrasound images includes:

[0018] If the target filling states of the bladder in the second set number of consecutive frames of bladder ultrasound images are different, determine the current filling state as the target filling state with the largest quantity.

[0019] In the above implementation process, through the analysis of multiple consecutive frames of images, false detection and missed detection caused by misjudgment of a single frame of image can be reduced.

[0020] Optionally, the determining the current filling state according to the target filling state of the bladder in multiple-frame bladder ultrasound images includes:

[0021] If the target filling states of the bladder in the third set number of consecutive frames of bladder ultrasound images are different, determine that the current filling state includes various different target filling states.

[0022] In the above implementation process, if most of the target filling states are different, it indicates that the filling state is changing. At this time, the detection network models corresponding to multiple filling states can be used for detection, which can improve the accuracy and stability of the detection result.

[0023] Optionally, the identifying the target filling state of the bladder in each frame of bladder ultrasound image includes:

[0024] Obtain the confidence levels of various filling states of the bladder in each frame of bladder ultrasound image recognized by the classification network model, where the filling states include non-filled state, standard filling state, and over-filled state;

[0025] Determine the filling state with the highest confidence as the target filling state.

[0026] In the above implementation process, determining the target filling state through the confidence level can determine the filling state of the bladder in each frame of image, and thus facilitate the accurate selection of the detection network model.

[0027] Optionally, the determining the corresponding detection network model according to the target filling state includes:

[0028] Determine the current filling state according to the target filling state and the corresponding confidence level of the bladder in multiple frames of bladder ultrasound images;

[0029] Determine the corresponding detection network model according to the current filling state.

[0030] In the above implementation process, determining the current filling state in combination with the confidence level can improve the accuracy and stability of recognition.

[0031] Optionally, the recognizing the target filling state of the bladder in each frame of bladder ultrasound image includes:

[0032] Recognize the target filling state of the bladder in each frame of bladder ultrasound image through the classification network model, where the filling states include non-filled state, standard filling state, and over-filled state;

[0033] Among them, the classification network model is the EfficientNet-B4 classification network, and / or the detection network model is the YOLOv9 detection network. The scheme of combining the EfficientNet-B4 classification network and the YOLOv9 detection network can achieve accurate classification of the filling state and accurate detection of bladder lesions.

[0034] Optionally, during the training process of the EfficientNet-B4 classification network, it is trained in a way of freezing the shallow feature extractor and fine-tuning the network parameters of the deep network. During the training process of the EfficientNet-B4 classification network, the parameters of the shallow feature extractor can be frozen, that is, the weights of these network layers are not updated, which can retain the ability of the pre-trained model to extract basic features and avoid the weakening of the generalization ability of these basic features due to over-training. By fine-tuning the parameters of the deep network, the high-level features of the network can better adapt to the specific requirements of medical images and improve the classification ability of the bladder filling state.

[0035] Optionally, the Anchor size of the YOLOv9 detection network corresponding to the standard filling state is the standard size, the Anchor size of the YOLOv9 detection network corresponding to the non-filling state is reduced by a set proportion compared to the standard size, and the Anchor size of the YOLOv9 detection network corresponding to the overfilling state is enlarged by a set proportion compared to the standard size.

[0036] In the above implementation process, in the non-filling state of the bladder, the lesion may be relatively small due to the relatively contracted shape of the bladder. Reducing the Anchor size of YOLOv9 by 50% can more precisely predict the boundary of the small lesion. In the overfilling state of the bladder, the lesion may be relatively large due to the relatively dilated shape of the bladder. Enlarging the Anchor size of YOLOv9 by 50% can better fit the size of the lesion, thereby improving the accuracy of detection.

[0037] In a second aspect, an embodiment of the present application provides a bladder lesion detection device, and the device includes:

[0038] An image acquisition module, configured to acquire multiple frames of bladder ultrasound images;

[0039] A state detection module, configured to identify the target filling state of the bladder in each frame of bladder ultrasound image;

[0040] A lesion detection module, configured to determine a corresponding detection network model according to the target filling state, and detect bladder lesions through the determined detection network model to obtain a detection result, where each filling state corresponds to a detection network model.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect as described above are run.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method provided in the first aspect as described above are run.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer program instructions, and when the computer program instructions are read and run by a processor, the steps in the method provided in the first aspect as described above are executed.

[0044] Other features and advantages of the present application will be described in the subsequent specification. Moreover, some of them will become apparent from the specification or can be understood by implementing the embodiments of the present application. The objectives and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the written specification, claims, and drawings. Brief Description of the Drawings

[0045] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 Flowchart of a method for detecting bladder lesions provided by an embodiment of the present application;

[0047] Figure 2 Specific example flowchart of a method for detecting bladder lesions provided by an embodiment of the present application;

[0048] Figure 3 Block diagram of a device for detecting bladder lesions provided by an embodiment of the present application;

[0049] Figure 4 Schematic diagram of the structure of an electronic device for executing the method for detecting bladder lesions provided by an embodiment of the present application. Detailed Description of the Embodiments

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application.

[0051] It should be noted that the terms "system" and "network" in the embodiments of the present invention can be used interchangeably. "Multiple" means two or more. In view of this, "multiple" can also be understood as "at least two" in the embodiments of the present invention. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " generally represents an "or" relationship between the associated objects before and after without special instructions.

[0052] It should also be noted that all actions of obtaining signals, information, or data in the present application are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and with the authorization given by the owner of the corresponding device.

[0053] An embodiment of the present application provides a method for detecting bladder lesions. By identifying the filling state of the bladder in each frame of bladder ultrasound images, important prior information is provided for subsequent lesion detection. A detection network model is used to detect the bladder in different filling states, and the detection network model is dynamically adjusted according to the identified filling state, enabling the detection network model to focus more on the fine detection of lesions, adapt to the changes in the imaging characteristics of the bladder in different filling states, effectively improve the accuracy of lesion detection, and reduce the probability of false detection or missed detection.

[0054] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for detecting bladder lesions provided by an embodiment of the present application. The method includes the following steps:

[0055] Step S110: Obtain multiple frames of bladder ultrasound images.

[0056] The multiple frames of bladder ultrasound images can be video frame images obtained by an ultrasound device. For example, the method of the present application is executed by an electronic device (the electronic device can be a device such as a terminal or a server). The electronic device can receive the video frame images collected by the ultrasound device. When the ultrasound device performs image collection, it collects a video stream, such as a video stream at 30 frames per second. The multiple frames of bladder ultrasound images here are video frame images, which can refer to the video frame images obtained per second, that is, the present solution can perform lesion detection on the video frame images obtained per second. Of course, it is also possible to obtain the video frame images within a period of time for lesion detection.

[0057] In some embodiments, the resolution of the original video frame images collected by the ultrasound device may be relatively large. To reduce the computational load, after receiving the original bladder ultrasound images, the electronic device can first perform adaptive resolution adjustment. For example, when the resolution of the original bladder ultrasound images exceeds a set value (such as 720p, and the specific value can be set according to actual needs), the original bladder ultrasound images can be resized. For example, the size of the original bladder ultrasound images can be reduced to 640*640 using an edge-preserving downsampling algorithm, and then the processed bladder ultrasound images can be subjected to subsequent detection. Of course, the specific reduced size can be flexibly set according to actual needs, which can not only retain the key details of the images but also reduce the computational load.

[0058] Among them, the edge-preserving downsampling algorithm is an algorithm that can retain the edge information of the image as much as possible while reducing the image resolution. The edge information can provide more key features for subsequent bladder lesion detection, thereby improving the accuracy of subsequent detection results. Reducing the image resolution can reduce the amount of calculation, and by using the edge-preserving downsampling algorithm to reduce the image resolution, the key details and edge information in the image can be retained at the same time.

[0059] Step S120: Identify the target filling state of the bladder in each frame of the bladder ultrasound image.

[0060] Among them, the filling state can include the non-filled state, the standard filling state, and the over-filled state. It can be understood that the target filling state can refer to one of these three filling states. Specifically, when identifying the target filling state of the bladder in the ultrasound image here, what can be obtained is the category of the filling state, that is, the above three filling states, or it can refer to the value of the filling state, that is, the values of various filling states, which refers to the confidence level in the subsequent embodiments. The identification method can be implemented by using corresponding image processing algorithms or machine learning models, etc.

[0061] In some embodiments, the filling state is determined according to the morphology of the bladder. The morphology of the bladder can be determined in combination with the bladder wall thickness, volume, and morphological characteristics. The calculation formula for the volume is V = 0.5 * major axis * minor axis * anteroposterior diameter. When classifying, when the wrinkled thickness of the bladder wall exceeds 3 mm and the volume is less than 100 ml, it is determined as the non-filled state; when the volume is between 100 - 400 ml and the wrinkled thickness of the bladder wall is 1 - 3 mm, it is determined as the standard filling state; when the volume is above 400 ml and the bladder wall is stretched and thinned to less than 1 mm, it is determined as the over-filled state.

[0062] It can be understood that in practical applications, the filling state can also be more finely differentiated according to actual needs, not limited to the three filling states provided in this solution.

[0063] In some embodiments, the target filling state of the bladder in each frame of the bladder ultrasound image can be identified through a classification network model. For example, if the input is video frame images, which are some temporally continuous images, these multiple frames of bladder ultrasound images can be input into the classification network model in the received order, and the classification network model can then sequentially identify the filling state of the bladder in each input image. The role of the classification network model is to extract key features from the input bladder ultrasound image, and classification can be performed based on the key features, that is, classified as the non-filled state, the standard filling state, or the over-filled state.

[0064] Step S130: Determine the corresponding detection network model according to the target filling state, and detect the bladder lesion through the determined detection network model to obtain the detection result.

[0065] After determining the target filling state of the bladder in each frame of the bladder ultrasound image, the corresponding detection network model can be determined according to the target filling state. Among them, each filling state corresponds to a detection network model. For example, the non-filling state corresponds to detection network model 1, the standard filling state corresponds to detection network model 2, and the overfilling state corresponds to detection network model 3. Here, each filling state corresponding to a detection network model can be understood as each filling state having a separate detection network model, or it can be understood that multiple filling states correspond to a basic detection network model. However, each filling state corresponds to a set of network parameters, and the network parameters of the basic detection network model can be adjusted according to the filling state. For example, when the target filling state is determined to be the non-filling state, the parameters of the basic detection network model can be adjusted to the network parameters corresponding to the non-filling state.

[0066] In some embodiments, after determining the target filling state of the bladder in each frame of the bladder ultrasound image, the frame of the bladder ultrasound image can be input into the corresponding detection network model for detection. For example, if the target filling state of the bladder in a certain frame of the bladder ultrasound image is determined to be the non-filling state, then this frame of the image can be continuously input into detection network model 1, and detection network model 1 can be used to detect the bladder lesions in this frame of the image and output the detection results. The detection results can include information such as the size, location, shape, and lesion classification of the bladder lesions. The lesion classification can be divided into classifications such as benign and malignant.

[0067] The detection network model can detect the bladder lesions by extracting the image features in the bladder ultrasound image and based on the image features, so as to obtain the detection results. The classification network model can identify the filling state of the bladder in each frame of the bladder ultrasound image. After determining the corresponding detection network model, the bladder lesions in each frame of the bladder ultrasound image can be detected through the corresponding detection network model, and the detection results can be obtained.

[0068] For the convenience of the user to view, the detection results can be output for display, so that the user can directly view the detection results of the bladder lesions in each frame of the bladder ultrasound image.

[0069] In the above implementation process, the classification network model is used to identify the filling state of the bladder in each frame of the bladder ultrasound image, providing important prior information for subsequent lesion detection. The detection network model is used to detect the bladder in different filling states, and the detection network model is dynamically adjusted according to the identified filling state, enabling the detection network model to focus more on the fine detection of the lesions, adapting to the changes in the imaging characteristics of the bladder in different filling states, effectively improving the accuracy of lesion detection, and reducing the probability of false detection or missed detection.

[0070] Based on the above embodiments, when the classification network model identifies the filling state of the bladder in each frame of bladder ultrasound image, a single-frame image may be affected by noise, occlusion, or other interference factors, resulting in inaccurate classification results. Therefore, the current filling state can be determined according to the target filling state of the bladder in multiple frames of bladder ultrasound images, and then the corresponding detection network model can be determined according to the current filling state, which can reduce the influence of these accidental factors and improve the stability of classification.

[0071] The multiple frames of bladder ultrasound images here can refer to the multiple frames of bladder ultrasound images obtained in the above step S110, or some of the bladder ultrasound images. In this implementation manner, the current filling state can be determined by combining the target filling state of the bladder in multiple frames of bladder ultrasound images.

[0072] In some embodiments, an image buffer can be set in the electronic device to store multiple frames of received bladder ultrasound images. For example, taking 15 frames as an example, the electronic device can perform image recognition and detection with a window of 15 frames. The electronic device can store 15 frames of bladder ultrasound images in the received multiple frames of bladder ultrasound images in chronological order in the image buffer, and detect the filling state of the bladder in each frame of bladder ultrasound image in chronological order. Then, a set number of bladder ultrasound images can be used as a unit for recognition. For example, 5 frames of images can be used as a unit for recognition. The classification network model can first identify the target filling state of the bladder in 5 frames of bladder ultrasound images. In this case, 5 target filling states can be obtained, and then the current filling state can be determined according to these 5 target filling states. For example, if the determined current filling state is the non-filled state, the detection network model is determined as the detection network model 1 corresponding to the non-filled state; if the determined current filling state is the standard filled state, the detection network model is determined as the detection network model 2 corresponding to the standard filled state.

[0073] In the above implementation process, by combining multiple frames of images to determine the current filling state, the problem that the classification result is inaccurate due to the influence of factors such as noise on a single-frame image can be reduced.

[0074] The following will introduce in detail several methods for determining the current filling state by combining the target filling state of the bladder in multiple frames of bladder ultrasound images.

[0075] (1) If the target filling states of the bladder in a continuous frame of bladder ultrasound images reaching the first set number are the same, then determine the current filling state as the target filling state.

[0076] For example, if the first set quantity is 5 frames (the specific value can be flexibly set according to actual requirements), the target filling state of the bladder in 5 consecutive frames of bladder ultrasound images can be detected first. If the target filling states of the bladder in these 5 frames of bladder ultrasound images are the same, such as all in the unfilled state, the system can determine that the current filling state is the unfilled state at this time, and then determine the detection network model 1 corresponding to the unfilled state. In this case, these 5 frames of bladder ultrasound images can be respectively input into the detection network model 1 for detection, so that the detection results of the bladder lesions in these 5 frames of bladder ultrasound images can be obtained.

[0077] In the above implementation process, by identifying whether there are the same target filling states reaching the set quantity, if so, it is determined that the filling state is stable, so that the accuracy and stability of the identification can be improved.

[0078] (2) If the target filling states of the bladder in the consecutive frames of bladder ultrasound images reaching the second set quantity are different, then determine the current filling state as the target filling state with the largest quantity.

[0079] The second set quantity here can be the same as or different from the above first set quantity. For example, when the second set quantity is also 5 frames, it is judged whether there are differences in the target filling states of the bladder in 5 consecutive frames of bladder ultrasound images. If there are differences, the target filling state with the largest quantity can be determined from these 5 target filling states. For example, among these 5 target filling states, 4 are in the unfilled state and 1 is in the standard filled state. At this time, it is determined that the current filling state is the unfilled state, and then the detection network model corresponding to the unfilled state can be determined, that is, the detection network model 1.

[0080] Among them, after the classification network model identifies the target filling state of the bladder in each frame of bladder ultrasound image, the classification result can be recorded in the image buffer. When the electronic device determines whether the target filling states of the bladder in the consecutive frames of bladder ultrasound images reaching the set quantity are the same or different, it can determine the current filling state, and then determine the detection network model corresponding to the current filling state.

[0081] Continuing with the above example, if the target filling states of the bladder in the previous 5 consecutive frames (e.g., frames 1 - 5) of the bladder ultrasound images are the same, after inputting these 5 frames of bladder ultrasound images into the detection network model 1 for detection, the classification network model can continue to detect the target filling states of the bladder in the subsequent 5 frames (i.e., frames 6 - 10) of the bladder ultrasound images, and then can continue to lock the filling state and determine the corresponding detection network model according to the above logic. For example, if 3 of the target filling states of the bladder in the latter 5 frames of bladder ultrasound images are in the standard filling state and 2 are in the overfilled state, the filling state can be locked as the standard filling state. At this time, the detection network model can be determined as the detection network model 2 corresponding to the standard filling state, and then these 5 frames of bladder ultrasound images can be respectively input into the detection network model 2 for lesion detection to obtain the detection results.

[0082] In some other embodiments, the second set number can be different from the first set number. For example, the electronic device can first determine whether there are the same target filling states in the first set number, such as whether the target filling states of the bladder in 5 consecutive frames of images are the same. If not, the current filling state is not determined at this time, but can continue to judge. For example, if the second set number is 15 frames, the target filling states of the bladder in the subsequent 10 frames of bladder ultrasound images can be continuously detected to obtain 15 target filling states. At this time, the filling state with the largest number can be determined from these 15 target filling states. For example, if the target filling state with the largest number is the standard filling state, the current filling state is determined as the standard filling state. Of course, if there are 7 non - filling states, 7 standard filling states, and 1 overfilled state, the current filling state can be determined to include the non - filling state and the standard filling state. In this case, the determined detection network model includes the network detection models corresponding to the non - filling state and the standard filling state.

[0083] When performing lesion detection, these 15 frames of bladder ultrasound images can be respectively input into the corresponding detection network models for detection. If there are multiple detection network models, each frame of bladder ultrasound image will be input into multiple detection network models for detection, and finally the detection results of multiple detection network models can be integrated.

[0084] In the above implementation process, through the analysis of multiple consecutive frames of images, misdetection and missed detection caused by misjudgment of single - frame images can be reduced.

[0085] (3) If the target filling states of the bladder in the consecutive frames of bladder ultrasound images reach the third set number and are different, it is determined that the current filling state includes various different target filling states.

[0086] The third set quantity may be the same as or different from the second set quantity. Continuing with the above example, taking the third set quantity as 5 frames, if the target filling states of the bladder in 5 consecutive frames of bladder ultrasound images are the same, then determine the current filling state in the above manner (1). Otherwise, the current filling state can be determined in manner (2) or manner (3). If determined in manner (3), assuming that the target filling states of the bladder in 5 frames of bladder ultrasound images include the standard filling state and the overfilling state, then the determined current filling state includes the standard filling state and the overfilling state. Assuming that the target filling states of the bladder in 5 frames of bladder ultrasound images include the non-filling state, the standard filling state, and the overfilling state, then the determined current filling state includes the non-filling state, the standard filling state, and the overfilling state. The subsequent determined detection network model is also the detection network model corresponding to various filling states.

[0087] In the above implementation process, if most of the target filling states are different, it indicates that the filling state is changing. At this time, the detection network models corresponding to multiple filling states can be used for detection, which can improve the accuracy and stability of the detection results.

[0088] Based on the above embodiments, in the manner of identifying the filling state, the classification network model can also output the confidence level of each filling state. Then, the final target filling state can be determined according to the confidence level. That is, the confidence levels of various filling states to which the bladder belongs in each frame of bladder ultrasound image identified by the classification network model can be obtained, and then the filling state with the highest confidence level is determined as the target filling state of the bladder in this frame of bladder ultrasound image.

[0089] Among them, the confidence level can represent the probability of each filling state. For example, the confidence levels of three filling states to which the bladder belongs in a certain frame of bladder ultrasound image are: non-filling state (0.7), standard filling state (0.2), overfilling state (0.1). In this case, the confidence level of the non-filling state is the highest, and at this time, it can be determined that the target filling state of the bladder in this frame of bladder ultrasound image is the non-filling state. In this way, the target filling state of the bladder in each frame of bladder ultrasound image can be determined.

[0090] In the above implementation process, by determining the target filling state through the confidence level, the filling state of the bladder in each frame of image can be determined, which is convenient for accurately selecting the detection network model.

[0091] Based on the above embodiments, in order to further improve the accuracy of classification and the accuracy of determining the detection network model, when determining the current filling state above, the current filling state can be determined according to the target filling state of the bladder in multiple frames of bladder ultrasound images and the corresponding confidence levels, and then the corresponding detection network model can be determined according to the current filling state.

[0092] The method of judgment in combination with confidence is introduced in detail below.

[0093] (4) If the target filling states of the bladder in consecutive frames of bladder ultrasound images reach the fourth set quantity and are the same, and the corresponding confidence levels all exceed the set threshold, then determine the current filling state as this target filling state.

[0094] For example, the target filling states of the bladder in 5 frames of bladder ultrasound images are all the same, such as all being unfilled states, and the confidence level of each unfilled state exceeds the set threshold (such as 0.85, and the specific value can be flexibly set according to actual needs). In this case, it can be determined that the current filling state is the unfilled state. However, if the target filling states of the bladder in 5 frames of bladder ultrasound images are all the same, such as all being unfilled states, but the confidence levels of more than half of the unfilled states reaching the first set quantity exceed the set threshold, for example, the confidence levels of 3 unfilled states exceed the set threshold, then it can also be determined that the current filling state is the unfilled state at this time.

[0095] If the confidence levels of the unfilled states do not reach a certain quantity and exceed the set threshold, for example, only the confidence levels of 2 unfilled states exceed the set threshold. In this case, the confidence levels of the target filling states of the bladder in the subsequent 5 frames of bladder ultrasound images can be continuously identified through the classification network model. At this time, 10 target filling states can be obtained, and then the current filling state can be judged by combining these 10 target filling states and the corresponding confidence levels. For example, if these 10 target filling states are all the same, such as still being the unfilled state, but the confidence levels of more than 5 unfilled states exceed the set threshold, then determine the current filling state as the unfilled state, otherwise continue to detect the target filling states and confidence levels of the bladder in other frames of bladder ultrasound images.

[0096] Of course, if the target filling states of the previous 5 frames or the subsequent 10 frames are different, then the current filling state can be determined according to the subsequent methods (2) or (5) or (6), such as selecting the target filling state with the largest quantity as the current filling state, or selecting the target filling state with the confidence level exceeding the set threshold as the current filling state.

[0097] (5) If the target filling states of the bladder in consecutive frames of bladder ultrasound images reach the fifth set quantity and are different, then count the target filling states with the confidence level exceeding the set threshold, and determine the current filling state as this target filling state.

[0098] For example, the target filling states of the bladder in 5 frames of bladder ultrasound images are different, including, for example, an unfilled state and a standard filled state. In this case, the filling states with a confidence level exceeding a set threshold (such as 0.85) can be counted. For example, if the confidence level of the unfilled state exceeds 0.85 while the confidence level of the standard filled state does not exceed 0.85, then the current filling state is determined to be the unfilled state. If the confidence levels of both the unfilled state and the standard filled state exceed 0.85, then the current filling state is determined to include the unfilled state and the standard filled state.

[0099] (6) If the target filling states of the bladder in the bladder ultrasound images of the sixth set number of consecutive frames are different and the confidence levels of more than a preset number are less than the set threshold, then the current filling state is determined to include various different target filling states.

[0100] For example, if it is recognized that the target filling states of the bladder in 15 frames of bladder ultrasound images are different, then count how many target filling states have a confidence level less than the set threshold (such as 0.85). If there are more than the set number (the set number can be half of the sixth set number or set according to actual requirements, such as 10), in this case, it means that the filling state of the bladder is changing, so the filling state cannot be clearly determined. At this time, various target filling states among these 15 target filling states can be directly used as the current filling state. If the 15 target filling states include the unfilled state and the standard filled state, then the current filling state includes the unfilled state and the standard filled state. If the 15 target filling states include the unfilled state, the standard filled state, and the overfilled state, then the current filling state includes the unfilled state, the standard filled state, and the overfilled state.

[0101] It can be understood that the above-mentioned fourth set number, fifth set number, and sixth set number can be flexibly set according to actual requirements. In a specific application, the electronic device can use 15 frames as a window for filling state recognition and detection. When the filling states are the same, the set number can be set to 5. When the filling states are different, the set number can be set to 15. In this way, the above-mentioned first set number and fourth set number can be 5, and the remaining set numbers can be 15. That is to say, if the filling states are the same, fewer frames of bladder ultrasound images can be used to determine the current filling state. If the filling states are different, more frames of bladder ultrasound images are required to determine the current filling state, thereby reducing the influence of noise and improving the stability and accuracy of recognition.

[0102] In the above implementation process, combining the confidence level to determine the current filling state can improve the accuracy and stability of recognition.

[0103] Based on the above embodiments, when identifying the filling state, if a classification network model is used to identify the target filling state of the bladder in each frame of the bladder ultrasound image, in some embodiments, the classification network model can be an EfficientNet-B4 classification network, and / or, the detection network model can be a YOLOv9 detection network.

[0104] Through its depthwise separable convolution and compound scaling techniques, the EfficientNet-B4 classification network can efficiently extract multi-scale features and has a high classification accuracy.

[0105] The YOLOv9 detection network has a high detection accuracy and also significantly reduces the number of parameters and the amount of computation. In this solution, the classification network model and the detection network model can be deployed on the device side for real-time operation.

[0106] In some embodiments, other models can also be used for the classification network model in this solution, such as the ResNet model, the DenseNet model, or other versions of the EfficientNet system model, etc. Other models can also be used for the detection network model, such as the Faster R-CNN model, the RetinaNet, or other versions of the YOLO system model, etc.

[0107] In the above implementation process, the combination of the EfficientNet-B4 classification network and the YOLOv9 detection network can achieve accurate classification of the filling state and accurate detection of bladder lesions.

[0108] Based on the above embodiments, the above classification network model and detection network model are obtained by being pre-trained with a large amount of data. During the training process, in order to ensure that the classification network model can learn the general image feature extraction ability, during the training of the EfficientNet-B4 classification network, the network parameters can be trained in a way of freezing the shallow feature extractor and fine-tuning the deep network.

[0109] Among them, the shallow feature extractor refers to the first few layers of the EfficientNet-B4 classification network, such as including the convolutional layer of the first layer and the MBConv blocks of the first stage, etc., which are mainly responsible for extracting the basic features of the image, such as edges, textures, etc. The deep network refers to the last few layers of the EfficientNet-B4 classification network, which are mainly responsible for extracting the high-level features of the image, such as semantic information, etc. The deep network can include the MBConv blocks, global average pooling layer, fully connected layer, etc. of the subsequent stages.

[0110] In addition, when training the EfficientNet-B4 classification network, optimizations such as multi-scale feature enhancement and anatomical prior guidance can also be introduced. For example, a dual-path attention mechanism can be embedded in the MBConv block in the EfficientNet-B4 classification network (such as the first MBConv block in stages 5-7 of EfficientNet-B4). One path is the spatial path, which uses dilated convolution to capture large-scale morphological features, and the other is the detail path, which uses 1*1 convolution to extract local textures. Finally, the features of the two paths are fused through learnable parameters. When the bladder changes from emptying (wall thickness > 3 mm) to overfilling (wall thickness < 1 mm), the wall folds change from complex to smooth. The dilated convolution expands the receptive field to 10*10 mm 2 , which can completely capture the ductility deformation in the late filling stage, while the 1*1 convolution focuses on the residual microfolds within the 3*3 mm 2 area.

[0111] Alternatively, an anatomical constraint module can be inserted after the first downsampling layer of the EfficientNet-B4 classification network. This module is a differentiable preprocessing layer used to encode medical prior knowledge into the feature space. Since the bladder wall thickness is collected during data acquisition, the bladder wall thickness distribution map can be converted into a feature mask through the differentiable rendering technology of this module and spatially weighted fused with the convolutional feature map after the first downsampling.

[0112] During the training process, a large number of images can be pre-collected. For example, during data acquisition, a high-frequency ultrasound probe can be used to obtain dynamic image sequences of the transverse and longitudinal sections of the bladder. For each subject, several time monitoring points such as the post-voiding moment (empty state), natural filling period (detected at every 50 ml gradient), and volume saturation period (>=400 ml, overfilling) can be set for image acquisition. And a water load test can be used to control the filling efficiency (500 ml / 20 minutes of uniform intake), and continuous volume monitoring can be carried out simultaneously to obtain full-cycle recording images from emptying to filling. These images are input into the classification network model and the detection network model for training.

[0113] The classification network model and the detection network model can be trained separately and independently. Of course, they can also be trained in combination, without specific restrictions.

[0114] After collecting a large number of bladder ultrasound images, in order to expand the training samples, operations such as rotation, flipping, and noise injection can also be used to preprocess the images. Moreover, an affine transformation simulation technique can be introduced. By designing an adjustable scaling matrix (scaling factor Δs ∈ [-0.3, 0.3]) and displacement parameters (Δx, Δy ∈ [-0.1W, 0.1H]), the images are preprocessed to reproduce the dynamic process of bladder expansion and contraction at the image space level. At the same time, the thin plate spline algorithm can be used to generate a non-rigid deformation field to simulate the local stretching effect of the bladder wall during overfilling and the mucosal fold characteristics in the emptying state, enabling the model to learn the invariant features in the continuous change of the organ's morphology.

[0115] In the above implementation process, during the training of the EfficientNet-B4 classification network, the parameters of the shallow feature extractor can be frozen, that is, the weights of these network layers are not updated. This can retain the pre-trained model's ability to extract basic features and avoid the weakening of the generalization ability of these basic features due to overtraining. By fine-tuning the parameters of the deep network, the high-level features of the network can better adapt to the specific requirements of medical images, improving the classification ability for the bladder filling state.

[0116] Based on the above embodiments, in order to enable the detection network model to adapt to the detection of lesions of different sizes, the Anchor sizes of the YOLOv9 detection network corresponding to different filling states can also be set differently. For example, the Anchor size of the YOLOv9 detection network corresponding to the standard filling state is the standard size, the Anchor size of the YOLOv9 detection network corresponding to the non-filling state is reduced by a set proportion compared to the standard size, and the Anchor size of the YOLOv9 detection network corresponding to the overfilling state is enlarged by a set proportion compared to the standard size.

[0117] The Anchor of the YOLOv9 detection network refers to the anchor box, which is a set of pre-set bounding boxes used to represent the possible positions and sizes of target objects of different sizes and shapes in the image. Each Anchor box corresponds to a predicted bounding box, and the model can predict the bounding box of the actual target object by adjusting the size and position of these Anchor boxes.

[0118] Among them, the standard size can be understood as the size of the Anchor of the original YOLOv9 detection network. Reducing the set proportion in the non-filling state can be to reduce the standard size by 50%, and the Anchor size of the YOLOv9 detection network corresponding to the non-filling state can be obtained. Enlarging the set proportion in the overfilling state can be to enlarge the standard size by 50%, and the Anchor size of the YOLOv9 detection network corresponding to the overfilling state can be obtained. Of course, the set proportion here can be flexibly set according to actual needs.

[0119] When detecting small lesions, these lesions are relatively small compared to the entire image, while the standard size of traditional Anchors may be relatively large, which makes it difficult for the model to accurately match and locate small lesions during detection, resulting in missed or misdetected cases. In the state of non-full bladder, the lesions may be relatively small due to the relatively contracted bladder shape. Reducing the Anchor size of YOLOv9 by 50% can more precisely predict the boundaries of small lesions. Therefore, for the detection network model in the non-full state, by reducing the Anchor size, the features and positions of these small lesions can be accurately captured because the reduced Anchor boxes better fit the size range of small lesions and can better fit these small targets on the feature map, thereby improving the sensitivity of detecting small lesions.

[0120] Conversely, when detecting large lesions, these lesions are relatively large compared to the entire image, while the standard size of traditional Anchors may be relatively small, which makes it difficult for the model to accurately match and locate large lesions during detection. In the state of over-full bladder, the lesions may be relatively large due to the relatively expanded bladder shape. Enlarging the Anchor size of YOLOv9 by 50% can better fit the size of the lesions, thereby improving the detection accuracy.

[0121] During the training process of the YOLOv9 detection network, since a YOLOv9 detection network is set for each of the three filling states, in order to reduce the training volume, these three YOLOv9 detection networks can achieve knowledge transfer by sharing the first 10-layer parameters of the backbone network. At the same time, an auxiliary task of filling state prediction can be incorporated into the loss function, that is, the YOLOv9 detection network can also identify the filling state, which can enhance the sensitivity of the model to morphological features.

[0122] The following combines the above solution and uses a specific example to illustrate the detection method of this solution. Specifically, it can be referred to Figure 2 as shown.

[0123] The electronic device first obtains multiple frames of bladder ultrasound images. Then, these images can be adjusted in resolution first, that is, resolution normalization. The adjusted images can be used for filling state recognition through a classification network model. First, taking 5 frames of images as a unit, it is detected whether the filling states corresponding to 5 consecutive frames of images are the same. If so, it enters the high-confidence mode. At this time, the corresponding detection network model is directly determined according to the current filling state, and then lesion detection is performed. If not, it is extended to the detection of 15 frames of images. It is detected whether the confidence levels of the filling states of more than 10 frames of images in the 15 frames of images are lower than 0.85. If not, a vote is taken on the filling states corresponding to the 15 frames of images, and then it enters the high-confidence mode, that is, the corresponding detection network model is determined according to the current filling state, and then lesion detection is performed. If so, it enters the transition state mode. At this time, multiple detection network models are used for hybrid detection, and the final detection result can be obtained by weighted summation.

[0124] For example, in the transition state mode, if it is determined that the filling states corresponding to 15 frames of images include the above three filling states, and at this time the detection network models also include the above three models, that is, the detection network model 1 corresponding to the non-filled state, the detection network model 2 corresponding to the standard filling state, and the detection network model 3 corresponding to the over-filled state. During detection, a certain frame of bladder ultrasound image can be input into these three detection network models respectively to obtain three detection results, and then these three detection results can be weighted and summed. The weights can be determined according to the confidence levels of the three filling states. For example, the classification detection network model can predict the confidence levels of the three filling states corresponding to each frame of image, and then the confidence levels of the three filling states in the 15 frames of images can be averaged to obtain the final confidence levels of the three filling states. For example, the final non-filled state: standard filling state: over-filled state = 0.7:0.2:0.1, and the weights are these confidence levels.

[0125] For example, the detection network model 1 outputs the detection result 1, the corresponding weight is 0.7, the detection network model 2 outputs the detection result 2, the corresponding weight is 0.2, and the detection network model 3 outputs the detection result 3, the corresponding weight is 0.1. When weighting, the final detection result = detection result 1 * 0.7 + detection result 2 * 0.2 + detection result 3 * 0.1. Among them, each detection result can include the location, size, shape, and classification of the lesion. When weighting, the detection data corresponding to the lesions with overlapping locations can be weighted and summed. The classification in the detection result can be presented by the classification probability, which is convenient for weighting processing. When using the hybrid model for detection, this method can be used to integrate the detection results, and finally it can be output to the user for display.

[0126] Please refer to Figure 3 , Figure 3The structural block diagram of a bladder lesion detection device provided by an embodiment of this application. This device can be a module, a program segment, or code on an electronic device. It should be understood that this device corresponds to the above Figure 1 method embodiment and can execute Figure 1 each step involved in the method embodiment. The specific functions of this device can be referred to the descriptions above. To avoid repetition, the detailed descriptions are appropriately omitted here.

[0127] Optionally, the device 200 includes:

[0128] An image acquisition module 210, configured to acquire multiple frames of bladder ultrasound images;

[0129] A state detection module 220, configured to identify the target filling state of the bladder in each frame of bladder ultrasound image;

[0130] A lesion detection module 230, configured to determine a corresponding detection network model according to the target filling state, and detect bladder lesions through the determined detection network model to obtain a detection result, where each filling state corresponds to a detection network model.

[0131] Optionally, the lesion detection module 230 is configured to determine the current filling state according to the target filling state of the bladder in multiple frames of bladder ultrasound images; and determine a corresponding detection network model according to the current filling state.

[0132] Optionally, the lesion detection module 230 is configured to determine the current filling state as the target filling state if the target filling states of the bladder in the first set number of consecutive frames of bladder ultrasound images are the same.

[0133] Optionally, the lesion detection module 230 is configured to determine the current filling state as the target filling state with the largest number if the target filling states of the bladder in the second set number of consecutive frames of bladder ultrasound images are different.

[0134] Optionally, the lesion detection module 230 is configured to determine that the current filling state includes various different target filling states if the target filling states of the bladder in the third set number of consecutive frames of bladder ultrasound images are different.

[0135] Optionally, the state detection module 220 is configured to obtain the confidence levels of various filling states to which the bladder belongs in each frame of bladder ultrasound image identified by the classification network model, where the filling states include an unfilled state, a standard filled state, and an overfilled state; and determine the filling state with the highest confidence level as the target filling state.

[0136] Optionally, the lesion detection module 230 is configured to determine the current filling state according to the target filling state of the bladder and the corresponding confidence level in multiple frames of bladder ultrasound images; and determine the corresponding detection network model according to the current filling state.

[0137] Optionally, the state detection module 220 is configured to identify the target filling state of the bladder in each frame of bladder ultrasound image through a classification network model, where the filling states include an unfilled state, a standard filling state, and an overfilled state;

[0138] Wherein, the classification network model is an EfficientNet-B4 classification network, and / or the detection network model is a YOLOv9 detection network.

[0139] Optionally, during the training process of the EfficientNet-B4 classification network, it is trained in a manner of freezing the shallow feature extractor and fine-tuning the network parameters of the deep network.

[0140] Optionally, the Anchor size of the YOLOv9 detection network corresponding to the standard filling state is the standard size, the Anchor size of the YOLOv9 detection network corresponding to the unfilled state is reduced by a set ratio compared to the standard size, and the Anchor size of the YOLOv9 detection network corresponding to the overfilled state is enlarged by a set ratio compared to the standard size.

[0141] It should be noted that those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here.

[0142] Please refer to Figure 4 , Figure 4 FIG. is a schematic structural diagram of an electronic device for implementing the method for detecting bladder lesions provided in an embodiment of the present application. The electronic device may include: at least one processor 310, such as a CPU, at least one communication interface 320, at least one memory 330, and at least one communication bus 340. Among them, the communication bus 340 is used to implement connection communication between these components. Among them, the communication interface 320 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 330 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. Optionally, the memory 330 may also be at least one storage device located far from the foregoing processor. The memory 330 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 310, the electronic device executes the above Figure 1 shown method process.

[0143] It can be understood that Figure 4 the structure shown is only schematic, and the electronic device may further include more or fewer components than those shown Figure 4 in it, or have a configuration different from that shown Figure 4 in it. Figure 4 Each component shown in it can be implemented by hardware, software, or a combination thereof.

[0144] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it executes the method process performed by the electronic device in the method embodiment shown Figure 1 in it.

[0145] This embodiment discloses a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments. For example, it includes:

[0146] Obtain multiple frames of bladder ultrasound images;

[0147] Identify the target filling state of the bladder in each frame of bladder ultrasound image;

[0148] Determine a corresponding detection network model according to the target filling state, and detect bladder lesions through the determined detection network model to obtain a detection result, where each filling state corresponds to a detection network model.

[0149] In summary, an embodiment of the present application provides a method, device, equipment, storage medium, and program product for detecting bladder lesions. By identifying the filling state of the bladder in each frame of bladder ultrasound image, important prior information is provided for subsequent lesion detection. The detection network model is used to detect the bladder in different filling states, and the detection network model is dynamically adjusted according to the identified filling state, so that the detection network model can focus more on the fine detection of lesions, adapt to the changes in the imaging characteristics of the bladder in different filling states, effectively improve the accuracy of lesion detection, and reduce the probability of false detection or missed detection.

[0150] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0151] In addition, the units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] Furthermore, in each embodiment of this application, the various functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0153] In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0154] The above description is only for the embodiments of this application and is not intended to limit the protection scope of this application. For those skilled in the art, this application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A method for detecting bladder lesions, characterized in that: The method comprises: Acquire multiple frames of bladder ultrasound images; identifying a target filling state of the bladder in each frame of the bladder ultrasound image; A corresponding detection network model is determined according to the target filling state, and bladder lesions are detected by using the determined detection network model to obtain a detection result, wherein each filling state corresponds to a detection network model.

2. The method according to claim 1, characterized in that Determining the corresponding detection network model according to the target filling state includes: determining a current filling state according to a target filling state of the bladder in multiple frames of bladder ultrasound images; A corresponding detection network model is determined according to the current filling state.

3. The method according to claim 2, characterized in that Determining the current filling state according to the target filling state of the bladder in the multiple frames of bladder ultrasound images includes: If the target filling state of the bladder in a first set number of consecutive frames of bladder ultrasound images is the same, the current filling state is determined to be the target filling state.

4. The method according to claim 2, characterized in that: Determining the current filling state according to the target filling state of the bladder in the multiple frames of bladder ultrasound images includes: If the target filling states of the bladder in the second set number of consecutive frames of bladder ultrasound images are different, the current filling state is determined to be the target filling state with the largest number.

5. The method according to claim 2, characterized in that: Determining the current filling state according to the target filling state of the bladder in the multiple frames of bladder ultrasound images includes: If the target filling states of the bladder in the third set number of consecutive frames of bladder ultrasound images are different, it is determined that the current filling state includes various target filling states.

6. The method according to claim 1, characterized in that The identifying the target filling state of the bladder in each frame of the bladder ultrasound image comprises: Obtaining the confidence of various filling states of the bladder in each frame of the bladder ultrasound image identified by the classification network model, wherein the filling states include a non-filling state, a standard filling state, and an over-filling state; The filling state with the highest confidence is determined as the target filling state.

7. The method according to claim 6, characterized in that Determining the corresponding detection network model according to the target filling state includes: Determine a current filling state according to a target filling state of the bladder in multiple frames of bladder ultrasound images and a corresponding confidence level; A corresponding detection network model is determined according to the current filling state.

8. The method according to claim 1, characterized in that The identifying the target filling state of the bladder in each frame of the bladder ultrasound image comprises: Identify the target filling state of the bladder in each frame of the bladder ultrasound image by a classification network model, wherein the filling state includes a non-filling state, a standard filling state, and an over-filling state; Among them, the classification network model is the EfficientNet-B4 classification network, and / or the detection network model is the YOLOv9 detection network.

9. The method according to claim 8, characterized in that During the training process of the EfficientNet-B4 classification network, the training is performed in a manner of freezing the shallow feature extractor and fine-tuning the network parameters of the deep network.

10. The method according to claim 8, characterized in that The Anchor size of the YOLOv9 detection network corresponding to the standard filling state is the standard size, the Anchor size of the YOLOv9 detection network corresponding to the underfilling state is reduced by a set ratio compared to the standard size, and the Anchor size of the YOLOv9 detection network corresponding to the overfilling state is enlarged by a set ratio compared to the standard size.

11. A bladder lesion detection device, characterized in that: The device comprises: An image acquisition module, used for acquiring multiple frames of bladder ultrasound images; a state detection module for identifying a target filling state of the bladder in each frame of the bladder ultrasound image; The lesion detection module is used to determine the corresponding detection network model according to the target filling state, and detect bladder lesions through the determined detection network model to obtain detection results, wherein each filling state corresponds to a detection network model.

12. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 10 is executed.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is executed.

14. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method according to any one of claims 1 to 10 is executed.

Images