Ultrasound device and ultrasound detection method

Abstract

The application discloses an ultrasonic device and an ultrasonic detection method, and belongs to the technical field of electronics. In the ultrasonic device, an ultrasonic probe collects an ultrasonic image of a biological organ; a processor encodes the ultrasonic image for n times of iterations; target operations are performed on the encoded image for n times of iterations, and the target operations include decoding; based on the image obtained through the target operations, an ultrasonic image after image segmentation is determined; based on the ultrasonic image after image segmentation, information of the biological organ is determined; the i-th target operation further includes updating a pixel in the image obtained through the i-th decoding based on the weight of at least one kind of information of the pixel; the weight of any information in the at least one kind of information is positively correlated with the influence degree of the any information on image segmentation; 1≤i≤n-1; and a display screen is used for displaying the ultrasonic image and the information of the biological organ. The application solves the problem that the processing effect of the ultrasonic device on the ultrasonic image is poor. The application is used for ultrasonic detection.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to an ultrasonic device and an ultrasonic testing method. Background Technology

[0002] With the development of electronic technology, the requirements for the image processing effect of ultrasound equipment are getting higher and higher.

[0003] In medical examinations, ultrasound equipment is used to acquire ultrasound images of internal organs. In related technologies, ultrasound equipment can use edge detection algorithms to detect the edges of organs in ultrasound images, segmenting the specific region where the organ is located within the image, and then obtaining corresponding information about the organ based on parameters of that region. The accuracy of determining the location of organs in ultrasound images is crucial for medical examinations.

[0004] However, in related technologies, ultrasound equipment has low accuracy in segmenting the region where organs are located in ultrasound images, poor processing effect of acquired ultrasound images, and poor accuracy in determining information about biological organs. Summary of the Invention

[0005] This application provides an ultrasound device and an ultrasound detection method, which can solve the problems of low image segmentation accuracy, poor image processing effect, and poor accuracy in determining information about biological organs by ultrasound devices. The technical solution is as follows:

[0006] On one hand, an ultrasonic device is provided, the ultrasonic device comprising:

[0007] An ultrasound probe is used to acquire ultrasound images of organs in living organisms.

[0008] Processor, used for:

[0009] The ultrasound image is encoded through n iterations, where n ≥ 2;

[0010] For the encoded image obtained through n iterations, a target operation is performed for n iterations, the target operation including decoding; wherein, the i-th target operation in the n-iteration target operation further includes: updating the i-th decoded image, the update being used to update the pixel based on the weight of at least one piece of information of the pixel in the i-th decoded image; the weight of any of the at least one pieces of information is positively correlated with the degree of influence of the at least one piece of information on image segmentation; 1≤i≤n-1;

[0011] Based on the image obtained from the target operation of the n iterations, the segmented ultrasound image is determined.

[0012] Based on the segmented ultrasound image, information about the organism's organs is determined;

[0013] A display screen is used to display the ultrasound images and information about the biological organs.

[0014] On the other hand, an ultrasonic testing method is provided for use in an ultrasonic device, the method comprising:

[0015] Acquire ultrasound images of organs in living organisms;

[0016] The ultrasound image is encoded through n iterations, where n ≥ 2;

[0017] For the encoded image obtained through n iterations, a target operation is performed for n iterations, the target operation including decoding; wherein, the i-th target operation in the n-iteration target operation further includes: updating the i-th decoded image, the update being used to update the pixel based on the weight of at least one piece of information of the pixel in the i-th decoded image; the weight of any of the at least one pieces of information is positively correlated with the degree of influence of the at least one piece of information on image segmentation; 1≤i≤n-1;

[0018] Based on the image obtained from the target operation of the n iterations, the segmented ultrasound image is determined.

[0019] Based on the segmented ultrasound image, information about the organism's organs is determined;

[0020] The ultrasound image and information about the organism's organs are displayed.

[0021] In another aspect, an ultrasonic device is provided, the ultrasonic device comprising: a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the steps executed by the processor in the above-described ultrasonic detection method.

[0022] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the steps executed by the processor in the above-described ultrasonic detection method.

[0023] The beneficial effects of the technical solution provided in this application include at least the following:

[0024] In this application, the ultrasound device can encode ultrasound images iteratively multiple times, and perform multiple iterative target operations on the encoded images obtained from n iterations. In each of these multiple iterative target operations, except for the last one, after decoding the image, the pixel is updated based on the weight of at least one piece of information in the decoded image, and the weight of this at least one piece of information is positively correlated with the degree of influence on image segmentation. In this way, information in the image that has a significant impact on the image segmentation result can be highlighted, while background and noise information that has a smaller impact on the image segmentation result can be reduced. This improves the accuracy of image segmentation, ensures better image processing performance of the ultrasound device, and ultimately improves the accuracy of identifying information about biological organs. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure of an ultrasonic device provided in an embodiment of this application;

[0026] Figure 2 This is a flowchart of an ultrasonic testing method provided in an embodiment of this application;

[0027] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of a data processing flow based on the two dimensions of channel and space, provided in an embodiment of this application;

[0029] Figure 5 This is a structural block diagram of another ultrasonic device provided in the embodiments of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0031] Currently, with the development of electronic technology, the requirements for the quality of information acquisition and processing are becoming increasingly stringent. In medical examinations, equipment such as ultrasound can be used to acquire images of internal organs, which can then be processed to determine relevant organ information, thereby enabling the examination of a person's health status. For example, medical examinations often require measuring bladder volume, a crucial parameter in urological clinical applications, reflecting the amount of urine contained in the bladder. Ultrasound equipment can acquire ultrasound images of the abdomen, segmenting these images to identify the bladder region, and then determining the bladder volume based on the size of this region. However, ultrasound images are noisy and have unclear boundaries, making segmentation difficult and resulting in low accuracy. Consequently, the accuracy of bladder volume determined from segmented ultrasound images is low, affecting the accuracy of medical examinations.

[0032] This application provides an ultrasonic device and an ultrasonic testing method that can perform relatively accurate image segmentation on ultrasonic images.

[0033] Figure 1 This is a schematic diagram of the structure of an ultrasonic device provided in an embodiment of this application, and Figure 1 The text also illustrates one application scenario for the ultrasound device. For example... Figure 1 As shown, the ultrasound device 10 may include: an ultrasound probe 101, a processor 102, and a display screen 103. The ultrasound probe 101 is used to acquire ultrasound images of biological organs. Acquiring the ultrasound image as described in this embodiment refers to acquiring information from the ultrasound image. The processor 102 is used to perform image segmentation on the ultrasound image. The processor 102 can also be used to determine information about the biological organ based on the segmented ultrasound image. The display screen 103 is used to display the ultrasound image and the determined information about the biological organ.

[0034] For example, the aforementioned biological organ can be an organ within the human body, such as the bladder, heart, or liver. Optionally, the biological organ can also be an organ within an animal; this application embodiment does not impose limitations. This application embodiment uses the bladder as an example. The processor 102 of the ultrasound device can perform image segmentation on the ultrasound image for the bladder. After image segmentation, based on the region where the bladder is located in the segmented image, bladder information is determined, such as bladder volume. The processor 102 can then control the display screen 103 to display the bladder volume. Optionally, after the ultrasound probe 101 acquires the ultrasound image, the display screen 103 can display the ultrasound image.

[0035] It should be noted that the scanning probe 101 can sequentially emit multiple ultrasound beams (i.e., transmit signals). The operator can aim the scanning probe 101 at a biological organ; the ultrasound beams emitted by the scanning probe 101 can penetrate the human body and be reflected off the organ. The scanning probe 101 can acquire the reflected ultrasound beams (i.e., echo signals) after a corresponding reception delay of the emitted ultrasound beams. The processor 102 can obtain organ information based on the echo signals, thereby generating an ultrasound image and performing image segmentation on the ultrasound image.

[0036] In this embodiment, the ultrasound probe, processor, and display screen in the ultrasound device 10 are interconnected via wiring. Optionally, the ultrasound probe, processor, and display screen can be independent. For example, the ultrasound probe can transmit the acquired image information to the ultrasound device and display screen via a network, and the ultrasound device can send the information of the biological organ to the display screen via the network for display. Optionally, the image segmentation process executed by the processor can also be executed on other independent devices with information processing capabilities to achieve image segmentation independently. In this case, the image segmentation target can be other ordinary color images or grayscale images. Such independent devices can be computers, servers, computers, and smartphones, etc., and this embodiment does not limit the scope.

[0037] Figure 2 This is a flowchart of an ultrasonic testing method provided in an embodiment of this application, which can be used in the aforementioned ultrasonic equipment. Figure 1 As shown, the method may include:

[0038] Step 201: Acquire ultrasound images of biological organs.

[0039] For example, the ultrasound device can acquire ultrasound images of cross-sections and longitudinal sections of biological organs, respectively. It should be noted that step 201 can be performed by the scanning probe 101 in the ultrasound device 10. For the method of acquiring ultrasound images by the scanning probe 101, please refer to the above description of the scanning probe 101; this embodiment will not repeat it further.

[0040] Step 202: Encode the ultrasound image through n iterations, where n ≥ 2.

[0041] The ultrasound device can perform multiple iterations of encoding on each ultrasound image acquired in step 201. That is, in these multiple encodings, the first encoding is for the entire pair of ultrasound images, and each subsequent encoding is for the image obtained from the previous encoding. It should be noted that encoding the image is equivalent to extracting its features. The ultrasound image can be an image of the target to be segmented, such as a biological organ like the bladder or heart. It should be noted that step 202 can be executed by the processor 102 in the ultrasound device 10.

[0042] Step 203: Perform n iterations of target operation on the encoded image obtained from the n iterations; the target operation includes decoding, and the i-th target operation in the n iterations of target operation further includes: updating the i-th decoded image, which is used to update the pixel based on the weight of at least one piece of information of the pixel in the i-th decoded image; 1≤i≤n-1.

[0043] An ultrasound device can perform n iterations of target operations on an image obtained after encoding an ultrasound image through n iterations. In these n target operations, the first target operation is performed on the image obtained after encoding through n iterations, and each subsequent target operation is performed on the image obtained from the previous target operation.

[0044] In this embodiment, the first n-1 target operations all include decoding the image and updating the decoded image. The last target operation only includes decoding the image obtained from the previous target operation and does not include an update process. Updating the decoded image is equivalent to updating each pixel in the image, and each pixel is updated based on the weight of at least one piece of information about that pixel. The images obtained after encoding and decoding are both feature maps, and the values ​​of the pixels in the images refer to the feature values ​​in the feature maps.

[0045] It should be noted that steps 202 and 203 pertain to the image segmentation process of the ultrasound image. The weight of any one of the at least one pieces of information in the pixels on which the update of the decoded image is based is positively correlated with the degree of influence of that information on image segmentation. It should also be noted that step 203 can be executed by the processor 102 in the ultrasound device 10.

[0046] Step 204: Based on the image obtained from the target operation of the n iterations, determine the ultrasound image after image segmentation.

[0047] Optionally, the ultrasound device can directly determine the image obtained from the n-iteration target operation as the segmented ultrasound image, that is, the image obtained after segmenting the ultrasound image. Optionally, the ultrasound device can also determine the segmented ultrasound image based on the image obtained from the n-iteration target operation and other information. It should be noted that step 204 can be executed by the processor 102 in the ultrasound device 10.

[0048] Step 205: Based on the segmented ultrasound image, determine the information of the organism's organs.

[0049] For example, an ultrasound device can acquire ultrasound images of the bladder in both transverse and longitudinal sections; image segmentation is performed on both ultrasound images, and information about the biological organ is determined based on the two images obtained from the image segmentation. In this embodiment, the biological organ is the bladder, and the information about the bladder is its volume. Optionally, the organ could also be other organs such as the heart, and the information about the biological organ could be other information.

[0050] It should be noted that step 205 can be executed by the processor 102 in the ultrasound device 10. Optionally, the processor 102 can calculate the bladder volume using a target formula, which is V = 0.5 * D1 * D2 * D3. Here, V represents the bladder volume, D1 represents the width of the bladder cross-section, D2 is the height of the bladder cross-section, and D3 is the length of the bladder longitudinal section. Optionally, a fixed value can be added to V to determine the bladder volume; this embodiment does not limit this.

[0051] Step 206: Display ultrasound images and information about the organism's organs.

[0052] It should be noted that step 206 can be performed by the display screen 103 in the ultrasound device 10. For example, the display screen 103 can display the ultrasound image at the center of the display area and display information about the organs of the organism at the edge of the display area.

[0053] In summary, the ultrasound detection method provided in this application allows the ultrasound device to encode ultrasound images iteratively multiple times, and to perform multiple iterative target operations on the encoded images obtained from n iterations. In each of these multiple iterative target operations, except for the last one, after decoding the image, the pixel is updated based on the weight of at least one piece of information in the decoded image, and the weight of this at least one piece of information is positively correlated with the degree of influence of image segmentation. This approach highlights information in the image that has a significant impact on the image segmentation result, while reducing background and noise information that has a smaller impact on the image segmentation result. This improves the accuracy of image segmentation, ensures better image processing performance of the ultrasound device, and ultimately enhances the accuracy of determining information about biological organs.

[0054] Figure 3 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This method can be used in the processor of the aforementioned ultrasound device to perform image segmentation on ultrasound images. Figure 3 Let's take n=5 as an example for illustration. Figure 3 As shown, the ultrasound equipment first encodes the ultrasound image A through 5 iterations, and then decodes the encoded image through 5 iterations to obtain the image segmentation result A' of the ultrasound image A. Figure 3 Taking the ultrasound image of the bladder as an example, the ultrasound image is a grayscale image. After image segmentation, a black and white image can be obtained, in which the white area is the area where the bladder is located.

[0055] For step 202 above, please refer to Figure 3 The process of encoding from the first to the fifth time in the process is described below with reference to the accompanying drawings.

[0056] It should be noted that in the n iterations of encoding the ultrasound image, each encoding step is used to extract image features, and the resulting image is a feature map of the ultrasound image. Shallow encoding (i.e., encoding performed earlier, such as the first and second encoding steps) has a smaller receptive field, primarily extracting local low-level features of the image, such as edges, shadows, and bright spots. Deep encoding (i.e., encoding performed later, such as the fifth encoding step) has a larger receptive field, primarily extracting global high-level features of the image, such as tissues, organs, and lesions. In the embodiments of this application, the image size is reduced after each encoding step, while the number of image channels increases, for example, by a factor of two.

[0057] In this embodiment, the encoding of any image includes: a first convolutional process and an auxiliary process executed sequentially, that is, auxiliary processing is performed on the result of the first convolutional process. It should be noted that in any two sequentially executed processes described in this embodiment, the latter process is a processing of the result obtained from the former process. In the encoding of the ultrasound image for the nth iteration, the auxiliary processing in the first encoding includes: downsampling processing. The downsampling processing in this embodiment can be pooling. It should be noted that after downsampling, the image size becomes smaller, and the image contains less data. The auxiliary processing in the (i+1)th encoding (that is, the encodings other than the first encoding) includes: a sequentially executed addition process and downsampling processing. The addition process is used to add the image after the first convolutional process to the image before the first convolutional process; the processing structure of this addition process is also the residual structure. Then, downsampling processing is performed on the image obtained after the addition process. Optionally, the addition process in the (i+1)th encoding may exist only once, or it may exist multiple times. When there are multiple addition processes, these processes are executed sequentially, and the output of each addition process is used as the input for the next addition process.

[0058] For example, such as Figure 3 As shown, the first convolutional processing in the first encoding can be implemented using a convolutional layer Conv[(7*7), (p, q), / 2]. Here, (7*7) refers to the size of the convolutional kernel used in this convolutional layer, p refers to the number of channels of the image input to this convolutional layer, q refers to the number of channels of the output image after the first convolutional processing, and " / 2" means that the size of the output image after the first convolutional processing is reduced to half of its original size. The downsampling processing in the first encoding can be implemented using a pooling layer MaxPool[(3*3), / 2]. Here, (3*3) refers to the size of the downsampling region, and " / 2" means that the size of the output image after the downsampling processing is reduced to half of its original size. Optionally, the stride of this downsampling processing is 2, and the padding amount is 1.

[0059] It should be noted that the size of the convolutional kernels in the convolutional layer, the number of channels, the size of the downsampling region in the pooling layer, and the reduction in image size can all be adjusted, and this application embodiment does not impose any limitations. The number of convolutional kernels in a convolutional layer determines the number of channels in the output image of that convolutional layer, and the number of channels in the output image of a convolutional layer can be equal to the number of convolutional kernels in that convolutional layer. For example, if p=1, q=16, the number of image channels in the input image is 1, and the convolutional layer includes 16 convolutional kernels, then the image output after the first convolutional processing in the first encoding of the ultrasound image has 16 image channels. In the embodiments of this application, each image channel of the image is used to characterize different features of the image, and the value of any image channel of a pixel in the encoded image refers to the feature value of that image channel.

[0060] In this embodiment, the combination of convolutional and pooling layers used in the first encoding is relatively simple, and the processing method of the first encoding is also relatively simple. The convolutional kernel size in this convolutional layer is relatively large, so the receptive field is relatively large; and the number of convolutional layers is relatively small (e.g., only one convolutional layer), so the output image can retain more detailed information in the ultrasound image, such as edges, shadows, and bright spots in the image.

[0061] Optionally, in this embodiment, the number of image channels in the first encoded image is less than 64, and the number of image channels in the first encoded image is 16. Since the number of image channels doubles with each encoding iteration, the entire processing structure for image processing becomes increasingly complex. In related technologies, the number of image channels in the image obtained after the first encoding is 64, and the number of image channels doubles with each subsequent encoding, resulting in high complexity of the processing structure. In this embodiment, the number of image channels in the initially encoded image is reduced. Even if the number of image channels increases to some extent with subsequent encoding iterations, the processing structure after the increase in the number of image channels is simpler than that of related technologies, with a smaller processing size and higher processing efficiency. Thus, this image processing method can be applied to computing devices with limited computing resources. Optionally, the ultrasound image is an ultrasound image, which has a low signal-to-noise ratio, and the number and types of targets to be detected are unique. This method of reducing the number of image channels can still ensure a high processing effect for the ultrasound image.

[0062] Please continue to refer to this. Figure 3 For the (i+1)th encoding (e.g. Figure 3 The encoding process (from the second to the fifth encoding) involves a first convolutional process implemented using at least one convolutional layer. Each convolutional layer corresponds to one first convolutional process, and multiple first convolutional processes can be performed in the (i+1)th encoding. Figure 3 This illustration uses four convolutional layers as an example. The structure of the convolutional layers can be the same in the second to fifth encoding processes. Figure 3Taking the third encoding as an example, the structure of its convolutional layers is illustrated. The convolutional layers in the (i+1)th encoding can form a group of convolutional layers with a residual structure. This residual structure is used to add the image after convolution to the image before convolution. For example... Figure 3 As shown, the four convolutional layers in the third encoding process form two groups of convolutional layers with residual structures. During the third encoding process, the image obtained from the second encoding first undergoes convolution and summation through the first group of convolutional layers with residual structures, then undergoes convolution and summation again through the second group of convolutional layers with residual structures, and finally undergoes downsampling through pooling layers.

[0063] Figure 3 Taking a convolutional layer group with two convolutional layers in each residual structure as an example. The two convolutional layers in the first residual structure's convolutional layer group are Conv[(3*3), (p, q), / 2] and Conv[(3*3), (q, q)], respectively. After passing through these two convolutional layers, the image size is reduced to half of its original size. The image obtained after passing through these two convolutional layers can be fused with the image obtained from the second encoding, and then input into the second residual structure's convolutional layer group. For example, the image obtained from the second encoding can also be convolved through the convolutional layer Conv[ / 2] to reduce the image size by 1 / 2. The image after being reduced in size by 1 / 2 is used to fuse with the image obtained after passing through these two convolutional layers. This fusion can be a direct addition of the two images (i.e., an addition process). The two convolutional layers in the second residual structure's convolutional layer group are both Conv[(3*3), (q, q)], respectively. After passing through these two convolutional layers, the image size remains unchanged. The images obtained after these two convolutional layers are directly added to the input image of the second residual structure's convolutional layer group, and then input into the pooling layer (i.e., ...). Figure 3 (MaxPool layer in the middle).

[0064] It should be noted that the explanation of the above convolutional layers should be inferred from the above description of the convolutional layers in the first encoding, and will not be repeated in this embodiment. Optionally, each group of convolutional layers may include only one convolutional layer, or it may include three, four, or even more convolutional layers; each encoding after the first encoding may also include only one group of convolutional layers with residual structures, or it may include three, four, or even more groups of convolutional layers with residual structures, and this embodiment does not impose any limitations. The use of convolutional layers with residual structures for encoding in this embodiment can ensure faster convergence of the processing results, which is beneficial for obtaining better image segmentation results.

[0065] Optionally, in the n-iteration encoding in this embodiment, at least one of the first convolutional processes in the encoding may employ a dilated convolution kernel. Using a dilated convolution kernel can increase the receptive field of the processed image, thereby enhancing the image segmentation effect. If this at least one encoding is the first two or three encodings in the n-iteration encoding, since the computation speed after using a dilated convolution kernel is slower, it can be used in fewer encodings. Optionally, the dilation rate (also called the dilation ratio) of the dilated convolution kernel can be 2 or 3.

[0066] For step 203 above, please refer to... Figure 3 The process of decoding from the first to the fifth time is described below with reference to the accompanying drawings, which will introduce the target operation of n iterations in the embodiments of this application.

[0067] In this embodiment, the target operation performed on the encoded image obtained through n iterations includes decoding the input image and updating the decoded image in each of the first n-1 target operations; the last target operation only includes decoding the input image and does not include the update process. The first decoding is performed on the encoded image obtained through n iterations, and each decoding operation other than the first decoding is a decoding of the image obtained from the previous target operation, that is, a decoding of the image obtained after updating the image obtained from the previous decoding. Updating the decoded image is equivalent to updating each pixel in the image.

[0068] In this embodiment, decoding for any image includes a second convolution process and an upsampling process. That is, in the target operation performed on the image obtained from n iterations of encoding, decoding in each target operation includes a second convolution process and an upsampling process. The second convolution process and the upsampling process can be performed sequentially, or the upsampling process can be performed first and then the second convolution process. This embodiment does not limit the execution order of the second convolution process and the upsampling process. In this embodiment, the image size increases and the number of image channels decreases after each decoding.

[0069] Optionally, upsampling can be performed using transposed convolution. The second convolution in each decoding iteration can be implemented using convolutional layers, with each convolutional layer corresponding to one second convolution operation; similarly, upsampling can be implemented using transposed convolutional layers, with each transposed convolutional layer corresponding to one upsampling operation. Each decoding iteration can perform one or more second convolution operations, or one or more upsampling operations. Figure 3In the first four decoding passes, two second convolution processes and one upsampling process are performed. Taking the fifth decoding pass as an example, it performs one second convolution process and two upsampling processes. The structures of the convolutional layers and transposed convolutional layers are the same in the first to fourth decoding passes. Figure 3 Taking the third decoding as an example, the structure of the convolutional layer and the transposed convolutional layer is illustrated.

[0070] For example, such as Figure 3 As shown, the second convolutional processing in the third decoding can be implemented using two convolutional layers Conv[(1*1), (p, q / 4)], and the upsampling processing can be implemented using a transposed convolutional layer TransConv[(3*3), p / 4, p / 4), *2]. Here, "*2" indicates that the size of the output image after corresponding to this convolutional layer or transposed convolutional layer is increased to twice the original size. For explanations of other parameters in the convolutional and transposed convolutional layers, please infer from the above description of the convolutional layers in the first encoding; these will not be repeated in the embodiments of this application. The transposed convolutional layer can be located between the two convolutional layers. First, the image obtained from the second target operation is processed by the convolutional layer Conv[(1*1), (p, q / 4)], then the image obtained from the second convolution is processed by the convolutional layer TransConv[(3*3), p / 4, p / 4), *2], and then the image obtained from the upsampled processing is processed by the convolutional layer Conv[(1*1), (p, q / 4)] again, and then the image obtained from the upsampled processing is processed by the second convolution again.

[0071] The second convolutional processing in the fifth decoding can be implemented using a convolutional layer Conv[(7*7), (q, q)]. The upsampling processing can be implemented using transposed convolutional layers TransConv[(3*3), (p, q), *2] and TransConv[(2*2), (q, 1), *2]. This convolutional layer can be located between the two transposed convolutional layers. First, the image obtained from the fourth target operation is upsampled using the transposed convolutional layer TransConv[(3*3), (p, q), *2]. Then, the image obtained from the upsampled processing is subjected to a second convolutional processing using the convolutional layer Conv[(7*7), (q, q)]. Finally, the image obtained from the second convolutional processing is subjected to a transposed convolutional processing again using the transposed convolutional layer TransConv[(2*2), (q, 1), *2]. It should be noted that the above convolutional layers and transposed convolutional layers are just examples. The size of the convolutional kernel, the number of each channel, and the amount of increase in image size can all be adjusted, and the embodiments of this application do not limit them.

[0072] The following describes the update process for the image obtained from the first n-1 decoding iterations. Updating the image means updating each pixel within that image.

[0073] In this embodiment, each pixel can be updated based on the weight of at least one piece of information for each pixel in the decoded image to update the decoded image. In this embodiment, the image obtained after each of the first n-1 decodings may include multiple image channels, such as the image obtained after the i-th decoding, which includes multiple image channels. Each pixel in the image includes the value of each of the multiple image channels. At least one piece of information for a pixel in the i-th decoded image may include at least one piece of information from the pixel's image channels and the pixel's position in the image. This embodiment takes as an example that at least one piece of information for a pixel may simultaneously include all image channels of the pixel and the pixel's position in the image, and updates the pixel based on both types of information. Optionally, the at least one piece of information may also include only all image channels of the pixel; or, the at least one piece of information may also include only the pixel's position in the image; or, the at least one piece of information may include some image channels of the pixel and the pixel's position in the image. This embodiment does not limit this.

[0074] Each image channel can have a corresponding weight, and each position in the image can also have a corresponding weight. In this embodiment, the pixel is updated based on the weights of each image channel of the pixel in the decoded image, and the weight of the pixel's position in the image. Each image channel of the image can correspond to a type of image feature, and the weight of the image channel is also the weight of the image feature. For example, for any image channel of a pixel, the updated value of that image channel is y = x * a * b; where x represents the value of that image channel of the pixel before the update, a represents the weight of that image channel, and b represents the weight of the pixel's position in the image.

[0075] This update method adds weights to the image feature map in two dimensions: channel and space. The space dimension is the width-height plane of the feature map. Figure 4 This is a schematic diagram illustrating a data processing flow based on the two dimensions of channel and space, provided in an embodiment of this application. For example... Figure 4 As shown, the original data U can be processed through two branches: the first branch processes the data based on spatial dimension weights, and the second branch processes the data based on channel dimension weights. The results of these two branches can be fused to obtain the final processed result U'.

[0076] In this embodiment, updating pixels in the image based on the weights of the image channels and the pixel's position within the image is essentially an attention mechanism. The weights of information more closely related to the target being segmented can be set higher to highlight information that significantly impacts the segmentation result; conversely, the weights of information less closely related to the target can be set lower to reduce noise and background information irrelevant to the segmentation. Consequently, during processing, the ultrasound device can focus on information with a greater impact on the segmentation result. Gradient descent is more effective and converges faster when processing high-weight information, resulting in better image processing performance. Furthermore, by reducing irrelevant parameters, image processing efficiency is improved, achieving higher image processing performance with fewer parameters.

[0077] In this embodiment, the image obtained from the i-th decoding in n iterations of decoding and the image obtained from the encoding in the ni-th target operation (i.e., the ni-th encoding) have the same image channels and the same image size. Since updating the decoded image based on weights does not change the image channels and image size, the image obtained from the i-th encoding and the image obtained from the ni-th decoding after weight updates have the same image channels and the same image size. Figure 3 In this example, n=5. The image obtained from the first encoding and the image obtained from the fourth decoding (based on weight updates) have the same image channels and the same image size. For the decoding and updating in the i-th target operation, this update can also be used to: add the i-th decoded image (i.e., the image obtained after the i-th weight update) to the ni-th encoded image to update the i-th decoded image. If the addition results in an auxiliary image, the (i+1)-th decoding is for this auxiliary image. This allows for the combined processing of local and global features of the encoded image, ensuring effective image processing.

[0078] Adding two images constitutes image fusion, which involves adding pixels one by one. The resulting image retains the same number of channels and size (also known as the size of the feature mapping tensor). In related technologies, the decoded and encoded images are fused by channel concatenation. This doubles the number of channels in the fused image, requiring more convolutional kernels (also known as convolutional filters) for processing, resulting in low efficiency. However, in this embodiment, the decoded and corresponding encoded images are added together, maintaining the same number of channels and thus improving image processing efficiency while reducing the size of the processing structure.

[0079] Optionally, the first encoding described above can also be called the initial processing, and the last decoding can also be called the final processing. It should be noted that the process of encoding the ultrasound image n times, and then performing n iterations of target operations on the encoded image, is also the image segmentation process for the ultrasound image. Optionally, the image obtained from the n iterations of target operations can be directly determined as the segmented ultrasound image, that is, the image obtained after image segmentation of the ultrasound image. Please refer to [reference needed]. Figure 3 The image A' obtained by the target operation in the n iterations is the image obtained after image segmentation of the ultrasound image A.

[0080] Optionally, in this embodiment, a semantic segmentation model can be used to implement the above-described ultrasound detection method to obtain the image obtained from n iterations of the target operation. For example, an ultrasound device can input an ultrasound image into a semantic segmentation model, and then obtain the image after image segmentation of the ultrasound image output by the semantic segmentation model. This semantic segmentation model is an undercomplete deep neural network model. The semantic segmentation model may include an encoding module, a decoding module, and an update module (or attention module). The structure of this semantic segmentation model can be referenced... Figure 3 , Figure 3 Each encoding process in the algorithm can correspond to an encoding module, each decoding process can correspond to a decoding module, and each weight-based update can correspond to an update module.

[0081] A semantic segmentation model can be trained on a training set, and then the ultrasound detection method described above can be executed using this model. For example, a training set can be constructed before training the semantic segmentation model. This can be done by acquiring training images (such as bladder images) using ultrasound equipment, and manually annotating key points in each training image, such as the edge regions of the target to be segmented (e.g., the bladder in the ultrasound image). To increase the model's generalization ability, data augmentation can be performed on the training data during training. Data augmentation can apply random geometric transformations such as rotation, stretching, shearing, translation, and flipping to the images, and can also randomly increase or decrease brightness, contrast, sharpness, and noise. Data augmentation can simulate images acquired by image acquisition equipment under different parameters and usage scenarios when the dataset is limited. The manually annotated images are then input into a deep neural network, and the resulting sample data is divided into a training set and a test set to train the neural network model. The training set is used for model training, and the test set is used for model performance testing.

[0082] In this embodiment of the application, based on the image obtained from the target operation of the n iterations, the ultrasound device can also determine the ultrasound image after image segmentation based on other information.

[0083] For example, ultrasound equipment is performing Figure 2 During steps 202 and 203, or in the process of... Figure 3 In the image processing method shown, the following steps can be performed in parallel: a first operation is performed on the ultrasound image for n iterations, the first operation including a third convolution process and an upsampling process. A second operation is performed on the image obtained from the first operation for n iterations, the second operation including a fourth convolution process and a downsampling process. In the embodiments of this application, the first operation can sequentially execute the third convolution process and the upsampling process, or sequentially execute the upsampling process and the third convolution process. The second operation can sequentially execute the fourth convolution process and the downsampling process, or sequentially execute the downsampling process and the fourth convolution process. Optionally, the first operation and the second operation for n iterations can be implemented based on an overcomplete neural network.

[0084] This first operation is also an encoding operation; for details on the first operation, please refer to the above. Figure 2 and Figure 3 The description of the encoding process involving n iterations is provided below. The third convolution process can be referenced in the above description of the first convolution process. The difference between this first operation and the encoding process described above is that downsampling is performed in the encoding operation, while upsampling is performed in this first operation. For details on the upsampling in the first operation, please refer to the above description of the upsampling in the decoding process; this application's embodiments will not repeat them here. The second operation is also a decoding operation; for details on the second operation, please refer to the above description of... Figure 2 and Figure 3 The following is a description of the decoding process after n iterations, where the fourth convolution process can be referred to the above description of the second convolution process. The difference between this second operation and the decoding process described above is that the decoding process performs upsampling, while this second operation performs downsampling. The downsampling process in the second operation can be referred to the above description of downsampling in the encoding process; it will not be repeated in the embodiments of this application.

[0085] After performing n iterations of the second operation and n iterations of the target operation, the ultrasound device can jointly determine the segmented ultrasound image based on the images obtained from the n iterations of the second operation and the n iterations of the target operation. For example, the images obtained from the n iterations of the second operation and the n iterations of the target operation can be fused, and then convolved to obtain the segmented ultrasound image. Optionally, the n first operations performed by the ultrasound device can correspond one-to-one with n encoding operations, and the n second operations can correspond one-to-one with n target operations. The ultrasound device can fuse the images obtained from each first operation with the corresponding encoding operation images, and perform the next first operation and encoding operation based on the fused image; it can also fuse the images obtained from each second operation with the corresponding target operation images, and perform the next second operation and target operation based on the fused image. This allows for the summarization and mixing of related information in each operation before processing, improving image processing efficiency.

[0086] In summary, the image processing method provided in this application embodiment allows the ultrasound device to encode ultrasound images iteratively multiple times, and to perform multiple iterative target operations on the encoded images obtained from n iterations. In each of these multiple iterative target operations, except for the last one, after decoding the image, the pixel is updated based on the weight of at least one piece of information in the decoded image, and the weight of this at least one piece of information is positively correlated with the degree of influence on image segmentation. This approach highlights information in the image that has a significant impact on the image segmentation result, while reducing background and noise information that has a smaller impact on the image segmentation result, thereby improving the accuracy of image segmentation and ensuring better image processing performance of the ultrasound device.

[0087] Figure 5 This is a structural block diagram of an ultrasonic device provided in an embodiment of this application. Figure 5 As shown, the ultrasound device 10 may include: a scanning probe 101, a processor 102, and a display screen 103. The processor 102 may include: an encoding module 1021, a processing module 1022, and a determination module 1023.

[0088] The encoding module 1021 is used to encode the ultrasound image through n iterations, where n ≥ 2. It should be noted that the execution process of the encoding module 1021 can be referred to the above. Figure 2 as well as Figure 3 The relevant descriptions of the encoding process in the previous embodiments will not be repeated in this application.

[0089] The processing module 1022 is used to perform n-iteration target operations on the encoded image obtained from n iterations. The target operations include decoding, and the i-th target operation in the n-iteration target operations further includes updating the i-th decoded image. This update is used to update the pixel based on the weights of at least one piece of information about the pixel in the i-th decoded image; 1 ≤ i ≤ n-1. The weight of any one of the at least one piece of information about the pixel on which the update of the decoded image is based is positively correlated with the degree of influence of that one piece of information on image segmentation.

[0090] For example, the processing module 1022 may include a decoding module 1022a and an update module 1022b. The decoding module 1022a is used to execute the decoding process in each target operation, and the update module 1022b is used to execute the update process in the target operation. It should be noted that the execution process of the decoding module 1022a can be referred to the above. Figure 2 as well as Figure 3 The relevant descriptions of the decoding process and the execution process of the update module 1022b can be found above. Figure 2 as well as Figure 3 The relevant descriptions of updating the decoded image are not repeated in the embodiments of this application.

[0091] The determination module 1023 is used to: determine the segmented ultrasound image based on the image obtained from the target operation over the n iterations. The execution process of the determination module 1023 can be referred to the above. Figure 2 as well as Figure 3 The relevant descriptions of the ultrasound images after image segmentation are not repeated in the embodiments of this application.

[0092] In summary, the ultrasound device provided in this application can encode ultrasound images iteratively multiple times, and perform multiple iterative target operations on the encoded images obtained from n iterations. In each of these multiple iterative target operations, except for the last one, after decoding the image, the pixel is updated based on the weight of at least one piece of information in the decoded image, and the weight of this at least one piece of information is positively correlated with the degree of influence on image segmentation. In this way, information in the image that has a significant impact on the image segmentation result can be highlighted, while background and noise information that has a smaller impact on the image segmentation result can be reduced, thereby improving the accuracy of image segmentation and ensuring better image processing performance of the ultrasound device.

[0093] In an exemplary embodiment, an electronic device is also provided, which may include a processor and a memory, the memory storing at least one instruction. The at least one instruction is configured to be executed by one or more processors to implement the steps performed by the processor in any of the above-described ultrasonic detection methods.

[0094] In an exemplary embodiment, a computer-readable storage medium is also provided, the storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded by a processor and executes the steps executed by the processor in any of the above-described ultrasonic detection methods.

[0095] In an exemplary embodiment, a computer program product is also provided, wherein the code in the computer program product is used to implement the steps executed by the processor in any of the above-described ultrasonic detection methods.

[0096] It should be noted that the ultrasound equipment provided in the above embodiments is only illustrated by the division of the above functional modules when performing image processing. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the equipment can be divided into different functional modules to complete all or part of the functions described above. The order of the steps in the method embodiments provided in this application can be appropriately adjusted, and the steps can also be added or removed according to the situation. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further. The ultrasound detection method embodiments, image processing method embodiments, and ultrasound equipment embodiments provided in this application can all be referred to each other, and this application does not limit them.

[0097] In this application, the term "comprising" as used throughout the specification and claims is an open-ended term and should therefore be interpreted as "comprising but not limited to". In the embodiments of this application, the terms "first", "second", etc., are used to distinguish identical or similar items with substantially the same function. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity or execution order. In the embodiments of this application, the term "at least one" means one or more, and the term "multiple" means two or more.

[0098] The term "and / or" as used in the embodiments of this application refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of A and B" means that A exists alone, A and B exist simultaneously, or B exists alone.

[0099] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An ultrasound apparatus, characterized by, The ultrasonic device comprises: an ultrasonic probe configured to acquire an ultrasonic image of a biological organ; a processor configured to: perform n times of iterative encoding on the ultrasonic image, n≥2; wherein in the n times of iterative encoding, a size of a convolution kernel in one or more shallow layer encodings performed in front is greater than a size of a convolution kernel in one or more deep layer encodings performed in back, a number of convolution layers in the one or more shallow layer encodings performed in front is less than a number of convolution layers in the one or more deep layer encodings performed in back, and a first convolution processing in the one or more shallow layer encodings performed in front adopts a hollow convolution kernel, and a first convolution processing in the one or more deep layer encodings performed in back does not adopt the hollow convolution kernel; perform n times of iterative target operations on images obtained by the n times of iterative encoding, the target operations comprising decoding; wherein an i-th target operation in the n times of iterative target operations further comprises updating an i-th decoded image, the updating being configured to update a pixel in the i-th decoded image based on a weight of at least one information of the pixel; a weight of any information in the at least one information is positively correlated with an influence degree of the any information on image segmentation; 1≤i≤n-1; determine the ultrasonic image after image segmentation based on images obtained by the n times of iterative target operations; determine information of the biological organ based on the ultrasonic image after image segmentation; a display screen configured to display the ultrasonic image and the information of the biological organ.

2. The ultrasound device of claim 1, wherein, The i-th decoded image comprises a plurality of image channels, and the at least one information of the pixel comprises at least one of each image channel of the pixel and a position of the pixel in the image.

3. The ultrasound device of claim 2, wherein, The at least one information of the pixel comprises the each image channel and the position; For any image channel of the pixel, a value y of the updated image channel is equal to x*a*b; wherein x represents a value of the image channel before updating, a represents a weight of the image channel, and b represents a weight of the position.

4. Ultrasonic apparatus according to any one of claims 1 to 3, characterized in that The i-th decoded image and an (n-i)-th encoded image have the same image channels and the same image size; for the decoding and the updating in the i-th target operation, the updating is further configured to: add the i-th decoded image after updating the pixel based on the weight of the at least one information to the (n-i)-th encoded image to update the i-th decoded image.

5. The ultrasonic apparatus according to any one of claims 1 to 3, characterized by The encoding for any image comprises a first convolution processing and an auxiliary processing performed in sequence; For the first encoding, the auxiliary processing comprises a down-sampling processing; For the i+1-th encoding, the auxiliary processing comprises an addition processing and a down-sampling processing performed in sequence, and the addition processing is configured to add the any image after the first convolution processing to the any image.

6. The ultrasound device of claim 5, wherein, The decoding for any image comprises a second convolution processing and an up-sampling processing; and the processor is further configured to: perform n times of first operations on the ultrasonic image, the first operations comprising third convolution processing and up-sampling processing; performing a second operation on the first operation result image of the n iterations, the second operation comprising a fourth convolution processing and a down-sampling processing; determining the segmented ultrasound image based on the second operation result image of the n iterations and the target operation result image of the n iterations.

7. The ultrasonic apparatus according to any one of claims 1 to 3, characterized by In the n encodings, the number of image channels of the first encoding result image is less than 64.

8. The ultrasonic apparatus according to any one of claims 1 to 3, characterized by, The processor is configured to: input the ultrasound image into a semantic segmentation model; obtain the target operation result image of the n iterations output by the semantic segmentation model.

9. An ultrasonic testing method characterized by, The method for an ultrasound device comprises: acquiring an ultrasound image of a biological organ; encoding the ultrasound image for n iterations, n≥2; wherein in the n iterations of encoding, the size of the convolution kernel in one or more preceding shallow encodings is greater than the size of the convolution kernel in one or more subsequent deep encodings, the number of convolution layers in one or more preceding shallow encodings is less than the number of convolution layers in one or more subsequent deep encodings, and the first convolution processing in one or more preceding shallow encodings uses a hollow convolution kernel, while the first convolution processing in one or more subsequent deep encodings does not use a hollow convolution kernel; performing a target operation on the n iteration encoding result image for n iterations, the target operation comprising decoding; wherein the i-th target operation in the n iteration target operation further comprises: updating the i-th decoding result image, the update being used to update the pixel based on the weight of at least one information of the pixel in the i-th decoding result image; the weight of any information in the at least one information is positively correlated with the influence degree of the any information on image segmentation; 1≤i≤n-1; determining the segmented ultrasound image based on the target operation result image of the n iterations; determining information of the biological organ based on the segmented ultrasound image; displaying the ultrasound image and the information of the biological organ.

Images