Image detection and model training method and device, medium and program product

Abstract

The embodiment of the invention provides an image detection method and device, a computer-aided diagnosis method and device, a model training method and device, a storage medium and a program product, and the image detection method is applied to a server side and comprises the steps that a medical image containing a to-be-detected target part is obtained, the medical image comprises an image slice sequence formed by a plurality of continuous image slices; according to the sequence of the image slices in the image slice sequence and the target part detection result of the previous image slice, carrying out target part tracking detection on the next image slice; and obtaining a target part detection result of the medical image according to a tracking detection result of the image slice. According to the embodiment of the invention, the method achieves the more accurate detection of the target part for the medical image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to an image detection and model training method thereof, a computer-aided diagnosis method, an electronic device, a computer storage medium, and a computer program product. Background Art

[0002] The development of AI (Artificial Intelligence) technology has led to its widespread application in various fields, with image processing being a key area. Furthermore, the use of AI for image processing is gaining increasing attention in the medical field, particularly in the detection and recognition of medical images.

[0003] However, at present, there are still some problems in the application of image processing in the medical field. Taking medical image detection as an example, it is limited by problems such as low contrast and low spatial resolution in imaging technology, making it difficult to accurately detect or identify certain tissues and parts in medical images even with the application of existing AI technology. Taking the human lymph nodes as an example, lymph nodes are widely distributed in the human body and are closely connected with adjacent tissues. Therefore, in the presentation of medical images such as CT, lymph nodes are usually similar in intensity to adjacent tissues and similar in size and shape, making it difficult to clearly distinguish them with existing AI technology. Furthermore, the accuracy of subsequent operations based on this detection or identification, such as auxiliary diagnosis related to lymph nodes, is also low. In addition, other imaging such as nodule imaging and prostate imaging also have the same problem. Summary of the Invention

[0004] In view of this, embodiments of the present application provide an image detection and model training thereof, as well as a computer-aided diagnosis solution to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of an embodiment of the present application, an image detection method is provided, which is applied to a server, and the method includes: obtaining a medical image containing a target part to be detected, wherein the medical image includes an image slice sequence formed by multiple continuous image slices; according to the order of the image slices in the image slice sequence, based on the target part detection result of the previous image slice, performing target part tracking detection on the subsequent image slice; and obtaining the target part detection result of the medical image based on the tracking detection result of the image slice.

[0006] According to a second aspect of an embodiment of the present application, another image detection method is provided, which is applied to a user device, and the method includes: displaying a first human-computer interaction interface, and receiving a medical image input by a user through the first human-computer interaction interface, wherein the medical image contains information of a target part to be detected; sending the medical image to a server, so as to track and detect the target part based on an image slice sequence formed by multiple continuous image slices corresponding to the medical image through an image detection model in the server, and obtain a target part detection result; receiving the target part detection result returned by the server, and displaying the target part detection result through a second human-computer interaction interface.

[0007] According to a third aspect of an embodiment of the present application, a computer-aided diagnosis method is provided, the method comprising: acquiring a three-dimensional medical image comprising a human body part to be detected, wherein the three-dimensional medical image comprises an image slice sequence formed by a plurality of continuous image slices; performing tracking detection of the human body part on a subsequent image slice according to the order of the image slices in the image slice sequence and based on the human body part detection result of the previous image slice; obtaining at least the size and position of the human body part in the three-dimensional medical image according to the tracking detection result of the image slice; and generating an auxiliary diagnosis suggestion for the human body part according to the size and position, as well as the standard size and standard position of the human body part.

[0008] According to a fourth aspect of an embodiment of the present application, a method for training an image detection model is provided, comprising: obtaining a medical image sample containing a target part, wherein the medical image sample can be divided into multiple continuous image sample slices to form an image sample slice sequence, and the target part of each image sample slice has annotation data; inputting the medical image sample into an image detection model to be trained, so that the image detection model performs target part tracking detection on a subsequent image sample slice according to the target part detection result of the previous image sample slice in the order of the image sample slices in the image sample slice sequence; and training the image detection model according to the tracking detection result of the image sample slice.

[0009] According to the fifth aspect of the embodiments of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect, the second aspect, the third aspect or the fourth aspect.

[0010] According to a sixth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect, the second aspect, the third aspect, or the fourth aspect is implemented.

[0011] According to the seventh aspect of the embodiments of the present application, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method described in the first aspect, the second aspect, the third aspect, or the fourth aspect.

[0012] According to the solution provided by the embodiment of the present application, for medical images that can be divided into multiple continuous image slices and form an image slice sequence, when it is necessary to detect the target parts contained therein, especially those parts with similar intensity, size, shape, and other unclear boundaries with adjacent tissues, which are difficult to distinguish, the image slice sequence is regarded as a time series, and a multi-target tracking method is used to achieve accurate detection of the target parts. On the one hand, this tracking method allows the detection results of the previous image slice to be effectively transferred to the next image slice, so that the target part detection for the next image slice has more accurate prior knowledge; on the other hand, the tracking detection method of the embodiment of the present application also performs relatively independent target part detection on the next image slice and obtains relatively independent detection results. Therefore, the final target part detection result determined based on the two parts of the results not only ensures accuracy, but also effectively avoids missed detection and false detection. Compared with the method of performing static detection on multiple image slices separately and then merging the detection results, it effectively avoids false detection of target parts caused by erroneous merging of detection results, as well as missed detection of target parts in some cases.

[0013] On this basis, the accuracy of subsequent possible operations, such as auxiliary diagnosis based on test results, will also be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] Figure 1 A schematic diagram of an exemplary system applicable to the embodiments of the present application;

[0016] Figure 2A Flowchart of the steps of a method for training an image detection model according to an embodiment of the present application;

[0017] Figure 2B for Figure 2AA structural block diagram of a model structure of an image detection model in the illustrated embodiment;

[0018] Figure 2C for Figure 2A A schematic diagram of an attention mask in the illustrated embodiment;

[0019] Figure 2D For Figure 2B Schematic diagram of the training process for training the image detection model shown;

[0020] Figure 3A is a flowchart of the steps of an image detection method according to an embodiment of the present application;

[0021] Figure 3B To adopt Figure 2D Schematic diagram of the process of image detection by the image detection model shown;

[0022] Figure 4A is a flowchart of another image detection method according to an embodiment of the present application;

[0023] Figure 4B for Figure 4A A schematic diagram of a scenario of the illustrated embodiment;

[0024] Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0026] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0027] Figure 1 An exemplary system applicable to the embodiment of the present application is shown. Figure 1 As shown, the system 100 may include a server 102, a communication network 104 and / or one or more user devices 106. Figure 1 The example in the figure is for multiple user devices.

[0028] The server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the server 102 can perform any suitable function. For example, in some embodiments, the server 102 can be used to perform image detection, such as detecting medical images. As an optional example, in some embodiments, the server 102 can obtain a medical image containing a target part to be detected, where the medical image comprises an image slice sequence formed by multiple consecutive image slices; perform target part tracking detection on a subsequent image slice according to the order of the image slices in the image slice sequence and based on the target part detection result of the previous image slice; and obtain the target part detection result of the medical image based on the tracking detection result of the image slice. As an optional example, in some embodiments, the server 102 is provided with an image detection model, and uses this image detection model to perform the above-mentioned image detection. As an optional example, in some embodiments, the server 102 can also train this image detection model. As another example, in some embodiments, the server 102 can interact with the user device 106, for example, receive images such as medical images sent by the user device 106, perform the aforementioned image detection on the image, and then send the detection results to the user device 106.

[0029] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The user device 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud service end 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user device 106 and the cloud service end 102, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of such links.

[0030] The user device 106 may include any one or more user devices suitable for interacting with a user and capable of presenting images and / or other information. In some embodiments, the user device 106 may display a first human-computer interaction interface and receive images input by the user through the first human-computer interaction interface, such as medical images, wherein the medical images contain information about the target part to be detected; send the medical images to a server, such as the aforementioned server 102, to perform the aforementioned medical image detection through the server 102, receive the target part detection results returned by the server 102, and display the target part detection results through a second human-computer interaction interface. In some embodiments, the user device 106 may include any suitable type of device. For example, in some embodiments, the user device 106 may include any appropriate form of medical auxiliary diagnostic equipment, a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of user device.

[0031] Based on the above system, the following describes the solution provided by the embodiment of the present application in the order of training the image detection model and implementing the image detection process. It should be noted that in multiple embodiments of the present application, medical images are used as examples, but those skilled in the art should understand that other 3D image forms similar to medical images, and images that can be cut into a continuous sequence of multiple image slices, can all be applied to the solution of the embodiment of the present application.

[0032] Below, refer to Figure 2A-2D , the training method of the image detection model of the embodiment of the present application is explained.

[0033] First, refer to Figure 2B , which shows a structural block diagram of the model structure of an exemplary image detection model. Figure 2B As can be seen in Figure 1, the exemplary image detection model includes a feature extraction backbone network, an encoder, and a decoder.

[0034] in:

[0035] The feature extraction backbone network is used to extract features from image sample slices corresponding to medical image samples during the training phase to obtain corresponding image features. During the inference phase, it is used to extract features from image slices corresponding to the medical image to be tested to obtain corresponding image features. The medical image samples used in the training phase can be obtained using any appropriate method and can be divided into multiple continuous image sample slices, which form an image sample slice sequence corresponding to the medical image sample. Similarly, during the inference phase, the medical image to be tested can also be divided into multiple continuous image slices to form an image slice sequence. In specific implementations, the feature extraction backbone network can be implemented as any appropriate network capable of extracting image features from medical images or medical image samples. For example, it can be implemented as a CNN (convolutional neural network), and further optionally, as a ResNet-50 network. When a ResNet network is used, a multi-scale feature fusion scheme can optionally be applied to the multiple ResNet blocks included in the ResNet network to leverage the rich contextual information of the 3D medical image to extract richer and more accurate image features.

[0036] The encoder is used to encode the image features and generate the encoded image features as detection query vectors (Detection queries). In one feasible way, the encoder of the embodiment of the present application can be implemented as an encoder based on the Transformer structure, which can also be called a Transformer Encoder. The encoder of the Transformer structure adopts an attention mechanism, which can not only extract features more efficiently, but also effectively capture the long-range dependencies of the data in the input sequence. In the Transformer, the query vector (Query) is a core component of the attention mechanism (Attention Mechanism), which, together with the key vector (Key) and the value vector (Value), forms the basis of attention calculation in the Transformer. Generally speaking, during the encoding and decoding process of the Transformer, the Query will perform a dot product operation with the Key at other positions to calculate the attention score, which can reflect the correlation or importance between the current position and other positions. Through the Query, the Transformer can flexibly focus on different parts of the input sequence and perform the next prediction or encoding task based on the important information of these parts. In an embodiment of the present application, the query vector will be divided into Track queries (tracking query vectors, used for tracking detection) and Detection queries (detection query vectors, used for target detection). Track queries are learnable query vectors that can perform complete tracking modeling for a target. In an embodiment of the present application, they are used to pass the detection results corresponding to the previous slice to the next slice, so that the next slice can track and detect the target part based on the vector; Detection queries are also learnable query vectors that can be used to detect the first appearance of the target. In an embodiment of the present application, they are used to perform relatively independent target part detection on the current slice, that is, target detection. Based on this, in the image detection model of an embodiment of the present application, the encoded image features output by the encoder are used as Detection queries, and subsequently, they can be input into the decoder together with Track queries for processing. In addition, in an embodiment of the present application, the image features input to the encoder have position encoding. In addition to generating the encoded vector, the encoder can also perform initial detection box prediction. In actual applications, the specific encoding structure of the encoder based on the Transformer structure can be set by those skilled in the art according to actual needs. For example, it can include 6 encoding blocks.

[0037] The decoder is used to perform mask attention processing based on Detection queries and the obtained Track queries to obtain target part prediction results. For multiple continuous image slices corresponding to medical images, when processing the first image slice, Track Queries can be empty or a set value; for other image slices except the first image slice, the detection result of the previous image slice in any two adjacent image slices will be passed to the next image slice as the Track Queries of the next image slice. In a feasible way, the decoder in the embodiment of the present application can be implemented as a decoder based on the Transformer structure, also known as TransformerDecoder, to better realize the prediction of target parts in medical images. On this basis, the decoder in the embodiment of the present application adopts a mask attention mechanism, through which the decoder decouples the prediction processing based on Detection Queries and Trackqueries in the decoder, avoids the influence of Trackqueries on Detection queries in the conventional Track queries priority matching method, and realizes more accurate target part tracking and detection, thereby obtaining more accurate prediction results. In the decoder, in addition to including multiple Transformer decoding blocks (exemplarily, 9 may be included), multiple shared FFN (feedforward neural network) detection heads across different decoding blocks are also included. Optionally, a query denoising branch can be introduced into the decoder (by introducing denoising queries and decoding them) to accelerate convergence and improve performance. Track queries are fed autoregressively from the previous slice output embedding of the last Transformer decoding block (before the last FFN detection head). However, it should be clear to those skilled in the art that decoders without query denoising branches are also applicable to the solutions of the embodiments of the present application.

[0038] A training method for the image detection model of the above structure is as follows Figure 2A As shown, it includes the following steps:

[0039] Step S202: Obtain a medical image sample containing the target part.

[0040] The medical image sample can be divided into multiple continuous image sample slices to form an image sample slice sequence, and the target part of each image sample slice has annotated data. In the embodiment of the present application, the target part can be a specific part, such as the prostate, or a part including multiple target objects, such as lymph nodes. For example, the axillary lymph node (part) includes multiple lymph node objects.

[0041] In the embodiments of the present application, whether it is a medical image sample or a medical image, it is in 3D form, that is, a 3D medical image sample or a 3D medical image, including but not limited to: medical CT (computed tomography), medical MRI (magnetic resonance imaging), ultrasonic 3D images, etc. These images can be divided into multiple continuous image slices or image sample slices along the Z axis (the axis perpendicular to the horizontal plane of the image when the image is placed horizontally), thereby forming an image slice sequence or an image sample slice sequence. During the training phase of the model, each image sample slice has annotation data to mark the target part therein to provide a reference for subsequent training.

[0042] It should be noted that the medical image or medical image sample may be an image or sample related to a part of the human body, or may be an image or sample related to other living bodies (such as animals, etc.).

[0043] In an optional manner, the medical image samples in this step may be medical image samples obtained after preprocessing the original medical images. Preprocessing can make it easier for subsequent models to process them, thereby improving the efficiency of model training. For example, CT (such as lymph node CT) can be standardized to a resolution of 0.8×0.8×2mm, and its intensity value can be clipped to a range of [-200, 300]. Alternatively, CT can be standardized to a resolution of 0.8×0.8×2mm, but the intensity value can be clipped to a range of [-1500, 500]. For another example, MRI (such as prostate MRI) can be standardized to a resolution of 0.5×0.5×3mm, and the intensity value can be clipped to a range of [0, 800].

[0044] Medical image samples may be divided into multiple consecutive image sample slices. However, during the training phase, adjacent slice pairs can be randomly sampled from the image sample slices corresponding to different medical image samples for model training. Optionally, consistent enhancements can be applied to both slices in each pair, such as random scaling, cropping, rotation, intensity scaling, and gamma adjustment.

[0045] Step S204: inputting the medical image sample into the image detection model to be trained, so that the image detection model performs target part tracking detection on the next image sample slice according to the order of the image sample slices in the image sample slice sequence and the target part detection result of the previous image sample slice.

[0046] In a related approach, each slice in the image sample slice sequence is treated as a static image. After performing target site detection on each slice, the multiple detection results corresponding to the multiple slices are merged to obtain the final detection result. However, unlike conventional detection methods such as pedestrian or vehicle detection, pedestrians or vehicles have clear boundaries and visual features, making them easy to detect and associate. However, for living parts such as human lymph nodes, their intensity, size, and shape are difficult to distinguish from the surrounding soft tissue. When these are treated as static images, merging multiple detection results can easily lead to problems such as mismerging. That is, two originally adjacent lymph nodes may merge to become a single large lymph node, resulting in a false detection. At the same time, missed detection problems may also occur. For example, as the slice sequence changes, a smaller lymph node becomes smaller and disappears. Using this related approach may result in missed detection. To avoid the above problems, the embodiments of the present application take into account the correlation and consistency between multiple slices in the slice sequence and propose a method for tracking detection of medical images. That is, the detection result of the previous slice is used as the tracking basis, and the target part is tracked and detected on the next slice. The tracking detection includes both tracking the target part and detecting new objects in the target part, thereby avoiding both false detection and missed detection.

[0047] In the training phase of the image detection model, in an optional manner, target part tracking detection of the subsequent image sample slice can be implemented as follows: performing first target part detection on the previous image sample slice to obtain the corresponding first target part detection result (the first target part detection of the first slice can be implemented based on target detection, and the first target part detection of the non-first slice can be implemented based on target detection and tracking detection); performing image feature extraction on the subsequent image sample slice to obtain the corresponding image feature; based on the first target part detection result and the image feature, performing tracking detection and target detection on the subsequent image sample slice respectively to obtain the corresponding second target part detection result and third target part detection result. In this way, the detection result of the previous slice (such as the detected lymph node) is tracked, and the subsequent slice can be detected separately to avoid missing the new object in the target part (such as a new lymph node) due to the change of the slice.

[0048] In one optional method, performing tracking detection and target detection on a subsequent image sample slice based on the first target part detection result and the image feature includes: splicing the first target part detection result and the image feature to obtain a spliced feature; and performing tracking detection and target detection on the subsequent image sample slice based on the spliced feature. By splicing the two features, the obtained features are richer and more comprehensive, providing a more accurate feature basis for subsequent tracking detection and target detection.

[0049] In adopting Figure 2B In the image detection model shown in the figure, the image features are obtained by processing the subsequent image sample slice through the feature extraction backbone network and the encoder, and the spliced features are sent to the decoder for tracking detection and target detection.

[0050] In a related method, after the two parts of the feature are spliced together, they will be sent to the decoder for simultaneous prediction and joint training. However, because Track queries have a higher matching priority than Detection queries, when Track queries and Detection queries predict the same object, such as a lymph node, Track queries have a higher matching priority for the recognition propagation of the object, which causes Detection queries to be penalized for errors during training, reducing the accuracy of Detection queries. To solve this problem, in an embodiment of the present application, when tracking detection and target detection are performed on the subsequent image sample slice based on the splicing features, tracking detection and target detection based on mask attention processing are performed on the subsequent image sample slice based on the splicing features, so as to ensure the independence of the Detection queries predictions and improve the accuracy of the entire detection.

[0051] In the specific implementation, the Transformer-based decoder adds an attention mask M=[m ij ]N×N, which can also be understood as multiplying the original SELF-ATTENTION matrix by the attention mask M. Where N=N track +N det . N track Indicates the number of Track queries, N det Indicates the number of Detection queries. In M, make the first N track ×N track The rows and columns of represent the tracking part, and the others represent the detection part. ij =1 means that the i-th query cannot see the j-th query; otherwise, m ij = 0. Then, the above attention mask M can be expressed as:

[0052]

[0053] For ease of understanding, refer to Figure 2C , where the matrix on the right shows an example of an attention mask, in which N track is 2, N detis 4, which forms a 6×6 attention mask matrix M. In this matrix, the upper left corner N track ×N track Part is 0 (blank square); similarly, the lower right corner N det ×N det In order to avoid the influence of Track queries on Detection queries, the other parts of the attention mask matrix M are all 1 (vertical squares). Figure 2C In the upper right corner N of the attention mask matrix M track ×N det The part is also 0. This is because the inventors found that the data of the attention mask matrix M is symmetrical about the diagonal. In actual calculations, the calculation can be performed using the data on the diagonal and below. Therefore, in order to improve the calculation efficiency and reduce the burden of data generation and processing of the attention mask matrix M, the upper right corner N of the attention mask matrix M is set to track ×N det Some of them are set to 0. However, those skilled in the art should understand that setting some of them to 1 is also applicable to the solution of the embodiment of the present application. Figure 2C The left side shows that in this way, on the one hand, the processing of Detection queries is not affected by Track queries, preventing information leakage from Track queries, thereby preventing Detection queries from being affected by Track queries; on the other hand, Track queries can access Detection queries to perform context alignment within the current slice. Thus, the decoder can obtain tracking detection and target detection based on Track queries and Detection queries for the target part (in the embodiment of the present application, these two parts of detection are referred to as tracking detection), and obtain corresponding detection results. This masked attention processing can be applied to each decoding block of the decoder.

[0054] Step S206: training the image detection model according to the tracking detection results of the image sample slices.

[0055] For a particular image sample slice, after obtaining its corresponding tracking detection result, a loss value is calculated using a preset loss function based on the detection result, the labeled data in the training sample, and the detection result of the previous image sample slice. The image detection model is then trained based on the loss value.

[0056] Because the embodiment of the present application adopts a tracking detection method and decouples tracking detection and target detection, in an optional method, when the detection results obtained above include three parts, this step can be implemented as: training the image detection model based on the first target part detection result, the second target part detection result and the third target part detection result.

[0057] Further optionally, training the image detection model based on the first target part detection result, the second target part detection result, and the third target part detection result may include: using a first loss function to perform loss calculation based on the first target part detection result and the second target part detection result to obtain a first loss calculation result; using a second loss function to perform loss calculation based on the second target part detection result and the target part annotation data to obtain a second loss calculation result; using a third loss function to perform loss calculation based on the third target part detection result and the target part annotation data to obtain a third loss calculation result; training the image detection model based on the first loss calculation result, the second loss calculation result, and the third loss calculation result. The first loss function can accurately measure the correlation between the two slices before and after; the second loss function and the third loss function can accurately measure the accuracy of the decoupled tracking detection and target detection. This also ensures better model training results.

[0058] Since the visual features of some target parts (such as the aforementioned lymph nodes, lung nodules, prostate, etc.) are mostly fuzzy and closely clustered together, it poses a challenge to the association between slices. For this reason, the similarity loss is introduced in the embodiment of this application. That is, the first loss function, which encourages Track queries belonging to the same instance across slices to be close in the latent space, while Track queries from different instances are pushed apart. Specifically, given adjacent slices x z-1 and x z , x z-1 The corresponding Track queries are expressed as x z The corresponding Track queries are expressed as The MLP layer φ (which can be used as a detection head) first projects these Track queries into a new latent space. Then, it uses pairwise dot products to generate similarity logits, which are then normalized using a sigmoid activation function to obtain the affinity matrix. Calculate the binary cross entropy loss, where 1 represents the same instance and 0 represents a different instance. For example, the first loss function can be expressed as:

[0059]

[0060] in, represents the binary cross entropy loss function.

[0061] It should be noted that, in the inference phase of the model, although this similarity measure is no longer used as a loss function, it can still be used as an auxiliary criterion for determining the instance association of the detected target parts during the inference process.

[0062] For tracking detection, in the embodiment of the present application, the instance correlation classification loss (where 1 represents the same instance and 0 represents different instances) is applied together with the associated detection box regression loss (based on the target part annotation data) as the second loss function to guide the association within the slice. For example, the second loss function can be expressed as:

[0063]

[0064] in, represents the second loss function; represents the instance relevance classification loss function, which can be implemented as a focal loss function; λ represents the detection box regression loss function, which can be implemented as a combination of the L1 loss function and the GIoU loss function. cls express The corresponding weight factor, λ box express Corresponding weight factors, in practical applications, these two weight factors can be appropriately set by those skilled in the art according to actual needs, such as setting λ cls is 1.0, λ box for 2.0, etc.

[0065] For target detection, since it has been decoupled from tracking detection, the solution of the embodiment of the present application determines the loss by calculating the pairwise matching cost between the decoder prediction output, i.e., the third target part detection result, and all the true value boxes (target part annotation data) in the slice. It applies bipartite graph matching together with the associated detection box regression loss (based on target part annotation data) as the third loss function. Exemplarily, the third loss function can be expressed as:

[0066]

[0067] in, represents the third loss function; represents the bipartite graph matching loss function; λ represents the detection box regression loss function, which can be implemented as a combination of the L1 loss function and the GIoU loss function. cls express The corresponding weight factor, λ box express In practical applications, these two weight factors can be appropriately set by those skilled in the art according to actual needs, such as setting λ cls is 1.0, λ box for 2.0, etc.

[0068] From the above, the total loss function used by the image detection model can be expressed as:

[0069]

[0070] After calculating the loss value using the above total loss function, the image detection model is trained based on this loss value. The above process is iterated until the training ends, such as reaching a preset number of training times or the loss value reaching a preset threshold.

[0071] The following, combined Figure 2D , the training process of the above image detection model is illustrated. It should be noted that, for the convenience of representation, Figure 2D The detection head portion of the model is not shown in the figure, but those skilled in the art can determine the position and function of the detection head in the model based on the above description. In addition, for the convenience of description, Figure 2D The processing of two adjacent image sample slices is shown in FIG. 1 , but those skilled in the art should understand that in actual applications, the model structure is as follows: Figure 2D shown on the right side of the middle, rather than including both Figure 2D Two parts on the left and right sides.

[0072] Depend on Figure 2D As can be seen from the figure, for two adjacent image sample slices in the image sample slice sequence of a medical image sample X, the first slice ( Figure 2D Indicated by x z-1 ), after being processed by the feature extraction backbone network, encoder and decoder in sequence, the corresponding detection result (first target part detection result) is obtained. In this example, as mentioned above, the feature extraction backbone network adopts the ResNet network; the encoder adopts the encoder based on the Transformer structure ( Figure 2D In the figure, it is shown as "Transformer Encoder"). For example, the encoder may include 6 BLOCKs (encoding blocks); the decoder adopts a decoder based on the Transformer structure ( Figure 2DIn the figure, it is shown as “Transformer Decoder”, and the decoder adopts the Mask Attention mechanism. For example, the Transformer Decoder may include 9 BLOCKs (decoding blocks), and Mask Attention is used in each BLOCK for calculation. If the slice x z-1 If it is the first slice in the image sample slice sequence of the medical image sample X, then the Track queries (tracking query, used for tracking detection) in the input Transformer Decoder can be empty; otherwise, it is x z-1 The detection result corresponding to the previous slice (the result of the decoding process performed by the Transformer Decoder based on Track queries and Detection queries (detection queries, used for target detection)). This example does not Figure 2D This part is clearly distinguished in the .

[0073] After being processed by Transformer Decoder, the prediction results include multiple, but can be passed to the next slice x z The valid detection results are those that are greater than the classification threshold (the classification threshold can be set by those skilled in the art according to actual needs, such as 0.8, etc.), slice x z-1 The corresponding valid test results are as follows Figure 2D As shown in the dotted box above the Transformer Decoder in the figure, the color blocks of different colors in the dotted box correspond to the detection boxes of the corresponding colors in the medical image above it.

[0074] Then, for x z-1 The last slice is processed, and the last slice is processed in Figure 2D Indicated by x z .Depend on Figure 2D As can be seen, slice x z First, after feature extraction processing by the feature extraction backbone network, we get x z The corresponding image features; then, the image features are input into the Transformer Encoder for encoding processing to generate the encoded image features (such as Figure 2D The encoded image features are used as Detection queries. At the same time, the image features from slice x are z-1 The features passed (i.e. slice x z-1The detection results of the second target part are used as Track queries, and the two are combined. In this example, the combination is implemented as splicing to form a splicing feature. Then, the splicing feature is input into the Transformer Decoder. After the Transformer Decoder performs attention calculation processing based on Mask Attention, the detection results corresponding to the Track queries (the second target part detection results) and the detection results corresponding to the Detection queries (the third target part detection results) are generated respectively.

[0075] On the basis of obtaining the three-part detection results, we can further use the preset loss function and combine it with the target part annotation data in the medical image sample X to obtain the corresponding loss value. Figure 2D As shown in , the loss functions preset in this example include: a first loss function for guiding the inter-slice tracking association training Second loss function for guiding intra-slice tracking detection training and for instructing object detection training within a slice These three loss functions form the total loss function for model training. After obtaining the loss value through these three loss functions, the parameters of the image detection model can be adjusted based on the loss value, that is, training can be performed.

[0076] As mentioned above, the training process is iterated until the training termination condition is reached.

[0077] It can be seen that through the model training method of this embodiment, on the one hand, the trained image detection model can use tracking to detect target parts to achieve more accurate and effective detection effects; on the other hand, the Mask Attention mechanism is used to decouple the processing of Track queries and Detection queries by the Transformer Decoder, thereby avoiding the influence of Track queries on the processing of Detection queries, thereby avoiding the occurrence of false detection or missed detection; on the other hand, through the three-part loss function, especially the first loss function used to calculate the similarity loss between slices and guide the training of tracking association between slices, the high accuracy of the detection results is guaranteed, and the cohesive association of target parts between consecutive slices is improved.

[0078] The following, combined Figure 3A and Figure 3B , the image detection method provided in the embodiment of the present application is described. In one feasible manner, the image detection method can be performed based on the image detection model trained as described above.

[0079] Reference Figure 3A , which shows the steps of an image detection method according to an embodiment of the present application, the image detection method includes the following steps:

[0080] Step S302: Acquire a medical image containing the target part to be detected.

[0081] As previously mentioned, the medical images in this embodiment may be 3D images containing the target body part to be detected (such as lymph nodes, lung nodules, prostate, etc.), including but not limited to CT, MRI, etc. These medical images can be divided into multiple continuous image slices along the Z axis to form an image slice sequence.

[0082] Step S304: performing target part tracking detection on the next image slice according to the order of the image slices in the image slice sequence and the target part detection result of the previous image slice.

[0083] When the tracking detection method is adopted, for any two adjacent slices in the image slice sequence, the target part detection result of the previous slice can be obtained first, and used as the tracking basis for the next slice. In the specific implementation, the target part detection result of the previous slice obtained will be passed to the processing process of the next slice. For the next slice, its image features are obtained, and then the target part detection result of the previous slice and the image features of the next slice are combined to perform detection based on the combined features. For example, in one feasible method, according to the target part detection result of the previous image slice, performing target part tracking detection on the next image slice may include: obtaining the target part detection result of the previous image slice and obtaining the image features of the next image slice; splicing the target part detection result and the image features to obtain splicing features; and performing target part tracking detection on the next image slice based on the splicing features.

[0084] In the detection process of the second slice, the target part detection results of the first slice can be used as the tracking query vector Track queries, and the image features of the second slice can be used as the detection query vector Detection queries. The two are spliced together to obtain the splicing features, and then the target part tracking detection of the second slice is performed based on the splicing features.

[0085] Because the matching priority of Track queries is higher than that of Detection queries in related methods, detection based on Detection queries is not accurate enough. To avoid this problem, in one feasible method, after obtaining the spliced features, the target part prediction based on mask attention processing can be performed on the spliced features, so as to perform target part tracking detection on the subsequent image slice based on the prediction results. Through mask attention, the processing of Track queries and Detection queries is decoupled, so that the processing of Track queries does not affect Detection queries, ensuring the relative independence of Detection queries processing, thereby making detection based on Detection queries more accurate, and therefore making the overall detection of target parts more accurate.

[0086] When implementing this step with the image detection model trained by the aforementioned training method, when detecting the latter slice of two adjacent image slices, first input the slice into the feature extraction backbone network of the image detection model for image feature extraction to obtain image features; input the image features into the encoder for encoding processing to obtain encoded image features, and the encoded image features will be used as Detection queries; at the same time, obtain the target part detection results of the previous slice, and use the target part detection results as Track queries; splice the Track queries and Detection queries to obtain spliced features; and then, the spliced features are input into the decoder for mask attention-based processing to achieve target part tracking detection of the latter slice and obtain detection results.

[0087] Step S306: Obtaining a target part detection result of the medical image according to the tracking detection result of the image slice.

[0088] Since the tracking detection is performed based on multiple continuous image slice sequences, the detection result of the last image slice in the image slice sequence is the target part detection result of the medical image corresponding to the image slice sequence.

[0089] For ease of explanation, the following Figure 3B , the above-mentioned medical image detection process is exemplified.

[0090] In this example, the image detection model used follows the Figure 2D The same structure as the model shown in Figure 2D Similarly, for ease of explanation, Figure 3BThe processing of two adjacent slices is also shown, but it does not mean that the model contains two parts of the structure at the same time. In actual application, the model only contains the structure of one side.

[0091] Depend on Figure 3B As can be seen in the figure, for two adjacent image slices x z-1 and x z , where x z-1 After being processed by the feature extraction backbone network, Transformer Encoder and Transformer Decoder, we get x z-1 The corresponding target part detection results, such as Figure 3B The box on the left side of the box without an "X". z-1 After the target part is detected, the image detection model is based on x z-1 The test results of x z Conduct tracking detection, such as Figure 3B As shown on the right side of the figure. z The image features are extracted by the feature extraction backbone network to obtain the image features. Then, the image features are sent to the Transformer Encoder for encoding processing to obtain the encoded image features. After obtaining the encoded image features, Figure 3B As shown in z-1 The processed detection results are used as Track queries and are spliced with the encoded image features as Detection queries to form spliced features. Then, the spliced features are sent to the Transformer Decoder. The Transformer Decoder performs masked attention calculations on Track queries and Detection queries based on the masked attention mechanism to obtain the detection results of tracking detection and the detection results of target detection based on the two parts respectively, and then obtains x based on these two parts of the detection results. z The corresponding target part detection results. Figure 3B As can be seen, after tracking detection based on Track queries, x z-1 One of the objects in the target part detection result no longer exists (maybe a false detection), while the target detection based on Detection queries found a new object. By using a detection head such as MLP to classify the two parts of the detection results, it can be determined that x z The corresponding target part detection results, such as Figure 3B As shown in the non-"X" box on the right side.

[0092] Through the above process, effective detection of the target part in the medical image can be achieved. After obtaining the test results, the user may also hope to obtain some auxiliary suggestions based on the test results. Based on this, in a feasible way, after obtaining the target part detection results of the medical image, it is also possible to generate auxiliary diagnosis suggestions for the target part based on the comparison results of the target part detection results and the imaging standards corresponding to the target part. Among them, the imaging standard is related to actual needs. Taking lymph nodes as an example, it may be the conventional position, size, etc. of the lymph nodes in a certain part. In this embodiment, there is no specific restriction on the imaging standard. Auxiliary diagnosis suggestions can be used to assist rather than determine the diagnosis. For example, a specific numerical value of the lymph node size, the result of comparing the numerical value with the standard numerical value, the positional relationship between the lymph node and the adjacent tissue or blood vessels, etc. are given.

[0093] Exemplarily, when the above process is applied to an auxiliary diagnosis scenario, it can become a computer-aided diagnosis method, which may include: obtaining a three-dimensional medical image (such as CT, MRI, etc.) containing a human body part to be detected (including but not limited to lymph nodes, lung nodules, prostate, etc.), wherein the three-dimensional medical image includes an image slice sequence formed by multiple continuous image slices (which can be obtained by cutting along the Z axis of the three-dimensional medical image); according to the order of the image slices in the image slice sequence, based on the human body part detection result of the previous image slice, tracking and detecting the human body part of the subsequent image slice; based on the tracking detection result of the image slice, at least obtaining the size and position of the human body part in the three-dimensional medical image; based on the size and position, as well as the standard size and standard position of the human body part, generating an auxiliary diagnosis suggestion for the human body part.

[0094] Under normal circumstances, human tissues and parts have relatively normal size and location ranges. If they significantly exceed or fall below these ranges, it may indicate a pathological condition. Based on this, using the standard size and location of human parts as a measurement standard can generate auxiliary diagnostic recommendations that are more practical and valuable.

[0095] It can be seen that through this embodiment, for medical images that can be divided into multiple continuous image slices to form an image slice sequence, when it is necessary to detect the target parts contained therein, especially those parts with similar intensity, size, shape, and other unclear boundaries with adjacent tissues, which are difficult to distinguish, the image slice sequence is regarded as a time series, and a multi-target tracking method is used to achieve accurate detection of the target parts. On the one hand, this tracking method allows the detection results of the previous image slice to be effectively transferred to the next image slice, so that the target part detection for the next image slice has more accurate prior knowledge; on the other hand, the tracking detection method in the embodiment of the present application will also perform relatively independent target detection on the next image slice and obtain relatively independent detection results. Therefore, the final target part detection result determined based on the two parts of the results not only ensures accuracy, but also effectively avoids missed detection and false detection. Compared with the method of performing static detection on multiple image slices separately and then merging the detection results, it effectively avoids false detection and missed detection of target parts caused by erroneous merging of detection results.

[0096] On this basis, the accuracy of subsequent possible operations, such as auxiliary diagnosis based on test results, will also be effectively improved.

[0097] Below, the above-mentioned medical imaging detection is applied to four different lymph node image data sets and compared with other methods in the related art to more clearly demonstrate the effect achieved by the image detection solution provided in the embodiment of the present application, as shown in Table 1 below.

[0098] Table 1

[0099]

[0100] In Table 1, the “Detection Method” column shows several methods used for lymph node detection, and the scheme of the embodiment of the present application is schematically referred to as “this scheme”. “NIH-LN”, “HN-LN”, “Eso-LN” and “Pan-LN” are the names of four data sets, respectively, and “Mean” is the average value of the effect achieved by the detection method on the four data sets. “AS” represents the average sensitivity of each patient above 1, 2, 4 and 8 FPs (focal lesions); “AP” represents the average accuracy at the 0.1IoU threshold in 3D images. Among them, nnUNet is a 3D segmentation, so there is only one FP point and no AP metric. In Table 1, the data with the best effect are marked in bold, and the data with the second best effect are underlined.

[0101] As shown in Table 1, the solution of the embodiment of the present application outperformed all other methods on the four datasets, outperforming the closest detection method LN-DETR (the latest 2.5D lymph node detector) by 2.7% in terms of average AS (62.66% vs. 59.96%) and average AP (52.68% vs. 50.31%).

[0102] The solution of the embodiment of the present application is then applied to a dataset of lung nodule images and a dataset of prostate images, and its performance is shown in Table 2 below.

[0103] Table 2

[0104]

[0105] In Table 1, the “Detection Method” column lists several detection methods used, and the scheme of the embodiment of the present application is illustrated as “this scheme”. “LIDC-IDRI” is a data set of CT images of lung nodules, and “ProstateX” is a data set of MRI images of prostate. “AS” represents the average sensitivity of each patient above 1, 2, 4 and 8 FPs (focal lesions); “AP” represents the average accuracy at the 0.1IoU threshold in 3D images. Among them, nnUNet is a 3D segmentation, so there is only one FP point and no AP metric. In Table 2, the data with the best effect are marked in bold, and the data with the second best effect are underlined.

[0106] It can be seen that the solution of the embodiment of the present application achieved the highest AS and AP among all the compared methods on the two tasks. On the LIDC-IDRI dataset, the solution of the embodiment of the present application exceeded the second highest LN-DETR by 1.22% in AS and exceeded 1.33% in AP. On the ProstateX dataset, compared with other detection methods, the solution of the embodiment of the present application also achieved higher performance with a larger accuracy gap. For example, it exceeded the closest Mask2Former by 3.95% in AS and exceeded 5.60% in AP. It should be noted that ProstateX is an MRI imaging dataset, and lesions in T2 scans usually show subtle intensity changes, which are more difficult to detect. As can be seen from Table 2 above, the solution of the embodiment of the present application demonstrated stronger performance on this dataset, further verifying the effectiveness and accuracy of the solution of the embodiment of the present application.

[0107] Next, the image detection method of the embodiment of the present application is described from the perspective of the user equipment. Figure 4A As shown, the following steps are included:

[0108] Step S402: displaying a first human-computer interaction interface, and receiving a medical image input by a user through the first human-computer interaction interface.

[0109] In order to facilitate users to use the image detection model, a human-computer interaction interface is provided on the user device side, through which users can upload medical images to be detected for processing by the server.

[0110] The medical image contains information about the target part to be detected. As mentioned above, the target part can be any appropriate part of the human body, including but not limited to lymph nodes, lung nodules, prostate, etc.

[0111] Step S404: sending the medical image to the server, receiving the target part detection result returned by the server based on target part tracking detection of the medical image, and displaying the target part detection result through the second human-computer interaction interface.

[0112] There is a data interaction channel between the user device and the server. The user device can upload medical images to the server through this channel. After the server performs target part tracking detection on the medical image and obtains the target part detection result, it returns the result to the user device for the user device to display to the user.

[0113] For example, the server can use the aforementioned image detection model to perform tracking detection of the target part based on an image slice sequence formed by multiple continuous image slices corresponding to the medical image, and obtain a target part detection result.

[0114] Specifically, for example, after a user device sends a medical image to a server, the server may first segment the medical image into multiple consecutive image slices to form an image slice sequence. Subsequently, the image detection model on the server may perform target part tracking detection based on the image slices in the image slice sequence and obtain detection results. The specific implementation of target part tracking detection in medical images using the image detection model on the server can be found in the description of the aforementioned embodiments and will not be repeated here.

[0115] After the server generates the target part detection results of the medical image, it will return them to the user device.

[0116] After receiving the target part detection results for the medical image from the server, the user device can display them to the user through a human-computer interaction interface, i.e., a second human-computer interaction interface. The target part detection results include, but are not limited to, the size, position, and relationship to adjacent tissues of the target part and / or objects within the target part.

[0117] On this basis, optionally, the image detection method of this embodiment may further include the following step S406.

[0118] Step S406: Obtain the imaging standard corresponding to the target part; and generate auxiliary diagnosis suggestions for the target part based on the comparison result between the target part detection result and the imaging standard.

[0119] The imaging standard corresponding to the target part can be obtained by the user device using any appropriate method, such as from local storage space or from a server database. This imaging standard indicates the normal shape, size, position, and other information of the target part. Based on this imaging standard, the difference between the value indicated by the target part detection result and the imaging standard can be determined to provide auxiliary diagnosis suggestions.

[0120] The following, combined Figure 4B , the above process is illustrated by way of example.

[0121] like Figure 4B As shown in , the first human-computer interaction interface of the user device is provided with a setting for the user to upload medical images. Figure 4B The figure shows an image input box where users can upload medical images by dragging and dropping, and a "select image" button for users to select medical images by browsing files.

[0122] In this example, it is assumed that the user drags the medical image to be detected (in this example, it is a CT image of the axillary lymph nodes) into the image input box by dragging. At the same time, the first human-computer interaction interface is also provided with a "Detect" button that triggers the uploading of medical images and the detection of the target part through the server. When the button is clicked, the medical image in the image input box is uploaded to the server. After receiving the medical image, the server first divides it and forms an image slice sequence. Then, the image detection model deployed in the server is called to track and detect it according to the slice order in the image slice sequence, and obtain the final target part detection result. Afterwards, the server returns the target part detection result to the user device. In this example, the target part detection result is set to include the position and size of multiple lymph nodes in the armpit.

[0123] After receiving the target part detection result, the user device displays it through the second human-computer interaction interface. In this example, the target part detection result is displayed together with the medical image so that the user can view it in conjunction with the text and picture.

[0124] Optionally, as indicated by the dotted arrow in the figure, the user device can also obtain an imaging standard corresponding to axillary lymph nodes (in this example, the imaging standard is obtained from the server), such as the size and location of normal axillary lymph nodes. The user device can then compare the locations and sizes of multiple detected axillary lymph nodes with the imaging standard, providing the comparison results and corresponding auxiliary suggestions. For example, the lymph node at location XXX exceeds the standard size by 10%, and so on.

[0125] This embodiment facilitates the use and interaction of the image detection model by users, reduces the complexity and burden of medical image processing operations for users, improves the efficiency of obtaining target part detection results, and enhances the user experience.

[0126] In addition, refer to Figure 5 According to the present application, an electronic device is also provided. The specific embodiments of the present application do not limit the specific implementation of the electronic device.

[0127] like Figure 5 As shown, the electronic device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0128] in:

[0129] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0130] The communication interface 504 is used to communicate with other electronic devices or servers.

[0131] The processor 502 is configured to execute the program 510 , and specifically may execute the relevant steps of any one of the above-mentioned method embodiments.

[0132] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0133] Processor 502 may be a CPU, a Graphics Processing Unit (GPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0134] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0135] The program 510 may include multiple computer instructions. Specifically, the program 510 may enable the processor 502 to execute operations corresponding to the method described in any of the aforementioned method embodiments through the multiple computer instructions.

[0136] The specific implementation of each step in program 510 can refer to the corresponding description of the corresponding steps and units in the above-mentioned method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working process of the above-mentioned devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, and will not be repeated here.

[0137] The present application also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the aforementioned method embodiments. The computer storage medium includes, but is not limited to, a compact disc read-only memory (CD-ROM), random access memory (RAM), a floppy disk, a hard disk, or a magneto-optical disk.

[0138] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any one of the above-mentioned multiple method embodiments.

[0139] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, user medical images, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0140] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0141] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded via a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., random access memory (RAM), read-only memory (ROM), flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown here.

[0142] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of the embodiments of this application.

[0143] The above implementation methods are only used to illustrate the embodiments of the present application, and are not intended to limit the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.

Claims

1. An image detection method, applied to a server, comprising: Acquiring a medical image containing a target part to be detected, wherein the medical image includes an image slice sequence formed by a plurality of continuous image slices; According to the order of the image slices in the image slice sequence, and based on the target part detection result of the previous image slice, the target part tracking detection is performed on the next image slice; According to the tracking detection result of the image slice, the target part detection result of the medical image is obtained.

2. The method according to claim 1, wherein The target part tracking detection is performed on the next image slice according to the target part detection result of the previous image slice, including: Obtaining the target part detection result of the previous image slice and obtaining the image features of the next image slice; Splicing the target part detection result and the image feature to obtain a splicing feature; Based on the stitching features, target part tracking and detection is performed on the subsequent image slice.

3. The method according to claim 2, wherein: The target part tracking detection is performed on the subsequent image slice based on the splicing feature, including: The target part prediction based on mask attention processing is performed on the splicing features to perform target part tracking detection on the subsequent image slice according to the prediction result.

4. The method according to claim 2 or 3, wherein: The step of splicing the target part detection result and the image feature to obtain a splicing feature includes: using the target part detection result as a tracking query vector and the image feature as a detection query vector to perform splicing to obtain a splicing feature; The target part tracking detection is performed on the next image slice based on the splicing feature, including: inputting the splicing feature into a decoder, and performing target part tracking detection on the next image slice through the decoder.

5. An image detection method, applied to a user device, comprising: Displaying a first human-computer interaction interface and receiving a medical image input by a user through the first human-computer interaction interface, wherein the medical image includes information of a target part to be detected; The medical image is sent to a server, a target part detection result returned by the server based on target part tracking detection of the medical image is received, and the target part detection result is displayed through a second human-computer interaction interface.

6. The method according to claim 5, wherein: The method further comprises: Acquiring an image standard corresponding to the target part; Based on the comparison result of the target part detection result and the imaging standard, an auxiliary diagnosis suggestion for the target part is generated.

7. A computer-aided diagnosis method, comprising: Acquire a three-dimensional medical image containing a human body part to be detected, wherein the three-dimensional medical image includes an image slice sequence formed by a plurality of continuous image slices; According to the order of the image slices in the image slice sequence, and based on the human body part detection result of the previous image slice, tracking detection of the human body part is performed on the next image slice; obtaining at least the size and position of the human body part in the three-dimensional medical image according to the tracking detection result of the image slice; An auxiliary diagnosis suggestion for the human body part is generated based on the size and position, as well as the standard size and standard position of the human body part.

8. A method for training an image detection model, comprising: Obtaining a medical image sample containing a target part, wherein the medical image sample can be divided into a plurality of continuous image sample slices to form an image sample slice sequence, and the target part of each image sample slice has annotation data; Inputting the medical image sample into the image detection model to be trained, so that the image detection model performs target part tracking detection on the subsequent image sample slice according to the target part detection result of the previous image sample slice in the order of the image sample slices in the image sample slice sequence; The image detection model is trained according to the tracking detection results of the image sample slices.

9. The method according to claim 8, wherein The target part tracking detection of the subsequent image sample slice according to the target part detection result of the previous image sample slice includes: performing a first target part detection on the previous image sample slice to obtain a corresponding first target part detection result; performing image feature extraction on the subsequent image sample slice to obtain a corresponding image feature; and performing tracking detection and target detection on the subsequent image sample slice based on the first target part detection result and the image feature to obtain a corresponding second target part detection result and a third target part detection result; The training of the image detection model according to the tracking detection result of the image sample slice includes: training the image detection model according to the first target part detection result, the second target part detection result and the third target part detection result.

10. The method according to claim 9, wherein: The step of performing tracking detection and target detection on the next image sample slice based on the first target part detection result and the image feature includes: splicing the first target part detection result and the image feature to obtain a splicing feature; Based on the stitching features, tracking detection and target detection are performed on the next image sample slice.

11. The method according to claim 10, wherein: The step of performing tracking detection and target detection on the next image sample slice based on the splicing features includes: Based on the splicing features, tracking detection and target detection based on mask attention processing are performed on the subsequent image sample slices.

12. The method according to any one of claims 9 to 11, wherein: The training of the image detection model according to the first target part detection result, the second target part detection result, and the third target part detection result includes: performing loss calculation using a first loss function according to the first target part detection result and the second target part detection result to obtain a first loss calculation result; performing loss calculation using a second loss function based on the second target part detection result and the target part annotation data to obtain a second loss calculation result; performing loss calculation using a third loss function according to the third target part detection result and the target part annotation data to obtain a third loss calculation result; The image detection model is trained according to the first loss calculation result, the second loss calculation result, and the third loss calculation result.

13. An electronic device comprising: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, where the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 12.

14. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer program product comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method according to any one of claims 1 to 12.

Images