Device and computer-readable storage medium for predicting prostate puncture tendency

Abstract

The present application discloses a device and a computer-readable storage medium for predicting the tendency of prostate puncture. The device includes: a processor; and a memory, in which program instructions for predicting the tendency of prostate puncture are stored. When the program instructions are executed by the processor, the device implements the following operations: obtaining the basic information of the subject and the text information of the prostate image; inputting the basic information and the text information of the prostate image into a natural language processing model to perform a first prostate puncture tendency prediction to obtain a first prediction result; obtaining the prostate image of the subject; inputting the prostate image of the subject into an image processing model to perform a second prostate puncture tendency prediction to obtain a second prediction result; and obtaining a final prediction result of predicting the tendency of prostate puncture based on the first prediction result and the second prediction result. Using the scheme of the present application, the accuracy of predicting the tendency of prostate puncture can be improved.

Description

Technical Field

[0001] The present application generally relates to the field of artificial intelligence technology. More specifically, the present application relates to a device and a computer-readable storage medium for predicting prostate puncture tendency. Background Art

[0002] Prostate biopsy is the gold standard for identifying prostate cancer, but it is an invasive procedure that brings discomfort to patients and has corresponding complications, sometimes even severe bacteremia. In addition, the clinical indications for prostate puncture are relatively broad. How to avoid unnecessary prostate puncture surgery is a major challenge in clinical practice and a focus of research. Clinically, clinical indicators such as patient age, family history, total prostate tumor antigen ("tPSA"), and total PSA / free PSA ratio have certain value in guiding prostate puncture and predicting prostate cancer, but the overall sensitivity is not satisfactory. Multi-parameter prostate magnetic resonance imaging ("mpMRI") is an important auxiliary examination in the diagnosis of prostate cancer. It calculates the PI-RADS score for the prostate based on mpMRI. The PI-RADS score includes 2-5 points, and each score predicts the probability of clinically significant prostate cancer of 6%, 12%, 48% and 72%, respectively. Clinicians usually give recommendations on whether to perform puncture based on the above factors. Different doctors may have different opinions on whether puncture is necessary based on their own clinical experience and subjective judgment. Therefore, a challenging question arises: how to reduce unnecessary punctures.

[0003] During the patient's visit, the doctor will often prescribe the corresponding laboratory tests and imaging examinations and other text information according to the situation. At present, natural language processing models (such as the natural language processing model GPT based on the Transformer architecture) have powerful text processing capabilities, which can comprehensively analyze various indicators and finally give prediction results. For example, by inputting text information such as the patient's basic clinical information, laboratory test results (such as tPSA, tPSA / fPSA) and mpMRI image description reports into the natural language processing model, the PI-RADS score and prostate puncture preference recommendation can be output to effectively avoid unnecessary punctures caused by subjective differences. However, the natural language processing model cannot make full use of image information. It can only process image text information and cannot extract the potential feature information of the image, which makes the results output by the natural language processing model inaccurate.

[0004] In view of this, there is an urgent need to provide a solution for predicting the tendency of prostate puncture. By adding an image processing model to process prostate images, the defect that the natural language processing model cannot fully utilize the image information can be compensated. At the same time, the prediction results of the natural language processing model and the image processing model can be combined to improve the accuracy of predicting the tendency of prostate puncture. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a solution for predicting the tendency of prostate puncture in multiple aspects.

[0006] In a first aspect, the present application provides a device for predicting a tendency for prostate puncture, comprising: a processor; and a memory, wherein program instructions for predicting a tendency for prostate puncture are stored, and when the program instructions are executed by the processor, the device implements the following operations: obtaining basic information of a subject and text information of a prostate image; inputting the basic information and the text information of the prostate image into a natural language processing model for a first prediction of a prostate puncture tendency, so as to obtain a first prediction result for predicting a tendency for prostate puncture; obtaining the prostate image of the subject; inputting the prostate image of the subject into an image processing model for a second prediction of a prostate puncture tendency, so as to obtain a second prediction result for predicting a tendency for prostate puncture; and obtaining a final prediction result for predicting a tendency for prostate puncture based on the first prediction result and the second prediction result.

[0007] In one embodiment, the basic information of the subject at least includes age information and / or prostate specific antigen test information.

[0008] In another embodiment, the text information of the prostate image at least includes diameter information and / or abnormal signal information of the prostate lesion.

[0009] In another embodiment, the image processing model includes an attention module and a residual module, and when the program instructions are executed by the processor, the device further implements the following operations to perform the second prostate puncture tendency prediction: the prostate image of the subject is input into the image processing model, and the attention module and the residual module in the image processing model are used to perform attention operations and residual operations on the prostate image in sequence to extract prostate features, so as to perform the second prostate puncture tendency prediction.

[0010] In another embodiment, when the program instructions are executed by the processor, the device further implements the following operations to perform the second prostate puncture tendency prediction: using the attention module in the image processing model to perform an attention operation on the prostate image to obtain an attention feature; and using the residual module in the image processing model to perform a residual operation on the attention feature to extract a prostate feature to perform the second prostate puncture tendency prediction.

[0011] In another embodiment, the attention module includes a position attention module and a channel attention module, and when the program instructions are executed by the processor, the device further implements the following operations to obtain the attention feature: use the position attention module and the channel attention module to perform position attention operations and channel attention operations on the prostate image in sequence to obtain position attention features and channel attention features.

[0012] In yet another embodiment, the residual module, the position attention module, and the channel attention module each include multiple groups.

[0013] In another embodiment, when the program instructions are executed by the processor, the device further implements the following operations to extract the prostate features: use each group of position attention modules, channel attention modules and residual modules to sequentially perform position attention operations, channel attention operations and residual operations on the prostate image to obtain each group of intermediate features; and use the next group of position attention modules, channel attention modules and residual modules to sequentially perform the next position attention operation, channel attention operation and residual operation on each group of intermediate features until the last group of intermediate features is obtained to extract the prostate features.

[0014] In another embodiment, when the program instructions are executed by the processor, the device further implements the following operations to obtain the final prediction result of predicting the tendency of prostate puncture: splicing the first prediction result and the second prediction result to obtain the final prediction result of predicting the tendency of prostate puncture.

[0015] In a second aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for predicting prostate puncture tendency, which, when executed by one or more processors, implement the operations performed by the apparatus as described in the multiple embodiments in the aforementioned first aspect.

[0016] Through the scheme for predicting the tendency of prostate puncture provided above, the embodiment of the present application processes the basic information of the subject and the text information of the prostate image through the natural language processing model to perform a first prostate puncture tendency prediction to obtain a first prediction result. The second prostate puncture tendency prediction is performed by processing the prostate image of the subject through the image processing model to obtain a second prediction result, and then the final prediction result is obtained by combining the first and second prediction results. Based on this, the embodiment of the present application adds an image processing model to process the prostate image, which makes up for the defect that the natural language processing model cannot fully utilize the image information, and by combining the prediction results of the natural language processing model and the image processing model, the accuracy of predicting the tendency of prostate puncture is greatly improved. Further, the image processing model of the embodiment of the present application includes an attention module and a residual module, and the attention module can also include a position attention module and a channel attention module. The attention module and the residual module perform attention operations and residual operations on the prostate image in sequence to extract prostate features, so that the prostate information is fully extracted and the accuracy of the second prediction result is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0018] Figure 1 is an exemplary structural block diagram showing an apparatus for predicting prostate puncture tendency according to an embodiment of the present application;

[0019] Figure 2 is an exemplary schematic diagram showing an overall method for predicting prostate puncture tendency according to an embodiment of the present application;

[0020] Figure 3 is an exemplary schematic diagram showing the processing of basic information and text information of a prostate image by a natural language processing model according to an embodiment of the present application;

[0021] Figure 4 is an exemplary schematic diagram showing the processing of a prostate image by an image processing model according to an embodiment of the present application;

[0022] Figure 5 is an exemplary schematic diagram showing a position attention module and a channel attention module according to an embodiment of the present application;

[0023] Figure 6 It is an exemplary structural block diagram showing a device for predicting prostate puncture tendency according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. It should be understood that the embodiments described in this specification are only partial embodiments provided by this application to facilitate a clear understanding of the solutions and to comply with legal requirements, and are not all embodiments that can implement this application. Based on the embodiments disclosed in this specification, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0025] Figure 1 FIG. 1 is an exemplary structural block diagram showing an apparatus 100 for predicting prostate puncture tendency according to an embodiment of the present application. Figure 1 As shown in , the device 100 may include a processor 101 and a memory 102. The processor 101 may include, for example, a general-purpose processor ("CPU") or a dedicated graphics processor ("GPU"), and the memory 102 stores program instructions that can be executed on the processor. In some embodiments, the memory 102 may include, but is not limited to, a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), and an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory).

[0026] Furthermore, the memory 102 may store program instructions for predicting the tendency of prostate puncture. When the program instructions are executed by the processor, the device 100 performs the following operations: obtaining the basic information of the subject and the text information of the prostate image, inputting the basic information and the text information of the prostate image into the natural language processing model for a first prediction of the tendency of prostate puncture, so as to obtain a first prediction result of predicting the tendency of prostate puncture. Then, obtaining the prostate image of the subject, inputting the prostate image of the subject into the image processing model for a second prediction of the tendency of prostate puncture, so as to obtain a second prediction result of predicting the tendency of prostate puncture, and then obtaining a final prediction result of predicting the tendency of prostate puncture based on the first prediction result and the second prediction result.

[0027] In some embodiments, the basic information of the above-mentioned subject may include but is not limited to age information and / or prostate-specific antigen test information (such as tPSA, tPSA / fPSA), and may also include the subject's disease history information, etc. The prostate image of the above-mentioned subject may be, for example, an mpMRI image, and the corresponding text information describes the lesion information present in the mpMRI image in text language. In an implementation scenario, the text information of the aforementioned prostate image may include but is not limited to the diameter information and / or abnormal signal information of the prostate lesion, wherein the aforementioned diameter information includes the three aspects of the length × width × height of the prostate lesion, such as 58mm×40mm×51mm, and the aforementioned abnormal signal information includes signal information such as T2wI, DWI, TIWI and ADC. When there are prostate lesions in the image, text information of the prostate image is formed by describing the lesion diameter information and / or abnormal signal information, for example, the diffuse abnormal signal of the prostate is manifested as T2wI low signal, DWI high signal, and ADC low signal; abnormal patchy signals can be seen on the right pubic bone and left sciatic bone, which are manifested as T2WI high signal, TIWI low signal, DWI high signal, and ADC signal reduction.

[0028] Based on the basic information of the subject and the text information of the prostate image obtained above, when the program instructions are executed by the processor, the device 100 further executes the input of the basic information and the text information of the prostate image into the natural language processing model to perform a first prostate puncture tendency prediction, so as to obtain a first prediction result related to the prediction of the prostate puncture tendency. In an implementation scenario, the natural language processing model can be, for example, a natural language processing model based on the Transformer architecture, such as GPT. In this scenario, the natural language processing model conducts a comprehensive analysis of the text content based on the input basic information of the subject and the text information of the prostate image, that is, performs a first prostate puncture tendency prediction, and finally outputs the first prediction result related to the prediction of the prostate puncture tendency. The aforementioned first prediction result includes puncture or no puncture. In some embodiments, the natural language processing model can also output a PI-RADS score.

[0029] As can be seen from the foregoing, the natural language processing model is only sensitive to text content, and thus cannot obtain the richer feature information contained in the prostate image. Therefore, in an embodiment of the present application, the prostate image is processed by adding an image processing model, and the prostate image of the subject is input into the image processing model for a second prostate puncture tendency prediction to obtain a second prediction result about the predicted prostate puncture tendency. In one embodiment, the above-mentioned image processing model can be, for example, a convolutional neural network model, and the image processing model can include an attention module and a residual module. When the program instructions are executed by the processor, the device further implements the following operations to perform a second prostate puncture tendency prediction: the prostate image of the subject is input into the image processing model, and the attention module and the residual module in the image processing model are used to perform attention operations and residual operations on the prostate image in sequence to extract prostate features, so as to perform a second prostate puncture tendency prediction.

[0030] Specifically, in an implementation scenario, the attention module in the image processing model can be used to perform an attention operation on the prostate image to obtain an attention feature, and then the residual module in the image processing model can be used to perform a residual operation on the attention feature to extract the prostate feature, so as to perform a second prostate puncture tendency prediction. In some embodiments, the aforementioned attention module may include a position attention module and a channel attention module, and the position attention operation and the channel attention operation are performed on the prostate image in sequence by using the position attention module and the channel attention module to obtain the position attention feature and the channel attention feature. In the embodiment of the present application, the aforementioned sequence can be understood as first performing a position attention operation on the prostate image using the position attention module to obtain the position attention feature, and then using the channel attention module to perform a channel attention operation on the position attention feature to obtain the channel attention feature. Furthermore, the residual module is used to perform a residual operation on the channel attention feature to extract the prostate feature, so as to perform a second prostate puncture tendency prediction to obtain a second prediction result.

[0031] In some embodiments, the above-mentioned residual module, position attention module and channel attention module each include multiple groups. In one embodiment, the above-mentioned device further uses each group of position attention modules, channel attention modules and residual modules to sequentially perform position attention operations, channel attention operations and residual operations on the prostate image to obtain each group of intermediate features, and then uses the next group of position attention modules, channel attention modules and residual modules to sequentially perform the next position attention operation, channel attention operation and residual operation on each group of intermediate features until the last group of intermediate features is obtained to extract prostate features. That is to say, after a group of residual modules, position attention modules and channel attention modules perform position attention operations, channel attention operations and residual operations, a group of intermediate features is obtained, and the next group of position attention operations, channel attention operations and residual operations are performed on the group of intermediate features again until the last group of intermediate features is obtained, which is the prostate features. This will be combined later. Figure 4 The operation of extracting prostate features is described in detail.

[0032] Based on the first prediction result and the second prediction result obtained above, the above device further obtains a final prediction result of predicting the tendency of prostate puncture by splicing the first prediction result and the second prediction result. In some embodiments, the final prediction result is whether prostate puncture is needed, for example, including 0 or 1, 1 indicates that prostate puncture is needed, and 0 indicates that prostate puncture is not needed.

[0033] Combined with the above description, it can be seen that the embodiment of the present application processes the basic information of the subject and the text information of the prostate image through the natural language processing model to obtain the first prediction result, and combines the image processing model with the second prediction result based on the prostate image to make up for the defect that the natural language processing model cannot fully utilize the image information, thereby improving the accuracy of predicting the tendency of prostate puncture. Furthermore, the embodiment of the present application extracts the global information of the image through multiple groups of position attention operations and channel attention operations, and uses residual fitting of the global information, so that the obtained prostate feature information is richer and more complete, thereby improving the accuracy of the second prediction result.

[0034] Figure 2 FIG. 2 is an exemplary schematic diagram showing an overall method for predicting prostate puncture tendency according to an embodiment of the present application. Figure 2As shown in , the basic information of the subject and the text information 201 of the prostate image are input into the natural language processing model 202 to perform a first prostate puncture tendency prediction, and obtain a first prediction result 203 about the predicted prostate puncture tendency. In an implementation scenario, the basic information of the subject may include age information, prostate-specific antigen test information (such as tPSA, tPSA / fPSA) or disease history information, etc. The text information of the prostate image may include diameter information and abnormal signal information about the lesion on the mpMRI image, etc. In an implementation scenario, the natural language processing model may be, for example, GPT. By inputting the aforementioned information into the natural language processing model for analysis and processing, the first prediction result can be directly obtained, including, for example, whether puncture is needed and the PI-RADS score.

[0035] The figure further shows that the prostate image 204 of the subject is input into the image processing model 205 for the second prostate puncture tendency prediction, and the second prediction result 206 about the predicted prostate puncture tendency is obtained. In an implementation scenario, the aforementioned image processing model can be, for example, a convolutional neural network model, and the image processing model can include an attention module 207 and a residual module 208. In this scenario, the attention module 207 and the residual module 208 perform attention operations and residual operations to extract prostate features in turn to achieve the second prostate puncture tendency prediction. In some embodiments, the aforementioned attention module 207 can include a position attention module 209 and a channel attention module 210. In the implementation scenario, the position attention operation is first performed on the prostate image via the position attention module 209 to obtain the position attention feature, and then the channel attention module 210 is used to perform the channel attention operation on the position attention feature to obtain the channel attention feature, and then the residual operation is performed via the residual module 208 to extract the prostate feature. Subsequently, a fully connected layer and a softmax function can be connected to output the second prediction result 206.

[0036] After obtaining the first prediction result 203 and the second prediction result 206, the first prediction result 203 and the second prediction result 206 are concatenated to form a vector. Similarly, a fully connected layer and a softmax function may be connected later to obtain a final prediction result 211. The final prediction result is whether prostate puncture is needed, for example, including 0 or 1.

[0037] Figure 3 FIG. 1 is an exemplary schematic diagram showing the processing of basic information and text information of prostate images by a natural language processing model according to an embodiment of the present application. Figure 3The left side in the middle shows the input end of the natural language processing model 202, which can input the subject's age information, prostate-specific antigen test information (such as tPSA, tPSA / fPSA), disease history information, prostate lesion diameter information and abnormal signal information, etc. As an example, the age of the examinee is 89; the text information of the prostate image is: enlarged prostate, about 58mm×40mm×51mm; diffuse abnormal signal of the left prostate, T2wI low signal, DWI high signal, ADC low signal; enhanced scanning shows that the lesion is obviously enhanced in the early stage, and the maximum diameter of the lesion is about 3.1cm; the left side of the prostate capsule is incomplete, and multiple nodules can be seen in the transition zone, which may be a manifestation of benign prostatic hyperplasia or prostate cancer; abnormal patchy signals can be seen in the right pubic bone and left sciatic bone, manifested as T2WI high signal, TIWI low signal, DWI high signal, and ADC signal reduction, which may indicate bone metastasis; the volume of the bilateral seminal vesicles is reduced, and the T2WI signal is weakened; the bladder is generally full, the bladder wall is not thick, and no abnormalities are found in the cavity; no enlarged lymph nodes are found in the pelvic wall and groin on both sides, and no obvious abnormal signals are found in the bones that make up the pelvis.

[0038] like Figure 3 The output of the natural language processing model is shown on the right side. Based on the above information, the natural language processing model performs the first prediction of the prostate biopsy tendency prediction and obtains the first prediction result, namely: according to the Pi-RADS scoring standard, the Pi-RADS score of the examinee is output as 5 points, and it is suggested that prostate cancer is highly suspected and bone metastasis may exist. Furthermore, combined with the patient's age and imaging manifestations, it is believed that the examinee has a high risk of prostate cancer and bone metastasis. Although biopsy can provide a more accurate diagnosis, considering the patient's age and potential complications of the biopsy process, it is recommended not to perform a prostate puncture.

[0039] Figure 4 FIG. 2 is an exemplary schematic diagram showing the processing of a prostate image by an image processing model according to an embodiment of the present application. Figure 4 As shown in , the image processing model may include a position attention module 209, a channel attention module 210 and a residual module 208. Furthermore, the position attention module 209, the channel attention module 210 and the residual module 208 may include multiple groups, for example, five groups are exemplarily shown in the figure. According to the foregoing, each group of corresponding intermediate features is obtained after each group of residual modules, position attention modules and channel attention modules perform position attention operations, channel attention operations and residual operations. Then, the corresponding intermediate features are subjected to the next group of position attention operations, channel attention operations and residual operations again until the last group of intermediate features is obtained.

[0040] For example, the first group is used to perform position attention operation, channel attention operation and residual operation on the prostate image 204 in sequence to obtain the intermediate feature 301. The second group is used to perform position attention operation, channel attention operation and residual operation on the intermediate feature 301 in sequence to obtain the intermediate feature 302. Similarly, intermediate features 303 and 304 and the last group of intermediate features, i.e., prostate features 305, can also be obtained. As mentioned above, by connecting a fully connected layer and a softmax function, the second prediction result 206 can be output. Further, the second prediction result 206 and the first prediction result 203 are concatenated to obtain the final prediction result 211.

[0041] Figure 5 is an exemplary schematic diagram showing a position attention module and a channel attention module according to an embodiment of the present application. Figure 5 Figure (a) shows the position attention module. Figure 5 Figure (b) shows the channel attention module. It should be understood that for the position attention module and the channel attention module, three feature vectors are obtained by performing attention operations on the above-mentioned prostate image or intermediate features, wherein the associated weights are determined according to two of the feature vectors, and the feature vector with the added weight is obtained by multiplying it with another feature vector. Furthermore, the attention feature can be obtained by adding the feature vector with the added weight to the original prostate image or intermediate feature.

[0042] For example, for the position attention module, it can first perform convolution 501 and convolution 502 on the above-mentioned prostate image 204 or intermediate features, and then perform position attention operations to obtain three position feature vectors, such as position feature vector 503, position feature vector 504 and position feature vector 505. Next, the position association weight 506 is determined according to the matrix multiplication of the position feature vector 504 and the position feature vector 505, and then the matrix product is performed with the position feature vector 501 to obtain the feature vector 507 with the position weight added, and it is restored to a three-dimensional feature 508. Finally, it is added with the original prostate image or intermediate feature to obtain the position attention feature 509.

[0043] For the channel attention module, it can pave the position attention feature 509 into two dimensions to perform the channel attention operation to obtain three channel feature vectors, such as channel feature vector 510, channel feature vector 511 and channel feature vector 512. Then, the channel association weight 513 is determined according to the matrix multiplication of the channel feature vector 510 and the channel feature vector 511, and then the matrix product is performed with the channel feature vector 512 to obtain the feature vector 514 with the added channel weight, and then it is restored to the three-dimensional feature 515. Finally, it is added to the original position attention feature 509 to obtain the channel attention feature 516. The residual operation is subsequently performed through the residual module to obtain the intermediate feature or the final prostate feature.

[0044] Based on the foregoing description, the embodiment of the present application adds an image processing model to make up for the defect that the natural language processing model cannot process images, performs a second prostate puncture tendency prediction to obtain a second prediction result, and combines the first prediction result of the natural language processing model to improve the accuracy of predicting the prostate puncture tendency. In addition, the embodiment of the present application also sets a position attention module, a channel attention module and a residual module to extract the global information of the image, thereby improving the accuracy of the second prediction result.

[0045] Figure 6 6 is an exemplary structural block diagram of a device 600 for predicting prostate puncture tendency according to an embodiment of the present application. It is understood that the device implementing the solution of the present application can be a single device (such as a computing device) or a multifunctional device including various peripheral devices.

[0046] like Figure 6 As shown in , the device of the present application may include a central processing unit or central processing unit ("CPU") 611, which may be a general-purpose CPU, a dedicated CPU, or other information processing and program execution unit. Further, the device 600 may also include a large-capacity memory 612 and a read-only memory ("ROM") 613, wherein the large-capacity memory 612 may be configured to store various types of data, including various basic information of the subject and text information of the prostate image, prostate image, various features, algorithm data, intermediate results, and various programs required to run the device 600. ROM 613 may be configured to store data and instructions required for power-on self-test of the device 600, initialization of various functional modules in the system, basic input / output drivers of the system, and booting the operating system.

[0047] Optionally, the device 600 may also include other hardware platforms or components, such as the tensor processing unit ("TPU") 614, the graphics processing unit ("GPU") 615, the field programmable gate array ("FPGA") 616, and the machine learning unit ("MLU") 617 shown. It is understood that although a variety of hardware platforms or components are shown in the device 600, they are only exemplary and not restrictive, and those skilled in the art can add or remove corresponding hardware according to actual needs. For example, the device 600 may only include a CPU, related storage devices, and interface devices to implement the method for predicting the prostate puncture tendency of the present application.

[0048] In some embodiments, in order to facilitate the transmission and interaction of data with an external network, the device 600 of the present application also includes a communication interface 618, so that it can be connected to a local area network / wireless local area network ("LAN / WLAN") 605 through the communication interface 618, and then connected to a local server 606 or connected to the Internet ("Internet") 607 through the LAN / WLAN. Alternatively or additionally, the device 600 of the present application can also be directly connected to the Internet or a cellular network through the communication interface 618 based on wireless communication technology, such as wireless communication technology based on the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the wireless communication technology. In some application scenarios, the device 600 of the present application can also access the server 608 and the database 609 of the external network as needed to obtain various known algorithms, data and modules, and can remotely store various data, such as various types of data or instructions for presenting, for example, basic information of the subject and text information of the prostate image, prostate image, various features, etc.

[0049] The peripheral devices of the device 600 may include a display device 602, an input device 603 and a data transmission interface 604. In one embodiment, the display device 602 may include, for example, one or more speakers and / or one or more visual displays, which are configured to perform voice prompts and / or image video display of the application for predicting the prostate puncture tendency. The input device 603 may include, for example, other input buttons or controls such as a keyboard, a mouse, a microphone, a gesture capture camera, etc., which are configured to receive input and / or user instructions of audio data. The data transmission interface 604 may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a firewire ("FireWire"), a PCI Express and a high-definition multimedia interface ("HDMI"), etc., which are configured to transmit and interact with other devices or systems. According to the scheme of the application, the data transmission interface 604 can receive the prostate image collected from the MRI acquisition device, and transmit the data or results including the prostate image or various other types to the device 600.

[0050] The CPU 611, mass storage 612, ROM 613, TPU 614, GPU 615, FPGA 616, MLU 617 and communication interface 618 of the device 600 of the present application can be interconnected through a bus 619, and data interaction with peripheral devices can be achieved through the bus. In one embodiment, through the bus 619, the CPU 611 can control other hardware components in the device 600 and its peripheral devices.

[0051] Combination of the above Figure 6 The device for predicting prostate puncture tendency that can be used to implement the present application is described. It should be understood that the device structure or architecture here is only exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application.

[0052] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium, which stores computer-readable instructions for predicting prostate puncture tendency. When the computer-readable instructions are executed by one or more processors, they can be used to implement the present application in combination with the accompanying drawings. Figure 1 A method for predicting propensity for prostate biopsy is described.

[0053] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0054] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0055] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0056] Although the implementation methods of the present application are as above, the contents described are only examples adopted to facilitate the understanding of the present application, and are not intended to limit the scope and application scenarios of the present application. Any technician in the technical field described in the present application can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of patent protection of the present application shall still be subject to the scope defined in the attached claims.

Claims

1. A device for predicting prostate puncture tendency, characterized in that: include: processor; as well as A memory having program instructions for predicting a prostate puncture tendency stored therein, wherein when the program instructions are executed by the processor, the device implements the following operations: Obtain the basic information of the subject and text information of the prostate image; Inputting the basic information and the text information of the prostate image into a natural language processing model to perform a first prostate puncture tendency prediction to obtain a first prediction result on the predicted prostate puncture tendency; Acquiring the prostate image of the subject; Inputting the prostate image of the subject into the image processing model to perform a second prostate puncture tendency prediction to obtain a second prediction result related to the predicted prostate puncture tendency; as well as obtaining a final prediction result of predicting the prostate puncture tendency based on the first prediction result and the second prediction result, The image processing model includes an attention module and a residual module, and when the program instructions are executed by the processor, the device further implements the following operations to perform the second prostate puncture tendency prediction: Inputting the prostate image of the subject into the image processing model, and using the attention module and the residual module in the image processing model to sequentially perform attention operation and residual operation on the prostate image to extract prostate features, so as to perform the second prostate puncture tendency prediction; The attention module includes a position attention module and a channel attention module, wherein the residual module, the position attention module and the channel attention module each include multiple groups; When the program instructions are executed by the processor, the device further implements the following operations to extract the prostate features: Using each group of position attention modules, channel attention modules and residual modules, sequentially performing position attention operations, channel attention operations and residual operations on the prostate image to obtain each group of intermediate features; as well as Use the next set of position attention modules, channel attention modules and residual modules to perform the next position attention operation, channel attention operation and residual operation on each set of intermediate features in sequence until the last set of intermediate features is obtained to extract the prostate features.

2. The device according to claim 1, characterized in that The basic information of the subject at least includes age information and / or prostate-specific antigen test information.

3. The device according to claim 1, characterized in that The text information of the prostate image at least includes diameter information and / or abnormal signal information of the prostate lesion.

4. The device according to claim 1, characterized in that When the program instructions are executed by the processor, the device further implements the following operations to perform the second prostate puncture tendency prediction: Using the attention module in the image processing model to perform an attention operation on the prostate image to obtain an attention feature; as well as The residual module in the image processing model is used to perform a residual operation on the attention feature to extract prostate features to perform the second prostate puncture tendency prediction.

5. The device according to claim 4, characterized in that When the program instructions are executed by the processor, the device further implements the following operations to obtain the attention feature: Use the position attention module and the channel attention module to perform position attention operations and channel attention operations on the prostate image in sequence to obtain position attention features and channel attention features.

6. The device according to claim 1, characterized in that When the program instructions are executed by the processor, the device further implements the following operations to obtain the final prediction result of predicting the prostate puncture tendency: The first prediction result and the second prediction result are spliced ​​together to obtain the final prediction result of predicting the tendency of prostate puncture.

7. A computer-readable storage medium, characterized in that: Computer-readable instructions for predicting the tendency of prostate puncture are stored thereon, and when the computer-readable instructions are executed by one or more processors, the operations performed by the device as described in any one of claims 1-6 are implemented.

Images