Image processing method, model generation method, equipment and storage medium
By constructing an image processing model, the feature extraction, classification and detection of X-ray images is solved, and the accuracy and efficiency of the analysis are improved.
Patent Information
- Application Number
- CN202411792000.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-18
AI Technical Summary
The lesions of the organ object in the X-ray image are not obvious, which affects the accuracy of the analysis of the organ object.
An image processing model including feature extraction module, classification module and detection module is constructed. The target X-ray image is processed through the pre-trained model, image features are extracted, and the organ object is classified and determined whether there is a lesion in it, and the location information of the lesion area is detected.
The accuracy and efficiency of the image processing model are improved, and it can effectively determine whether the organ object has lesions and accurately detect the location of the lesion area.
Smart Images
Figure CN119648676A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing, and in particular to an image processing method, a model generation method, a device and a storage medium. Background Art
[0002] Medical images are important auxiliary information that helps in disease diagnosis and organ function status analysis. For example, the functional status of an organ object can be learned from X-ray images.
[0003] However, there are cases where when an organ object has a lesion, the lesion is not obvious in the X-ray image. For example, when an intestinal organ object has intussusception, the features and areas of the intussusception are not obvious in the X-ray image, thus affecting the accuracy of the organ object analysis. Summary of the invention
[0004] The embodiments of the present application provide an image processing method, a model generation method, a device and a storage medium, which are used to solve the problem that the pathological condition of an organ object in an X-ray image is not obvious, which affects the accuracy of the organ object analysis.
[0005] In a first aspect, the present application provides an image processing method, comprising: Acquire a target X-ray image, wherein the target X-ray image is obtained by photographing a target organ object of a target user; Inputting the target X-ray image into an image processing model, so as to extract image features in the target X-ray image using a feature extraction module in the image processing model; the image processing model is pre-trained; Based on the image features, a target probability value of the presence of a lesion in the target organ object is obtained using a classification module in the image processing model, and target position information of a lesion area of the target organ object is obtained using a detection module in the image processing model; Wherein, the target probability value and the target position information are used to analyze the target organ object of the target user; Among them, the image processing model uses a first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are trained as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is taken for the target organ object of the sample user, and the second sample X-ray image is taken when the lesion area of the target organ object is processed.
[0006] In a second aspect, the present application provides an image processing method, comprising: Acquire a target abdominal X-ray image, where the target abdominal X-ray image is obtained by photographing an intestinal organ object of a target user; Inputting the target abdominal X-ray image into an image processing model, so as to extract image features in the target abdominal X-ray image using a feature extraction module in the image processing model; the image processing model is pre-trained; Based on the image features, a target probability value of the intestinal organ object having a lesion is obtained by using a classification module in the image processing model, and target position information of the lesion area of the intestinal organ object is obtained by using a detection module in the image processing model; Wherein, the target probability value and the target location information are used to analyze the intestinal organ object of the target user; Among them, the image processing model uses a first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are trained as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is taken for the target organ object of the sample user, and the second sample X-ray image is taken when the lesion area of the target organ object is processed.
[0007] In a third aspect, the present application provides a model generation method, comprising: Construct an image processing model including a feature extraction module, a classification module, and a detection module; Acquire a training data sample set, wherein the training data sample set includes a first sample X-ray image and a second sample X-ray image, wherein the first sample X-ray image is obtained by photographing a target organ object of a sample user, and the second sample X-ray image is obtained by photographing a lesion area of the target organ object after processing; Determine first position information of the lesion area of the target organ object in the first sample X-ray image based on second position information of the lesion area of the target organ object marked in the second sample X-ray image; The image processing model is trained using the first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are used as training labels to obtain model parameters of the image processing model.
[0008] In a fourth aspect, the present application provides a computing device comprising a storage component and a processing component; the storage component stores one or more computer program instructions, the computer program instructions are called and executed by the processing component, and the processing component executes the one or more computer program instructions to implement the image processing method described in the first aspect, the image processing method described in the second aspect, or the model generation method described in the third aspect.
[0009] In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a computer to implement the image processing method described in the first aspect, the image processing method described in the second aspect, or the model generation method described in the third aspect.
[0010] In a sixth aspect, the present application provides a computer program product storing a computer program, which, when executed by a computer, implements the image processing method described in the first aspect, the image processing method described in the second aspect, or the model generation method described in the third aspect.
[0011] In the embodiment of the present application, by constructing an image processing model including a feature extraction module, a classification module and a detection module, the image processing model is trained by using the first sample X-ray image of the sample user as a training sample, the first probability value of the target organ object having a lesion in the first sample X-ray image and the first position information of the lesion area as training labels, and the X-ray image can be used to realize multi-task processing of determining whether the target organ object has a lesion, and detecting the position of the lesion area when the lesion exists, thereby improving the efficiency and accuracy of the image processing model. In addition, the first position information in the training label is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image. Since the second sample X-ray image is obtained by shooting the lesion area of the target organ object, the second position information corresponding to the lesion area is accurate, and the first position information in the first sample X-ray image determined thereby is accurate, further improving the accuracy of the image processing model.
[0012] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0014] Figure 1 A flow chart of an embodiment of a model generation method provided by the present application is shown; Figure 2 A schematic diagram showing the structure of an embodiment of an image processing model provided by the present application is shown; Figure 3 A flowchart of an embodiment of an image processing method provided by the present application is shown; Figure 4 A flowchart of another embodiment of an image processing method provided by the present application is shown; Figure 5 A schematic diagram showing the structure of an embodiment of a model generation device provided by the present application is shown; Figure 6 A schematic diagram showing the structure of an embodiment of an image processing device provided by the present application is shown; Figure 7 A schematic structural diagram of another embodiment of an image processing device provided by the present application is shown; Figure 8 A schematic diagram of the structure of an embodiment of a computing device provided by the present application is shown. DETAILED DESCRIPTION
[0015] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0016] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.
[0017] The technical solution of the embodiment of the present application is applicable to the field of image processing, especially the field of medical image processing. Medical images are important auxiliary information to help diagnose diseases and analyze the functional status of organs. For example, the functional status of organ objects can be known based on X-ray images. However, when an organ object has a lesion, the lesion is not obvious in the X-ray image. For example, when an intestinal organ object has a lesion such as intussusception, the features and areas of intussusception in the X-ray image are not obvious, thereby affecting the accuracy of the organ object analysis.
[0018] In order to solve the above technical problems, the inventors have thought that for organ objects with lesions, when the lesion is treated, for example, when intussusception is treated by enema repositioning technology, the location of intussusception can be displayed in the X-ray image taken. Therefore, if the X-ray image taken during the treatment of the lesion can be used to assist in the lesion, such as the annotation of the location of the lesion area, so as to train the image processing model, the accuracy of image recognition can be significantly improved.
[0019] Based on this, the inventors proposed the technical solution of the present application, including: obtaining a target X-ray image, the target X-ray image being obtained by shooting for a target organ object of a target user; inputting the target X-ray image into an image processing model, so as to extract image features in the target X-ray image by using a feature extraction module in the image processing model; the image processing model is pre-trained; based on the image features, a target probability value of the existence of a lesion in the target organ object is obtained by using a classification module in the image processing model, and target position information of the lesion area of the target organ object is obtained by detecting by using a detection module in the image processing model; wherein the target probability value and the target position information are used to analyze the target organ object of the target user; wherein the image processing model is trained using a first sample X-ray image as a training sample, a first probability value of the existence of a lesion in the target organ object in the first sample X-ray image and first position information of the lesion area of the target organ object as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in a second sample X-ray image, the first sample X-ray image is obtained by shooting for the target organ object of the sample user, and the second sample X-ray image is obtained by shooting when the lesion area of the target organ object is processed.
[0020] By constructing an image processing model including a feature extraction module, a classification module and a detection module, the image processing model is trained by using the first sample X-ray image of the sample user as a training sample, the first probability value of the target organ object having a lesion in the first sample X-ray image and the first position information of the lesion area as training labels, and the X-ray image can be used to realize multi-task processing of determining whether the target organ object has a lesion, and detecting the position of the lesion area when the lesion exists, thereby improving the accuracy of the image processing model. In addition, the first position information in the training label is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image. Since the second sample X-ray image is obtained by shooting the lesion area of the target organ object, the second position information corresponding to the lesion area is accurate, and the first position information in the first sample X-ray image determined thereby is accurate, further improving the accuracy of the image processing model.
[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0022] The technical solution of the embodiment of the present application can be applied to a system architecture including a user end and a server end, and a connection is established between the user end and the server end through a network. The network provides a medium for a communication link between the user end and the server end. The network can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0023] The client can interact with the server through the network to send X-ray images or receive target probability values and target position information of the target organ object in the X-ray image.
[0024] Among them, the user end can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language5, Hypertext Markup Language Version 5) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application. The user end can be deployed in an electronic device and needs to rely on the device to run or some apps in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. For ease of understanding, various other types of applications can usually be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0025] The server side may include servers that provide various services, such as servers for background training that provide support for the model.
[0026] It should be noted that the server can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.
[0027] It should be noted that the image processing method and model generation method provided in the embodiments of the present application are generally executed by the server. However, in other embodiments of the present application, the user end may also have similar functions to the server, thereby executing the image processing method and model generation method provided in the embodiments of the present application. In other embodiments, the image processing method and model generation method provided in the embodiments of the present application may also be jointly executed by the user end and the server.
[0028] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described in this article within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).
[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0030] It should be noted that the technical solution of the embodiment of the present application is applicable to a network virtual environment. The users described generally refer to "virtual users". Real users can register a user account in the server through registration to obtain a user identity in the network environment.
[0031] The technical solution of the embodiment of the present application is implemented through an image processing model, and the process of model generation is explained below. Figure 1 This is a flowchart of an embodiment of a model generation method provided in the present application. The method may include the following steps.
[0032] 101: Construct an image processing model including a feature extraction module, a classification module and a detection module.
[0033] Among them, the feature extraction module can be used to extract features from the input image. Optionally, the feature extraction module can be implemented using ResNet50 (ResidualNetwork50, a deep convolutional neural network based on a residual network architecture). Of course, other neural network modules with feature extraction functions can also be used to implement it, and this application does not limit this.
[0034] The classification module can be used to identify image features and determine the probability value of the target organ object having a lesion. Optionally, the feature extraction module can output a feature vector map, and the classification module can perform global pooling processing on the feature vector map to obtain a feature vector, which is processed by a sigmoid function (an activation function of a neural network that maps variables to between 0 and 1) through sequentially connected linear layers and sigmoid layers to obtain the probability value of the target organ object having a lesion.
[0035] The detection module can be used to detect image features and obtain the location information of the lesion area of the target organ object. Optionally, the detection module can be implemented using the Faster R-CNN (Faster Region-based Convolutional Neural Network, a deep learning algorithm for target detection) model. Of course, other neural network models with position detection functions can also be used to implement it without limitation.
[0036] For ease of understanding, Figure 2 FIG. 2 shows a schematic diagram of the structure of an image processing model in an embodiment. Figure 2 As shown, the image processing model may be composed of a feature extraction module 201, a classification module 202 and a detection module 203. The classification module 202 and the detection module 203 are connected in parallel and are connected after the feature extraction module 201.
[0037] 102: Obtain a training data sample set, where the training data sample set includes a first sample X-ray image and a second sample X-ray image.
[0038] In this embodiment, the training data sample set may be composed of a sample pair of a first sample X-ray image and a second sample X-ray image, and the sample pair may be divided based on sample users. Specifically, the sample user may refer to a sample patient, and the first sample X-ray image and the second sample X-ray image of the sample patient may be obtained from the medical record data generated by the sample patient during the diagnosis and treatment process or the medical record front page data recorded in the medical record front page.
[0039] Among them, the X-ray image refers to an image obtained by taking X-rays. The first sample X-ray image and the second sample X-ray image can be obtained by taking the same target organ object of the sample user or the body area where the target organ object is located. Taking the target organ object as an intestinal organ object as an example, the first sample X-ray image and the second sample X-ray image can both be abdominal X-ray images. Specifically, the second sample X-ray image is obtained by taking the diseased area of the target organ object after processing, and the first sample X-ray image can be obtained by taking the diseased area without the processing. Taking the intestinal organ object with intussusception as an example, the second sample X-ray image can be obtained by taking the intussusception by using the enema reduction technology to process the intussusception, wherein the location of the intussusception area in the second sample X-ray image is clear and accurate.
[0040] 103: Determine first position information of the lesion area of the target organ object in the first sample X-ray image based on second position information of the lesion area of the target organ object marked in the second sample X-ray image.
[0041] For the convenience of description, the position information of the lesion area in the second sample X-ray image may be referred to as the second position information, and the position information of the lesion area in the first sample X-ray image may be referred to as the first position information.
[0042] The second position information of the lesion area of the target organ object may be annotated in the second sample X-ray image by manual annotation, or the second position information of the lesion area may be identified by using a recognition model, without limitation.
[0043] In this embodiment, since the first sample X-ray image and the second sample X-ray image are both obtained by photographing the same target organ object or the same body region where the target organ object is located, the first position information in the first sample X-ray image can be determined based on the second position information annotated in the second sample X-ray image. For example, it can be achieved by using a variety of methods such as manual inference annotation and registration transformation, which will be explained in subsequent embodiments.
[0044] 104: Using the first sample X-ray image as a training sample, a first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and first position information of the lesion area of the target organ object as training labels, training an image processing model, and obtaining model parameters of the image processing model.
[0045] The first probability value can be determined by ultrasonic detection, processing of the lesion area, etc., and can be obtained from the case data or the medical record homepage data of the sample user. The first probability value and the second position information marked above can be used as training labels, and the first sample X-ray image can be used as a training sample to train the image processing model. Specifically, the training sample can be input into the image processing model, the output data can be compared with the training label, the loss value can be calculated, and the model parameters can be adjusted based on the loss value until the loss requirements are met.
[0046] In this embodiment, by constructing an image processing model including a feature extraction module, a classification module and a detection module, the image processing model is trained by using the first sample X-ray image of the sample user as a training sample, the first probability value of the target organ object having a lesion in the first sample X-ray image and the first position information of the lesion area as training labels, and the X-ray image can be used to realize multi-task processing of determining whether the target organ object has a lesion, and detecting the position of the lesion area when the lesion exists, thereby improving the accuracy of the image processing model. In addition, the first position information in the training label is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image. Since the second sample X-ray image is obtained by shooting the lesion area of the target organ object, the second position information corresponding to the lesion area is accurate, and the first position information in the first sample X-ray image determined thereby is accurate, further improving the accuracy of the image processing model.
[0047] In some embodiments, a method for training an image processing model and obtaining model parameters of the image processing model may include: extracting image features of the first sample X-ray image using a feature extraction module; Based on the image features, a probability value of the presence of a lesion in the target organ object is obtained using a classification module, and a first loss value is determined based on the classification result and the first probability value; Based on the image features, using a detection module to detect and obtain position information of a lesion area of the target organ object, and determining a second loss value based on the detection result and the first position information; Based on the first loss value and the second loss value, a target loss value is determined, and the target loss value is used to adjust the model parameters of the image processing model.
[0048] Among them, the probability value obtained by the classification module can be used as a classification result, compared with the first probability value in the training label, and a first loss value can be determined. For example, a cross entropy loss function can be used to calculate the first loss value, which will not be elaborated.
[0049] The position information obtained by the detection module can be used as a detection result and compared with the first position information in the training label to determine the second loss value. For example, the second loss value can be calculated using a square loss function, which will not be elaborated.
[0050] According to the first loss value and the second loss value, a weighted process can be used to determine the target loss value, for example, the target loss value L = L1 + L2, where L can represent the target loss value, L1 can represent the first loss value, and L2 can represent the second loss value. Can represent hyperparameters.
[0051] In this embodiment, by adopting multi-task processing of obtaining probability values through classification and obtaining position information through detection, the target loss value is jointly determined based on the first loss value corresponding to the classification task and the second loss value corresponding to the detection task, and the model parameters are adjusted according to the target loss value, so that the classification task and the detection task can influence each other, learn common features, and further improve the accuracy of the image processing model.
[0052] The following is an explanation of the process of determining the first position information in the first sample X-ray image.
[0053] In some embodiments, based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the method for determining the first position information of the lesion area of the target organ object in the first sample X-ray image may include: Determining transformation parameters for registering and transforming the second sample X-ray image to the first sample X-ray image; Based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first position information of the lesion area of the target organ object in the first sample X-ray image is obtained by performing transformation according to the transformation parameters.
[0054] Among them, an image registration algorithm, such as a transformation matrix, etc., can be used to perform the registration transformation, and this application does not impose any restrictions on this.
[0055] Taking the case where the target organ object is an intestinal organ object and the lesion area is an intussusception area as an example, the second position information of the intussusception area in the second sample X-ray image may refer to the position information of the intussusception area bounding box. According to the transformation parameters obtained by the registration transformation, the second position information may be transformed to obtain the position information of the intussusception area bounding box in the first sample X-ray image, i.e., the first position information.
[0056] By performing image registration transformation processing, transformation parameters are obtained. According to the transformation parameters, the first position information of the lesion area in the first sample X-ray image can be determined based on the second position information of the lesion area in the second sample X-ray image, thereby achieving the determination of the position of the lesion area in the first sample X-ray image and improving the accuracy of position determination.
[0057] To further improve the accuracy, in some embodiments, the above method may further include: Determine a target region frame where the target organ object is located in the first sample X-ray image; The image outside the target area frame in the first sample X-ray image is cropped to obtain a processed first sample X-ray image, and the processed first sample X-ray image is used as a training sample.
[0058] Taking the target organ object as an intestinal organ object as an example, the target region where the target organ object is located may be the abdominal region. The position information of the abdominal region bounding box in the first sample X-ray image may be determined, and the image region outside the bounding box may be cropped to avoid interference caused by other information outside the target region frame, such as text information carried by the X-ray image.
[0059] Optionally, in order to avoid other interfering information, such as the presence of equipment used to treat intussusception using enema reduction techniques in the area below the inferior iliac spine, the position information of the left and right inferior iliac spines can also be determined, and the area below the line connecting the left inferior iliac spine and the right inferior iliac spine can be cropped. The cropped first sample X-ray image can be used as a training sample to train the image processing model.
[0060] In this embodiment, the first sample X-ray image is cropped to remove the area outside the target area frame where the target organ object is located, thereby avoiding the influence of other interference information on the detection without affecting the detection of the lesion area of the target organ object, thereby further improving the accuracy of the image processing model.
[0061] In some embodiments, in the above process, the method of determining the target region frame where the target organ object is located in the first sample X-ray image may include: The first sample X-ray image is input into the recognition model to use the recognition model to identify the target area frame where the target organ object is located; wherein the recognition model can be trained based on the third sample X-ray image and the target area frame where the target organ object is located marked in the third sample X-ray image.
[0062] Wherein, the recognition model may include another feature extraction module and a recognition module. The feature extraction module may also be implemented using ResNet50 or other neural network models to extract features of the third sample X-ray image. The recognition module may identify the location information of the target area box where the target organ object is located based on the extracted features, and compare it with the marked target area box to determine the loss value, and adjust the model parameters of the recognition model according to the loss value until the loss requirements are met. Wherein, the third sample X-ray image is also obtained by shooting the same target organ object or the body area where the target organ object is located, and the third sample X-ray image may be the same as or different from the first sample X-ray image.
[0063] Optionally, the position information of the left and right inferior iliac spines marked in the third sample X-ray image can be used as training to train the recognition model, so that the first sample X-ray image is input into the recognition model, and the recognition model can be used to identify the target area box where the target organ object is located and the position information of the left and right inferior iliac spines.
[0064] In this embodiment, by training the recognition model, the recognition model is used to determine the position information of the target area box where the target organ object in the first sample X-ray image is located, thereby improving the accuracy of determining the position of the target area box, thereby improving the accuracy of cropping the first sample X-ray image, and improving the accuracy of the image processing model.
[0065] Of course, the position information of the above-mentioned target area frame or the left and right inferior iliac spines can also be determined by manual observation and recognition, and this application does not impose any restrictions on this.
[0066] The following is a brief description of the technical solution of the present application from the perspective of image processing.
[0067] like Figure 3 As shown, it is a flowchart of an embodiment of an image processing method provided by the present application, and the method may include the following steps.
[0068] 301: Acquire a target X-ray image, where the target X-ray image may be obtained by photographing a target organ object of a target user.
[0069] 302: Input the target X-ray image into the image processing model to extract image features in the target X-ray image using a feature extraction module in the image processing model.
[0070] 302: Based on the image features, a classification module in the image processing model is used to obtain a target probability value of the presence of a lesion in the target organ object, and a detection module in the image processing model is used to detect and obtain target position information of the lesion area of the target organ object.
[0071] The image processing model is trained using a first sample X-ray image as a training sample, a first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and first position information of the lesion area of the target organ object as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is obtained by shooting the target organ object of the sample user, and the second sample X-ray image is obtained by shooting the lesion area of the target organ object after processing. The specific training process is as follows: Figure 1 The illustrated embodiment has been described in detail and will not be described again in detail here.
[0072] The target probability value and target location information can be used to analyze the target organ object of the target user. For example, when the target probability value is higher than the preset probability value, it can be determined that a lesion has occurred, and the target location information can be used as the location information of the lesion area. When the target probability value is lower than the preset probability value, it can be indicated that no lesion has occurred, and the target location information does not need to be used as the location information of the lesion area, and so on.
[0073] In this embodiment, by constructing an image processing model including a feature extraction module, a classification module and a detection module, the image processing model is trained by using the first sample X-ray image of the sample user as a training sample, and the first probability value of the target organ object having a lesion in the first sample X-ray image and the first position information of the lesion area as training labels, so that the X-ray image can be used to realize multi-task processing of determining whether the target organ object has a lesion, and detecting the position of the lesion area when the lesion exists, thereby improving the efficiency and accuracy of the image processing model. In addition, the first position information in the training label is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image. Since the second sample X-ray image is obtained by shooting the lesion area of the target organ object, the second position information corresponding to the lesion area is accurate, and the first position information in the first sample X-ray image determined thereby is accurate, further improving the accuracy of the image processing model.
[0074] In some embodiments, the above method may further include: Inputting the target X-ray image into the recognition model, so as to use the recognition model to recognize a target region frame where the target organ object in the target X-ray image is located; The image outside the target area frame in the target X-ray image is cropped to obtain a processed target X-ray image.
[0075] At this time, inputting the target X-ray image into the image processing model may include: The processed target X-ray image is input into the image processing model.
[0076] The training process of the recognition model can also be referred to Figure 1 The description of the illustrated embodiment will not be repeated here.
[0077] In this embodiment, the first sample X-ray image is cropped to remove the area outside the target area frame where the target organ object is located, thereby avoiding the influence of other interference information on the detection without affecting the detection of the lesion area of the target organ object, thereby further improving the accuracy of image processing.
[0078] The technical solution of the present application is described below by taking an abdominal X-ray image as an example.
[0079] like Figure 4 FIG. 1 is a flow chart of another embodiment of an image processing method provided by the present application. The method may include the following steps.
[0080] 401: Acquire a target abdominal X-ray image, where the target abdominal X-ray image can be acquired by photographing an intestinal organ object of a target user.
[0081] 402: Inputting the target abdominal X-ray image into the image processing model, so as to extract image features in the target abdominal X-ray image using a feature extraction module in the image processing model; 403: Based on the image features, a classification module in the image processing model is used to obtain a target probability value of the intestinal organ object having a lesion, and a detection module in the image processing model is used to detect and obtain target position information of the lesion area of the intestinal organ object.
[0082] The image processing model is trained using a first sample X-ray image as a training sample, a first probability value of the presence of a lesion in a target organ object in the first sample X-ray image and first position information of a lesion area of the target organ object as training labels, the first position information is determined based on second position information of the lesion area of the target organ object marked in a second sample X-ray image, the first sample X-ray image is obtained by shooting the target organ object of the sample user, and the second sample X-ray image is obtained by shooting the lesion area of the target organ object after processing.
[0083] The target probability value and the target location information can be used to analyze the intestinal organ object of the target user. For example, when the target probability value is higher than a preset probability value, it can be determined that intussusception occurs, and the target location information can be used as the location information of intussusception; when the target probability value is lower than the preset probability value, it can be indicated that intussusception does not occur, and the target location information does not need to be used as the location information of intussusception, and so on.
[0084] In this embodiment, by constructing an image processing model including a feature extraction module, a classification module and a detection module, the image processing model is trained by using the first sample X-ray image of the sample user as a training sample, and the first probability value of the target organ object having a lesion in the first sample X-ray image and the first position information of the lesion area as training labels, so that the X-ray image can be used to realize multi-task processing of determining whether the target organ object has a lesion, and detecting the position of the lesion area when the lesion exists, thereby improving the efficiency and accuracy of the image processing model. In addition, the first position information in the training label is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image. Since the second sample X-ray image is obtained by shooting the lesion area of the target organ object, the second position information corresponding to the lesion area is accurate, and the first position information in the first sample X-ray image determined thereby is accurate, further improving the accuracy of the image processing model.
[0085] like Figure 5 As shown, it is a structural schematic diagram of an embodiment of an image processing device provided by the present application, and the device may include the following modules.
[0086] The first acquisition module 501 is used to acquire a target X-ray image, where the target X-ray image is acquired by photographing a target organ object of a target user; A first extraction module 502, for inputting the target X-ray image into the image processing model, so as to extract image features in the target X-ray image using a feature extraction module in the image processing model; A first detection module 503 is used to obtain a target probability value of the presence of a lesion in the target organ object based on the image features by using a classification module in the image processing model, and to obtain target position information of the lesion area of the target organ object by using a detection module in the image processing model; Among them, the target probability value and the target position information are used to analyze the target organ object of the target user; Among them, the image processing model uses a first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are trained as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is taken for the target organ object of the sample user, and the second sample X-ray image is taken when the lesion area of the target organ object is processed.
[0087] In some embodiments, the apparatus may further include: A first recognition module is used to input the target X-ray image into the recognition model so as to use the recognition model to recognize the target region frame where the target organ object in the target X-ray image is located; A cropping module is used to crop the image outside the target area frame in the target X-ray image to obtain a processed target X-ray image; The first extraction module 502 can be specifically used to input the processed target X-ray image into the image processing model, so as to utilize the feature extraction module in the image processing model to extract the image features in the target X-ray image.
[0088] Figure 5 The image processing apparatus shown can be used to perform Figure 3 The implementation principle and technical effect of the image processing method shown are not described in detail. The specific way in which each module performs operations in the image processing device in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.
[0089] like Figure 6 As shown, it is a structural schematic diagram of an embodiment of an image processing device provided by the present application, and the device may include the following modules.
[0090] The second acquisition module 601 is used to acquire a target abdominal X-ray image, where the target abdominal X-ray image is acquired by photographing an intestinal organ object of a target user; A second extraction module 602 is used to input the target abdomen X-ray image into the image processing model to extract image features in the target abdomen X-ray image using a feature extraction module in the image processing model; The second detection module 603 is used to obtain the target probability value of the intestinal organ object having a lesion based on the image features by using the classification module in the image processing model, and to obtain the target position information of the lesion area of the intestinal organ object by using the detection module in the image processing model; Among them, the target probability value and the target location information are used to analyze the intestinal organ object of the target user; Among them, the image processing model uses a first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are trained as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is taken for the target organ object of the sample user, and the second sample X-ray image is taken when the lesion area of the target organ object is processed.
[0091] Figure 6 The image processing apparatus shown can be used to perform Figure 4 The implementation principle and technical effect of the image processing method shown are not described in detail. The specific way in which each module performs operations in the image processing device in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.
[0092] like Figure 7 As shown, it is a structural schematic diagram of an embodiment of a model generation device provided by the present application, and the device may include the following modules.
[0093] A construction module 701 is used to construct an image processing model including a feature extraction module, a classification module and a detection module; The third acquisition module 702 is used to acquire a training data sample set, where the training data sample set includes a first sample X-ray image and a second sample X-ray image, where the first sample X-ray image is obtained by photographing a target organ object of a sample user, and the second sample X-ray image is obtained by photographing a lesion area of the target organ object after processing; A first determination module 703 is used to determine first position information of the lesion area of the target organ object in the first sample X-ray image based on second position information of the lesion area of the target organ object marked in the second sample X-ray image; The training module 704 is used to train the image processing model using the first sample X-ray image as a training sample, the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image, and the first position information of the lesion area of the target organ object as training labels, and obtain the model parameters of the image processing model.
[0094] In some embodiments, the training module 704 can be specifically used to extract image features of a first sample X-ray image using a feature extraction module; based on the image features, obtain a probability value of the presence of lesions in the target organ object using a classification module, and determine a first loss value based on the classification result and the first probability value; based on the image features, obtain position information of the lesion area of the target organ object using a detection module, and determine a second loss value based on the detection result and the first position information; determine a target loss value based on the first loss value and the second loss value, and use the target loss value to adjust the model parameters of the image processing model.
[0095] In some embodiments, the first determination module 703 can be specifically used to determine the transformation parameters for aligning and transforming the second sample X-ray image to the first sample X-ray image; based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first position information of the lesion area of the target organ object in the first sample X-ray image is obtained by transforming according to the transformation parameters.
[0096] In some embodiments, the apparatus may further include: A second determination module is used to determine a target region frame where the target organ object is located in the first sample X-ray image; The processing module is used to crop the image outside the target area frame in the first sample X-ray image to obtain a processed first sample X-ray image, and use the processed first sample X-ray image as a training sample.
[0097] In some embodiments, the second determination module can be specifically used to input the first sample X-ray image into a recognition model so as to use the recognition model to identify the target area box where the target organ object is located; wherein the recognition model is trained based on the third sample X-ray image and the target area box where the target organ object is located marked in the third sample X-ray image.
[0098] Figure 7 The model generation apparatus shown can be used to perform Figure 1 The implementation principle and technical effect of the model generation method shown are not described in detail. The specific way in which each module performs operations in the model generation device in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.
[0099] like Figure 8 , which is a schematic diagram of a structure of an embodiment of a computing device provided by the present application, the device may include a storage component 801 and a processing component 802; The storage component 801 can be used to store one or more computer program instructions, wherein the one or more computer program instructions are called and executed by the processing component 802 to implement Figure 1 The model generation method shown, or Figure 3 The image processing method shown, or Figure 4 The image processing method shown.
[0100] Of course, the above computing device may also include other components, such as input / output interfaces, communication components, etc.
[0101] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices.
[0102] It should be noted that the above computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. It can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device.
[0103] Optionally, the computing device may also be implemented as an electronic device. An electronic device may refer to a device used by a user and having the functions of surfing the Internet, computing, communicating, etc. required by the user, such as a mobile phone, a tablet computer, a personal computer, a wearable device, etc. It is understandable that the electronic device may also include a display component, an input / output interface, a communication component, and other components, which will not be described in detail.
[0104] In one or more of the above embodiments, the processing component may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0105] The storage component is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0106] The display component may be an electroluminescent (EL) element, a liquid crystal display or a micro display having a similar structure, or a retinal direct display or a similar laser scanning display.
[0107] The present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a computer, Figure 1 The model generation method shown, or Figure 3 The image processing method shown, or Figure 4 The computer readable medium may be included in the computing device described in the above embodiments; or may exist independently without being assembled into the computing device.
[0108] The computer-readable storage medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above.
[0109] The present application also provides a computer program product, which includes a computer program carried on a computer-readable storage medium, and when the computer program is executed by a computer, Figure 1 The model generation method shown, or Figure 3The image processing method shown, or Figure 4 The image processing method shown.
[0110] In such an embodiment, the computer program may be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, various functions defined in the system of the present application are performed.
[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An image processing method, characterized in that: include: Acquire a target X-ray image, wherein the target X-ray image is obtained by photographing a target organ object of a target user; Inputting the target X-ray image into an image processing model to extract image features in the target X-ray image using a feature extraction module in the image processing model; Based on the image features, a target probability value of the presence of a lesion in the target organ object is obtained using a classification module in the image processing model, and target position information of a lesion area of the target organ object is obtained using a detection module in the image processing model; Wherein, the target probability value and the target position information are used to analyze the target organ object of the target user; Among them, the image processing model uses a first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are trained as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is taken for the target organ object of the sample user, and the second sample X-ray image is taken when the lesion area of the target organ object is processed.
2. The method according to claim 1, characterized in that Also includes: Inputting the target X-ray image into a recognition model, so as to use the recognition model to recognize a target region frame where a target organ object in the target X-ray image is located; Crop the image outside the target area frame in the target X-ray image to obtain a processed target X-ray image; The inputting the target X-ray image into the image processing model comprises: The processed target X-ray image is input into the image processing model.
3. An image processing method, characterized in that: include: Acquire a target abdominal X-ray image, where the target abdominal X-ray image is obtained by photographing an intestinal organ object of a target user; Inputting the target abdominal X-ray image into an image processing model, so as to extract image features in the target abdominal X-ray image using a feature extraction module in the image processing model; Based on the image features, a target probability value of the intestinal organ object having a lesion is obtained by using a classification module in the image processing model, and target position information of the lesion area of the intestinal organ object is obtained by using a detection module in the image processing model; Wherein, the target probability value and the target location information are used to analyze the intestinal organ object of the target user; Among them, the image processing model uses a first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are trained as training labels, the first position information is determined based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first sample X-ray image is taken for the target organ object of the sample user, and the second sample X-ray image is taken when the lesion area of the target organ object is processed.
4. A model generation method, characterized in that: include: Construct an image processing model including a feature extraction module, a classification module, and a detection module; Acquire a training data sample set, wherein the training data sample set includes a first sample X-ray image and a second sample X-ray image, wherein the first sample X-ray image is obtained by photographing a target organ object of a sample user, and the second sample X-ray image is obtained by photographing a lesion area of the target organ object after processing; Determine first position information of the lesion area of the target organ object in the first sample X-ray image based on second position information of the lesion area of the target organ object marked in the second sample X-ray image; The image processing model is trained using the first sample X-ray image as a training sample, and the first probability value of the presence of a lesion in the target organ object in the first sample X-ray image and the first position information of the lesion area of the target organ object are used as training labels to obtain model parameters of the image processing model.
5. The method according to claim 4, characterized in that The step of training the image processing model to obtain model parameters of the image processing model includes: extracting image features of the first sample X-ray image using the feature extraction module; Based on the image features, using the classification module to obtain a probability value of the target organ object having a lesion, and determining a first loss value based on the classification result and the first probability value; Based on the image features, using the detection module to detect and obtain the position information of the lesion area of the target organ object, and determining a second loss value based on the detection result and the first position information; Based on the first loss value and the second loss value, a target loss value is determined, and the model parameters of the image processing model are adjusted using the target loss value.
6. The method according to claim 4, characterized in that The determining, based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first position information of the lesion area of the target organ object in the first sample X-ray image comprises: Determining transformation parameters for registering and transforming the second sample X-ray image to the first sample X-ray image; Based on the second position information of the lesion area of the target organ object marked in the second sample X-ray image, the first position information of the lesion area of the target organ object in the first sample X-ray image is obtained by performing transformation according to the transformation parameters.
7. The method according to claim 4, characterized in that Also includes: Determine a target region frame where the target organ object is located in the first sample X-ray image; The image outside the target area frame in the first sample X-ray image is cropped to obtain a processed first sample X-ray image, and the processed first sample X-ray image is used as a training sample.
8. The method according to claim 7, characterized in that The step of determining a target region frame where the target organ object is located in the first sample X-ray image comprises: The first sample X-ray image is input into the recognition model to use the recognition model to identify the target area frame where the target organ object is located; wherein the recognition model is trained based on the third sample X-ray image and the target area frame where the target organ object is located marked in the third sample X-ray image.
9. A computing device, characterized in that It comprises a storage component and a processing component; the storage component stores one or more computer program instructions, the computer program instructions are called and executed by the processing component, and the processing component executes the one or more computer program instructions to implement the image processing method according to any one of claims 1 to 2, the image processing method according to claim 3, or the model generation method according to any one of claims 4 to 8.
10. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program is executed by a computer to implement the image processing method according to any one of claims 1 to 2, the image processing method according to claim 3, or the model generation method according to any one of claims 4 to 8.