An image processing method, device, computer equipment and storage medium

Abstract

Embodiments of the present application disclose an image processing method and device, computer equipment and a storage medium. Embodiments of the present application obtain a plurality of sample image pairs, determine pixel difference information of a first sample image and a second sample image, and determine a target pixel from the pixels of the first sample image according to the pixel difference information. The first sample image is processed based on the position of the target pixel in the first image to obtain a processed first sample image, and the processed first sample image is marked with a change area of the observed object. A preset network model is trained according to the processed first sample image to obtain a trained model. The trained model is used to process a to-be-processed image to obtain a target image corresponding to the to-be-processed image. The scheme can accurately predict the change of the observed object in the image by constructing a network model, and can improve the prediction accuracy of the image change.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an image processing method and device, computer equipment and storage medium. BACKGROUND

[0002] With the rapid development of computer technology, its application in the medical field is also more and more widely. In the medical field, in order to meet the needs of disease diagnosis, treatment plan making, etc., it is often necessary to scan the patient's body to determine the condition of each organ inside the patient's body. Cerebral hemorrhage, which belongs to "stroke", is a common and serious brain complication of middle-aged and elderly patients with hypertension. Cerebral hemorrhage is a non-traumatic intracerebral hemorrhage caused by vascular rupture, which has a sudden onset and a very high mortality rate, and is one of the leading causes of death in the elderly.

[0003] In the related art, cerebral hemorrhage is mainly detected by performing a head CT (computed tomography) examination on the patient to obtain a CT image, so that the doctor can analyze the patient's condition according to the CT image.

[0004] In the research and practice of the related art, the present inventor found that in the prior art, when the doctor analyzes the patient's condition according to the CT image, the doctor mainly observes the CT image to subjectively diagnose the patient's condition, but this method is limited to understanding the patient's current condition and has little effect on the subsequent treatment of the patient's condition. SUMMARY

[0005] The embodiments of the present application provide an image processing method and device, computer equipment and storage medium, which can improve the prediction accuracy of image changes.

[0006] The embodiments of the present application provide an image processing method, comprising:

[0007] Obtaining a plurality of sample image pairs, each sample image pair comprising a first sample image before a change of a same observation object and a second sample image after the change of the observation object;

[0008] Determining pixel difference information of the first sample image and the second sample image, and determining a target pixel from pixels of the first sample image according to the pixel difference information;

[0009] Processing the first sample image based on a position of the target pixel in the first image to obtain a processed first sample image, the processed first sample image being marked with a change region of the observation object;

[0010] Training a preset network model according to the processed first sample image to obtain a trained model;

[0011] The target image corresponding to the to-be-processed image is obtained by processing the to-be-processed image based on the trained model.

[0012] Correspondingly, the application further provides an image processing device, comprising:

[0013] An acquisition unit is configured to acquire a plurality of sample image pairs, each sample image pair comprising a first sample image before a change of an observation object and a second sample image after the change of the observation object.

[0014] A determination unit is configured to determine pixel difference information of the first sample image and the second sample image, and determine a target pixel from pixels of the first sample image according to the pixel difference information.

[0015] A first processing unit is configured to process the first sample image based on a position of the target pixel in the first image to obtain a processed first sample image, and the processed first sample image is marked with a change region of the observation object.

[0016] A training unit is configured to train a preset network model according to the processed first sample image to obtain a trained model.

[0017] A second processing unit is configured to process a to-be-processed image based on the trained model to obtain a target image corresponding to the to-be-processed image, and the target image is a predicted image after the change of the to-be-processed image.

[0018] In some embodiments, the determination unit comprises:

[0019] An alignment subunit is configured to align the first sample image and the second sample image to obtain a correspondence between a first pixel in the first sample image and a second pixel in the second sample image.

[0020] A first calculation subunit is configured to calculate a difference value of color parameters of the first pixel and the second pixel based on the correspondence to obtain the pixel difference information of the first sample image and the second sample image.

[0021] In some embodiments, the determination unit further comprises:

[0022] A first determination subunit is configured to determine, from the first sample image, a first pixel corresponding to the difference value when the difference value of the color parameters of the first pixel and the second pixel is greater than a preset difference value to obtain a pixel set.

[0023] A second determination subunit is configured to determine a first observation region in the first sample image, and a color parameter difference value of a pixel in the first observation region and a pixel in another region of the first sample image is greater than a preset threshold value.

[0024] The third determining sub-unit is configured to determine a second observation region in the second sample image, wherein a color parameter difference between a pixel in the second observation region and a pixel in another region of the second sample image is greater than a preset threshold value.

[0025] The fourth determining sub-unit is configured to determine a first difference region between the first observation region and the second observation region from the first sample image.

[0026] The extracting sub-unit is configured to extract a pixel located in the first difference region from the pixel set to obtain a target pixel.

[0027] In some embodiments, the aligning sub-unit is specifically configured to:

[0028] determine a first observation region in the first sample image and a second observation region in the second sample image;

[0029] extract feature information of the first observation region and feature information of the second observation region;

[0030] align the first image and the second image to the same angle according to the feature information of the first observation region and the feature information of the second observation region.

[0031] In some embodiments, the training unit comprises:

[0032] The input sub-unit is configured to input the processed first sample image into a preset network model for feature extraction to obtain feature data of the first sample image.

[0033] The generating sub-unit is configured to generate a predicted image of the first sample image according to the feature data.

[0034] The second calculating sub-unit is configured to calculate a similarity between the first sample image and the predicted image to obtain a first similarity.

[0035] The third calculating sub-unit is configured to calculate a similarity between a second difference region and a third difference region to obtain a second similarity, wherein the second difference region is a difference part between the predicted image and the first sample image, and the third difference region is a difference part between the first sample image and the second sample image.

[0036] The converging sub-unit is configured to determine a target similarity based on the first similarity and the second similarity, and adjust model parameters of the preset network model according to the target similarity until the preset network model converges to obtain a trained model.

[0037] In some embodiments, the converging sub-unit is specifically configured to:

[0038] determine a product of the first similarity and the second similarity, and a sum of the first similarity and the second similarity.

[0039] calculating a ratio of the product and the sum;

[0040] determining the target similarity based on a product of the ratio and a preset coefficient.

[0041] In some embodiments, the second calculating subunit is specifically configured to:

[0042] performing image segmentation processing on the predicted image to obtain a third observation region in the predicted image, pixels in the third observation region having a color parameter difference greater than a preset threshold with pixels in other regions in the third sample image;

[0043] determining an overlap degree between the third observation region and the first observation region in the first sample image to obtain a first similarity.

[0044] In some embodiments, the acquisition unit comprises:

[0045] The acquisition subunit is configured to acquire images of at least one observation object at different time points to obtain a plurality of sample images.

[0046] The selection subunit is configured to select sample images belonging to the same observation object from the plurality of sample images, and combine the sample images belonging to the same observation object in pairs to obtain a plurality of sample image pairs.

[0047] In some embodiments, the apparatus further comprises:

[0048] The first identifying unit is configured to identify an observation object in the first sample image, and extract a region where the observation object is located from the first sample image to obtain a first target sample image.

[0049] The second identifying unit is configured to identify an observation object in the second sample image, and extract a region where the observation object is located from the second sample image to obtain a second target sample image.

[0050] In some embodiments, the determining unit further comprises:

[0051] The fifth determining subunit is configured to determine pixel difference information of the first target sample image and the second target sample image.

[0052] Correspondingly, the embodiments of the present application further provide a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the image processing method provided by any of the embodiments of the present application.

[0053] Correspondingly, the embodiments of the present application further provide a storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the image processing method as above.

[0054] The embodiment of the present application further provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions stored in a storage medium. The processor of the terminal reads the computer instructions from the storage medium, and the processor executes the computer instructions, so that the terminal executes the image processing method provided in various optional implementation manners of the above aspect.

[0055] The embodiment of the present application trains the network model according to the sample training set by taking the images of the same observation object collected at different time points as the sample training set, obtains the trained model, predicts the change of the observation object in the to-be-processed image through the trained model, and outputs the to-be-processed image after the change, so that the prediction accuracy of the image change can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0057] Figure 1 The scene schematic diagram of the image processing system provided by the embodiment of the present application.

[0058] Figure 2 The flowchart of the image processing method provided by the embodiment of the present application.

[0059] Figure 3 The corresponding relationship diagram of the image pair provided by the embodiment of the present application.

[0060] Figure 4 The network model structure diagram provided by the embodiment of the present application.

[0061] Figure 5 The flowchart of another image processing method provided by the embodiment of the present application.

[0062] Figure 6 The cropped image diagram provided by the embodiment of the present application.

[0063] Figure 7 The diagram of predicting the development of the local region of the image provided by the embodiment of the present application.

[0064] Figure 8 The structure block diagram of the image processing device provided by the embodiment of the present application.

[0065] Figure 9 The structure block diagram of another image processing device provided by the embodiment of the present application.

[0066] Figure 10 Another structural block diagram of an image processing device provided by an embodiment of the present application.

[0067] Figure 11 A structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0069] The embodiments of the present application provide an image processing method, device, computer device and storage medium. Specifically, the embodiments of the present application provide an image processing device suitable for a computer device. The computer device can be a terminal or a server, etc. The server can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers. The terminal can be a tablet computer, notebook computer, desktop computer, etc., but is not limited thereto, and the embodiments of the present application do not limit this.

[0070] Please refer to Figure 1 , Figure 1 A scene schematic diagram of an image processing system provided by an embodiment of the present application, including an image device and a server. The image device and the server can be connected through a network, which includes routers, gateways and other network entities.

[0071] The image device can collect sample images and send the sample images to the server for processing. The server can obtain a plurality of sample image pairs, each sample image pair including a first sample image before a change of a same observation object and a second sample image after the change; determine pixel difference information of the first sample image and the second sample image, and determine a target pixel from pixels of the first sample image according to the pixel difference information; process the first sample image based on a position of the target pixel in the first image to obtain a processed first sample image; train a preset network model according to the processed first sample image to obtain a trained model; process a to-be-processed image based on the trained model to obtain a target image corresponding to the to-be-processed image, the target image being a predicted image after a change of the to-be-processed image.

[0072] It should be noted that Figure 1The scene diagram of the image processing system shown is merely an example, and the image processing system and the scene described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the image processing system evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0073] Based on the above problems, the first image processing method, device, computer device and storage medium provided by the embodiments of the present application can improve the security of identity information verification. The following will be described in detail. It should be noted that the order of the following embodiments is not limited as the preferred order of the embodiments.

[0074] With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields. For example, robots, intelligent medical treatment, etc. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0075] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system, so that the machine has the functions of perception, reasoning and decision making.

[0076] Computer vision technology (CV) is a technology that uses cameras and computers to replace human eyes to identify and measure targets, and further processes images to make computer processing more suitable for human observation or transmission to instrument detection, trying to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition and other technologies, and also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0077] Machine learning (ML) is a multi-disciplinary subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0078] In the scheme of the present application, computer vision technology is adopted to pre-process the sample images, and a training data set is determined based on the processed sample images. Then a network model is constructed through machine learning, the network model is trained according to the training data set, and finally the change of the observed object in the image is predicted based on the trained model, thereby improving the image processing efficiency.

[0079] As shown in Figure 2 , Figure 2 A flowchart of an image processing method provided by an embodiment of the present application is shown. The specific process of the image processing method can be as follows:

[0080] 101. Obtain a plurality of sample image pairs.

[0081] The sample image pair refers to an image set composed of two sample images, and the sample image pair includes a first sample image before the change of the same observed object and a second sample image after the change. In the embodiment of the present application, the plurality of sample image pairs can include a plurality of sample image pairs corresponding to the same observed object, or a plurality of sample image pairs corresponding to different observed objects.

[0082] The observed object refers to an object that will change over time, and the change can include various ways, such as size change, shape change, color change, etc.

[0083] For example, the observed object can be a living being, and the size or shape of each living being at different growth stages is different; or the observed object can be a colored liquid dropped into water, which will diffuse and change shape after being dropped into water, etc.

[0084] In some embodiments, in order to ensure the effectiveness of the sample images, the step of "obtaining a plurality of sample image pairs" can include the following operations:

[0085] Collect images of at least one observed object at different time points to obtain a plurality of sample images;

[0086] Select sample images belonging to the same observed object from the plurality of sample images, and combine the sample images belonging to the same observed object two by two to obtain a plurality of sample image pairs.

[0087] Since different observed objects require different lengths of time to change, the images of the observed objects can be collected according to the change time of different observed objects. The interval time for collecting the images of the observed objects can also be determined according to the change speed of different observed objects, such as setting a shorter interval time for collecting images of observed objects with a fast change speed, and setting a longer interval time for collecting images of observed objects with a slow change speed.

[0088] For example, the change time length of the observation object is predetermined, which can be 5 days. If the time of collecting the image of the observation object for the first time is January 1, 2020, the time of collecting the image of the observation object for the next time can be January 6, 2020.

[0089] Alternatively, when the change time length of the observation object is not determined, a longer time interval can be set, and the effective change image of the observation object can be collected to avoid collecting a large number of invalid change images and wasting resources.

[0090] Specifically, a plurality of sample images can be obtained by photographing images of a plurality of observation objects and images of the same observation object at different time points, or the sample images can be obtained by obtaining images of the observation object from a picture database, and the like.

[0091] Each sample image pair includes two sample images, i.e., a first sample image and a second sample image, which are images of the same observation object. The collection time of the first image is before the collection time of the second sample image.

[0092] In the embodiments of the present application, in order to make the first sample image and the second sample image in the same sample image pair have obvious differences, so as to ensure the accuracy of subsequent model training, two sample images with a long time interval can be selected from a plurality of sample images as a sample image pair.

[0093] For example, the images of the first observation object in the plurality of sample images include image A, image B, and image C, etc. The collection time of image A can be January 1, 2020, the collection time of image B can be January 4, 2020, and the collection time of image C can be January 6, 2020. Then, image A and image C can be selected to obtain the sample image pair corresponding to the first observation object.

[0094] 102. Determine the pixel difference information of the first sample image and the second sample image, and determine the target pixel from the pixels of the first sample image according to the pixel difference information.

[0095] The pixel difference information includes the feature difference between the pixels in the first sample image and the pixels in the second sample image. The feature difference can include a plurality of types, such as color difference, etc.

[0096] In some embodiments, in order to quickly obtain the pixel difference information of the first sample image and the second sample image, the step of “determining the pixel difference information of the first sample image and the second sample image” can include the following process:

[0097] aligning the first sample image with the second sample image to obtain a correspondence between the first pixel in the first sample image and a second pixel in the second sample image;

[0098] calculating a difference value of the color parameters of the first pixel and the second pixel based on the correspondence to obtain pixel difference information of the first sample image and the second sample image.

[0099] In some embodiments, the aligning of the first sample image with the second sample image can employ an image registration technique. Image registration is a process of matching and superimposing two or more images acquired at different times, by different sensors (imaging devices), or under different conditions (weather, illumination, camera position and angle, etc.).

[0100] Specifically, the process of the image registration technique is as follows: first, feature extraction is performed on two images to obtain feature points; similarity measurement is performed to find matched feature point pairs; then, image space coordinate transformation parameters are obtained through the matched feature point pairs; finally, image registration is performed by the coordinate transformation parameters. Among them, feature extraction is the key in the registration technique, and accurate feature extraction provides guarantee for the successful performance of feature matching. The image registration method can be summarized as relative registration and absolute registration: relative registration refers to selecting one image as a reference image, and registering other related images with it, and its coordinate system is arbitrary. Absolute registration refers to defining a control grid first, and registering all images relative to this grid, that is, completing the geometric correction of each component image to realize the unification of the coordinate system.

[0101] In the embodiments of the present application, in order to ensure the success rate of the alignment of the first sample image and the second sample image, a relative registration method can be used. In the relative registration method, how to determine the registration function mapping relationship between multiple images is the key to image registration. Usually, a suitable polynomial is used to fit the translation, rotation and affine transformation between two images, thereby converting the image registration function mapping relationship into how to determine the coefficients of the polynomial, and finally converting it into how to determine the registration control point (RCP). Specifically, according to the method of how to determine the RCP and the image information used in image registration, the image registration method can be divided into three main categories: gray-scale information-based method, transform domain method and feature-based method, wherein the feature-based method can be further divided into several categories according to different feature attributes used.

[0102] In some embodiments, in order to improve the efficiency of image alignment, the step of "aligning the first sample image with the second sample image" can include the following operations:

[0103] determining a first observation region in the first sample image and a second observation region in the second sample image;

[0104] extracting feature information of the first observation region and feature information of the second observation region;

[0105] aligning the first image and the second image to the same angle according to the feature information of the first observation region and the feature information of the second observation region.

[0106] The observation region refers to a local region that needs to be detected by the observation object, and the local region can be a change region of the observation object. In the sample image, the difference between the color parameters of the pixels in the observation region and the color parameters of the pixels in the non-observation region is greater than a preset threshold. Then, the first observation region in the first sample image refers to a local region that needs to be detected by the observation object in the first sample image, and the second observation region in the second sample image refers to a local region that needs to be detected by the observation object in the second sample image.

[0107] For example, the change of the face of user A needs to be detected, and images (including face regions) of user A at different times can be collected to obtain a first sample image and a second sample image. The observation object is user A, and the first sample image and the second sample image are images of user A. Then, the observation regions in the first sample image and the second sample image can be face regions of user A.

[0108] The feature information refers to image features of the observation region. The image features mainly include color features, texture features, shape features, and spatial relationship features. The color features are global features, which describe the surface properties of the scene corresponding to the image or image region. The texture features are also global features, which also describe the surface properties of the scene corresponding to the image or image region. The shape features have two types of representation methods, one is contour features, and the other is region features. The contour features of the image are mainly for the outer boundary of the object, and the region features of the image are related to the entire shape region. The spatial relationship features refer to the mutual spatial positions or relative direction relationships between multiple targets segmented from the image. These relationships can be divided into connection / adjacency relationships, intersection / overlap relationships, and inclusion / containment relationships.

[0109] In the embodiments of the present application, in order to save the time of image processing and improve the efficiency of image processing, the color features can be used as the feature information. Then, the color features of the first observation region are extracted to obtain the feature information of the first observation region, and the color features of the second observation region are extracted to obtain the feature information of the second observation region.

[0110] When the feature information of the first observation region and the feature information of the second observation region are extracted, the similarity measure of the first observation region and the second observation region is determined by the feature information, and then the matched feature point pairs are found according to the similarity measure; then the image space coordinate transformation parameters are obtained by the matched feature point pairs; finally, the first sample image and the second sample image are registered by the coordinate transformation parameters, and the first sample image and the second sample image are registered to the same angle, so that the positions of the first observation region and the second observation region are aligned, so as to determine the accurate change position of the observation region in the observation object.

[0111] When the first sample image and the second sample image are aligned, the corresponding relationship of the first sample image and the second sample image can be obtained. The corresponding relationship can be the corresponding relationship of the first pixel in the first sample image and the second pixel in the second sample image.

[0112] The first pixel is all the pixels in the first sample image, and the second pixel is all the pixels in the second sample image.

[0113] For example, please refer to Figure 3 , Figure 3 A corresponding relationship diagram of an image pair provided by an embodiment of the present application. The image pair includes a first sample image P1 and a second sample image P2. The first sample image P1 and the second sample image P2 are aligned to obtain the corresponding relationship as shown in Figure 3 . The first sample image P1 includes first pixels: P11, P12, P13, P14, P15, P16, P17, P18 and P19. The second sample image P2 includes first pixels: P21, P22, P23, P24, P25, P26, P27, P28 and P29. The corresponding relationship of the first pixel in the first sample image and the second pixel in the second sample image is: P11 corresponds to P21, P12 corresponds to P22, P13 corresponds to P23, P14 corresponds to P24, P15 corresponds to P25, P16 corresponds to P26, P17 corresponds to P27, P18 corresponds to P28, and P19 corresponds to P29.

[0114] The color parameter refers to the color value of the pixel, and the color value can be an RGB (red, green, blue) value. The difference value of the color parameter of the first pixel and the second pixel is calculated, that is, the difference value of the RGB value of the first pixel and the second pixel in the corresponding relationship is calculated.

[0115] For example, as Figure 3According to the corresponding relationship between each first pixel in the first sample image P1 and each second pixel in the second sample image P2, the RGB difference value between P11 and P21, the RGB difference value between P12 and P22, the RGB difference value between P13 and P23, the RGB difference value between P14 and P24, the RGB difference value between P15 and P25, the RGB difference value between P16 and P26, the RGB difference value between P17 and P27, the RGB difference value between P18 and P28, and the RGB difference value between P19 and P29 are calculated respectively. The difference information of the first sample image and the second sample image is obtained according to the RGB difference values of all corresponding pixel points.

[0116] In some embodiments, in order to avoid the influence of the change of the non-observation area in the first sample image on the result of the change of the observation area, the step of "determining the target pixel from the pixels of the first sample image according to the pixel difference information" can include the following operations:

[0117] When the difference value of the color parameter of the first pixel and the second pixel is greater than the preset difference value, the first pixel corresponding to the difference value is determined from the first sample image to obtain a pixel set;

[0118] The first observation area is determined in the first sample image;

[0119] The second observation area is determined in the second sample image;

[0120] The first difference area between the first observation area and the second observation area is determined from the first sample image;

[0121] The pixels located in the first difference area are extracted from the pixel set to obtain the target pixel.

[0122] The preset difference value is used to determine the change size of the first pixel to the second pixel. When the difference value of the first pixel and the second pixel is greater than the preset difference value, it can be considered that the change of the first pixel to the second pixel is large, which exceeds the acceptable change range. When the difference value of the first pixel and the second pixel is less than or equal to the preset difference value, it can be considered that the change of the first pixel to the second pixel is small, which does not exceed the acceptable change range.

[0123] Further, by comparing the difference value of the first pixel and the second pixel with the preset difference value, the first pixel corresponding to the difference value greater than the preset difference value can be determined, so as to obtain the pixel set. In the embodiments of the present application, the preset difference value is not limited, and the preset difference value can be set to different values according to actual conditions.

[0124] For example, the difference value of P11 and P21 is C1, the difference value of P12 and P22 is C2, the difference value of P13 and P23 is C3, the difference value of P14 and P24 is C4, the difference value of P15 and P25 is C5, the difference value of P16 and P26 is C6, the difference value of P17 and P27 is C7, the difference value of P18 and P28 is C8, and the difference value of P19 and P29 is C9. The difference values of the first pixel and the second pixel are compared with the preset difference value. The difference values greater than the preset difference value can be C1, C2, C4, C5 and C9. The first pixels corresponding to C1, C2, C4, C5 and C9 are P11, P12, P14, P15 and P19 respectively, and a pixel set is obtained.

[0125] The first observation area and the second observation area are defined as described above. The color parameter difference between the pixels in the first observation area and the pixels in other areas of the first sample image is greater than a preset threshold. The color parameter difference between the pixels in the second observation area and the pixels in other areas of the second sample image is greater than the preset threshold.

[0126] The second sample image is an image obtained after a certain period of time from the first sample image. The second observation area in the second sample image is obtained by changing the first observation area in the first sample image after a certain period of time. The change of the observation area can be shape change, size change or color change, etc. The first difference area can be determined according to the difference in shape, size or color between the first observation area and the second observation area.

[0127] Specifically, the non-overlapping area between the first observation area and the second observation area is determined. When the non-overlapping area is located in the first sample image, the position of the non-overlapping area in the first sample image is determined to obtain the first difference area. When the non-overlapping area is located in the second sample image, the corresponding position of the non-overlapping area in the first sample image is determined based on the correspondence between the first sample image and the second sample image to obtain the first difference area.

[0128] The target pixel is the main focus in model training. In the embodiments of the present application, in order to focus on the change of the first observation area in the first sample image in subsequent model training, and avoid the influence of the change of the area outside the first observation area in the first sample image on the first observation area, the change area of the first observation area and the second observation area in the first sample image, i.e. the first difference area, can be determined.

[0129] Further, the pixels located in the difference area are determined from the pixel set of the first sample image, i.e. the target pixels are obtained.

[0130] In some embodiments, in order to improve the image processing efficiency, before the step of "determining the pixel difference information of the first sample image and the second sample image", the following step can also be included:

[0131] identify the observation object in the first sample image, and extract the region where the observation object is located from the first sample image to obtain the first target sample image;

[0132] identify the observation object in the second sample image, and extract the region where the observation object is located from the second sample image to obtain the second target sample image;

[0133] Then the step of "determining the pixel difference information of the first sample image and the second sample image" can include the following flow:

[0134] determine the pixel difference information of the first target sample image and the second target sample image.

[0135] For example, the first sample image can be processed by image recognition technology, and the second sample image can be processed by image recognition technology.

[0136] Among them, image recognition refers to using computers to process, analyze and understand images to identify various different patterns of targets and objects, which is a practical application of deep learning algorithm. The traditional image recognition process is divided into four steps: image acquisition → image preprocessing → feature extraction → image recognition.

[0137] Specifically, the observation object can be identified from the first sample image according to the characteristics of the observation object, and then the region other than the observation object in the first sample image is cropped to obtain the first target sample image; and the observation object is identified from the second sample image, and the region other than the observation object in the second sample image is cropped to obtain the first target sample image. By this way, the region irrelevant to the observation object in the first sample image and the second sample image is removed, and the time for processing the irrelevant region is saved, so that the image processing speed can be improved.

[0138] 103, based on the position of the target pixel in the first image, process the first sample image to obtain a processed first sample image.

[0139] In the embodiments of the present application, processing the first sample image based on the position of the target pixel in the first image can be achieved by performing region weighting processing on the first sample image. Region weighting refers to applying different weights to the regions where the target pixel and other pixels in the first sample image are located. Specifically, the target pixel and other pixels in the first sample image other than the target pixel can be weighted respectively according to the preset weight.

[0140] The preset weight can include a first weight and a second weight, the first weight can be greater than the second weight, and the first weight and the second weight can be set according to actual conditions. The target pixels in the first sample image are weighted according to the first weight, and other display in the first sample image except the target pixels is weighted according to the second weight, to obtain a processed first sample image.

[0141] In the embodiment of the application, by applying the weight to the target pixel region in the first image, the focus of the network model in the subsequent model training process can be focused on the change part of the first observation region in the first sample image, and the influence of the change of the region outside the first observation region in the first sample image on the model training result is weakened.

[0142] 104. Training a preset network model according to the processed first sample image to obtain a trained model.

[0143] The preset network model refers to a network structure used for training data, and the preset network model can include multiple types. For example, in the embodiment of the application, the preset network model can be a 3D-Unet structure. The main feature of the 3D-Unet structure is the connection of low-level high-resolution features and up-sampled features. By fusing the low-level high-resolution feature map into the up-sampled feature map, the accuracy of the final network model can be improved.

[0144] Please refer to Figure 4 , Figure 4 A network model structure diagram is provided in the embodiment of the application. In Figure 4 The 3D-Unet network structure diagram shown in the figure includes an encoding path (left side) that is a “down-sampling stage” in a traditional network and a decoding path (right side) that is an “up-sampling stage”, and each path has four resolution levels.

[0145] Specifically, the encoding path includes two 3x3x3 convolutions at each layer, each followed by a ReLU (Rectified Linear Unit, linear rectifier function) layer, and then a 2x2x2 maximum pooling layer with a step of 2 in each direction; in the decoding path, each layer includes a 2x2x2 deconvolution layer with a step of 2, followed by two 3x3x3 convolution layers, each followed by a ReLU layer.

[0146] Among them, through concat, that is, shortcut (direct connection, dense connection of network layers, shortcut can cross many layers and can exist at the same time, by dividing the network into block and limiting the output channel number of each layer to reduce parameters and reduce computational complexity), the layers of the same resolution in the encoding path are passed to the decoding path to provide the original high-resolution features. The last layer is a 1x1x1 convolution layer that can reduce the number of output channels, and the last output channel number is the number of label categories.

[0147] BN (batch normalization) is used before ReLU, which is a standardization process for the input data of each layer during the training of the neural network. Traditional neural networks only standardize the sample before inputting it into the input layer (subtract the mean and divide the standard deviation) to reduce the difference between samples. BN is based on this, not only standardizes the input data of the input layer, but also standardizes the input of each hidden layer. In addition, the 3D U-Net network uses a weighted softmax loss function to set unmarked pixels to 0 so that the network can learn more about the marked pixels, thereby achieving universal characteristics.

[0148] In some embodiments, in order to improve the efficiency of model training, the step of "training the preset network model according to the processed first sample image to obtain a trained model" can include the following operations:

[0149] Input the processed first sample image into the preset network model for feature extraction to obtain feature data of the first sample image;

[0150] Generate a predicted image of the first sample image according to the feature data;

[0151] Calculate the similarity between the first sample image and the predicted image to obtain a first similarity;

[0152] Calculate the similarity between the second difference area and the third difference area to obtain a second similarity;

[0153] Determine the target similarity based on the first similarity and the second similarity, and adjust the model parameters of the preset network model according to the target similarity until the preset network model converges to obtain a trained model.

[0154] For example, the processed first sample image is input to the input end of the preset network model, the image is extracted for features through two 3x3x3 convolutions of each layer in the encoding path of the preset network model, and then the image size is recovered according to the extracted image features through a 2x2x2 deconvolution layer with a step of 2 and two 3x3x3 convolution layers in each layer in the decoding path of the preset network model, so that a prediction image with the same dimension as the first sample image is obtained at the output end of the preset network model. The dimension can refer to the size of the image, that is, the size of the prediction image is consistent with that of the first sample image.

[0155] The first similarity refers to the overlap degree of the first sample image and the second sample image.

[0156] In some embodiments, in order to improve the calculation efficiency, the step of "calculating the similarity between the first sample image and the prediction image to obtain the first similarity" can include the following process:

[0157] The prediction image is subjected to image segmentation processing to obtain a third observation region in the prediction image.

[0158] The overlap degree of the third observation region and the first observation region in the first sample image is determined to obtain the first similarity.

[0159] The image segmentation is a technology and process of dividing an image into a plurality of specific regions with unique properties and proposing a target of interest. It is a key step from image processing to image analysis. Existing image segmentation methods mainly include the following categories: threshold-based segmentation method, region-based segmentation method, edge-based segmentation method, and segmentation method based on specific theory, etc. From a mathematical point of view, image segmentation is a process of dividing a digital image into mutually disjoint regions. The process of image segmentation is also a marking process, that is, pixels belonging to the same region are assigned the same number.

[0160] In the embodiments of the present application, in order to improve the operation efficiency, the prediction image can be subjected to segmentation processing by using the gray threshold segmentation method. The gray threshold segmentation method is a most commonly used parallel region technology, and it is the most commonly used in image segmentation. The threshold segmentation method is actually a transformation from the input image f to the output image g as follows:

[0161] Wherein, T is the threshold value, g(i,j)=1 for the image element of the object, and g(i,j)=0 for the image element of the background. The key of the threshold segmentation algorithm is to determine the threshold value, and if a suitable threshold value is determined, the image can be accurately segmented. After the threshold value is determined, the threshold value is compared with the gray value of each pixel, and the pixel segmentation can be performed in parallel for each pixel, and the segmentation result directly gives the image region.

[0162] The third observation region in the prediction image can be obtained by segmenting the prediction image. The observation region is a local region of the observation object that needs to be detected in the above step. Therefore, the third observation region in the prediction image is the region after the change of the first observation region in the first sample image.

[0163] The overlap degree can be calculated by the Dice coefficient. The Dice coefficient is a function for evaluating similarity, and is usually used to calculate the similarity or overlap degree of two samples. The calculation formula of the Dice coefficient can be as follows:

[0164]

[0165] In the above formula, Dice represents the overlapping region of sample a and sample b, i.e., the overlap degree, Va represents the region of sample a, and Vb represents the region of sample b.

[0166] For example, the first observation region can be V1, and the third observation region can be V3. The Dice value of V1 and V3 can be calculated by the above calculation formula of the Dice coefficient, and the first similarity of the first sample image and the prediction image can be obtained.

[0167] The second difference region is the difference between the prediction image and the first sample image, and the third difference region is the difference between the first sample image and the second sample image.

[0168] Specifically, the similarity between the second difference region and the third difference region, i.e., the overlap degree between the second difference region and the third difference region, can be calculated by the Dice coefficient.

[0169] For example, the second difference region between the prediction image and the first sample image can be CV1, and the third difference region between the first sample image and the second sample image can be CV3. The Dice value of CV1 and CV3 can be calculated by the calculation formula of the Dice coefficient, and the second similarity between the second difference region and the third difference region can be obtained.

[0170] The target similarity is used to evaluate the index of the network model, and the target similarity can be calculated by a preset loss function.

[0171] In some embodiments, in order to improve the model training efficiency, the step of "determining the target overlap degree value based on the first overlap degree value and the second overlap degree value" can include the following operations:

[0172] determining the product of the first similarity and the second similarity, and the sum of the first similarity and the second similarity;

[0173] calculating the ratio of the product and the sum;

[0174] The target similarity is determined based on a product of the ratio and a preset coefficient.

[0175] In the embodiments of the present application, the preset loss function can be a DiceLoss loss function. The DiceLoss loss function is a network model that uses a dice coefficient as a loss function to evaluate the loss when performing image segmentation using deep learning.

[0176] For example, the DiceLoss loss function is as follows:

[0177] Avg = 2 * (D neg * Dpos) / (D neg + Dpos)

[0178] In the above formula, Avg represents the target similarity, Dneg represents the first similarity, and Dpos represents the second similarity. The target similarity can be obtained by calculating the first similarity and the second similarity according to the DiceLoss loss function.

[0179] 105. The trained model is used to process the to-be-processed image to obtain a target image corresponding to the to-be-processed image.

[0180] The to-be-processed image refers to an image of a target observation object, and the target image refers to a predicted image of the target observation object after a change. The to-be-processed image can be collected in various ways, such as being photographed by an imaging device or downloaded from the Internet.

[0181] Specifically, the to-be-processed object is input into the trained model, and after feature extraction and other processing of the to-be-processed object by the trained model, a target image corresponding to the to-be-processed image is output, and a change prediction result of the target observation object is obtained. The trained model generated by the present application can accurately predict the development of the observation object in the image, thereby improving the accuracy of entity development prediction.

[0182] The embodiment of the application discloses an image processing method, which comprises the following steps: acquiring a plurality of sample image pairs, wherein the sample image pair comprises a first sample image and a second sample image of the same observation object before and after a change; determining pixel difference information of the first sample image and the second sample image, and determining a target pixel from the pixels of the first sample image according to the pixel difference information; processing the first sample image based on the position of the target pixel in the first image to obtain a processed first sample image, wherein the processed first sample image is marked with a change area of the observation object; training a preset network model according to the processed first sample image to obtain a trained model; and processing a to-be-processed image based on the trained model to obtain a target image corresponding to the to-be-processed image. In this way, the embodiment of the application trains a network model by taking images of the same observation object collected at different time points as a sample training set, trains the network model according to the sample training set to obtain a trained model, predicts the change of the observation object in a to-be-processed image through the trained model, and outputs the to-be-processed image after the change, so that the prediction accuracy of the image change can be improved.

[0183] According to the above introduction, the image processing method of the application will be further illustrated by examples.

[0184] With the development of computer technology and the improvement of artificial intelligence algorithm effect, artificial intelligence will be more and more widely used in the medical industry.

[0185] Cerebral hemorrhage, which belongs to a kind of "stroke", is a common and serious brain complication of middle-aged and elderly patients with hypertension. Cerebral hemorrhage is a non-traumatic intracerebral hemorrhage caused by rupture of blood vessels, which has a sudden onset and a very high mortality rate, and is one of the fatal diseases of the elderly.

[0186] Through the image processing method in the application, a network model based on deep learning is designed to predict the development of cerebral hematoma in patients, providing reference information for subsequent treatment plans.

[0187] In this embodiment, the image processing method will be applied to the detection of cerebral hemorrhage in medicine as an example. Please refer to Figure 5 , Figure 5 The flowchart of another image processing method provided by the embodiment of the application is shown. The specific process can be as follows:

[0188] 201, the computer device acquires a plurality of images of a patient to obtain a plurality of sample images.

[0189] Wherein, the patient refers to a patient suffering from a cerebral hemorrhage condition, and the image refers to a brain image of the patient. The brain image can be a CT (Computed Tomography) image, which is composed of a certain number of pixels with different gray scales from black to white arranged in a matrix. These pixels reflect the X-ray absorption coefficient of the corresponding voxel. The size and number of pixels obtained by different CT devices are different. The size can be 1.0*1.0mm, 0.5*0.5mm, etc.; the number can be 256*256, i.e.65536, or 512*512, i.e.262144, etc. Obviously, the smaller the pixel, the more the number, the more detailed the image, i.e. high spatial resolution.

[0190] Specifically, CT is to scan a certain thickness of a layer of a human body by an X-ray beam, receive the X-ray through the layer by a detector, convert it into visible light, then convert it into an electric signal by photoelectric conversion, and then convert it into a digital signal by an analog / digital converter and input it into a computer for processing. The processing of image formation is as follows: divide the selected layer into a plurality of cuboids with the same volume, which are called voxels. The information obtained by scanning is calculated to obtain the X-ray attenuation coefficient or absorption coefficient of each voxel, which is then arranged into a matrix, i.e. a digital matrix, which can be stored in a magnetic disk or an optical disk. Each number in the digital matrix is converted into a small block with different gray scales from black to white by a digital / analog converter, i.e. a pixel, and arranged in a matrix, i.e. a CT image. Therefore, the CT image is a reconstructed image. The X-ray absorption coefficient of each voxel can be calculated by different mathematical methods.

[0191] For example, the head of the patient is scanned by a CT device to collect the CT image of the patient's head, and then the CT image is imported into a computer device to obtain the image of the patient.

[0192] Wherein, the CT image of the patient's head can be collected at different time points to obtain a plurality of sample images.

[0193] 202, the computer device combines the plurality of sample images two by two to obtain a sample image pair.

[0194] Wherein, the sample image pair includes a first sample image and a second sample image. The first sample image is a CT image of the patient collected at a previous time node, and the second sample image is a CT image of the patient collected at a subsequent time node.

[0195] For example, the collected patient images include: image A, image B, image C, and image D. The collection time of image A can be January 10, 2020, the collection time of the patient's image B can be January 20, 2020, the collection time of image C can be January 12, 2020, and the collection time of image D can be January 15, 2020. Then image A can be taken as a first sample image, image B can be taken as a second sample image, and a first sample image pair can be constructed; image A can be taken as a first sample image, image C can be taken as a second sample image, and a second sample image pair can be constructed; image A can be taken as a first sample image, image D can be taken as a second sample image, and a third sample image pair can be constructed; image C can be taken as a first sample image, image B can be taken as a second sample image, and a fourth sample image pair can be constructed; image C can be taken as a first sample image, image D can be taken as a second sample image, and a fifth sample image pair can be constructed; image D can be taken as a first sample image, image B can be taken as a second sample image, and a sixth sample image pair can be constructed.

[0196] 203、The computer device pre-processes the first sample image and the second sample image in the sample images to obtain a processed first sample image and a processed second sample image.

[0197] In the scheme of the present application, the purpose of pre-processing the first sample image and the second sample image is to align the first sample image and the second sample image, and remove the differences other than the changes caused by the development of the non-hematoma region.

[0198] Specifically, the image cropping method can be used to remove the regions other than the brain tissue in the first sample image and the second sample image. Wherein, removing the regions other than the brain tissue can include head cropping, bone removal, and obtaining a cropped brain image.

[0199] For example, please refer to Figure 6 , Figure 6 A cropped image provided by an embodiment of the present application. Figure 6 The left image is a patient brain CT image, that is, a first sample image or a second sample image. The sample image includes an observation region. By cropping the first sample image or the second sample image, the observation region in the sample image is extracted to obtain the right cropped image, that is, the observation region image. The region in the patient brain image that is irrelevant to the hematoma region change is removed, and the subsequent processing efficiency is improved.

[0200] Further, the image registration technology is used to register the first sample image and the second sample image to the same angle, so that the positions of the hematoma regions in the first sample image and the second sample image are aligned.

[0201] In some embodiments, since different CT devices may have different scan layer thicknesses when acquiring CT images, the CT images with different scan layer thicknesses may affect the training results in the training process. In order to avoid the influence of CT images with different scan layer thicknesses on the training results in the training process, the multiple sample images are resampled to the same layer thickness, and all the data are fixed to the same dimension by padding and cropping.

[0202] The resampling refers to resampling the CT images to a new physical space size. In CT, each pixel has a corresponding physical space size. In order to align the physical size of each pixel of the two images, the resampling is used to align the two images. The shape of the aligned sample is also not completely consistent, and therefore padding or cropping is used to make the size of the two CTs consistent.

[0203] Further, the sample images are standardized. Image standardization is a centering process of data by mean removal. According to the convex optimization theory and the knowledge related to data probability distribution, the data centering conforms to the data distribution law, and the generalization effect after training is more easily achieved. Data standardization is one of the common methods of data preprocessing. By standardizing the sample images, the value of each pixel in the image is mapped to the range of [-1, 1]. The value range is the distribution range of the value of the pixel on the image.

[0204] 204, the computer device inputs the first sample image into the preset network model for processing to generate a predicted image corresponding to the first sample image.

[0205] The preset network model refers to a network model used for deep learning. In the embodiments of the present application, in order to improve the training efficiency of the model, a 3D U-Net network can be used.

[0206] Biomedical images are usually block-shaped, that is, they are composed of many slices. If a 2D image processing model is used to process 3D, there is a problem that the biomedical images have to be sent into the designed model for training in groups (including training data and labeled data). In this case, the training efficiency is reduced, and the data preprocessing method is relatively complicated.

[0207] The 3D U-Net network can take the whole picture as input to the model for training. In addition, the 3D U-Net uses a weighted softmax loss function to set the unlabeled pixel points to 0 so that the network can learn more only from the labeled pixel points, thereby achieving the characteristics of universality.

[0208] Specifically, the first sample image is input into the network from the input end of the preset network model, and the first sample image is processed by down-sampling and up-sampling through the preset network model, so that a prediction image with the same dimension as the first sample image can be obtained at the network prediction end. The down-sampling and up-sampling of the first sample image through the preset network model can refer to the description of the previous embodiment, and will not be repeated here.

[0209] 205、The computer device optimizes the preset network model based on a preset loss function to obtain a trained model.

[0210] In the embodiments of the present application, the image data of the first sample image after processing is extracted to obtain sample training data, and the preset network model is trained based on the sample training data.

[0211] During the training process, the sample training data is weighted. The weight of the weighting is composed of two parts. The first part is called positive sample weighting. In the present scheme, the sample training data is divided into positive sample data and negative sample data according to the judgment standard of hematoma enlargement (absolute change greater than 3ml or relative change rate greater than 33%), and the positive sample data is multiplied by a larger weight during the training process. The second part is called regional weighting. In the present scheme, the mask (a image with the same size as the image, with only 0 and 1 on it, where 1 represents hematoma) obtained by labeling the hematoma is used to calculate the difference mask between the first sample image and the second sample image, and a larger weight is applied in this region, so that the model focuses on the change part of the hematoma.

[0212] The prediction target of the present scheme is to accurately depict the hematoma region in the second sample image. Therefore, a hematoma segmentation method is used to segment the hematoma region of the prediction image. Specifically, the hematoma segmentation tool can be used to segment the hematoma of the prediction image to obtain the mask of the hematoma region in the prediction image.

[0213] Further, after the sample training data is weighted, the preset loss function is used for constraint, so that the constrained region of the preset network model focuses on the change region of the sample with obvious change as much as possible, and the generalization ability of the network affected by the difference between the images caused by the development of other non-hematoma regions is reduced.

[0214] The preset loss function can be a DiceLoss loss function, as follows:

[0215] Avg=2*(D neg *Dpos) / (D neg +Dpos)

[0216] In the above formula, Dneg refers to the degree of overlap of the hematoma region of the predicted image with the hematoma region of the first sample image, and Dpos refers to the degree of overlap of the first difference region and the second difference region, the first difference region being the difference region between the predicted image and the first sample image, and the second difference region being the difference region between the first sample image and the second sample image. The similarity Avg is calculated by a preset loss function, and the similarity Avg is taken as an index for evaluating the model.

[0217] When the preset network model has been basically fitted to the sample training data, the model parameter with the best prediction result is selected from the pre-prepared verification set as the finally trained model, and the trained model is obtained.

[0218] 206, the computer device obtains a to-be-predicted image of a target patient, and processes the to-be-detected image based on the trained model to obtain a target predicted image of the target patient.

[0219] The to-be-predicted image refers to the current brain CT image of the target patient, and the to-be-predicted image is input into the trained model for processing. The target predicted image is output by the trained model for predicting the change of the hematoma region in the to-be-predicted image. The target predicted image is an image after the change of the hematoma region.

[0220] For example, please refer to Figure 7 , Figure 7 A schematic diagram of predicting the development of a local region of an image is provided in an embodiment of the present application. Figure 7 In the above formula, Dneg refers to the degree of overlap of the hematoma region of the predicted image with the hematoma region of the first sample image, and Dpos refers to the degree of overlap of the first difference region and the second difference region, the first difference region being the difference region between the predicted image and the first sample image, and the second difference region being the difference region between the first sample image and the second sample image. The similarity Avg is calculated by a preset loss function, and the similarity Avg is taken as an index for evaluating the model.

[0221] In the above formula, Dneg refers to the degree of overlap of the hematoma region of the predicted image with the hematoma region of the first sample image, and Dpos refers to the degree of overlap of the first difference region and the second difference region, the first difference region being the difference region between the predicted image and the first sample image, and the second difference region being the difference region between the first sample image and the second sample image. The similarity Avg is calculated by a preset loss function, and the similarity Avg is taken as an index for evaluating the model.

[0222] The embodiment of the application discloses an image processing method, which comprises the following steps: collecting multiple images of a patient to obtain multiple sample images; combining the multiple sample images two by two by a computer device to obtain sample image pairs; preprocessing a first sample image and a second sample image in the sample images by the computer device to obtain a processed first sample image and a processed second sample image; inputting the first sample image into a preset network model by the computer device to process the first sample image and generate a predicted image corresponding to the first sample image; optimizing the preset network model based on a preset loss function by the computer device to obtain a trained model; obtaining a to-be-predicted image of a target patient by the computer device, and processing the to-be-detected image based on the trained model to obtain a target predicted image of the target patient. In this way, based on a deep full convolution network design, a network model is trained by collecting a large number of CT images of patients at different time points, so that the model can finally predict the possible future state of a single CT image, and the future development state of a lesion is displayed, which has important significance for the medical field.

[0223] In order to better implement the image processing method provided by the embodiment of the application, the embodiment of the application further provides an image processing device based on the above-mentioned image processing method. The meanings of the terms are the same as those in the above-mentioned image processing method, and specific implementation details can be referred to the description in the method embodiment.

[0224] Please refer to Figure 8 , Figure 8 The structure block diagram of the image processing device provided by the embodiment of the application comprises:

[0225] The acquisition unit 301 is configured to acquire multiple sample image pairs, and each sample image pair comprises a first sample image before a change of an observation object and a second sample image after the change of the observation object.

[0226] The determination unit 302 is configured to determine pixel difference information of the first sample image and the second sample image, and determine a target pixel from pixels of the first sample image according to the pixel difference information.

[0227] The first processing unit 303 is configured to process the first sample image based on a position of the target pixel in the first image to obtain a processed first sample image, and the processed first sample image is marked with a change region of the observation object.

[0228] The training unit 304 is configured to train a preset network model according to the processed first sample image to obtain a trained model.

[0229] The second processing unit 305 is configured to process a to-be-processed image based on the trained model to obtain a target image corresponding to the to-be-processed image, and the target image is a predicted image after the change of the to-be-processed image.

[0230] In some embodiments, referring to Figure 9 , Figure 9 Another structural block diagram of an image processing apparatus provided by an embodiment of the present application is shown in FIG. 3. The determining unit 302 can include:

[0231] The alignment sub-unit 3021 is configured to align the first sample image and the second sample image to obtain a correspondence between a first pixel in the first sample image and a second pixel in the second sample image.

[0232] The first calculation sub-unit 3022 is configured to calculate a difference value of color parameters of the first pixel and the second pixel based on the correspondence to obtain pixel difference information of the first sample image and the second sample image.

[0233] In some embodiments, referring to Figure 10 , Figure 10 Another structural block diagram of an image processing apparatus provided by an embodiment of the present application is shown in FIG. 3. The determining unit 302 can include:

[0234] The first determination sub-unit 3023 is configured to determine, from the first sample image, a first pixel corresponding to the difference value when the difference value of the color parameters of the first pixel and the second pixel is greater than a preset difference value to obtain a pixel set.

[0235] The second determination sub-unit 3024 is configured to determine a first observation region in the first sample image, wherein a pixel in the first observation region has a color parameter difference value greater than a preset threshold value from pixels in other regions of the first sample image.

[0236] The third determination sub-unit 3025 is configured to determine a second observation region in the second sample image, wherein a pixel in the second observation region has a color parameter difference value greater than a preset threshold value from pixels in other regions of the second sample image.

[0237] The fourth determination sub-unit 3026 is configured to determine, from the first sample image, a first difference region between the first observation region and the second observation region.

[0238] The extraction sub-unit 3027 is configured to extract, from the pixel set, a pixel located in the first difference region to obtain a target pixel.

[0239] In some embodiments, the alignment sub-unit 3021 can be specifically configured to:

[0240] determine a first observation region in the first sample image and a second observation region in the second sample image;

[0241] extract feature information of the first observation region and feature information of the second observation region;

[0242] The first image and the second image are aligned to the same angle according to the characteristic information of the first observation area and the characteristic information of the second observation area.

[0243] In some embodiments, the training unit 304 can include:

[0244] The input subunit is configured to input the processed first sample image into a preset network model for feature extraction to obtain feature data of the first sample image.

[0245] The generation subunit is configured to generate a predicted image of the first sample image according to the feature data.

[0246] The second calculation subunit is configured to calculate the similarity between the first sample image and the predicted image to obtain a first similarity.

[0247] The third calculation subunit is configured to calculate the similarity between the second difference area and the third difference area to obtain a second similarity, wherein the second difference area is a difference part of the predicted image and the first sample image, and the third difference area is a difference part of the first sample image and the second sample image.

[0248] The convergence subunit is configured to determine a target similarity based on the first similarity and the second similarity, and adjust the model parameters of the preset network model according to the target similarity until the preset network model converges to obtain a trained model.

[0249] In some embodiments, the convergence subunit can be specifically configured to:

[0250] Determine the product of the first similarity and the second similarity, and the sum of the first similarity and the second similarity.

[0251] Calculate the ratio of the product and the sum.

[0252] Determine the target similarity based on the product of the ratio and a preset coefficient.

[0253] In some embodiments, the second calculation subunit can be specifically configured to:

[0254] Perform image segmentation processing on the predicted image to obtain a third observation area in the predicted image, and the color parameter difference between the pixels in the third observation area and the pixels in other areas in the third sample image is greater than a preset threshold.

[0255] Determine the overlap degree between the third observation area and the first observation area in the first sample image to obtain the first similarity.

[0256] In some embodiments, the acquisition unit 301 can include:

[0257] The acquisition subunit is configured to acquire images of at least one observation object at different time points to obtain a plurality of sample images.

[0258] The selection subunit is configured to select sample images belonging to the same observation object from the plurality of sample images, and combine the sample images belonging to the same observation object two by two to obtain a plurality of sample image pairs.

[0259] In some embodiments, the apparatus can further include:

[0260] The first identification unit is configured to identify the observation object in the first sample image, and extract a region where the observation object is located from the first sample image to obtain a first target sample image.

[0261] The second identification unit is configured to identify the observation object in the second sample image, and extract a region where the observation object is located from the second sample image to obtain a second target sample image.

[0262] In some embodiments, the determination unit further includes:

[0263] The fifth determination subunit is configured to determine pixel difference information of the first target sample image and the second target sample image.

[0264] Embodiments of the present application disclose an image processing apparatus. A plurality of sample image pairs are obtained by an acquisition unit 301. The sample image pair includes a first sample image before a change of an observation object and a second sample image after the change of the observation object. A determination unit 302 determines pixel difference information of the first sample image and the second sample image, and determines a target pixel from pixels of the first sample image according to the pixel difference information. A first processing unit 303 processes the first sample image based on a position of the target pixel in the first image to obtain a processed first sample image. The processed first sample image is marked with a change region of the observation object. A training unit 304 trains a preset network model according to the processed first sample image to obtain a trained model. A second processing unit 305 processes a to-be-processed image based on the trained model to obtain a target image corresponding to the to-be-processed image. The target image is a predicted image after the change of the to-be-processed image. In this way, the prediction accuracy of the image change can be improved.

[0265] Embodiments of the present application also provide a computer device. The computer device can be a server, as shown in Figure 11 The server can include a processor 701 with one or more processing cores, a memory 702 with one or more computer readable storage media, a power supply 703, and an input unit 704. Those skilled in the art can understand that the server can include other components, and the components are not limited to the above-mentioned components.

[0266] The server can include a processor 701 with one or more processing cores, a memory 702 with one or more computer readable storage media, a power supply 703, and an input unit 704. Those skilled in the art can understand that the server can include other components, and the components are not limited to the above-mentioned components.Figure 11 The server structure shown in the figure is not intended to limit the server, and can include more or fewer components than shown, or combine certain components, or arrange different components. Among them:

[0267] The processor 701 is the control center of the server, which connects various parts of the server through various interfaces and lines, and performs various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 702 and calling data stored in the memory 702, thereby overall detecting the server. Optionally, the processor 701 can include one or more processing cores; preferably, the processor 701 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 701.

[0268] The memory 702 can be used to store software programs and modules, and the processor 701 executes various functions and data processing by running the software programs and modules stored in the memory 702. The memory 702 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the server, etc. In addition, the memory 702 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 702 can also include a memory controller to provide access for the processor 701 to the memory 702.

[0269] The server also includes a power supply 703 for supplying power to various components, and preferably the power supply 703 can be logically connected to the processor 701 through a power management system, so as to realize the functions of managing charging, discharging and power consumption management through the power management system. The power supply 703 can also include one or more than one direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, etc. Any component.

[0270] The server can also include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0271] Although not shown, the server can also include a display unit and the like, which will not be described here. In particular, in the present embodiment, the processor 701 in the server will load the executable file corresponding to the process of one or more application programs into the memory 702 according to the following instructions, and run the application program stored in the memory 702 by the processor 701, thereby realizing various functions, as follows:

[0272] Obtain a plurality of sample image pairs, the sample image pair comprising a first sample image and a second sample image of the same observation object before and after the change;

[0273] Determine the pixel difference information of the first sample image and the second sample image, and determine the target pixel from the pixels of the first sample image according to the pixel difference information;

[0274] Based on the position of the target pixel in the first image, the first sample image is processed to obtain a processed first sample image, and the processed first sample image is marked with the change area of the observation object;

[0275] According to the processed first sample image, the preset network model is trained to obtain a trained model;

[0276] Based on the trained model, the to-be-processed image is processed to obtain a target image corresponding to the to-be-processed image.

[0277] The specific implementation of each operation can refer to the foregoing embodiments, which will not be described here.

[0278] As can be seen from the above, the server of the present embodiment can realize the steps of image processing, and improve the security of identity information verification.

[0279] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a storage medium and loaded and executed by a processor.

[0280] Therefore, the embodiments of the present application provide a storage medium, which stores a plurality of instructions, the instructions can be loaded by a processor to execute the steps in any data processing method provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0281] Obtain a plurality of sample image pairs, the sample image pair comprising a first sample image and a second sample image of the same observation object before and after the change; determine the pixel difference information of the first sample image and the second sample image, and determine the target pixel from the pixels of the first sample image according to the pixel difference information; process the first sample image based on the position of the target pixel in the first image to obtain a processed first sample image, and the processed first sample image is marked with the change area of the observation object; train a preset network model according to the processed first sample image to obtain a trained model; process the to-be-processed image based on the trained model to obtain the target image corresponding to the to-be-processed image.

[0282] The specific implementation of each operation can refer to the foregoing embodiments, which will not be described here.

[0283] The storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0284] Due to the instructions stored in the storage medium, the steps of any image processing method provided in the embodiments of the present application can be executed, and thus the beneficial effects of any image processing method provided in the embodiments of the present application can be achieved. Details are described in the foregoing embodiments, which will not be described here.

[0285] The embodiments of the present application also provide a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the terminal reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the terminal executes the image processing method provided in various optional implementation manners of the above aspects.

[0286] The image processing method, device, computer device and storage medium provided in the embodiments of the present application are described in detail above, and the principle and implementation manner of the present application are described by applying specific examples in this paper. The above embodiment is only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as the limitation of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Multiple sample image pairs are acquired, each sample image pair including a first sample image before the change in the same brain hemorrhage area and a second sample image after the change, wherein the sample image is a CT image containing the brain hemorrhage area; Determine the pixel difference information between the first sample image and the second sample image, and determine the target pixel from the pixels of the first sample image based on the pixel difference information, wherein the pixel difference information includes the color difference between the pixels in the first sample image and the second sample image; Based on the position of the target pixel in the first sample image, the first sample image is processed to obtain a processed first sample image, and the processed first sample image is marked with the changed area of ​​the cerebral hemorrhage area. The preset network model is trained based on the processed first sample image to obtain the trained model. The preset network model includes an encoding path, a decoding path, and a loss function module. The encoding path is directly connected to the layers of the same resolution in the decoding path. The loss function module sets the pixels in the non-changing regions of the processed first sample image to zero. The trained model is used to process the CT image to be processed to obtain the target image corresponding to the CT image to be processed. The target image is the predicted CT image after the CT image to be processed has been changed.

2. The method according to claim 1, characterized in that, Determining the pixel difference information between the first sample image and the second sample image includes: Align the first sample image with the second sample image to obtain the correspondence between the first pixel in the first sample image and the second pixel in the second sample image; Based on the correspondence, the difference in color parameters between the first pixel and the second pixel is calculated to obtain pixel difference information between the first sample image and the second sample image.

3. The method according to claim 2, characterized in that, Determining the target pixel from the pixels of the first sample image based on the pixel difference information includes: When the difference between the color parameters of the first pixel and the second pixel is greater than a preset difference value, the first pixel corresponding to the difference value is determined from the first sample image to obtain a pixel set; A first observation region is determined in the first sample image, wherein the color parameter difference between the pixels in the first observation region and the pixels in other regions of the first sample image is greater than a preset threshold. A second observation region is determined in the second sample image, wherein the color parameter difference between the pixels in the second observation region and the pixels in other regions of the second sample image is greater than a preset threshold. From the first sample image, determine the first difference region between the first observation region and the second observation region; The target pixel is obtained by extracting the pixels located in the first difference region from the pixel set.

4. The method according to claim 2, characterized in that, Aligning the first sample image with the second sample image includes: Determine a first observation region in the first sample image and a second observation region in the second sample image; Extract the feature information of the first observation area and the feature information of the second observation area; Based on the feature information of the first observation area and the feature information of the second observation area, the first sample image and the second sample image are aligned to the same angle.

5. The method according to claim 1, characterized in that, The step of training a preset network model based on the processed first sample image to obtain a trained model includes: The processed first sample image is input into the preset network model for feature extraction to obtain the feature data of the first sample image; Generate a predicted image of the first sample image based on the feature data; Calculate the similarity between the first sample image and the predicted image to obtain a first similarity. Calculate the similarity between the second difference region and the third difference region to obtain the second similarity, wherein the second difference region is the difference between the predicted image and the first sample image, and the third difference region is the difference between the first sample image and the second sample image; The target similarity is determined based on the first similarity and the second similarity, and the model parameters of the preset network model are adjusted according to the target similarity until the preset network model converges to obtain the trained model.

6. The method according to claim 5, characterized in that, Determining the target similarity based on the first similarity and the second similarity includes: Determine the product of the first similarity and the second similarity, and the sum of the first similarity and the second similarity; Calculate the ratio of the product to the sum; The target similarity is determined based on the product of the ratio and a preset coefficient.

7. The method according to claim 5, characterized in that, The step of calculating the similarity between the first sample image and the predicted image to obtain a first similarity includes: The predicted image is segmented to obtain a third observation region in the predicted image. The color parameter difference between the pixels in the third observation region and the pixels in other regions of the third sample image is greater than a preset threshold. The overlap between the third observation region and the first observation region in the first sample image is determined to obtain the first similarity.

8. The method according to claim 1, characterized in that, The acquisition of multiple sample image pairs includes: Images of at least one cerebral hemorrhage area were acquired at different time points to obtain multiple sample images; Select sample images belonging to the same brain hemorrhage region from the multiple sample images, and combine the sample images belonging to the same brain hemorrhage region in pairs to obtain multiple sample image pairs.

9. The method according to claim 1, characterized in that, Before determining the pixel difference information between the first sample image and the second sample image, the method further includes: The brain hemorrhage area in the first sample image is identified, and the area where the brain hemorrhage area is located is extracted from the first sample image to obtain the first target sample image; The brain hemorrhage area in the second sample image is identified, and the area where the brain hemorrhage area is located is extracted from the second sample image to obtain the second target sample image; Determining the pixel difference information between the first sample image and the second sample image includes: Determine the pixel difference information between the first target sample image and the second target sample image.

10. An image processing apparatus, characterized in that, The device includes: The acquisition unit is used to acquire multiple sample image pairs, wherein the sample image pair includes a first sample image before the change in the same brain hemorrhage area and a second sample image after the change, and the sample image is a CT image containing the brain hemorrhage area; A determining unit is configured to determine pixel difference information between the first sample image and the second sample image, and to determine a target pixel from the pixels of the first sample image based on the pixel difference information, wherein the pixel difference information includes color differences between the pixels in the first sample image and the second sample image; The first processing unit is configured to process the first sample image based on the position of the target pixel in the first sample image to obtain a processed first sample image, wherein the processed first sample image is marked with the changed area of ​​the cerebral hemorrhage region. The training unit is used to train a preset network model based on the processed first sample image to obtain a trained model. The preset network model includes an encoding path, a decoding path, and a loss function module. The encoding path is directly connected to layers of the same resolution in the decoding path. The loss function module sets the pixels in the unchanging regions of the processed first sample image to zero. The second processing unit is used to process the CT image to be processed based on the trained model to obtain a target image corresponding to the CT image to be processed. The target image is a CT image containing the brain hemorrhage region after the predicted changes to the CT image to be processed.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the program, it implements the image processing method as described in any one of claims 1 to 9.

12. A storage medium, characterized in that, The storage medium stores a plurality of instructions adapted for loading by a processor to execute the image processing method according to any one of claims 1 to 9.

Images