Image segmentation processing method based on deep learning, system thereof, and electronic device

Abstract

The present invention provides an image segmentation processing method based on deep learning, a system thereof, and an electronic device, which can be used to train a sub-graph fine segmentation model using a sub-graph containing a target object after dividing the image to be processed into sub-graphs, repeat the training until the sub-graph fine segmentation model reaches a preset segmentation index, and then segment the sub-graph containing the target object to obtain a sub-graph mask corresponding to the sub-graph. Finally, the sub-graph mask is spliced ​​based on the sub-graph coordinates to obtain the segmentation result of the image to be processed. The above-mentioned image segmentation processing method based on deep learning can realize transfer learning, reduce network training time, and improve segmentation accuracy in the process of iterative field of view focusing through a sub-graph cutting method and a macro to micro analysis method. The present invention can be widely applied to image segmentation processing with multiple target objects having multiple scales and / or high original image resolution but a small proportion of target objects, and is particularly suitable for medical image segmentation tasks.

Description

Technical field

[0001] The present invention relates to the field of image processing, and in particular to an image segmentation processing method based on deep learning, a system thereof, and an electronic device [Background Technology]

[0002] With the continuous development of artificial intelligence, the demand for image processing is increasing. In order to better analyze and process images, the existing method is to use feature extractors and other methods to extract corresponding image features. However, due to the limitations of existing image analysis and processing technologies, for images of multi-target and multi-scale target objects, or images with high original image resolution but a small proportion of target objects, the data processing time required is long and it is difficult to meet the user's needs for high-precision image analysis and processing.

[0003] Therefore, there is an urgent need to provide a new technical solution that can effectively solve the above-mentioned image analysis and processing. [Summary of the invention]

[0004] In order to solve the technical problems existing in existing image analysis and processing, the present invention provides an image segmentation processing method based on deep learning, a system thereof, and an electronic device.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: an image segmentation processing method based on deep learning, which comprises the following steps: step S1: providing an image to be processed having at least one target object, and performing sub-graph segmentation on the image to be processed to obtain a sub-graph containing the target object and its sub-graph coordinates; step S2, using the sub-graph containing the target object to train a sub-graph fine segmentation model, repeating the training until the sub-graph fine segmentation model reaches a preset segmentation index, and then segmenting the sub-graph containing the target object to obtain a sub-graph mask corresponding to the sub-graph; and step S3, combining the sub-graph coordinates and the sub-graph mask to perform sub-graph mask splicing to obtain the segmentation result of the image to be processed.

[0006] Preferably, in step S2, the variables involved in the current training are used as initial values ​​of the variables for the next training, and each training includes multiple iterations.

[0007] Preferably, the step S2 specifically includes the following steps: step S21, initializing the variables involved in the sub-image fine segmentation model; step S22, using the sub-image containing the target object as training data, updating the variables at the beginning of each training and using the cropped image and mask to train the sub-image fine segmentation model, after multiple iterations until convergence or when the number of iterations is equal to the preset maximum number of iterations, then stopping the current training and obtaining new variables; step S23, determining whether to continue or stop training based on whether the sub-image fine segmentation model reaches the preset segmentation index; and step S24, saving the segmentation results under the corresponding variables in the current sub-image fine segmentation model.

[0008] Preferably, in the above step S2, the variables involved in the sub-graph fine segmentation model include a combination of cropping coefficient, segmentation index coefficient, number of training times, and network parameters.

[0009] Preferably, in the above step S23, the preset segmentation index includes that the number of training times is equal to the maximum number of training times or the segmentation index coefficient of the current training times is less than the previous segmentation index coefficient.

[0010] Preferably, the cropped image and mask in step S22 can be obtained by performing center cropping on the sub-image containing the target object and the mask corresponding thereto based on the center position of the target object to obtain the cropped image and mask.

[0011] Preferably, during the training process, the cropping coefficient decreases as the number of training times increases; the center position of the current training is calculated based on the segmentation result of the previous training.

[0012] In order to solve the above technical problems, the present invention provides another technical solution as follows: an image segmentation system based on deep learning, comprising: a sub-graph segmentation module, used to provide an image to be processed having at least one target object, and perform sub-graph segmentation on the image to be processed to obtain a sub-graph containing the target object and its sub-graph coordinates; a sub-graph fine segmentation module, used to train a sub-graph fine segmentation model using a sub-graph containing the target object, repeat the training until the sub-graph fine segmentation model reaches a preset segmentation index, and then segment the sub-graph containing the target object to obtain a sub-graph mask corresponding to each sub-graph; and an image segmentation mask generation module, used to combine the sub-graph coordinates and the sub-graph mask to perform sub-graph mask splicing to obtain the segmentation result of the image to be processed.

[0013] In order to solve the above technical problems, the present invention provides another technical solution as follows: an electronic device, which includes a storage unit and a processing unit, the storage unit is used to store computer programs, and the processing unit is used to execute the steps of the above-mentioned deep learning-based image segmentation processing method through the computer program stored in the storage unit.

[0014] Compared with the prior art, the image segmentation processing method based on deep learning and its system and electronic device provided by the present invention have the following beneficial effects:

[0015] The present invention provides an image segmentation processing method based on deep learning, which can be used to train a sub-graph fine segmentation model using a sub-graph containing a target object after dividing the image to be processed into sub-graphs, thereby obtaining a sub-graph mask corresponding to the sub-graph, and finally performing sub-graph mask splicing based on the sub-graph coordinates to obtain the segmentation result of the image to be processed. The above-mentioned image segmentation processing method based on deep learning can realize transfer learning from macro to micro in the process of iterative field of view focusing through sub-graph cutting, thereby effectively reducing network training time and improving segmentation accuracy. The image segmentation processing method based on deep learning provided by the present invention can be widely used in image segmentation processing with multiple target objects having multiple scales and / or high original image resolution but a small proportion of target objects, and is particularly suitable for small target segmentation tasks in medical images.

[0016] In the deep learning-based image segmentation processing method provided by the present invention, based on step S2, the variables involved in the current training are used as the initial values ​​of the variables for the next training. Each training includes multiple iterations, thereby obtaining corresponding refined segmentation results, further improving the effectiveness of training, reducing network training time, and improving segmentation accuracy. The training can also be automatically determined based on the actual training results.

[0017] Furthermore, in the above-mentioned step S21, the variable corresponding to the sub-graph fine segmentation model is limited and the preset segmentation index is defined as the number of training times of the current training is equal to the maximum number of training times; or the segmentation index coefficient of the current training is smaller than the size of the segmentation index coefficient of the previous training, which can make the sub-graph fine segmentation model have wider applicability and controllability.

[0018] The step of obtaining the cropped image and mask in step S22 can ensure that the target object can always be in the center of the cropped image during the cropping process of training, so that in the iterative field of view focusing process, the accuracy of image segmentation processing based on the target object can still be maintained, and the segmentation processing speed can be improved. During the training process, the cropping coefficient decreases as the number of training times increases. Therefore, as the training progresses, the side lengths of the cropped image and mask will also shorten as the cropping coefficient changes, thereby achieving further focusing of the image field of view. When the center position of the current training is related to the segmentation result of the previous training, and the variables involved in the current training are used as the initial values ​​of the variables of the next training, it is possible to ensure that transfer learning is achieved during the training process. If multiple iterations are included in a single training, it is possible to further improve the refinement and accuracy of the segmentation based on the iterative field of view focusing.

[0019] In step S1 of the deep learning-based image segmentation method provided by the present invention, sub-image segmentation of the image to be processed includes first coarsely segmenting the image to be processed into multiple target objects, thereby determining the bounding boxes and center positions of the target objects through the coarse segmentation; and then performing sub-image segmentation based on the bounding boxes and center positions of the target objects. Based on these processing steps, the accuracy and speed of sub-image segmentation of the image to be processed can be improved.

[0020] The present invention also provides an image segmentation system and an electronic device based on deep learning, which have the same beneficial effects as the above-mentioned image segmentation processing method based on deep learning, can effectively reduce the network training time for high-precision image segmentation and improve the segmentation accuracy, and are particularly suitable for segmentation tasks in medical images.

Brief Description of the Drawings

[0021] Figure 1A This is a flowchart of the steps of the image segmentation processing method based on deep learning provided by the first embodiment of the present invention.

[0022] Figure 1B yes Figure 1A , in step 1 of the deep learning-based image segmentation processing method, a flow chart of the specific steps for sub-dividing the image to be processed is shown.

[0023] Figure 2A yes Figure 1B FIG. 4 is a schematic diagram of a rough segmentation operation in step S01.

[0024] Figure 2B is Figure 2A Schematic diagram of sub-image segmentation after determining the bounding box and center position of the target object based on the shown basis.

[0025] Figure 2C is Figure 2B A schematic diagram of cutting out corresponding subgraphs based on the shown.

[0026] Figure 3 It is a flowchart of specific implementation steps of step S2 shown in FIG1 .

[0027] Figure 4 yes Figure 3 FIG. 5 is a flow chart showing the specific implementation steps of step S22.

[0028] Figure 5 yes Figure 3 FIG. 5 is a flow chart showing the specific implementation steps of step S23.

[0029] Figure 6This is a schematic diagram of the functional modules of the deep learning-based image segmentation processing system provided in the second embodiment of the present invention.

[0030] Figure 7 yes Figure 6 Schematic diagram of the functional modules of the subgraph segmentation module shown in .

[0031] Figure 8 It is a schematic diagram of functional modules of an electronic device provided by the third embodiment of the present invention.

[0032] Description of the accompanying symbols: S, image; S0, S1 and S2, sub-images; P1, the center position of sub-image S1; P2, the center position of sub-image S2;

[0033] 20. Image segmentation system based on deep learning; 21. Sub-image segmentation module; 22. Sub-image fine segmentation module; 23. Image segmentation mask generation module; 211. Image coarse segmentation module; 212. Segmentation module;

[0034] 30. Electronic device; 31. Storage unit; 32. Processing unit. [Specific implementation method]

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] See also Figure 1A The first embodiment of the present invention provides an image segmentation processing method S10 based on deep learning, which includes the following steps:

[0037] Step S1: providing an image to be processed having at least one target object, and performing sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and its sub-image coordinates;

[0038] Step S2: using the sub-image containing the target object to train a sub-image fine segmentation model, repeating the training until the sub-image fine segmentation model reaches a preset segmentation index, segmenting the sub-image containing the target object to obtain a sub-image mask corresponding to the sub-image; and

[0039] Step S3: combining the sub-image coordinates with the sub-image masks and performing sub-image mask splicing to obtain a segmentation result of the image to be processed.

[0040] Based on the above-mentioned deep learning-based image segmentation processing method S10, it is possible to obtain image segmentation results of the required accuracy in two application scenarios, one of which is that multiple target objects correspond to images of multiple scales, and the other is that the original image has a high resolution, while the corresponding target objects account for a relatively small proportion of the image. The above-mentioned method can be implemented based on the target object, from large to small, from macro to micro, thereby improving the image segmentation accuracy. It should be noted that the images of multiple scales mentioned in the present invention can be understood as target objects that can be processed separately at different scales. In some embodiments, the scale referred to can also be understood as multiple resolutions.

[0041] See also Figure 1B In the above step S1, the sub-image segmentation of the image to be processed specifically includes the following steps:

[0042] Step S01: Input the image to be processed into a coarse segmentation model for coarse segmentation to obtain the bounding box and center position of the corresponding target object; and

[0043] Step S02: performing sub-image segmentation based on the bounding box and center position of the corresponding target object.

[0044] Specifically, in the above step S1, the coarse segmentation model can be obtained based on training, and the original training image training set with segmentation annotations is used to train the initial coarse segmentation model. In order to facilitate processing, the memory limit value of the training image can be set. If the memory size of the original training image exceeds the memory limit value, the original training image can be processed accordingly using the original length and width proportionally reduced version of the image data. Among them, the original training image with segmentation annotations and the image to be processed are of the same type of image and have similar target objects to be segmented. The segmentation annotations can correspond to different target objects. It is particularly suitable for small target segmentation tasks in medical images. For example, in the segmentation of tracheal medical images, the progress can be improved by 1.2% compared with the case of a simple Un et network training once, and the corresponding training time can be saved by about 30%.

[0045] In the above step S01, the coarse segmentation model includes but is not limited to: UNet, SegNet, DeepLab, FCN, ENet, LinkNet, PSPN, etc.

[0046] In order to better illustrate the above-mentioned deep learning-based image segmentation processing method S10, the following example is given:

[0047] like Figure 2AAs shown in FIG, the image input of the coarse segmentation model in the above step S01 is set to image S, wherein the image S is a long rectangle with a length a and a width b. The image S includes N (N=1, 2....n-1, n, where n is a positive integer) target objects, such as Figure 2A As shown in , N=2.

[0048] In the above step S02, the bounding box and center position of the target object can be understood as the area range of the target object and the center position within the corresponding area range. For example, taking the medical image of the articular cartilage area as an example, if the target objects in the image to be processed are patellar cartilage, femoral cartilage and tibial cartilage, then the bounding box corresponds to the boundary area range of the patellar cartilage, femoral cartilage and tibial cartilage. The maximum length of the bounding box is set to a max .

[0049] Combine Figure 2B As shown in , the corresponding two target objects are a triangle and a circle, the corresponding center positions are P1 and P2, and the corresponding bounding boxes correspond to the edges of the triangle and circle respectively. Set a subgraph S0 with a side length of I, where the value range of I is (a max, a). For example, the corresponding side length I can be 0.5 (a max +a), in other embodiments, the side length of the subgraph S0 may also be other specific values.

[0050] Use subgraph S0 to split the image into subgraphs, such as Figure 2C As shown in , the center positions of the segmented sub-graphs S1 and S2 of each sub-graph S0 correspond to P1 and P2, which are set as the center position of each target object after coarse segmentation.

[0051] Correspondingly, in the process of sub-graph segmentation of the image S, the sub-graph coordinates of the corresponding target image can be obtained based on the center position of the corresponding target object.

[0052] Continue as Figure 2A-2C As shown in , the image S includes two target objects, and the two target objects correspond to different scales. The scale described in the present invention can be understood as being based on the image scale space theory, that is, different target objects in the same image may have different optimal scales.

[0053] The preset segmentation index in the above step S2 refers to the segmentation standard corresponding to the sub-image fine segmentation model in the sub-image fine segmentation process for different sub-images and target objects of different scales, so as to obtain the segmentation effect required by the user. Figure 2A-2C As shown in , even for different target objects in the same image, the preset segmentation indicators corresponding to the sub-image fine segmentation model may be different during the training process of sub-image fine segmentation, which can be set based on experience values ​​or standards.

[0054] In step S2, the variables involved in the current training can be used as the initial values ​​of the variables for the next training. The variables involved in the sub-graph fine segmentation model include a combination of the cropping coefficient K, the segmentation index coefficient Dice, the number of training times j, and the network parameter M.

[0055] Specifically, in this embodiment, the cropping coefficient K can be understood as the size relationship between the side length of the image obtained by center cropping the image and the side length of the initial sub-image during each training process. The size of the cropping coefficient K can reflect the size of the image obtained by cropping.

[0056] The segmentation index coefficient Dice can be used to evaluate image segmentation performance. In some specific implementations, the segmentation index coefficient Dice can be expressed as:

[0057]

[0058] The predicted segmented region area and the actual segmented region area are respectively represented as the segmented region area predicted based on the current cropping coefficient K and the actual segmented region area.

[0059] The training number j represents the actual number of training times when a sub-graph containing a target object is used to train a sub-graph fine segmentation model, wherein each training may include multiple iterations.

[0060] The network parameter M can be represented as a variable in the subgraph fine segmentation model. In this embodiment, taking the U-net network as an example, the network parameter M can be understood as a variable that can reflect the characteristics of the U-net network.

[0061] See also Figure 3 In the above step S2, a sub-image fine segmentation model is trained using the sub-image containing the target object. After the training is repeated until the sub-image fine segmentation model reaches a preset segmentation index, the sub-image containing the target object is segmented to obtain a sub-image mask corresponding to the sub-image. Specifically, the following steps are included:

[0062] Step S21, initializing the variables involved in the subgraph fine segmentation model;

[0063] Step S22: Using the sub-image containing the target object as training data, at the beginning of each training, updating the variables and using the cropped image and mask to train the sub-image fine segmentation model, after multiple iterations until convergence or the current number of iterations equals the preset maximum number of iterations, then stopping the current training and obtaining new variables;

[0064] Step S23, determining whether to continue or stop training based on whether the sub-graph fine segmentation model reaches a preset segmentation index; and

[0065] Step S24: Save the refined segmentation results of the corresponding variables in the current subgraph refined segmentation model.

[0066] Specifically, in the above step S22, the cropped image and mask are used to perform center cropping on the sub-image containing the target object and the mask corresponding thereto based on the center position of the target object, so as to obtain the cropped image and mask.

[0067] Specifically, in the above steps, the image and mask are center-cropped, and the network parameter M of the current sub-image fine segmentation model is used as the initial value of the network parameter for the current training. The sub-image fine segmentation model is trained using the cropped version of the image and mask. After multiple iterations until convergence or the current number of iterations is equal to the preset maximum number of iterations, the current training is stopped and new variables are obtained, wherein the new variables include the network parameter M j And the corresponding segmentation index coefficient Dice.

[0068] In the above step S23, the preset segmentation index includes the number of training times equal to the maximum number of training times J max ; or the segmentation index coefficient of the current training is smaller than the segmentation index coefficient of the previous training.

[0069] In step S24, the variable corresponding to the current sub-graph fine segmentation model can be expressed as the network parameter M j and the crop factor K j .

[0070] Specifically, see Figure 4 , the above step S22 can be further divided into:

[0071] Step S221, based on the initialization values ​​of the corresponding variables in the obtained sub-image fine segmentation model; specifically, the cropping coefficient K0 can be initialized to 1.0; the segmentation index coefficient Dice can be initialized to 0; the number of training times j = 0, and the maximum number of training times J of the sub-image fine segmentation model for initialization training is J. max Usually it can be set to an integer between 3 and 8. Randomly initialize the network parameters M of the sub-graph fine segmentation model for training j =M0.

[0072] Step S222: Perform the first training and update the number of training times 1 = 0 + 1; calculate the cropping coefficient K1 = K0 * alpha value; wherein the alpha value can be a decimal between 0.5 and 0.95, and the specific value selection may be related to the type of image to be cut and the accuracy requirement;

[0073] Step S223: perform center cropping on the image and mask. Specifically, the side length of the image to be processed is shortened by the cropping factor K1.

[0074] Step S224, M 0＝1-1 (That is, M j denoted as the network parameters of the jth training) as the initial values ​​of the network parameters of the first iteration of the training, and use the cropped version of the image and mask to train the sub-graph fine segmentation model until convergence or the number of iterations of the training (epoch) reaches E max The network parameters after M j The corresponding segmentation index coefficient Dic e;

[0075] Among them, E max The maximum number of iterations in each training session is in the range of 10 to 20. The maximum number of iterations in each training session can be determined based on the specific model to be trained and its training data. This is only an example and is not intended to be a specific limitation of the present invention.

[0076] It should be further explained that during the above training process, the cropping coefficient K decreases as the number of training times increases. That is, during the training process of the sub-image fine segmentation model, the side length of the sub-image will be shortened as the cropping coefficient K becomes smaller.

[0077] When the center position of the current training is related to the segmentation result of the previous training, the initial values ​​of the network parameters of the current training are consistent with the network parameters after the previous training is completed.

[0078] Specifically, see Figure 5 In the above step S23, a corresponding comparison is performed based on the segmentation index coefficient of step S23, which specifically includes the following steps:

[0079] Step S231, determine whether the current training number j is equal to the maximum training number J max If not, proceed to step S232; if yes, proceed to step S233;

[0080] Step S232, determine whether the segmentation index coefficient of the current training number is less than the segmentation index coefficient of the previous training. If so, enter step S233; if not, update the number of training times and return to step S22.

[0081] Step S233, stop training.

[0082] In the above method, the clipping coefficient K can be adjusted according to the increase or decrease of the segmentation index coefficient and the number of training times.

[0083] Based on the high-precision deep learning image segmentation method using iterative field of view focusing in step S3 above, the segmented sub-images can be effectively finely segmented, thereby finely segmenting the target objects in the sub-images obtained based on the coarse segmentation. The method provided by the present invention is particularly suitable for image segmentation of multiple target objects of different scales in the same image to be processed. After the image to be processed is coarsely segmented, the single target object can be further segmented based on the sub-image fine segmentation model, and the image and its mask can be continuously cropped based on training. Transfer learning is achieved during the iterative field of view focusing process, reducing network training time and improving segmentation accuracy.

[0084] Continue to combine Figure 3 As shown in , in step S3, the sub-image coordinates and the sub-image mask are combined to obtain the segmentation result of the image to be processed, wherein the segmentation result of the image to be processed is specifically a refined segmentation mask of the image to be processed.

[0085] The deep learning-based image segmentation method provided in this embodiment can also generate high-precision segmentation masks in the following two scenarios: one is an image with multiple target objects at multiple scales, and the other is an image with high original resolution and a small proportion of target objects. The deep learning-based image segmentation processing method proposed in this article aims to achieve transfer learning through a divide-and-conquer and macro-to-micro approach in an iterative field of view focusing process, reducing network training time and improving segmentation accuracy.

[0086] See also Figure 6 A second embodiment of the present invention provides an image segmentation system 20 based on deep learning, which includes:

[0087] A sub-image segmentation module 21 is configured to provide an image to be processed having at least one target object, and perform sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and its sub-image coordinates;

[0088] A sub-image fine segmentation module 22 is configured to train a sub-image fine segmentation model using the sub-image containing the target object, repeatedly train the sub-image fine segmentation model until the sub-image fine segmentation model reaches a preset segmentation index, and then segment the sub-image containing the target object to obtain a sub-image mask corresponding to each sub-image; and

[0089] The image segmentation mask generation module 23 combines the sub-image coordinates and the sub-image masks to perform sub-image mask splicing to obtain a segmentation result of the image to be processed.

[0090] In order to obtain better image segmentation effect and improve segmentation accuracy, such as Figure 7 As shown in , the subgraph segmentation module 21 further includes:

[0091] An image coarse segmentation module 211 is configured to input a provided image to be processed having at least one target object into a coarse segmentation model for coarse segmentation to obtain a bounding box and a center position of the target object; and

[0092] The segmentation module 212 is configured to segment the image to be processed into sub-images based on the bounding box and the center position of the target object to obtain sub-images containing the target object and the sub-image coordinates thereof.

[0093] Specifically, in this embodiment, the relevant contents of the image coarse segmentation and sub-image fine segmentation model are consistent with those in the first embodiment above, and are not repeated here.

[0094] See also Figure 8 The third embodiment of the present invention provides an electronic device 30, which includes a storage unit 31 and a processing unit 32. The storage unit 31 is used to store computer programs, and the processing unit 32 is used to execute the specific steps of the deep learning-based image segmentation processing method described in the first embodiment through the computer program stored in the storage unit 31.

[0095] In some specific embodiments of the present invention, the electronic device 30 can be hardware or software. When the electronic device 30 is hardware, it can be various electronic devices with a display screen and supporting video playback, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Gro up Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Gro up Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, laptop computers and desktop computers, etc. When the electronic device 30 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0096] The storage unit 31 includes a read-only memory (ROM), a random access memory (RAM), and a storage portion such as a hard disk. The processing unit 32 can perform various appropriate actions and processes based on the programs stored in the read-only memory (ROM) or loaded into the random access memory (RAM). The random access memory (RAM) also stores various programs and data required for the operation of the electronic device 30.

[0097] The electronic device 30 may further include an input unit (not shown) such as a keyboard and a mouse; an output unit (not shown) such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; and a communication unit (not shown) such as a network interface card (NIC) such as a LAN card or a modem. The communication unit 35 performs communication processing via a network such as the Internet.

[0098] In particular, according to embodiments disclosed herein, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments disclosed herein may include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via a communication component.

[0099] When the computer program is executed by the processing unit 32, the above functions defined in the training method of the neural network model with anti-counterfeiting function of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] In this application, a computer-readable storage medium may also be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0101] The computer program code for carrying out the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0102] The flow charts and block diagrams in the accompanying drawings of the present invention illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementation schemes, the functions marked in the box can also occur in a different order than those marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is determined based on the functions involved. It should be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0103] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be arranged in a processor.

[0104] As another aspect, a fourth embodiment of the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and the programs specifically include: providing an image to be processed having at least one target object, performing sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and its sub-image coordinates; using the sub-image containing the target object to train a sub-image fine segmentation model, when the sub-image fine segmentation model reaches a preset segmentation index, segmenting the sub-image containing the target object to obtain a sub-image mask corresponding to each sub-image; and combining the sub-image coordinates with the sub-image mask to perform sub-image mask splicing to obtain a segmentation result of the image to be processed.

[0105] Compared with the prior art, the deep learning-based image segmentation processing method, system, and electronic device provided by the present invention have the following beneficial effects:

[0106] The present invention provides an image segmentation processing method based on deep learning, which can be used to train a sub-graph fine segmentation model using a sub-graph containing a target object after dividing the image to be processed into sub-graphs, thereby obtaining a sub-graph mask corresponding to the sub-graph, and finally performing sub-graph mask splicing based on the sub-graph coordinates to obtain the segmentation result of the image to be processed. The above-mentioned image segmentation processing method based on deep learning can realize transfer learning in the process of iteratively training the sub-graph fine segmentation model from macro to micro through the sub-graph cutting method, thereby effectively reducing the network training time and improving the segmentation accuracy. The image segmentation processing method based on deep learning provided by the present invention can be widely used in image segmentation processing with multiple target objects having multiple scales and / or the original image has high resolution but the target object ratio is small, and is particularly suitable for segmentation tasks in medical imaging.

[0107] The present invention provides an image segmentation system and an electronic device based on deep learning, which have the same beneficial effects as the above-mentioned image segmentation processing method based on deep learning. They can effectively reduce the network training time for high-precision image segmentation and improve the segmentation accuracy. They are particularly suitable for segmentation tasks in medical images.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep learning-based image segmentation processing method, characterized by: It includes the following steps: Step S1: providing an image to be processed having at least one target object, and performing sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and its sub-image coordinates; Step S2: Obtain a sub-image fine segmentation model and initialize the variables involved in the sub-image fine segmentation model. The variables involved in the sub-image fine segmentation model include a combination of a cropping coefficient, a segmentation index coefficient, a number of training times, and network parameters. The size of the cropping coefficient reflects the size of the image obtained by cropping, and the segmentation index coefficient is expressed as: The predicted segmentation area and the true segmentation area are respectively expressed as the predicted segmentation area based on the current cropping coefficient and the true segmentation area. The sub-image containing the target object is used as training data. At the beginning of each training, the variables are updated and the cropped image and mask are used to train the sub-image fine segmentation model. During the training process, the cropping coefficient decreases as the number of training times increases, and the side lengths of the cropped image and mask shorten as the cropping coefficient changes. The training is repeated until the sub-image fine segmentation model reaches a preset segmentation index. The sub-image containing the target object is segmented to obtain a sub-image mask corresponding to the sub-image. The preset segmentation index includes that the number of training times is equal to the maximum number of training times or the segmentation index coefficient of the training time is less than the segmentation index coefficient of the previous training. and Step S3: combining the sub-image coordinates with the sub-image masks and performing sub-image mask splicing to obtain a segmentation result of the image to be processed.

2. The image segmentation processing method based on deep learning as claimed in claim 1, characterized in that: In step S2, the variables involved in the current training are used as initial values ​​of the variables for the next training, and each training includes multiple iterations.

3. The image segmentation processing method based on deep learning as claimed in claim 1, characterized in that: The cropped image and mask can be obtained by the following steps: shortening the side length of the image to be processed by the cropping factor based on the center position of the target object.

4. The image segmentation processing method based on deep learning as claimed in claim 3, characterized in that: The center position of the current training is calculated based on the segmentation results of the previous training.

5. The image segmentation processing method based on deep learning as claimed in claim 1, characterized in that: In the above step S1, the sub-image segmentation of the image to be processed specifically includes the following steps: Step S01: Input the image to be processed into a coarse segmentation model for coarse segmentation to obtain the bounding box and center position of the corresponding target object; and Step S02: performing sub-image segmentation based on the bounding box and center position of the corresponding target object.

6. A deep learning-based image segmentation system, characterized by: The image segmentation system based on deep learning includes: A sub-image segmentation module is configured to provide an image to be processed having at least one target object, and perform sub-image segmentation on the image to be processed to obtain a sub-image containing the target object and its sub-image coordinates; The sub-image fine segmentation module is used to obtain a sub-image fine segmentation model and initialize the variables involved in the sub-image fine segmentation model. The variables involved in the sub-image fine segmentation model include a combination of a cropping coefficient, a segmentation index coefficient, a training number, and network parameters. The size of the cropping coefficient reflects the size of the image obtained by cropping. The segmentation index coefficient is expressed as: The predicted segmentation area and the true segmentation area are respectively expressed as the predicted segmentation area based on the current cropping coefficient and the true segmentation area. The sub-image containing the target object is used as training data. At the beginning of each training, the variables are updated and the sub-image fine segmentation model is trained using the cropped image and mask. During the training process, the cropping coefficient decreases as the number of training times increases, and the side lengths of the cropped image and mask shorten as the cropping coefficient changes. The training is repeated until the sub-image fine segmentation model reaches a preset segmentation index. The sub-image containing the target object is segmented to obtain a sub-image mask corresponding to each sub-image. The preset segmentation index includes that the number of training times is equal to the maximum number of training times or the segmentation index coefficient of the training time is less than the segmentation index coefficient of the previous training. and The image segmentation mask generation module is used to combine the sub-image coordinates and the sub-image mask to perform sub-image mask splicing to obtain the segmentation result of the image to be processed.

7. An electronic device, characterized in that: The electronic device includes a storage unit and a processing unit, the storage unit is used to store a computer program, and the processing unit is used to execute the steps of the deep learning-based image segmentation processing method according to any one of claims 1 to 5 through the computer program stored in the storage unit.

Images