Cervical vertebra segmentation method and device based on edge pyramid and cross feature fusion

Abstract

The present invention provides a cervical vertebra segmentation method and device based on edge pyramid and cross feature fusion, the method comprising: obtaining a cervical vertebra medical image to be segmented; inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes; based on a plurality of PEEM modules in the deep learning network model, performing edge extraction operations on the first encoded image respectively to obtain a plurality of second encoded images of different sizes; based on a plurality of CFM modules in the deep learning network model, performing cross feature fusion operations on the plurality of first encoded images and the second encoded images respectively to obtain a plurality of third encoded images of different sizes; based on a plurality of third encoded images of different sizes and a plurality of decoded images of different sizes, performing fusion operations, and outputting the segmentation result of the cervical vertebra medical image. The method can make the fused feature map more accurate, reduce the loss of detail features, and improve the segmentation effect.

Description

Technical Field

[0001] The present invention relates to the medical field, and in particular to a cervical vertebra segmentation method and device based on edge pyramid and cross feature fusion. Background Art

[0002] With the continuous development of technology, artificial intelligence technology is increasingly being applied to the medical field. Taking the cervical spine as an example, the introduction of artificial intelligence technology has saved doctors a lot of time and improved surgical efficiency. However, due to the diversity of cervical spine lesions, it is easy to cause low accuracy of cervical spine segmentation, especially in the segmentation of detailed features, and the segmentation deviation is large, resulting in a bad experience.

[0003] Therefore, how to solve the above problems needs to be considered. Summary of the invention

[0004] The present invention provides a cervical vertebra segmentation method and device based on edge pyramid and cross feature fusion to solve the above problems.

[0005] In a first aspect, the present invention provides a cervical vertebra segmentation method based on edge pyramid and cross feature fusion, comprising:

[0006] Acquire a cervical spine medical image to be segmented;

[0007] Inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes;

[0008] Based on a plurality of pyramid feature extraction PEEM modules in the deep learning network model, edge extraction operations are respectively performed on the first encoded image to obtain a plurality of second encoded images of different sizes;

[0009] Based on a plurality of cross-feature fusion CFM modules in the deep learning network model, respectively perform a cross-feature fusion operation on the plurality of first encoded images and the second encoded images to obtain a plurality of third encoded images of different sizes;

[0010] Based on the plurality of third encoded images of different sizes and the plurality of decoded images of different sizes, a fusion operation is performed to output a segmentation result of the cervical vertebra medical image;

[0011] The sizes of the multiple decoded images correspond one-to-one to the sizes of the multiple first encoded images.

[0012] Optionally, the deep learning network model includes a five-layer network structure, the first layer of the network structure is a network layer that inputs the cervical vertebra medical image to be segmented, and the sizes of the first encoded images of the first layer of the network structure to the fifth layer of the network structure decrease successively;

[0013] The first to fourth network structures are all provided with the pyramid feature extraction module and the cross feature fusion module;

[0014] The cross-feature fusion module in the current network layer is used to perform a cross-feature fusion operation on the second coded image of the current network layer and the first coded image of other network layers to obtain a third coded image of the current network layer;

[0015] The current network layer is any network layer from the first network structure to the fourth network structure, and the other network layers include multiple network layers from the first network structure to the fourth network structure excluding the current network layer.

[0016] Optionally, the PEEM module includes a four-layer sub-network structure;

[0017] Each layer of the sub-network structure is used to perform edge extraction operation, average pooling operation, addition operation and splicing operation on the first coded image of the current network layer to obtain a second coded image;

[0018] Among them, the sub-network structures in different layers have different sizes for average pooling operations.

[0019] Optionally, the calculation process of the edge extraction operation is as follows:

[0020] G(x,y)=x(i,j)+y(i,j)

[0021]

[0022] Wherein, G is the gradient calculation result of the first encoded image, Edge is the edge calculation result, x(i, j) is the edge of the first encoded image in the x direction, and y(i, j) is the edge of the first encoded image in the y direction.

[0023] Optionally, x(i,j) and t(i,j) are calculated as follows:

[0024]

[0025]

[0026] Wherein, img is the edge extracted image, and (i, j) is the coordinate of the first encoded image.

[0027] Optionally, the cross feature fusion CFM module includes:

[0028] a first branch structure and a second branch structure;

[0029] The first branch structure is used to perform a Softmax operation on the second encoded image of the current network layer to obtain a first feature parameter;

[0030] The second branch structure is used to perform a feature fusion operation on the multiple first encoded images in the other network layer to obtain a first feature map;

[0031] A feature fusion operation is performed on the first feature map, the first feature parameter, and the second encoded image of the current network layer to obtain a third encoded image corresponding to the current network layer.

[0032] Optionally, the segmentation loss function adopted by the deep learning network model includes at least one of the following:

[0033] CELoss loss function; DiceLoss loss function.

[0034] In a second aspect, the present invention provides a cervical vertebra segmentation device based on edge pyramid and cross feature fusion, comprising:

[0035] An acquisition module, used for acquiring a cervical spine medical image to be segmented;

[0036] An input module, used for inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes;

[0037] An extraction module, configured to perform edge extraction operations on the first encoded image based on a plurality of pyramid feature extraction PEEM modules in the deep learning network model, to obtain a plurality of second encoded images of different sizes;

[0038] A fusion module, configured to perform a cross-feature fusion operation on the plurality of first encoded images and the second encoded images respectively based on a plurality of cross-feature fusion CFM modules in the deep learning network model, so as to obtain a plurality of third encoded images of different sizes;

[0039] A processing module, configured to perform a fusion operation based on the plurality of third encoded images of different sizes and the plurality of decoded images of different sizes, and output a segmentation result of the cervical vertebra medical image;

[0040] The sizes of the multiple decoded images correspond one-to-one to the sizes of the multiple first encoded images.

[0041] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the cervical vertebra segmentation method based on edge pyramid and cross feature fusion as described above is implemented.

[0042] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the cervical vertebra segmentation method based on edge pyramid and cross feature fusion as described above is implemented.

[0043] The above technical solution of the present invention has at least the following beneficial effects:

[0044] The cervical vertebra segmentation method based on edge pyramid and cross feature fusion provided by the present invention performs edge extraction operation on first coded images of different sizes through the PEEM module, so that the edge feature information of the obtained first coded image can be more accurate, and the edge feature information and overall feature information of the obtained third coded image can be more accurate by performing cross feature fusion operation on multiple first coded images and second coded images through the CFM module. In addition, since the obtained third coded image retains relatively accurate edge feature information and overall feature information, the feature map obtained by fusion based on the third coded image and the decoded image can be made more accurate, thereby obtaining an accurate segmentation result of the cervical vertebra medical image, reducing the loss of detail features, and improving the segmentation effect of the cervical vertebra medical image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic flow chart of a cervical vertebra segmentation method based on edge pyramid and cross feature fusion provided by the present invention;

[0047] Figure 2 A schematic diagram of the structure of a deep learning network model provided by the present invention;

[0048] Figure 3 A schematic diagram of the structure of a PEEM module provided by the present invention;

[0049] Figure 4 A schematic diagram of the structure of a cross-feature fusion CFM module provided by the present invention;

[0050] Figure 5 A schematic diagram of a module of a cervical vertebra segmentation device based on edge pyramid and cross feature fusion provided by the present invention;

[0051] Figure 6 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein.

[0054] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0055] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0056] It should be understood that in the present invention, "plurality" refers to two or more than two. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "Contains A, B and C", "Contains A, B, C" means that A, B, and C are all included, "Contains A, B or C" means that one of A, B, and C is included, and "Contains A, B and / or C" means that any one, any two, or any three of A, B, and C are included.

[0057] It should be understood that in the present invention, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based only on A, but B can also be determined based on A and / or other information. A and B match when the similarity between A and B is greater than or equal to a preset threshold.

[0058] Depending on the context, "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting."

[0059] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0060] See also Figure 1 , is a flow chart of a cervical vertebra segmentation method based on edge pyramid and cross feature fusion provided by the present invention, the cervical vertebra segmentation method comprises the following steps:

[0061] S11: Acquire a cervical spine medical image to be segmented.

[0062] The cervical spine medical image adopts the DICOM format and is input as a 2.5D image, which fully considers the correspondence between different slices of the CT image.

[0063] S12: Inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes.

[0064] It should be noted that the encoding operation includes convolution Conv3x3 operation + batch normalization BN operation + nonlinear Relu operation + maximum pooling Max Pooling operation, and multiple downsampling is performed to finally obtain multiple first encoded images of different sizes.

[0065] S13: Based on multiple pyramid feature extraction (PEEM) modules in the deep learning network model, edge extraction operations are performed on the first encoded image respectively to obtain multiple second encoded images of different sizes.

[0066] By performing edge extraction operations on first encoded images of different sizes, edge feature information of the first encoded images of different sizes can be retained.

[0067] S14: Based on the multiple cross-feature fusion CFM modules in the deep learning network model, cross-feature fusion operations are performed on the multiple first encoded images and the second encoded images respectively to obtain multiple third encoded images of different sizes.

[0068] By performing a cross-feature fusion operation on a plurality of first coded images and second coded images through the CFM module, the feature information retained in the obtained third coded image can be enriched, thereby avoiding the loss of detail features.

[0069] S15: performing a fusion operation based on the plurality of third encoded images of different sizes and the plurality of decoded images of different sizes, and outputting a segmentation result of the cervical vertebra medical image;

[0070] The sizes of the multiple decoded images correspond one-to-one to the sizes of the multiple first encoded images.

[0071] The cervical vertebra segmentation method based on edge pyramid and cross feature fusion provided by the present invention performs edge extraction operation on first coded images of different sizes through the PEEM module, so that the edge feature information of the obtained first coded image can be more accurate, and the edge feature information and overall feature information of the obtained third coded image can be more accurate by performing cross feature fusion operation on multiple first coded images and second coded images through the CFM module. In addition, since the obtained third coded image retains relatively accurate edge feature information and overall feature information, the feature map obtained by fusion based on the third coded image and the decoded image can be made more accurate, thereby obtaining an accurate segmentation result of the cervical vertebra medical image, reducing the loss of detail features, and improving the segmentation effect of the cervical vertebra medical image.

[0072] For example, Figure 2 The figure shows a schematic diagram of the structure of a deep learning network model provided by the present invention. The deep learning network model includes a five-layer network structure, the first layer of the network structure is a network layer for inputting the cervical spine medical image to be segmented, and the sizes of the first encoded images of the first layer of the network structure to the fifth layer of the network structure decrease successively.

[0073] It should be noted that the first coded images corresponding to the first layer network structure to the fifth layer network structure are represented by E1 to E5 respectively.

[0074] The first to fourth network structures are all provided with the pyramid feature extraction module and the cross feature fusion module, wherein the cross feature fusion module in the current network layer is used to perform cross feature fusion operations on the second encoded image of the current network layer and the first encoded images of other network layers to obtain the third encoded image of the current network layer.

[0075] It should be noted that the second coded images corresponding to the first to fourth network structures are represented by E1' to E4', respectively. The third coded images corresponding to the first to fourth network structures are represented by E1* to E4*, respectively.

[0076] The current network layer is any network layer from the first network structure to the fourth network structure, and the other network layers include multiple network layers from the first network structure to the fourth network structure excluding the current network layer.

[0077] For example, the pyramid feature extraction module and the cross feature fusion module are located in the first layer of the network structure. The pyramid feature extraction module only performs an edge extraction operation on the first encoded image in the first layer of the network structure to obtain the second encoded image E1', and the cross feature fusion module is used to perform a cross feature fusion operation on the second encoded image E1', the first encoded image E2 in the second layer of the network structure to the fourth layer of the network structure, the first encoded image E3 and the first encoded image E4 to obtain the third encoded image E1*.

[0078] For example, see Figure 3 , is a schematic diagram of the structure of a PEEM module provided by the present invention. The PEEM module includes a four-layer sub-network structure; each layer of the sub-network structure is used to perform edge extraction operation, average pooling operation, addition operation and splicing operation on the first coded image of the current network layer to obtain a second coded image; wherein the sub-network structures of different layers have different sizes for average pooling operation.

[0079] It should be noted that in Figure 3 In the figure, the first encoded image is represented by Ei, and the value of i is 1, 2, 3, and 4. Optionally, i=1 is used as an example for explanation. First, the first encoded image E1 is subjected to edge extraction operations through different sub-network structures, and then the results are subjected to average pooling operations of different sizes, wherein the average pooling operations of the first layer of sub-network structure to the fourth layer of sub-network structure are represented by Avg_1, Avg_2, Avg_3, and Avg_4, respectively. Avg_1 is 2*2 pooling, Avg_2 is 4*4 pooling, Avg_3 is 8*8 pooling, and Avg_4 is 16*16 pooling. Through the average pooling operations in different sub-network structures, edge information at multiple levels can be extracted to the maximum extent. Next, the average pooling results are added to the first encoded image E1 respectively, and finally the results obtained by adding the four layers of sub-network structures are concatenated and Concat is performed to obtain the second encoded image E1'.

[0080] Specifically, the calculation process of the edge extraction operation is as follows:

[0081] G(x,y)=x(i,j)+y(i,j)

[0082]

[0083] Wherein, G is the gradient calculation result of the first encoded image, Edge is the edge calculation result, x(i, j) is the edge of the first encoded image in the x direction, and y(i, j) is the edge of the first encoded image in the y direction.

[0084] Furthermore, x(i,j) and y(i,j) are calculated as follows:

[0085]

[0086]

[0087] Wherein, img is the edge extracted image, and (i, j) is the coordinate of the first encoded image.

[0088] Next, see Figure 4 , is a schematic diagram of the structure of a cross-feature fusion CFM module provided by the present invention. The cross-feature fusion CFM module includes:

[0089] A first branch structure and a second branch structure; the first branch structure is used to perform a Softmax operation on the second encoded image of the current network layer to obtain a first feature parameter.

[0090] Specifically, after the second encoded image is subjected to a Softmax operation, a characteristic parameter σ is obtained, and then a 1-σ operation is performed to obtain σ', where σ' is the first characteristic parameter.

[0091] The second branch structure is used to perform a feature fusion operation on the multiple first encoded images in the other network layers to obtain a first feature map.

[0092] Specifically, the second branch structure includes multiple sub-branch structures, each of which is used to perform a Softmax operation on the first encoded image to obtain a feature parameter σ. The feature parameter σ is multiplied with the first encoded image in each sub-branch structure to obtain multiple encoded feature images. The multiple encoded feature images are then added to obtain the first feature image EE. The encoded feature image EE and the first feature parameter σ'

[0093] A feature fusion operation is performed on the first feature map, the first feature parameter, and the second encoded image of the current network layer to obtain a third encoded image corresponding to the current network layer.

[0094] Specifically, the first feature map EE is multiplied by the first feature parameter σ' to obtain the second feature map EET, and the second feature map EET is added to the second encoded image to perform feature fusion, and finally the third encoded image Ei* is obtained.

[0095] Furthermore, the calculation process of obtaining the third encoded image Ei* is as follows:

[0096] σ′=1-Softmax(Ei′), E * =EET+Ei′. The variables in the formula correspond to Figure 4 The variables of each feature map and intermediate value in , i represents the corresponding feature layer number, i = 1, 2, 3, 4, j = 1, 2, 3, 4, and j ≠ i, Represents the dot product.

[0097] For example, the segmentation loss function used by the deep learning network model includes at least one of the following:

[0098] CELoss loss function; DiceLoss loss function.

[0099] Among them, the expression of CELoss loss function is:

[0100] CELoss=-[ylogy′+(1-y)log(1-y′)]

[0101] The expression of DiceLoss loss function is:

[0102]

[0103] If the deep learning network model uses the above two loss functions, the expression of the loss function is:

[0104] Loss=α·CELoss+(1-α)·DiceLoss

[0105] Among them, y is the label value, y' is the predicted value, and α is the loss weight coefficient

[0106] Based on the same technical concept as the above-mentioned cervical vertebra segmentation method based on edge pyramid and cross feature fusion, the present invention provides a cervical vertebra segmentation device based on edge pyramid and cross feature fusion. The cervical vertebra segmentation device has the same function as the cervical vertebra segmentation method, which will not be described in detail here.

[0107] See also Figure 5 , is a schematic diagram of a module of a cervical vertebra segmentation device based on edge pyramid and cross feature fusion provided by the present invention, the cervical vertebra segmentation device comprising:

[0108] An acquisition module 51 is used to acquire a cervical vertebra medical image to be segmented;

[0109] An input module 52, used for inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes;

[0110] An extraction module 53 is used to perform edge extraction operations on the first encoded image based on multiple pyramid feature extraction PEEM modules in the deep learning network model to obtain multiple second encoded images of different sizes;

[0111] A fusion module 54 is used to perform a cross-feature fusion operation on the multiple first encoded images and the second encoded images based on multiple cross-feature fusion CFM modules in the deep learning network model to obtain multiple third encoded images of different sizes;

[0112] A processing module 55, configured to perform a fusion operation based on the plurality of third encoded images of different sizes and the plurality of decoded images of different sizes, and output a segmentation result of the cervical vertebra medical image;

[0113] The sizes of the multiple decoded images correspond one-to-one to the sizes of the multiple first encoded images.

[0114] Optionally, the deep learning network model includes a five-layer network structure, the first layer of the network structure is a network layer that inputs the cervical vertebra medical image to be segmented, and the sizes of the first encoded images of the first layer of the network structure to the fifth layer of the network structure decrease successively;

[0115] The first to fourth network structures are all provided with the pyramid feature extraction module and the cross feature fusion module;

[0116] The cross-feature fusion module in the current network layer is used to perform a cross-feature fusion operation on the second coded image of the current network layer and the first coded image of other network layers to obtain a third coded image of the current network layer;

[0117] The current network layer is any network layer from the first network structure to the fourth network structure, and the other network layers include multiple network layers from the first network structure to the fourth network structure excluding the current network layer.

[0118] Optionally, the PEEM module includes a four-layer sub-network structure;

[0119] Each layer of the sub-network structure is used to perform edge extraction operation, average pooling operation, addition operation and splicing operation on the first coded image of the current network layer to obtain a second coded image;

[0120] Among them, the sub-network structures in different layers have different sizes for average pooling operations.

[0121] Optionally, the calculation process of the edge extraction operation is as follows:

[0122] G(x,y)=x(i,j)+y(i,j)

[0123]

[0124] Wherein, G is the gradient calculation result of the first encoded image, Edge is the edge calculation result, x(i, j) is the edge of the first encoded image in the x direction, and y(i, j) is the edge of the first encoded image in the y direction.

[0125] Optionally, x(i,j) and y(i,j) are calculated as follows:

[0126]

[0127]

[0128] Wherein, img is the edge extracted image, and (i, j) is the coordinate of the first encoded image.

[0129] Optionally, the cross feature fusion CFM module includes:

[0130] a first branch structure and a second branch structure;

[0131] The first branch structure is used to perform a Softmax operation on the second encoded image of the current network layer to obtain a first feature parameter;

[0132] The second branch structure is used to perform a feature fusion operation on the multiple first encoded images in the other network layer to obtain a first feature map;

[0133] A feature fusion operation is performed on the first feature map, the first feature parameter, and the second encoded image of the current network layer to obtain a third encoded image corresponding to the current network layer.

[0134] Optionally, the segmentation loss function adopted by the deep learning network model includes at least one of the following:

[0135] CELoss loss function; DiceLoss loss function.

[0136] See next Figure 6 , which is a schematic structural diagram of an electronic device provided by the present invention.

[0137] The electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the cervical vertebra segmentation method based on edge pyramid and cross feature fusion provided by the above methods.

[0138] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0139] Another aspect of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the cervical vertebra segmentation method based on edge pyramid and cross feature fusion as described above is implemented.

[0140] Computer readable storage medium can be a tangible device that can keep and store instructions used by instruction execution device. Computer readable storage medium can be, for example, but not limited to, electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as punch card or convex structure in groove with instructions stored thereon, and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0141] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0142] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0143] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0144] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0145] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0146] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0147] Note that, unless otherwise directly stated, all features disclosed in this specification (including any attached claims, abstracts and drawings) may be replaced by alternative features for achieving the same, equivalent or similar purposes. Therefore, unless otherwise explicitly stated, each feature disclosed is only an example of a group of equivalent or similar features. Where used, further, preferably, further and more preferably are simple beginnings for elaborating another embodiment based on the aforementioned embodiment, and the content of the further, preferably, further or more preferably followed by the combination with the aforementioned embodiment constitutes a complete construction of another embodiment. Several further, preferably, further or more preferably settings following the same embodiment can be arbitrarily combined to form another embodiment.

[0148] It should be understood by those skilled in the art that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A cervical vertebra segmentation method based on edge pyramid and cross feature fusion, characterized in that: include: Acquire a cervical spine medical image to be segmented; Inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes; Based on a plurality of pyramid feature extraction PEEM modules in the deep learning network model, edge extraction operations are respectively performed on the first encoded image to obtain a plurality of second encoded images of different sizes; Based on a plurality of cross-feature fusion modules in the deep learning network model, respectively perform a cross-feature fusion operation on the plurality of first encoded images and the second encoded images to obtain a plurality of third encoded images of different sizes; Based on the plurality of third encoded images of different sizes and the plurality of decoded images of different sizes, a fusion operation is performed to output a segmentation result of the cervical vertebra medical image; wherein the sizes of the plurality of decoded images of different sizes correspond one-to-one to the sizes of the plurality of first encoded images of different sizes; The deep learning network model includes a five-layer network structure, the first layer of the network structure is a network layer for inputting the cervical vertebra medical image to be segmented, and the sizes of the first encoded images of the first layer of the network structure to the fifth layer of the network structure decrease in sequence; The first to fourth network structures are all provided with the pyramid feature extraction module and the cross feature fusion module; The cross-feature fusion module in the current network layer is used to perform a cross-feature fusion operation on the second coded image of the current network layer and the first coded image of other network layers to obtain a third coded image of the current network layer; The current network layer is any network layer from the first network structure to the fourth network structure, and the other network layers include multiple network layers from the first network structure to the fourth network structure excluding the current network layer; The cross feature fusion module comprises: a first branch structure and a second branch structure; The first branch structure is used to perform a Sofimax operation on the second coded image of the current network layer to obtain a first feature parameter; The second branch structure is used to perform a feature fusion operation on the multiple first encoded images in the other network layer to obtain a first feature map; A feature fusion operation is performed on the first feature map, the first feature parameter, and the second encoded image of the current network layer to obtain a third encoded image corresponding to the current network layer.

2. The cervical vertebra segmentation method based on edge pyramid and cross feature fusion according to claim 1, characterized in that: The PEEM module includes a four-layer sub-network structure; Each layer of the sub-network structure is used to perform edge extraction operation, average pooling operation, addition operation and splicing operation on the first coded image of the current network layer to obtain a second coded image; Among them, the sub-network structures in different layers have different sizes for average pooling operations.

3. The cervical vertebra segmentation method based on edge pyramid and cross feature fusion according to claim 2, characterized in that: The calculation process of the edge extraction operation is as follows: G(x,y)=x(i,j)+y(i,j) Wherein, G is the gradient calculation result of the first encoded image, Edge is the edge calculation result, x(i, j) is the gradient of the first encoded image in the x direction, and y(i, j) is the gradient of the first encoded image in the y direction.

4. The cervical vertebra segmentation method based on edge pyramid and cross feature fusion according to claim 3, characterized in that: x(i, j) and y(i, j) are calculated as follows: Wherein, img is the edge-extracted image, and (i, j) is the coordinate of the first encoded image.

5. The cervical vertebra segmentation method based on edge pyramid and cross feature fusion according to claim 1, characterized in that: The segmentation loss function used by the deep learning network model includes at least one of the following: CELoss loss function; DiceLoss loss function.

6. A cervical vertebra segmentation device based on edge pyramid and cross feature fusion, characterized in that: include: An acquisition module, used for acquiring a cervical spine medical image to be segmented; An input module, used for inputting the cervical vertebra medical image to be segmented into a deep learning network model for encoding operation to obtain a plurality of first encoded images of different sizes; An extraction module, configured to perform edge extraction operations on the first encoded image based on a plurality of pyramid feature extraction PEEM modules in the deep learning network model, to obtain a plurality of second encoded images of different sizes; A fusion module, configured to perform a cross-feature fusion operation on the plurality of first encoded images and the second encoded images respectively based on a plurality of cross-feature fusion modules in the deep learning network model, so as to obtain a plurality of third encoded images of different sizes; A processing module, configured to perform a fusion operation based on the plurality of third encoded images of different sizes and the plurality of decoded images of different sizes, and output a segmentation result of the cervical vertebra medical image; wherein the sizes of the plurality of decoded images of different sizes correspond one-to-one to the sizes of the plurality of first encoded images of different sizes; The deep learning network model includes a five-layer network structure, the first layer of the network structure is a network layer for inputting the cervical vertebra medical image to be segmented, and the sizes of the first encoded images of the first layer of the network structure to the fifth layer of the network structure decrease in sequence; The first to fourth network structures are all provided with the pyramid feature extraction module and the cross feature fusion module; The cross-feature fusion module in the current network layer is used to perform a cross-feature fusion operation on the second coded image of the current network layer and the first coded image of other network layers to obtain a third coded image of the current network layer; The current network layer is any network layer from the first network structure to the fourth network structure, and the other network layers include multiple network layers from the first network structure to the fourth network structure excluding the current network layer; The cross feature fusion module comprises: a first branch structure and a second branch structure; The first branch structure is used to perform a Softmax operation on the second encoded image of the current network layer to obtain a first feature parameter; The second branch structure is used to perform a feature fusion operation on the multiple first encoded images in the other network layer to obtain a first feature map; A feature fusion operation is performed on the first feature map, the first feature parameter, and the second encoded image of the current network layer to obtain a third encoded image corresponding to the current network layer.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the cervical vertebra segmentation method based on edge pyramid and cross feature fusion as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cervical vertebra segmentation method based on edge pyramid and cross feature fusion as described in any one of claims 1 to 5 is implemented.

Images