Skeleton extraction method and system, and storage medium

Abstract

The application provides a skeleton extraction method and system and a storage medium, wherein the method steps comprise: step S100, training a data set, obtaining a polygon contour and a skeleton key point of an object in an original image; taking the polygon contour as a domain constraint, triggering wave propagation connection between the skeleton key points, obtaining a preliminary skeleton, and performing an inflation operation to obtain a coarse-grained skeleton; and step S200, establishing a W-shaped network, taking a distance conversion grayscale image of the original image as an input of a first encoder, taking the coarse-grained skeleton as an input of a second encoder, taking an original real skeleton as a label, and adopting a hybrid loss function with a weight value decreasing layer by layer to complete training, so as to perform skeleton extraction. In this way, the problem of skeleton fracture caused by large shape and scale changes of different parts of the object in the process of extracting the skeleton is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to skeleton extraction technology in deep learning, in particular to a skeleton extraction method and system based on a coarse skeleton and a W-shaped network, and a storage medium. BACKGROUND

[0002] Skeleton extraction is a basic but important processing step in many computer vision tasks, aiming to extract the skeleton from the foreground object to represent its shape topology, so as to reduce data redundancy. As a compact, intuitive and effective image representation method, skeleton has a wide application prospect in the fields of agricultural production, medical image processing, digital museum and virtual reality.

[0003] The existing binary image skeleton extraction algorithm is divided into two categories: traditional method and deep learning method. The traditional skeleton extraction algorithm is easily affected by the edge noise of the binary image, and the over-branching situation occurs, so a complex post-processing step is needed. However, the deep learning method can correctly guide the generation process of the skeleton through the label in the training process, and reduce the over-branching. However, different parts of the foreground object have different shape scales.

[0004] In the neural network architecture, the receptive field size is different at different network stages due to the influence of the convolution kernel size and downsampling. With the continuous downsampling of the image, the image size becomes smaller, and in the case of the same convolution kernel size, the receptive field becomes larger. Therefore, the shallow stage of the network pays more attention to the skeleton features of the part with small shape scale, while the deep layer of the network pays more attention to the skeleton information of the part with large shape scale. This leads to the problem that the skeleton is prone to break at the part with large shape scale change, as shown in the accompanying Figure 8 SUMMARY

[0005] Therefore, the main purpose of the present application is to provide a skeleton extraction method and system, and a storage medium, to solve the problem of skeleton breakage caused by the large shape scale change of different parts of the object in the skeleton extraction process.

[0006] In order to achieve the above purpose, according to the first aspect of the present application, a skeleton extraction method is provided, and the steps thereof include:

[0007] Step S100: training a data set to obtain the polygon contour and skeleton key points of the object in the original image; taking the polygon contour as the domain constraint, triggering wave propagation connection between the skeleton key points to obtain a preliminary skeleton, and performing an inflation operation to obtain a coarse skeleton;

[0008] ​Step S200 establishes a W-shaped network, takes the distance transform gray image of the original image as the input of the first encoder, takes the coarse-grained skeleton as the input of the second encoder, takes the original real skeleton as the label, and adopts a hybrid loss function with the weight decreasing layer by layer to complete the training, so as to perform skeleton extraction.

[0009] The skeleton extraction method according to claim 1, wherein the step of obtaining the polygon contour and the skeleton key point in step S100 comprises:

[0010] Step S110 extracts the skeleton key point of the object in the original image as the first real label and the polygon contour of the object as the second real label.

[0011] Step S120 establishes a polygon contour extraction network, takes the binary image of the original image as the input, and takes the second real label as the label to perform training and obtain the polygon contour.

[0012] Step S130 establishes a key point element extraction network, takes the distance transform gray image of the original image as the input, and takes the first real label as the label to perform training and obtain the skeleton key point.

[0013] In a possible preferred implementation, the step of extracting the skeleton key point in step S110 comprises: decomposing the original real skeleton into a starting point, a connecting point and a skeleton branch; uniformly sampling all skeleton branches to obtain skeleton branch points; and taking the starting point, the connecting point and the skeleton branch points as the skeleton key points.

[0014] In a possible preferred implementation, the step of extracting the polygon contour in step S110 comprises: connecting all the obtained starting points to form a closed shape and filling to obtain the polygon contour.

[0015] In a possible preferred implementation, the step of establishing the W-shaped network comprises:

[0016] Step S210 sets at least two parallel encoders and two parallel decoders, and the network as a whole is divided into at least five levels from top to bottom, and the same level has the same number of channels and resolution;

[0017] Step S220 sets the same level of each encoder to realize feature extraction through downsampling; and sets different levels of each encoder to realize feature splicing through upsampling after the resolution decreases from top to bottom.

[0018] Step S230 sets the same level of each decoder to realize feature recovery through upsampling; and sets different levels of each decoder to realize feature splicing through downsampling.

[0019] Step S240 sets the feature fusion of each path and the same level encoder and decoder node through dense connection.

[0020] In a possible preferred embodiment, the hybrid loss function with the weight decreasing layer by layer is:

[0021] ;

[0022] Wherein = ;

[0023] And , ;

[0024] Wherein represents the weight adjustment parameter of the sample loss of different categories, represents the loss adjustment parameter of the easy-to-classify sample, represents the true label, represents the predicted value; as the adjustment parameter to prevent the divisor and the dividend being 0.

[0025] In a possible preferred embodiment, each encoder and decoder node comprises two convolution units, and each convolution unit is composed of a convolution layer with a convolution kernel size of 3*3, a Batch Normalization layer, and a ReLU layer.

[0026] Corresponding to the above method, according to the second aspect of the present application, a skeleton extraction system is also provided, which comprises:

[0027] A storage unit for storing programs comprising the steps of the skeleton extraction method as claimed in any one of the above, for the data set training unit, the transmission unit, the processing unit, and timely calling execution;

[0028] A data set training unit for obtaining the acquisition polygon contour and the skeleton key point of the object in the original image, and converting the original image into a distance transform gray image, and sending it to the processing unit through the transmission unit;

[0029] A processing unit for triggering wave propagation connection of the skeleton key point with the polygon contour as the domain constraint, obtaining a preliminary skeleton, and performing an inflation operation to obtain a coarse-grained skeleton, taking the coarse-grained skeleton as the input of the second path encoder in the W-type network, taking the distance transform gray image as the input of the first path encoder, and taking the original real skeleton as the label, and using the hybrid loss function with the weight decreasing layer by layer to complete the training of the W-type network;

[0030] The transmission unit is also used to obtain the image that needs to extract the skeleton, so as to input the trained W-type network of the processing unit to extract the skeleton of the object in the image.

[0031] In a possible preferred implementation, the data set training unit comprises an image processing subunit, a polygon contour extraction network subunit, and a key point element extraction network subunit, wherein:

[0032] The image processing subunit is configured to obtain a distance transform grayscale image from the original image binary image through Euclidean distance calculation, and extract the skeleton key points of the object in the original image as the first real label and the polygon contour of the object as the second real label.

[0033] The polygon contour extraction network subunit is configured to take the original image binary image as input and the second real label as label for training to obtain the polygon contour.

[0034] The key point element extraction network subunit is configured to take the distance transform grayscale image of the original image as input and the first real label as label for training to obtain the skeleton key points.

[0035] Corresponding to the above method, according to a third aspect of the present application, a computer readable storage medium is also provided, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the skeleton extraction method according to any one of the above.

[0036] Through the skeleton extraction method and system provided by the present application, and the storage medium, the label can guide the skeleton generation process, effectively avoiding the over-branching problem caused by image edge noise, and through the design of the coarse-grained skeleton extraction of the present application, the preliminary skeleton can be obtained and positioned, thereby effectively solving the skeleton breaking problem caused by the large shape and size difference.

[0037] In addition, it is worth mentioning that, in addition to the above steps, through the innovative W-shaped network design of the present application, the position information of the coarse-grained skeleton and the topological information of the distance transform can be fully integrated, so that each side output can learn the skeleton information within the size range that it can extract, thereby effectively enhancing the extraction capability of the network for different shapes and sizes of the target, further detecting the shape and size changes, and thereby solving the skeleton breaking problem caused by the large shape and size changes of different parts of the object during the skeleton extraction process. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application illustrated in the drawings and their descriptions are used to explain the present application and are not intended to limit the present application. In the drawings:

[0039] Figure 1 The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application illustrated in the drawings and their descriptions are used to explain the present application and are not intended to limit the present application. In the drawings:

[0040] Figure 2 Logical architecture diagram for the skeleton extraction method of the present application;

[0041] Figure 3 Skeleton key point acquisition process diagram in the skeleton extraction method of the present application;

[0042] Figure 4 Polygon contour acquisition process diagram in the skeleton extraction method of the present application;

[0043] Figure 5 Coarse-grained skeleton generation process diagram in the skeleton extraction method of the present application;

[0044] Figure 6 W-type network structure diagram in the skeleton extraction method of the present application;

[0045] Figure 7 Skeleton extraction result diagram of a hand binary image by the skeleton extraction method of the present application;

[0046] Figure 8 Skeleton extraction result diagram of a hand binary image by the prior art method, wherein the skeleton is broken;

[0047] Figure 9 Structure diagram of the skeleton extraction system of the present application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solutions of the present application, the specific technical solutions of the present application will be described clearly and completely below in conjunction with the embodiments, so as to help those skilled in the art to further understand the present application. Obviously, the embodiments described in the present application are only a part of the embodiments of the present application, but not all the embodiments. It should be pointed out that, for those skilled in the art, the embodiments in the present application and the features in the embodiments can be combined with each other without departing from the concept of the present application and without being in conflict with each other. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts should belong to the disclosure and protection scope of the present application.

[0049] Furthermore, the terms "first", "second", "S100", "S200" and the like in the description and in the claims of the present application and the drawings, are used for distinguishing between similar objects, not necessarily for describing a sequential or chronological order. It is to be understood that the use of such terms herein is merely for distinguishing between two separate objects that are described, and does not necessarily indicate a requirement that one object precede another object in time. The terms "comprises", "comprising", "includes", "including", "has", "having" and their conjugates, as used herein, are used in their broadest sense and are intended to encompass the presence of one or more elements or components, without excluding the presence of other elements or components. Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. By the term "about" used herein is meant an acceptable degree of error for the quantitative values employed in the application under study. Generally, the term "about" is used herein to indicate that a value is within a degree of error of a value that is expected to be within the range of values expected to be obtained by a given method. The term "consisting essentially of to avoid the implication that the applications succumb to the jurisdiction of the Patent Cooperation Treaty.

[0050] It should be noted that in the example of the skeleton extraction method of the present application, the data set involved is at least partially derived from the CVPR2019 skeleton extraction competition data set , such as the original image binary graph and the skeleton real label composed of the original real skeleton as input data.

[0051] Please refer to Figures 1 to 7 To solve the problem of skeleton fracture caused by large shape and size changes of different parts of the object during skeleton extraction, the present application provides a skeleton extraction method, the steps of which include:

[0052] Step S100: training a data set to obtain the polygon contour of the object in the original image and the skeleton key point; using the polygon contour as a domain constraint, triggering wave propagation connection between the skeleton key points to obtain a preliminary skeleton, and performing an inflation operation to obtain a coarse-grained skeleton.

[0053] The step of training a data set to obtain a polygon contour and a skeleton key point in step S100 includes:

[0054] Step S110: extracting the skeleton key point of the object in the original image as the first real label and the polygon contour of the object as the second real label.

[0055] Specifically, the step of extracting the skeleton key point in step S110 includes: decomposing the original real skeleton into three parts, including: a starting point, a connection point and a skeleton branch, as shown in Figure 3 (a). Wherein, the skeleton starting point is represented by a white hollow circle, the connection point is represented by a white solid circle, and the line represents the skeleton branch, i.e. the connection branch between the starting points, the starting points and the connection points, and the connection points.

[0056] After that, all skeleton branches are uniformly sampled respectively, as shown in Figure 3 (b), to obtain skeleton branch points, and then the pixels of the starting points, the pixels of the connecting points and the skeleton graph of the skeleton branch points are taken as the skeleton key points. The pixel values of the skeleton key points are marked as 1, and the pixel values of the remaining pixels are marked as 0, so that the skeleton key point true label, i.e. the first true label, can be obtained, as shown in Figure 3 (c).

[0057] The extraction step of the polygon contour includes: connecting all the obtained starting points to form a closed shape and filling, such as filling the pixel value as 1 and the background pixel value as 0, to obtain the polygon contour true label, i.e. the second true label, as shown in Figure 4 .

[0058] Step S120 establishes a polygon contour extraction network, takes the binary graph of the original image as the input, and takes the second true label as the label for training. After the training is completed, the weight file is read, the polygon contour extraction is performed, and the polygon contour can be obtained.

[0059] Step S130 establishes a key point element extraction network, takes the distance conversion gray graph of the original image as the input, and takes the first true label as the label for training. After the training is completed, the weight file is read, the skeleton key point extraction is performed, and the skeleton key point can be obtained.

[0060] After that, as shown in Figure 5 (a), the vertices of the polygon contour obtained in step 120 are combined with the skeleton key points obtained in step S130 to form a domain constraint, and then as shown in Figure 5 (b), the vertices are taken as the starting points to trigger the wave propagation in the neighborhood to find the adjacent key points and connect them until all the key points obtained by the propagation are obtained, and the preliminary skeleton is obtained. Finally, the preliminary skeleton is dilated to obtain the coarse-grained skeleton, as shown in Figure 5 (c), so as to realize the preliminary positioning of the skeleton.

[0061] Step S200 establishes a W-type network, performs Euclidean distance calculation on the binary graph of the original image to obtain a distance conversion gray graph, takes the distance conversion gray graph as the input of the first encoder, takes the coarse-grained skeleton as the input of the second encoder, takes the original true skeleton as the label, and adopts a hybrid loss function with decreasing weights layer by layer to complete the training, so as to perform skeleton extraction.

[0062] Specifically, the establishment step of the W-type network includes:

[0063] Step S210 sets up at least two parallel encoders and two parallel decoders. The network is divided into at least five layers from top to bottom, and each layer has the same number of channels and resolution.

[0064] Step S220 sets up feature extraction between different levels of the same channel of each encoder by downsampling; sets the resolution between different levels of different channels of each encoder to decrease from top to bottom, and performs feature stitching after upsampling.

[0065] Step S230 sets up upsampling between different levels of the same channel of each decoder to achieve feature recovery; sets downsampling between different levels of different channels of each decoder to perform feature stitching.

[0066] Step S240 sets all encoder and decoder nodes at the same level to perform feature fusion through dense connections.

[0067] like Figure 6 As shown, the W-type network in this example consists of two parallel encoders and two parallel decoders. The network is divided into five stages from top to bottom: stage1, stage2, stage3, stage4, and stage5. Each stage has the same number of channels and resolution. In the figure, the number of channels and resolution are represented by the number of channels. high Width, respectively 64 256 256, 128 128 128, 256 64 64, 512 32 32, 1024 16 16.

[0068] The first encoder takes as input a distance-transformed grayscale image of the original image, and internally includes... , , , as well as Five nodes. The input to the second encoder is a coarse-grained skeleton, which includes... , , , as well as Five nodes.

[0069] Furthermore, feature extraction is achieved between different levels within the same encoder path through downsampling. For example, in the first encoder path... Depend on Obtained through downsampling operation, Depend on Obtained through downsampling, Depend on Obtained through downsampling, Depend on Obtained through downsampling.

[0070] Meanwhile, due to the different resolutions between different paths and levels of the encoder, feature concatenation is required after upsampling. For example, in one encoder path... After upsampling and resolution adjustment, it needs to be compared with the two encoders. Perform feature splicing. After upsampling and Perform feature splicing.

[0071] Feature recovery is achieved between different levels within the same path of the decoder through upsampling. For example, in one decoder... Depend on Obtained through upsampling, Depend on Obtained through upsampling, Depend on Obtained through upsampling, Depend on Obtained through upsampling.

[0072] In this process, different paths and levels of the decoder need to be connected by feature concatenation after downsampling. For example, in one decoder path... After downsampling and resolution adjustment, it needs to be compared with the two-channel decoder. Perform feature splicing. It needs to be downsampled and then compared with The splicing is also underway.

[0073] Simultaneously, encoder and decoder nodes at the same level perform feature fusion through dense connections. Ultimately, the four shallow feature maps from the two decoders ( , , as well as ) respectively fed into convolution kernels with a size of 1 A fully connected layer of 1 and a sigmoid classifier serve as the four outputs of this network, namely... , , and The resolutions of these four outputs are 256. 256, 128 128, 64 64 and 32 32.

[0074] Wherein in the present example, each encoder and decoder node comprises two convolution units, each of which is composed of a convolution layer with a kernel size of 3 3, a Batch Normalization layer (normalization), and a ReLU layer (activation function).

[0075] In this way, through the W-shaped network structure, the encoder can continuously fuse the position information of the coarse-grained skeleton and the topological information contained in the distance transform in the feature extraction process, and the decoder can recover the features in the continuous feature interaction, thereby effectively solving the problem of difficult extraction of the skeleton at a large shape scale.

[0076] Further, the original real skeleton is taken as a label, and three downsampling operations with a step size of 2 are performed on it, to obtain four labels with different resolutions, i.e. , , and The resolutions of the four labels are 256 256, 128 128, 64 64, and 32 32, respectively.

[0077] Then, the four outputs , , and are respectively calculated with the real skeleton graphs , , and with corresponding resolutions to obtain Loss.

[0078] Since the pixels of the skeleton key points in the image are not evenly distributed with the background pixels, a hybrid loss function is preferably used in the present example. The hybrid loss function Loss is represented by formula (1):

[0079]

[0080] In formula (1), the represents a Focal Loss , which is mainly proposed for uneven sample categories and difficult classification samples, and the calculation process can be represented by formula (2):

[0081]

[0082] In formula (2), the has the effect of weighting the Loss of samples of different categories. When the number of positive samples is small, the weight of the Loss of the positive samples is increased; when the predicted value When larger (i.e. belonging to easy classification samples), by reducing The loss of easy classification samples can be reduced, so that the model can pay more attention to difficult classification samples.

[0083] In formula (1) Indicates the Dice Loss , mainly aiming at the problem of small foreground ratio. DiceLoss is essentially a set similarity measure function, which is used to measure the overlapping part of two samples, and the calculation process can be represented by formula (3):

[0084]

[0085] Where Indicates the real label; Indicates the predicted value; Used to prevent the case where the divisor and the dividend are 0.

[0086] The final Is obtained by weighted addition of four side outputs, and since the real skeleton will lose part of the feature information after down-sampling, the weight design is gradually decreased according to the level, which is represented by formula (4):

[0087]

[0088] In this way, through the multi-scale side output mode, each side output can learn the skeleton information in the scale range that it can extract, effectively enhancing the network's extraction ability for different shapes and scales of the target, and providing more possibilities for extracting continuous and unbroken skeletons.

[0089] Finally, after the W-shaped network training is completed, the weight file is read, and the skeleton extraction can be performed, as shown in Figure 8 , the unbreakable skeleton can be finally obtained.

[0090] On the other hand, please refer to Figure 9 , corresponding to the above method, the present application also provides a skeleton extraction system, which comprises:

[0091] A storage unit for storing programs including steps of any of the above skeleton extraction methods, for the data set training unit, the transmission unit, the processing unit, and timely calling execution.

[0092] The data set training unit is used to obtain the polygon contour and the skeleton key point of the object in the original image, and convert the original image into a distance transform grayscale image, and send it to the processing unit through the transmission unit.

[0093] The processing unit is used to take the polygon contour as a domain constraint, to connect the skeleton key points by trigger wave propagation, to obtain a preliminary skeleton, to perform an inflation operation to obtain a coarse skeleton, to take the coarse skeleton as an input of a second encoder in a W-shaped network, to take a distance transform grayscale map as an input of a first encoder, and to take an original real skeleton as a label, and to use a hybrid loss function with a weight decreasing layer by layer to complete the training of the W-shaped network.

[0094] The transmission unit is further used to obtain an image in which a skeleton needs to be extracted, so as to input the image into the W-shaped network trained by the processing unit to extract the skeleton of the object in the image.

[0095] In the preferred example, the data set training unit comprises an image processing subunit, a polygon contour extraction network subunit, and a key point element extraction network subunit, wherein:

[0096] The image processing subunit is used to obtain a distance transform grayscale map by calculating the Euclidean distance of a binary image of an original image, and to extract skeleton key points of the object in the original image as a first real label and a polygon contour of the object as a second real label.

[0097] The polygon contour extraction network subunit is used to take the binary image of the original image as an input and the second real label as a label to perform training, so as to obtain the polygon contour.

[0098] The key point element extraction network subunit is used to take the distance transform grayscale map of the original image as an input and the first real label as a label to perform training, so as to obtain the skeleton key points.

[0099] Corresponding to the above method, the application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the skeleton extraction method in any one of the above embodiments.

[0100] In summary, by using the skeleton extraction method and system and the storage medium provided by the application, the label can guide the skeleton generation process, the problem of excessive branch caused by image edge noise can be effectively avoided, and by using the design of the coarse skeleton extraction, the preliminary skeleton can be obtained, the basic position of the preliminary skeleton can be located, and the problem of skeleton fracture in the case of large shape and size difference can be effectively solved.

[0101] In addition, it is worth mentioning that, in addition to the above steps, through the design of the W-shaped network of the present application, the position information of the coarse-grained skeleton and the topological information of the distance transformation can be fully integrated to utilize the multi-scale side output, so that each side output can learn the skeleton information within the scale range that it can extract, thereby effectively enhancing the network's extraction capability for different shapes and scales of the target, further detecting shape and scale changes, and thereby solving the problem of skeleton breaking caused by large shape and scale changes of different parts of the object in the skeleton extraction process.

[0102] The preferred embodiments of the application disclosed above are only used to help explain the application. The preferred embodiments do not describe all the details and limit the application to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the present application. The embodiments are selected and described in detail in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and utilize the application. The application is limited by the claims and their full scope and equivalents, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

[0103] Those skilled in the art can understand that, in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can also be realized by logically programming the method steps to make the system, device and each module thereof provided by the present application in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for realizing various programs can also be considered as structures in the hardware component; the modules for realizing various functions can also be considered as both software programs for realizing methods and structures in the hardware component.

[0104] In addition, all or part of the steps of the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a program stored in a storage medium, including a plurality of instructions for causing a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0105] Moreover, any combination of the various embodiments of the present embodiments can be made, as long as it does not deviate from the spirit of the present embodiments, and it should also be considered as disclosed by the present embodiments.

[0106] CITATIONS

[0107] [1] Demir I, Hahn C, Leonard K, et al. SkelNetOn 2019 Dataset and Challenge on Deep Learning for Geometric Shape Understanding [J]. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019: 1143-1151.

[0108] (Demir I, Hahn C, Leonard K, et al. SkelNetOn 2019 Dataset and Challenge on Deep Learning for Geometric Shape Understanding [J]. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019: 1143-1151.)

[0109] [2] Lin T Y, Goyal P, Girshick R, et al. Focal Loss for Dense Object Detection [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 42(2): 2999-3007.

[0110] (Lin T Y, Goyal P, Girshick R, et al. Focal Loss for Dense Object Detection [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 42(2): 2999-3007.)

[0111] [3] Li X, Sun X, Meng Y, et al. Dice Loss for Data Imbalanced NLP Tasks [J]. arXiv Preprint arXiv:1911.02855 2019.

[0112] (Li X, Sun X, Meng Y, et al. Dice Loss for Data-imbalanced NLP Tasks[J]. arXiv preprints arXiv:1911.02855, 2019).

Claims

1. A skeleton extraction method, comprising the following steps: Step S100: Train the dataset and obtain the polygonal outlines and skeletal key points of objects in the original image; Using the polygonal contour as the domain constraint, trigger wave propagation is performed between the key points of the skeleton to obtain the preliminary skeleton, and then an expansion operation is performed to obtain the coarse-grained skeleton. Step S200 establishes a W-type network, using the distance-transformed grayscale image of the original image as the input of the first encoder, the coarse-grained skeleton as the input of the second encoder, and the original real skeleton as the label. After training, a hybrid loss function with progressively decreasing weights is used for skeleton extraction. The hybrid loss function with progressively decreasing weights is as follows: ； in = ; and , ; in The weight adjustment parameter represents the loss of samples from different categories. The Loss adjustment parameter represents the number of easily classifiable samples. Indicates the true label, Indicates the predicted value; This is an adjustment parameter used to prevent the divisor and dividend from being zero.

2. The skeleton extraction method according to claim 1, wherein step S100, obtaining the polygonal contour and skeleton key points, includes: Step S110 extracts the skeleton key points of the object in the original image as the first ground truth label and the polygonal outline of the object as the second ground truth label; Step S120: Establish a polygon contour extraction network, using the binary image of the original image as input and the second real label as the label for training, to obtain polygon contours. Step S130 establishes a key point element extraction network, using the distance-transformed grayscale image of the original image as input and the first real label as the label for training, to obtain skeleton key points.

3. The skeleton extraction method according to claim 2, wherein the skeleton key point extraction step S110 includes: The original real skeleton is broken down into starting points, connection points, and skeleton branches; The skeleton branch points are obtained by uniformly sampling all skeleton branches respectively; The starting point, connection point, and skeleton branch point are designated as the skeleton key points.

4. The skeleton extraction method according to claim 3, wherein the polygon contour extraction step in step S110 includes: Connect all the obtained starting points to form a closed shape, and fill it to obtain a polygonal outline.

5. The skeleton extraction method according to claim 1, wherein the step of establishing the W-shaped network includes: Step S210 sets up at least two parallel encoders and two parallel decoders. The network is divided into at least five layers from top to bottom, and each layer has the same number of channels and resolution. Step S220 sets up feature extraction between different levels of the same channel of each encoder through downsampling; sets the resolution between different levels of different channels of each encoder to decrease from top to bottom, and performs feature stitching after upsampling; Step S230 sets up upsampling to restore features between different levels of the same channel in each decoder; sets downsampling to stitch features between different levels of different channels in each decoder. Step S240 sets all encoder and decoder nodes at the same level to perform feature fusion through dense connections.

6. The skeleton extraction method according to claim 5, wherein, Each encoder and decoder node contains two convolutional units, and each convolutional unit consists of a convolutional layer with a kernel size of 3×3, a batch normalization layer, and a ReLU layer.

7. A skeleton extraction system, comprising: A storage unit is used to store a program including the skeleton extraction method steps as described in any one of claims 1 to 6, so that the dataset training unit, transmission unit, and processing unit can retrieve and execute it as needed. The dataset training unit is used to obtain the polygonal outlines of objects in the original image and obtain the skeleton key points, and convert the original image into a distance-transformed grayscale image, which is then sent to the processing unit via the transmission unit. The processing unit is used to connect the key points of the skeleton with trigger wave propagation based on the polygonal contour as the domain constraint, obtain the preliminary skeleton, and perform dilation operation to obtain the coarse-grained skeleton. After obtaining the coarse-grained skeleton, it is used as the input of the second encoder in the W-type network. At the same time, the distance transformation grayscale image is used as the input of the first encoder, and the original real skeleton is used as the label. The training of the W-type network is completed by using a hybrid loss function with progressively decreasing weights. The transmission unit is also used to acquire the image from which the skeleton needs to be extracted, and input it into the W-shaped network trained by the processing unit to extract the skeleton of the object in the image.

8. The skeleton extraction system according to claim 7, wherein the dataset training unit comprises: Image processing subunit, polygon contour extraction network subunit, key point element extraction network subunit, where: The image processing subunit is used to calculate the distance transformation grayscale image from the binary image of the original image using Euclidean distance; and to extract the skeleton key points of the object in the original image as the first ground truth label and the polygonal outline of the object as the second ground truth label. The polygon contour extraction network subunit is used to train the network by taking the binary image of the original image as input and the second ground truth label as the label, in order to obtain polygon contours. The keypoint element extraction network subunit is used to train the network by taking the distance-transformed grayscale image of the original image as input and the first ground truth label as the label, in order to obtain the skeleton keypoints.

9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the skeleton extraction method as claimed in any one of claims 1 to 6.

Images