Semi-supervised training method, system and electronic device for soft bag segmentation network

Abstract

The present application relates to a semi-supervised training method, system and electronic device for a soft bag segmentation network, wherein the semi-supervised training method for the soft bag segmentation network comprises: training an instance segmentation network according to pre-acquired soft bag labeled data to obtain a soft bag coarse segmentation network, and training a fine segmentation network according to labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network; predicting unlabeled data through the soft bag coarse segmentation network to obtain coarse pseudo labels, and processing the coarse pseudo labels according to the soft bag fine segmentation network to obtain fine pseudo labels; extracting features from the unlabeled data and the labeled data through the soft bag coarse segmentation network to obtain a first segmentation result of the labeled data, and segmenting the unlabeled data according to the soft bag coarse segmentation network to obtain a second segmentation result; supervising the first segmentation result according to the label of the labeled data, and supervising the second segmentation result according to the fine pseudo labels, so as to train the soft bag coarse segmentation network to obtain the soft bag segmentation network.

Description

Technical Field

[0001] The present application relates to the field of semi-supervised training, and in particular to a semi-supervised training method, system and electronic equipment for a soft bag segmentation network. Background Art

[0002] In the destacking scenario of soft bags, only by accurately separating all the overlapping soft bags can accurate coordinates be provided for the subsequent grabbing of the soft bags.

[0003] The current soft bag segmentation method is usually based on deep learning methods, which require a large amount of soft bag data to be very finely labeled and trained to meet the accuracy requirements. In addition, when encountering new soft bag materials, the model trained using historical data is not general enough, and the segmentation effect on the new soft bag is not good. The new soft bag data can only be labeled again and the model can be retrained with the previous data. For the problem of small amount of data, a semi-supervised method can be adopted to effectively reduce the amount of data labeling. However, semi-supervised methods usually have poor prediction effects on unlabeled data, resulting in low quality of the pseudo-labels provided. Low-quality pseudo-labels will affect the segmentation effect of the model during the model iteration process, resulting in increasingly poor segmentation effect of the model.

[0004] Existing soft bag segmentation methods require a large amount of accurate labeled data and the segmentation effect is poor. Summary of the invention

[0005] The embodiments of the present application provide a semi-supervised training method, system and electronic device for a soft bag segmentation network, so as to at least solve the problem in the related art that a large amount of accurate labeled data is required and the segmentation effect is poor.

[0006] In a first aspect, an embodiment of the present application provides a semi-supervised training method for a soft bag segmentation network, characterized by comprising:

[0007] Training an instance segmentation network according to pre-acquired soft bag labeled data to obtain a soft bag coarse segmentation network, and training a fine segmentation network according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network;

[0008] Predicting the pre-acquired unlabeled data of the soft bag by the soft bag coarse segmentation network with frozen weights to obtain a coarse pseudo label, and processing the coarse pseudo label according to the soft bag fine segmentation network to obtain a fine pseudo label;

[0009] Performing feature extraction on the unlabeled data and the labeled data through the soft bag coarse segmentation network, obtaining a first segmentation result of the labeled data based on the extraction result, and segmenting the unlabeled data according to the soft bag coarse segmentation network with unfrozen weights to obtain a second segmentation result;

[0010] The first segmentation result is supervised according to the label of the labeled data, and the second segmentation result is supervised according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain a soft bag segmentation network.

[0011] In one embodiment, the extracting features of the unlabeled data and the labeled data by the soft bag coarse segmentation network, and obtaining a first segmentation result of the labeled data based on the extraction result, includes:

[0012] Extracting features of the unlabeled data by using the soft bag coarse segmentation network with unfrozen weights to obtain unlabeled data features;

[0013] Extracting features of the labeled data by using the soft bag coarse segmentation network with unfrozen weights to obtain labeled data features;

[0014] A cross-attention between the labeled data feature and the unlabeled data feature is calculated, a regeneration feature of the labeled data is determined according to the cross-attention, and a first segmentation result is obtained according to the regeneration feature.

[0015] In one embodiment, the calculating the cross attention between the features of the labeled data and the features of the unlabeled data includes:

[0016]

[0017] Among them, q represents the query in the attention calculation, k represents the key, v represents the value, and w q 、w k and w v Both represent feature transformation weights, M l A two-dimensional matrix representing the features of labeled data, represents a two-dimensional matrix of unlabeled data features, s represents the cross attention matrix between the labeled data features and the unlabeled data features, represents the softmax function, represents the instance normalization function.

[0018] In one embodiment, determining the regeneration features of the labeled data according to the cross attention includes:

[0019]

[0020] Among them, c represents the number of channels of unlabeled data, It means to find the index of the previous channels for the value of the dimension where the unlabeled data feature in the cross attention matrix is ​​located. represents the value of the cross attention obtained according to the index, represents the value of v obtained according to the index, represents the rebirth feature, w out represents the feature transformation weight.

[0021] In one embodiment, supervising the first segmentation result according to the label of the labeled data, and supervising the second segmentation result according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain a soft bag segmentation network, comprises:

[0022] Determine a first loss according to the first segmentation result and the label of the corresponding labeled data;

[0023] Determine a second loss according to the second segmentation result and the corresponding fine pseudo-label;

[0024] A total loss is determined according to the first loss and the second loss, and the soft bag coarse segmentation network is iteratively trained based on the total loss.

[0025] In one embodiment, the training of a fine segmentation network according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network includes:

[0026] Predicting the labeled data through the soft bag coarse segmentation network with frozen weights to obtain a prediction result;

[0027] Randomly destroy the predicted result according to a preset probability to obtain a destruction result;

[0028] A fine segmentation network is trained according to the destruction results and the labels of the corresponding labeled data to obtain a soft bag fine segmentation network.

[0029] In one embodiment, the soft bag coarse segmentation network and the fine segmentation network are arbitrary segmentation network models.

[0030] In a second aspect, an embodiment of the present application provides a semi-supervised training system for a soft bag segmentation network, comprising:

[0031] An acquisition module is used to train an instance segmentation network according to pre-acquired labeled data of a soft bag to obtain a soft bag coarse segmentation network, and to train a fine segmentation network according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network;

[0032] Label module: used to predict the pre-acquired unlabeled data of the soft bag by freezing the weight of the soft bag coarse segmentation network to obtain a coarse pseudo label, and process the coarse pseudo label according to the soft bag fine segmentation network to obtain a fine pseudo label;

[0033] Segmentation module: used for performing feature extraction on the unlabeled data and the labeled data through the soft bag coarse segmentation network, obtaining a first segmentation result of the labeled data based on the extraction result, and segmenting the unlabeled data according to the soft bag coarse segmentation network with unfrozen weights to obtain a second segmentation result;

[0034] Training module: used to supervise the first segmentation result according to the label of the labeled data, and supervise the second segmentation result according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain a soft bag segmentation network.

[0035] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the semi-supervised training method for the soft bag segmentation network as described in the first aspect above is implemented.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the semi-supervised training method for the soft bag segmentation network as described in the first aspect above.

[0037] The semi-supervised training method, system and electronic device of a soft bag segmentation network provided by the embodiments of the present application have at least the following technical effects.

[0038] The semi-supervised training method proposed in this application reduces the requirement for the amount of labeled data and adds a fine segmentation network. The fine segmentation network is used to improve the quality of the pseudo-labels obtained by the soft bag coarse segmentation network, so as to optimize the obtained pseudo-labels and improve the quality of the training data, thereby improving the segmentation effect without the need for a large amount of labeled data.

[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 is a flow chart of a semi-supervised training method for a soft bag segmentation network provided in an embodiment of the present application;

[0042] Figure 2 This is a schematic diagram of the effect of the traditional semi-supervised training segmentation method;

[0043] Figure 3is a schematic diagram of the effect of the semi-supervised training method of the soft bag segmentation network provided in an embodiment of the present application;

[0044] Figure 4 is a structural block diagram of a semi-supervised training system for a soft bag segmentation network according to an embodiment of the present application;

[0045] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0047] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0048] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] In a first aspect, an embodiment of the present application provides a semi-supervised training method for a soft bag segmentation network. Figure 1 is a flow chart of a semi-supervised training method for a soft bag segmentation network provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0051] Step S101 , training an instance segmentation network according to pre-acquired soft bag labeled data to obtain a soft bag coarse segmentation network, and training a fine segmentation network according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network.

[0052] Optionally, the instance segmentation network and the fine segmentation network are trained by a fully supervised method, specifically including: randomly initializing the instance segmentation network, using the labeled data of the soft bag and the corresponding labels to train the instance segmentation network until it converges, and obtaining a soft bag coarse segmentation network. The loss function during training can select any single loss function or loss function combination suitable for instance segmentation. The soft bag coarse segmentation network obtained by training predicts the labeled data, and the fine segmentation network is trained based on the prediction results and the corresponding labels to obtain a soft bag fine segmentation network. Among them, the label of the labeled data includes the location information of the soft bag.

[0053] In one example, step S101 includes:

[0054] Step S1011, predicting the labeled data through the soft bag coarse segmentation network with frozen weights to obtain a prediction result.

[0055] Step S1012: randomly destroy the prediction result according to a preset probability to obtain a destruction result.

[0056] Step S1013: training the fine segmentation network according to the destruction result and the label of the corresponding labeled data to obtain a soft bag fine segmentation network.

[0057] Optionally, in order to avoid interfering with the training of the coarse segmentation network, the weights of the coarse segmentation network are frozen, and only the soft bag coarse segmentation network is used to predict the labeled data to obtain the prediction mask for the fine segmentation network training. When training the fine segmentation network, the destroyed mask (i.e., the destruction result) is used as input, and the label of the labeled data is used as output. The fine segmentation network is trained and the network parameters are updated until the fine segmentation network converges. The loss function during training is:

[0058]

[0059] Among them, CE represents cross entropy loss, L1 represents L1 loss, Represents the mask after destruction, y l Indicates the label corresponding to the labeled data. It should be noted that the loss function used to train the fine segmentation network can be determined based on the characteristics of the training data and the model characteristics in actual application, and is not limited to this example. In this way, the fine segmentation network is trained through the destroyed mask to improve the robustness of the fine segmentation network.

[0060] In one example, the soft bag coarse segmentation network and the fine segmentation network are any segmentation network models. Optionally, the semi-supervised training method of the soft bag segmentation network proposed in the present application does not limit the network model used, and can be selected according to actual needs.

[0061] Step S102 , predicting the pre-acquired unlabeled data of the soft bag through the soft bag coarse segmentation network with frozen weights to obtain coarse pseudo labels, and processing the coarse pseudo labels according to the soft bag fine segmentation network to obtain fine pseudo labels.

[0062] Optionally, in order to avoid interfering with the training of the coarse segmentation network, the weights of the coarse segmentation network are frozen, and only the soft bag coarse segmentation network is used to predict the labeled and unlabeled data to obtain a coarse pseudo label. The soft bag fine segmentation network obtains a fine pseudo label based on the coarse pseudo label. In this way, the quality of the pseudo label obtained by the soft bag coarse segmentation network is improved by the fine segmentation network, thereby improving the quality of the training data.

[0063] Step S103, extracting features of the unlabeled data and the labeled data through the soft bag coarse segmentation network, and obtaining a first segmentation result of the labeled data based on the extraction result, and segmenting the unlabeled data according to the soft bag coarse segmentation network with unfrozen weights to obtain a second segmentation result.

[0064] Optionally, the features of the unlabeled data and the labeled data are extracted through a soft bag coarse segmentation network, and the cosine similarity between the features of the labeled data and the features of the unlabeled data is calculated, and the features of the labeled data are replaced with the features of the unlabeled data with higher similarity to the features of the labeled data, wherein the data amounts of the labeled data and the unlabeled data are the same.

[0065] In one example, step S103 includes:

[0066] Step S301 , extracting features of unlabeled data through a soft bag coarse segmentation network with unfrozen weights to obtain unlabeled data features.

[0067] Optionally, the unlabeled data is represented as x u , then the feature representation of the unlabeled data extracted during the inference process is , where c represents the number of channels of the feature map of each image, h represents the height of the feature map, and w represents the width of the feature map. u Convert to a two-dimensional matrix , and stored in a queue Q of length N.

[0068] Step S302 , extracting features from the labeled data using a soft bag coarse segmentation network with unfrozen weights to obtain features of the labeled data.

[0069] Optionally, labeled data is represented as x l , the feature representation of the labeled data extracted by the soft bag coarse segmentation network is , where c represents the number of channels of the feature map of each image, h represents the height of the feature map, and w represents the width of the feature map. l Convert to a two-dimensional matrix , merge the feature matrices in the queue Q of unlabeled data features into .

[0070] Step S303, calculate the cross attention between the labeled data features and the unlabeled data features, determine the regeneration features of the labeled data according to the cross attention, and obtain the first segmentation result according to the regeneration features.

[0071] Optionally, the cosine similarity between the features of the labeled data and the features of the unlabeled data is calculated to obtain features of the unlabeled data having a higher similarity to the features of the labeled data, and the features of the labeled data are replaced with the obtained features of the unlabeled data.

[0072] In one example, step S303 includes:

[0073]

[0074] Among them, q represents the query in the attention calculation, k represents the key, v represents the value, and w q 、w k and w v Both represent feature transformation weights, M l A two-dimensional matrix representing the features of labeled data, represents the two-dimensional matrix of unlabeled data features, s represents the cross attention matrix between labeled data features and unlabeled data features, represents the softmax function, represents the instance normalization function.

[0075] Optionally, , , , w q 、w k and w v Represents the feature transformation weight, c represents the number of channels of the feature map of each image, h represents the height of the feature map, and w represents the width of the feature map. Represents the similarity matrix between the c channels of N unlabeled data features and the c channels of the current labeled data features.

[0076] In one example, determining the regeneration features of the labeled data according to the cross attention in step S303 includes:

[0077]

[0078] Among them, c represents the number of channels of unlabeled data, It means to find the index of the previous channels for the value of the dimension where the unlabeled data feature in the cross attention matrix is ​​located. represents the value of the cross attention obtained according to the index, represents the value of v obtained according to the index, represents the rebirth feature, w out represents the feature transformation weight.

[0079] Optionally, represents the feature transformation weight, It means to calculate the index of the top c of the dimension where the c channels of unlabeled data in the similarity matrix s are located, and obtain the values ​​of s and v according to the index of top c, and get , , among which, the rebirth feature Characterize the features after replacing the features of labeled data with the features of unlabeled data.

[0080] In this way, step S103 generates features of labeled data using features of unlabeled data according to the soft bag coarse segmentation network, and supervises the generated features through the labels of the labeled data to improve the quality of training data, thereby improving the segmentation accuracy of the soft bag segmentation network.

[0081] Step S104 , supervising the first segmentation result according to the label of the labeled data, and supervising the second segmentation result according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain a soft bag segmentation network.

[0082] Optionally, a soft bag coarse segmentation network is trained according to the joint loss of the first segmentation result and the second segmentation result to obtain a soft bag segmentation network. In this way, the soft bag coarse segmentation network uses the features of the unlabeled data to generate features of the labeled data, and the generated features are supervised by the labels of the labeled data to improve the quality of the training data and the segmentation accuracy of the soft bag segmentation network. By optimizing the obtained pseudo labels, the quality of the training data is improved, thereby improving the segmentation effect without requiring a large amount of labeled data.

[0083] In one example, step S104 includes:

[0084] A first loss is determined according to the first segmentation result and the label of the corresponding labeled data; a second loss is determined according to the second segmentation result and the corresponding fine pseudo-label; a total loss is determined according to the first loss and the second loss, and the soft bag coarse segmentation network is iteratively trained based on the total loss.

[0085] Optionally, the first segmentation result is supervised by the label of the labeled data, and the second segmentation result is supervised by the fine pseudo-label for training. The loss function during training is expressed as:

[0086]

[0087] Among them, L represents the total loss, represents the first loss, y l The label representing the labeled data, represents the first segmentation result determined according to the rebirth feature, represents the second loss, θ1 represents the soft bag coarse segmentation network, x u represents unlabeled data, Indicates a fine pseudo label.

[0088] Figure 2 This is a schematic diagram of the effect of the traditional semi-supervised training segmentation method. Figure 3Schematic diagram of the effect of the semi-supervised training method of the soft bag segmentation network provided in the embodiment of the present application, refer to Figure 2 and Figure 3 It can be seen that the segmentation network obtained by the traditional semi-supervised segmentation method is prone to segmenting parts of other soft bags together with the target soft bag during segmentation, resulting in low segmentation accuracy, while the segmentation accuracy of the soft bag segmentation network obtained by the training method adopted in this application is obviously higher. And when training the method adopted in this application, each new soft bag only needs to be labeled with 10 frames of data for semi-supervised training to achieve the effect of labeling 200 frames of data for each soft bag for full supervised training.

[0089] In summary, the present application improves the quality of pseudo-labels obtained by the soft bag coarse segmentation network through a fine segmentation network, thereby improving the quality of training data. The soft bag coarse segmentation network uses the features of unlabeled data to generate features of labeled data, and supervises the generated features through the labels of labeled data to improve the quality of training data and the segmentation accuracy of the soft bag segmentation network. The semi-supervised training method proposed in the present application reduces the requirements for the amount of labeled data, and adds a fine segmentation network to optimize the obtained pseudo-labels and improve the quality of training data, thereby improving the segmentation effect without the need for a large amount of labeled data.

[0090] In a second aspect, the embodiments of the present application provide a semi-supervised training system for a soft bag segmentation network. Figure 4 is a structural block diagram of a semi-supervised training system for a soft bag segmentation network according to an embodiment of the present application. Figure 4 As shown, the system includes:

[0091] Acquisition module 100: used to train the instance segmentation network according to the pre-acquired soft bag labeled data to obtain a soft bag coarse segmentation network, and to train the fine segmentation network according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network.

[0092] The label module 200 is used to predict the pre-acquired unlabeled data of the soft bag to obtain a rough pseudo label through the soft bag coarse segmentation network with frozen weights, and process the coarse pseudo label according to the soft bag fine segmentation network to obtain a fine pseudo label.

[0093] Segmentation module 300: used to extract features of unlabeled data and labeled data through a soft bag coarse segmentation network, obtain a first segmentation result of the labeled data based on the extraction result, and segment the unlabeled data according to the soft bag coarse segmentation network with unfrozen weights to obtain a second segmentation result.

[0094] Training module 400: used to supervise the first segmentation result according to the label of the labeled data, and supervise the second segmentation result according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain the soft bag segmentation network.

[0095] In one example, the segmentation module 300 includes:

[0096] The first extraction unit is used to extract features of the unlabeled data by using a soft bag coarse segmentation network with unfrozen weights to obtain unlabeled data features.

[0097] The second extraction unit is used to extract features of the labeled data through a soft bag coarse segmentation network with unfrozen weights to obtain features of the labeled data.

[0098] Segmentation unit: used to calculate the cross attention between the labeled data features and the unlabeled data features, determine the reborn features of the labeled data according to the cross attention, and obtain the first segmentation result according to the reborn features.

[0099] In one example, the segmentation unit includes:

[0100]

[0101] Among them, q represents the query in the attention calculation, k represents the key, v represents the value, and w q 、w k and w v Both represent feature transformation weights, M l A two-dimensional matrix representing the features of labeled data, represents the two-dimensional matrix of unlabeled data features, s represents the cross attention matrix between labeled data features and unlabeled data features, represents the softmax function, represents the instance normalization function.

[0102] In one example, the segmentation unit includes:

[0103]

[0104] Among them, c represents the number of channels of unlabeled data, It means to find the number of previous channels index for the value of the dimension where the unlabeled data feature in the cross attention matrix is ​​located. Represents the value of the cross attention obtained according to the index, Indicates the value of v obtained according to the index, Indicates the rebirth feature, w out represents the feature transformation weight.

[0105] In one example, the training module 400 includes:

[0106] A first loss is determined according to the first segmentation result and the label of the corresponding labeled data.

[0107] A second loss is determined according to the second segmentation result and the corresponding fine pseudo-label.

[0108] The total loss is determined according to the first loss and the second loss, and the soft bag coarse segmentation network is iteratively trained based on the total loss.

[0109] In one example, the acquisition module 100 includes:

[0110] The soft bag coarse segmentation network with frozen weights is used to predict the labeled data and obtain the prediction results.

[0111] The predicted results are randomly destroyed according to the preset probability to obtain the destruction results.

[0112] The fine segmentation network is trained according to the destruction results and the labels of the corresponding labeled data to obtain the soft bag fine segmentation network.

[0113] In one example, the soft bag coarse segmentation network and the fine segmentation network in the acquisition module 100 are arbitrary segmentation network models.

[0114] In summary, the present application improves the quality of pseudo-labels obtained by the soft bag coarse segmentation network through a fine segmentation network, thereby improving the quality of training data. The soft bag coarse segmentation network uses the features of unlabeled data to generate features of labeled data, and supervises the generated features through the labels of the labeled data to improve the quality of training data and the segmentation accuracy of the soft bag segmentation network. The semi-supervised training system proposed in the present application reduces the requirements for the amount of labeled data, and adds a fine segmentation network to optimize the obtained pseudo-labels and improve the quality of training data, thereby improving the segmentation effect without the need for a large amount of labeled data.

[0115] In a third aspect, an embodiment of the present application provides an electronic device, Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the semi-supervised training method of the soft bag segmentation network provided in the first aspect is implemented. Figure 5 The electronic device 60 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0116] The electronic device 60 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 60 may include but are not limited to: at least one processor 61, at least one memory 62, and a bus 63 connecting different system components (including the memory 62 and the processor 61).

[0117] The bus 63 includes a data bus, an address bus, and a control bus.

[0118] The memory 62 may include a volatile memory, such as a random access memory (RAM) 621 and / or a cache memory 622 , and may further include a read-only memory (ROM) 623 .

[0119] The memory 62 may also include a program / utility 625 having a set (at least one) of program modules 624, such program modules 624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0120] The processor 61 executes various functional applications and data processing by running the computer program stored in the memory 62, such as the semi-supervised training method of the soft bag segmentation network provided in the first aspect of the present application.

[0121] The electronic device 60 may also communicate with one or more external devices 64 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 65. Furthermore, the model-generated device 60 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 66. As shown, the network adapter 66 communicates with other modules of the model-generated device 60 via a bus 63. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated device 60, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0122] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0123] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the semi-supervised training method for the soft bag segmentation network provided in the first aspect.

[0124] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0125] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the semi-supervised training method for the soft bag segmentation network provided in the first aspect.

[0126] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0127] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A semi-supervised training method for a soft bag segmentation network, characterized in that: include: According to the pre-acquired soft bag labeled data, an instance segmentation network is trained to obtain a soft bag coarse segmentation network, and according to the labeled data and the soft bag coarse segmentation network, a fine segmentation network is trained to obtain a soft bag fine segmentation network, wherein the labeled data and the unlabeled data are image data, and the fine segmentation network is trained according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network, including: The labeled data is predicted by the soft bag coarse segmentation network with frozen weights to obtain a prediction result, the prediction result is randomly destroyed according to a preset probability to obtain a destroyed result, and a fine segmentation network is trained according to the destroyed result and the label of the corresponding labeled data to obtain a soft bag fine segmentation network; Predicting the pre-acquired unlabeled data of the soft bag by the soft bag coarse segmentation network with frozen weights to obtain a coarse pseudo label, and processing the coarse pseudo label according to the soft bag fine segmentation network to obtain a fine pseudo label; Performing feature extraction on the unlabeled data and the labeled data through the soft bag coarse segmentation network, obtaining a first segmentation result of the labeled data based on the extraction result, and segmenting the unlabeled data according to the soft bag coarse segmentation network with unfrozen weights to obtain a second segmentation result; The first segmentation result is supervised according to the label of the labeled data, and the second segmentation result is supervised according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain a soft bag segmentation network.

2. A semi-supervised training method for a soft bag segmentation network according to claim 1, characterized in that: The step of extracting features from the unlabeled data and the labeled data through the soft bag coarse segmentation network and obtaining a first segmentation result of the labeled data based on the extraction result includes: Extracting features of the unlabeled data by using the soft bag coarse segmentation network with unfrozen weights to obtain unlabeled data features; Extracting features of the labeled data by using the soft bag coarse segmentation network with unfrozen weights to obtain labeled data features; A cross-attention between the labeled data feature and the unlabeled data feature is calculated, a regeneration feature of the labeled data is determined according to the cross-attention, and a first segmentation result is obtained according to the regeneration feature.

3. The semi-supervised training method for a soft bag segmentation network according to claim 2, characterized in that: The calculating the cross attention between the features of the labeled data and the features of the unlabeled data includes: Among them, q represents the query in the attention calculation, k represents the key, v represents the value, and w q 、w k and w v Both represent feature transformation weights, M l A two-dimensional matrix representing the features of labeled data, represents a two-dimensional matrix of unlabeled data features, s represents the cross attention matrix between the labeled data features and the unlabeled data features, represents the softmax function, represents the instance normalization function.

4. The semi-supervised training method for a soft bag segmentation network according to claim 3, characterized in that: The determining the regeneration features of the labeled data according to the cross attention comprises: Among them, c represents the number of channels of unlabeled data, (s,v,c) represents the number of indexes of the previous channels for the value of the dimension where the unlabeled data feature in the cross attention matrix is ​​located. represents the value of the cross attention obtained according to the index, represents the value of v obtained according to the index, represents the rebirth feature, w out represents the feature transformation weight.

5. The semi-supervised training method for a soft bag segmentation network according to claim 2, characterized in that: The step of supervising the first segmentation result according to the label of the labeled data and supervising the second segmentation result according to the fine pseudo label to train the soft bag coarse segmentation network to obtain a soft bag segmentation network includes: Determine a first loss according to the first segmentation result and the label of the corresponding labeled data; Determine a second loss according to the second segmentation result and the corresponding fine pseudo-label; A total loss is determined according to the first loss and the second loss, and the soft bag coarse segmentation network is iteratively trained based on the total loss.

6. The semi-supervised training method for a soft bag segmentation network according to claim 1, characterized in that: The soft bag coarse segmentation network and the fine segmentation network are arbitrary segmentation network models.

7. A semi-supervised training system for a soft bag segmentation network, characterized in that: include: An acquisition module is used to train an instance segmentation network according to pre-acquired soft bag labeled data to obtain a soft bag coarse segmentation network, and train a fine segmentation network according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network, wherein the labeled data and the unlabeled data are image data, and the fine segmentation network is trained according to the labeled data and the soft bag coarse segmentation network to obtain a soft bag fine segmentation network, including: The labeled data is predicted by the soft bag coarse segmentation network with frozen weights to obtain a prediction result, the prediction result is randomly destroyed according to a preset probability to obtain a destroyed result, and a fine segmentation network is trained according to the destroyed result and the label of the corresponding labeled data to obtain a soft bag fine segmentation network; Label module: used to predict the pre-acquired unlabeled data of the soft bag through the soft bag coarse segmentation network with frozen weights to obtain a coarse pseudo label, and process the coarse pseudo label according to the soft bag fine segmentation network to obtain a fine pseudo label; Segmentation module: used for performing feature extraction on the unlabeled data and the labeled data through the soft bag coarse segmentation network, obtaining a first segmentation result of the labeled data based on the extraction result, and segmenting the unlabeled data according to the soft bag coarse segmentation network with unfrozen weights to obtain a second segmentation result; Training module: used to supervise the first segmentation result according to the label of the labeled data, and supervise the second segmentation result according to the fine pseudo-label, so as to train the soft bag coarse segmentation network to obtain a soft bag segmentation network.

8. An electronic device, characterized in that: include Memory, processor, and A computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the semi-supervised training method for the soft bag segmentation network according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the semi-supervised training method for the soft bag segmentation network according to any one of claims 1 to 6 is implemented.

Images