Training methods and devices, storage media and electronic devices for image recognition networks

By training the image recognition network with and without labeled sample images, adjusting the network parameters, and generating reference labels, the problem of poor training results caused by limited labeled samples is solved, thus improving the recognition performance of the image recognition network.

CN117011579BActive Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211483321.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-24
Publication Date
2026-11-17
Estimated Expiration
2042-11-24

AI Technical Summary

Technical Problem

In existing technologies, training image recognition networks using limited labeled samples results in poor training performance.

Method used

The image recognition network is trained by acquiring a first sample image with instance labels and a second sample image without instance labels. The network parameters are adjusted to generate reference labels. When the recognition loss value reaches a threshold condition, the network parameters are determined to have reached the convergence condition. The training set is then updated using the second sample image with reference labels.

Benefits of technology

It improves the training performance of image recognition networks, solves the problem of poor training results caused by insufficient labeled samples, and achieves more robust image recognition results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117011579B_ABST
    Figure CN117011579B_ABST
Patent Text Reader

Abstract

This invention discloses a training method, apparatus, storage medium, and electronic device for an image recognition network. The method includes: inputting a first sample image and a second sample image into an image recognition network to obtain a recognition loss value, wherein the first sample image carries instance labels; adjusting the network parameters in the image recognition network if the recognition loss value does not reach a threshold condition, and using the adjusted network parameters to determine a reference label generation network; inputting the second sample image into the reference label generation network to obtain reference instance labels generated for the second sample image; updating the second image sample set using the second sample image carrying the reference instance labels; and determining the image recognition network containing the current network parameters as the target image recognition network that has reached convergence if the recognition loss value reaches a threshold condition. This invention solves the technical problem of poor training performance in existing training methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a training method and apparatus for an image recognition network, a storage medium, and an electronic device. Background Technology

[0002] In the field of image recognition technology, there are recognition tasks that require the use of neural network models to segment objects in an image that belong to a specific category set, thereby providing different labels for individual instances of objects belonging to the same category. These tasks are usually called instance segmentation tasks.

[0003] In related techniques, before training a network to perform instance segmentation tasks, it is usually necessary to obtain manually labeled image samples to train the initial model. However, due to the large amount of work involved in labeling the instances in the images, the number of labeled samples available for training during the initial training process is very limited. Training with limited labeled samples can lead to overfitting in the instance segmentation network. In other words, the method of training the model using limited labeled samples has the problem of poor training performance.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a training method and apparatus, storage medium and electronic device for image recognition networks, to at least solve the technical problem that related technologies using limited labeled samples for network training have poor training effects.

[0006] According to one aspect of the present invention, a training method for an image recognition network is provided, comprising: acquiring a first image sample set and a second image sample set, wherein the first image sample set includes a plurality of first samples, each of the first samples including a first sample image carrying an instance label, the instance label being used to indicate the instance category of an instance object in the first sample image and the image position of the instance object in the first sample image; the second image sample set includes a plurality of second samples, each of the second samples including a second sample image without an instance label; inputting the first sample image acquired from the first image sample set and the second sample image acquired from the second image sample set into an image recognition network in a training state to obtain a recognition loss value, wherein... The aforementioned image recognition network is used to identify the instance category of an instance object in a sample image and the image position of the instance object in the sample image. If the recognition loss value does not reach a threshold, the network parameters in the image recognition network are adjusted, and a reference label generation network is determined using the adjusted network parameters. The reference label generation network is used to generate reference instance labels for the input image. The second sample image is input into the reference label generation network to obtain reference instance labels generated for the second sample image. The second image sample set is updated using the second sample image carrying the reference instance labels. If the recognition loss value reaches a threshold, the image recognition network containing the current network parameters is determined as the target image recognition network that has reached the convergence condition.

[0007] According to another aspect of the present invention, a training apparatus for an image recognition network is provided, comprising: an acquisition unit, configured to acquire a first image sample set and a second image sample set, wherein the first image sample set includes a plurality of first samples, each of the first samples including a first sample image carrying an instance label, the instance label indicating the instance category of an instance object in the first sample image and the image position of the instance object in the first sample image; the second image sample set includes a plurality of second samples, each of the second samples including a second sample image without an instance label; and a training unit, configured to input the first sample images acquired from the first image sample set and the second sample images acquired from the second image sample set into an image recognition network in a training state to obtain a recognition loss value. The image recognition network is used to identify the instance category of an instance object in a sample image and the image position of the instance object in the sample image; the update unit is used to adjust the network parameters in the image recognition network when the recognition loss value has not reached a threshold condition, and to determine a reference label generation network using the adjusted network parameters, wherein the reference label generation network is used to generate reference instance labels for the input image; the second sample image is input into the reference label generation network to obtain reference instance labels generated for the second sample image; the second image sample set is updated using the second sample image carrying the reference instance labels; the determination unit is used to determine the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition when the recognition loss value reaches a threshold condition.

[0008] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described training method for the image recognition network at runtime.

[0009] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the image recognition network described above.

[0010] According to another aspect of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the above-described image recognition network training method through the computer program.

[0011] In this embodiment of the invention, a first sample image and a second sample image are input into an image recognition network to obtain a recognition loss value, wherein the first sample image carries an instance label; if the recognition loss value does not reach a threshold condition, the network parameters in the image recognition network are adjusted, and a reference label generation network is determined using the adjusted network parameters; the second sample image is input into the reference label generation network to obtain a reference instance label generated for the second sample image; the second image sample set is updated using the second sample image carrying the reference instance label; if the recognition loss value reaches a threshold condition, the image recognition network containing the current network parameters is determined as the target image recognition network that has reached the convergence condition. This method trains the image recognition network using both labeled and unlabeled samples. If convergence is not achieved, a new network is generated using reference labels determined from the image recognition network. This network then generates samples for training the image recognition network in the next training round. This solves the problem in related technologies where only a small number of labeled samples can be used to train the network. By training the image recognition network with a small number of labeled and a large number of unlabeled image samples, the image recognition performance of the trained network is improved. This also solves the technical problem of poor training effect in related technologies that use limited labeled samples for network training. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0013] Figure 1 This is a schematic diagram of the hardware environment for an optional image recognition network training method according to an embodiment of the present invention;

[0014] Figure 2 This is a flowchart of an optional image recognition network training method according to an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of an optional image recognition network training method according to an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of another optional label determination model training method according to an embodiment of the present invention;

[0017] Figure 5 This is a schematic diagram of another optional label determination model training method according to an embodiment of the present invention;

[0018] Figure 6This is a schematic diagram of another optional label determination model training method according to an embodiment of the present invention;

[0019] Figure 7 This is a schematic diagram of another optional label determination model training method according to an embodiment of the present invention;

[0020] Figure 8 This is a flowchart of another optional label determination model training method according to an embodiment of the present invention;

[0021] Figure 9 This is a schematic diagram of the structure of an optional image recognition network training device according to an embodiment of the present invention;

[0022] Figure 10 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] The following explains the terminology used in this application:

[0026] Instance segmentation: Segmenting objects in an image that belong to a specific set of categories, providing different labels for individual instances of objects belonging to the same category;

[0027] Mask: a delimiter or segmentation label;

[0028] Single-stage: Instance segmentation focuses on pixels by treating instance segmentation as a bottom-up pixel allocation problem, and generally uses a global mask;

[0029] Two-stage: A top-down approach that first detects and then segments the local mask within the detection box is called two-stage instance segmentation, such as the Mask R-CNN model;

[0030] Semi-supervised learning: The training data has a small amount of manually labeled data (Dlabeled, abbreviated as Dl), and the rest are unlabeled samples (Dunlabeled, abbreviated as Du).

[0031] End-to-end: Training is completed in one go, without the need for human intervention in the intermediate training process;

[0032] Pseudo label: A label that is not manually labeled (and may contain noise) for unlabeled samples, obtained through methods such as models and rules.

[0033] SOLO: An advanced single-stage instance partitioning method;

[0034] Strong aug: Strong data augmentation, such as splicing, mosaic, etc.;

[0035] Weak data augmentation, such as color jitter or flipping;

[0036] According to one aspect of the present invention, a method for training an image recognition network is provided. As an optional implementation, the above-described image recognition network training method can be applied, but is not limited to, to applications such as... Figure 1 The training system shown is an image recognition network consisting of server 102 and terminal device 104. Figure 1As shown, server 102 is connected to terminal device 104 via network 110. This network may include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). The wireless network includes Bluetooth, Wi-Fi, and other networks enabling wireless communication. The terminal device may include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, PDAs, MIDs (Mobile Internet Devices), PADs, desktop computers, smart TVs, in-vehicle devices, etc. The terminal device may have a client installed, such as a search client with image recognition functionality or an instant messaging client with image recognition functionality. The terminal device also includes a display, a processor, and a memory. The display can show the program interface of the search client or instant messaging client with image recognition functionality, as well as the image to be recognized. The processor can process the image using a trained recognition network. The memory stores the image to be recognized. Understandably, the image recognition network can be trained first on server 102. Once training is complete, server 102 can send the trained target image recognition network to terminal device 104 via network 110. After receiving the trained target image recognition network, terminal device 104 can perform image recognition on the user-provided image to determine the type of instances included in the image and their locations within the image. Server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud server. The server includes a database and a processing engine. The database may include a set of sample images for training; the processing engine is used to execute the training process of the image recognition network.

[0037] According to one aspect of the present invention, the training system of the image recognition network described above may further perform the following steps: Server 102 performs steps S102 to S108 to obtain a first image sample set and a second image sample set, wherein the first image sample set includes a plurality of first samples, each first sample including a first sample image carrying an instance label, the instance label being used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image; the second image sample set includes a plurality of second samples, each second sample including a second sample image without an instance label; the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set are input into the image recognition network in the training state to obtain a recognition loss value, wherein the image recognition network is used to identify the instance category and the instance position of the instance object in the sample image. Example: The image location of the object in the sample image; if the recognition loss value does not reach the threshold condition, the network parameters in the image recognition network are adjusted, and the reference label generation network is determined using the adjusted network parameters. The reference label generation network is used to generate reference instance labels for the input image; the second sample image is input into the reference label generation network to obtain the reference instance labels generated for the second sample image; the second image sample set is updated using the second sample image carrying the reference instance labels; if the recognition loss value reaches the threshold condition, the image recognition network containing the current network parameters is determined as the target image recognition network that has reached the convergence condition; then, the server 102 executes step S110 and sends the target image recognition network to the terminal device 104 through network 110; finally, the terminal device 104 executes step S112 and performs image recognition using the target image recognition network.

[0038] In this embodiment of the invention, a first sample image and a second sample image are input into an image recognition network to obtain a recognition loss value, wherein the first sample image carries an instance label; if the recognition loss value does not reach a threshold condition, the network parameters in the image recognition network are adjusted, and a reference label generation network is determined using the adjusted network parameters; the second sample image is input into the reference label generation network to obtain a reference instance label generated for the second sample image; the second image sample set is updated using the second sample image carrying the reference instance label; if the recognition loss value reaches a threshold condition, the image recognition network containing the current network parameters is determined as the target image recognition network that has reached the convergence condition. This method trains the image recognition network using both labeled and unlabeled samples. If convergence is not achieved, a new network is generated using reference labels determined from the image recognition network. This network then generates samples for training the image recognition network in the next training round. This solves the problem in related technologies where only a small number of labeled samples can be used to train the network. By training the image recognition network with a small number of labeled and a large number of unlabeled image samples, the image recognition performance of the trained network is improved. This also solves the technical problem of poor training effect in related technologies that use limited labeled samples for network training.

[0039] The above is merely an example, and no limitation is made in this embodiment.

[0040] As an optional implementation method, such as Figure 2 As shown, the training method for the above image recognition network includes the following steps:

[0041] S202, Obtain the first image sample set and the second image sample set;

[0042] The first image sample set includes multiple first samples, each of which includes a first sample image carrying an instance label. The instance label is used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image. The second image sample set includes multiple second samples, each of which includes a second sample image without an instance label.

[0043] S204, the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set are input into the image recognition network in the training state to obtain the recognition loss value;

[0044] Among them, the image recognition network is used to identify the instance category of the instance object in the sample image and the image position of the instance object in the sample image;

[0045] S206, if the recognition loss value does not reach the threshold condition, the network parameters in the image recognition network are adjusted, and the reference label generation network is determined using the adjusted network parameters; wherein, the reference label generation network is used to generate reference instance labels for the input image; the second sample image is input into the reference label generation network to obtain the reference instance labels generated for the second sample image; the second image sample set is updated using the second sample image carrying the reference instance labels.

[0046] S208, if the recognition loss value reaches the threshold condition, determine the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition.

[0047] In step S202 above, the sample images included in the first image sample set and the second image sample set may include at least two regions. The first region is the image region occupied by the instance object to be identified in this embodiment, and the second region is other image regions besides the instance object. Figure 3 For example, the object occupying the central area of ​​the image that represents the image's subject is the instance object, such as... Figure 3 The instance objects in the image are the two zebras located in the center; while in addition to Figure 3 The area outside the zebra object shown is the area occupied by the image background, such as... Figure 3 The background area in the image is mainly occupied by grass and trees.

[0048] It is understood that, in this embodiment, the instance label can be used to indicate the instance category of the instance object and the position of the instance object in the sample image. For example, according to Figure 3 The instance object in can be determined Figure 3 The instance object in the code corresponds to the category "zebra". Assuming the category code for "zebra" is "100", then "100" is used as... Figure 3 The instance category in the instance tag; assuming Figure 3 In the case where the pixel set occupied by the instance object "Zebra" includes "(100pt, 110pt), (100pt, 111pt), (100pt, 112pt), (100pt, 113pt)...", the above pixel set can be used as... Figure 3 The image location in the instance label. The labeling method above is only an example and does not limit the specific labeling method in this embodiment.

[0049] It should be noted that the first sample in the first image sample mentioned above can be a first sample image carrying an instance label, and the instance label can be a real annotation result. The second sample in the second image sample can be a second sample image without an instance label. Furthermore, the second image sample mentioned above can be a sample image without any annotation result, or it can be a sample image carrying a "pseudo-label". The pseudo-label can be a recognition result output by a label generation model based on the second sample image, and the confidence level of the recognition result is less than or equal to 1.

[0050] Furthermore, in step S204 above, a first sample image is obtained from the first sample image and a second sample image is obtained from the second image sample set. Then, the first sample image carrying instance labels and the second sample image without instance labels are input into the image recognition network in the training state to obtain the training loss.

[0051] It is understood that after obtaining the first sample image from the first sample image set and the second sample image from the second image sample set, certain preprocessing operations can be performed on the first and second sample images before inputting the preprocessed images into the image recognition network. These preprocessing operations may include, but are not limited to, image enhancement processing of the first and second sample images, and image feature fusion processing of the first and second sample images. This embodiment does not limit the specific operation methods of the above preprocessing operations.

[0052] Optionally, in step S206 above, if the recognition loss value does not reach the threshold condition, the parameters of the image recognition network can be adjusted first, and then a reference label generation network for generating reference instance labels can be determined based on the adjusted network parameters. It is understood that, in this embodiment, after obtaining the reference label generation network, the second sample image in the second image sample set that does not carry instance labels can be recognized based on the reference label generation network to obtain a reference instance label corresponding to the second sample image.

[0053] Furthermore, having obtained the reference instance label corresponding to the second sample image, we can return to step S204 above and further train the image recognition network based on the second sample image carrying the reference instance label and the first sample image carrying the instance label.

[0054] The image recognition model described above can be a single-stage image recognition model, which can directly output image recognition results based on image features, such as the SOLO model and the condinst model.

[0055] The above-described embodiments of this application involve inputting a first sample image and a second sample image into an image recognition network to obtain a recognition loss value, wherein the first sample image carries an instance label; if the recognition loss value does not reach a threshold condition, adjusting the network parameters in the image recognition network and using the adjusted network parameters to determine a reference label generation network; inputting the second sample image into the reference label generation network to obtain a reference instance label generated for the second sample image; updating the second image sample set using the second sample image carrying the reference instance label; and if the recognition loss value reaches a threshold condition, determining the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition. This method trains the image recognition network using both labeled and unlabeled samples. If convergence is not achieved, a new network is generated using reference labels determined from the image recognition network. This network then generates samples for training the image recognition network in the next training round. This solves the problem in related technologies where only a small number of labeled samples can be used to train the network. By training the image recognition network with a small number of labeled and a large number of unlabeled image samples, the image recognition performance of the trained network is improved. This also solves the technical problem of poor training effect in related technologies that use limited labeled samples for network training.

[0056] The image recognition model in steps S202 to S208 above will be further explained below. As an optional approach, after determining that the image recognition network containing the current network parameters has reached convergence when the recognition loss value reaches a threshold condition, the method further includes:

[0057] S1, acquire the target image to be identified;

[0058] S2, extract image features from the target image based on the feature extraction layer of the target image recognition network;

[0059] S3, input the image features into the category recognition layer of the target image recognition network to determine the target instance category of the instance objects included in the target image;

[0060] S4, input the image features into the location recognition layer of the target image recognition network to determine the target image location of the instance object included in the target image;

[0061] S5 determines the target instance category and target image location as the recognition result of the target image.

[0062] In one specific way, the aforementioned target image recognition network can be a SOLO network used to perform instance segmentation tasks on an image to obtain the types of instance objects included in the image and the image positions occupied by the instance objects.

[0063] The following combination Figure 4 , Figure 5 The image recognition network described above will be further explained.

[0064] It should be noted that in related technologies, a two-stage instance segmentation algorithm is typically used to solve the instance segmentation problem, for example... Figure 4 Figure (a) illustrates a two-stage instance segmentation algorithm, Mask R-CNN. Its logic for instance segmentation involves first determining the bounding box containing the instance object, and then detecting the type and specific location of the instance object within that bounding box based on the determined bounding box range. In this multi-stage detection method, if the bounding box shifts, local prediction cannot accurately predict the mask containing the instance object.

[0065] In this embodiment, the instance segmentation algorithm used is single-order SOLO, which is boxffee, meaning it performs local segmentation of instances without relying on predicted bounding boxes, and the result of local segmentation does not depend on the prediction accuracy of the bounding boxes. For example... Figure 4 As shown in Figure (b), the prediction of the category of the instance object and the prediction of the mask do not depend on the prediction of the detection box; the prediction of the mask is global.

[0066] Furthermore, such as Figure 5 As shown, the SOLO algorithm determines a specific instance based on its position and size. Therefore, SOLO is based on a Fully Convolutional Network (FCN) and has two branches: a class prediction branch and a mask prediction branch. The feature F at the instance's location is convolved to obtain the class prediction, and this feature is also convolved with the mask feature to obtain the mask prediction of the instance at that location. Correspondingly, in this embodiment, the supervised loss L of SOLO... s It is divided into two parts, namely L c and L m , where L s =L c +L m Loss L c Used for supervising classification branching, L m Used to supervise the segmentation of mask branches. Further, the aforementioned loss L... c Using focal loss, L m Dice loss is used.

[0067] The above-described embodiments of this application obtain a target image to be identified; extract image features from the target image using the feature extraction layer of the target image recognition network; input the image features into the category recognition layer of the target image recognition network to determine the target instance category of the instance object included in the target image; input the image features into the position recognition layer of the target image recognition network to determine the target image position of the instance object included in the target image; and determine the target instance category and target image position as the recognition result of the target image. Thus, a semi-supervised training method is adopted on the single-stage example segmentation model, which can significantly improve the model training speed and the accuracy of the example segmentation result.

[0068] As an optional implementation, the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set are input into the image recognition network in the training state to obtain the recognition loss value, including:

[0069] S1, generate a third sample image based on the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set, wherein the third sample image includes the first instance object in the first sample image and the second instance object in the second sample image;

[0070] S2, extract image features of the third sample image based on the image recognition network in the training state, and identify the predicted instance category of the instance object included in the third sample image and the predicted image position of the instance object in the third sample image based on the image features.

[0071] S3. Determine the recognition loss value based on the prediction results of the image recognition network in training state and the instance label corresponding to the third sample image.

[0072] In this embodiment, after obtaining the first sample image and the second sample image, a third sample image can be generated based on the first sample image and the second sample image. The generated third sample image is then input into the image recognition network to obtain a prediction result for the third sample image. In this embodiment, the third sample image includes the instance objects from the first sample image.

[0073] Next, the recognition loss value is determined based on the prediction result of the third sample image and the instance label corresponding to the third sample image. It can be understood that since the third sample image includes the instance objects in the first sample image, the instance label corresponding to the third sample image can also be determined based on the instance label corresponding to the first sample image.

[0074] Through the above-described embodiments of this application, a third sample image is generated based on a first sample image obtained from a first image sample set and a second sample image obtained from a second image sample set; image features of the third sample image are extracted based on an image recognition network in training state, and the predicted instance category of the instance object included in the third sample image and the predicted image position of the instance object in the third sample image are identified based on the image features; the recognition loss value is determined based on the prediction result of the image recognition network in training state and the instance label corresponding to the third sample image, and thus, even when only a small number of labeled samples can be obtained, the image recognition network is trained by combining labeled samples and unlabeled samples, thereby obtaining a more robust model and solving the technical problem of overfitting in existing training methods due to too few actual labeled samples.

[0075] As an optional implementation, the above-described generation of the third sample image based on the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set includes:

[0076] S1, Obtain the instance image corresponding to the first instance object in the first sample image;

[0077] S2, Add an instance image to the target location in the second sample image;

[0078] S3, the second sample image carrying the instance image is determined as the third sample image;

[0079] S4-1, if the second sample image has a corresponding reference instance label, determine the instance label corresponding to the third sample image based on the reference instance label corresponding to the second sample image and the instance label corresponding to the first sample image.

[0080] S4-2, if the instance object in the second sample image does not have a corresponding reference instance label, determine the instance label corresponding to the third sample image based on the instance label corresponding to the first sample image.

[0081] The following combination Figure 6 The above methods will be explained. For example... Figure 6 As shown, the first sample image displays a first instance object, and the first instance object has a corresponding manually labeled instance tag; simultaneously, the second sample image displays a second instance object, which does not have a corresponding manually labeled instance tag. Further, the instance image corresponding to the first instance object in the first sample image is added to the second sample image, thereby obtaining... Figure 6 The third sample image in the image includes both the first instance object and the second instance object, wherein the first instance object has a corresponding label.

[0082] Furthermore, if the second instance image in the second sample image has a corresponding reference instance label, the corresponding reference instance label of the second instance image can also be used as the instance label of the third sample image. The reference instance label can be an instance label predicted by the model.

[0083] Optionally, before adding the instance image at the target location of the second sample image, the above also includes one of the following:

[0084] S1, determine the target location in the region of the second sample image that does not include the instance object;

[0085] S2, obtain the reference instance position of the instance object in the first sample image; determine the image region in the second sample image corresponding to the reference instance position as the target position.

[0086] It is understandable that in the first method of determining the target location, the target location can be determined in the second sample image excluding the location of the instance object, thereby avoiding the interference of instance object stacking on the instance segmentation result. In the above method, the corresponding instance label can be updated according to the actual instance location. In the second method of determining the target location, the target location can be determined directly in the corresponding location of the second sample image according to the specific location of the instance object in the first sample image. Therefore, in the above method, the instance label corresponding to the first sample image can be transferred to the third sample image.

[0087] Through the above-described embodiments of this application, an instance image corresponding to a first instance object in a first sample image is obtained; an instance image is added to a target position in a second sample image; the second sample image carrying the instance image is determined as a third sample image; if the second sample image has a corresponding reference instance label, the instance label corresponding to the third sample image is determined based on the reference instance label corresponding to the second sample image and the instance label corresponding to the first sample image; if the instance object in the second sample image does not have a corresponding reference instance label, the instance label corresponding to the third sample image is determined based on the instance label corresponding to the first sample image. Thus, a labeled third sample image is generated based on the first sample image carrying the label and the second sample image without the label, thereby expanding the number of samples that can be used for training. Furthermore, when only a small number of labeled samples can be obtained, the image recognition network is trained by combining labeled samples and unlabeled samples, thereby obtaining a more robust model and solving the technical problem of overfitting in existing training methods due to insufficient actual labeled samples.

[0088] As an optional implementation, before determining the second sample image carrying the instance image as the third sample image, the method further includes:

[0089] S1, perform image enhancement processing on the second sample image;

[0090] S2, using the reference label generation network corresponding to the image recognition network before parameter adjustment, the second sample image after image enhancement is identified to obtain the classification confidence and region confidence of each of the multiple candidate instance objects;

[0091] S3, obtain the product of the classification confidence and region confidence of each of the multiple candidate instance objects, and determine the product of each of the multiple candidate instance objects as the label confidence of each of the multiple candidate instance objects;

[0092] S4, determine the candidate instances whose label confidence is greater than or equal to the confidence threshold as the second instance objects included in the second sample image, and determine the reference instance label corresponding to the second sample image based on the classification confidence and region confidence corresponding to the second instance object.

[0093] The following describes the method for determining the reference example label corresponding to the second sample image in this embodiment.

[0094] In this embodiment, the second sample image may first undergo image enhancement processing. This image enhancement may include, but is not limited to, a weak data enhancement process, such as color dithering or image flipping. This embodiment does not limit the specific method of image enhancement.

[0095] After image enhancement processing is performed on the second sample image without instance labels, it can be recognized using a reference label generation network corresponding to the image recognition network before parameter adjustment. It should be noted that in this embodiment, multiple training processes can be included. In each training process, a corresponding reference label generation model can be configured for the currently trained image recognition model, and a corresponding reference instance label is configured for the second sample image based on the recognition result of this reference label generation model.

[0096] The following describes a specific process for generating a third sample image.

[0097] First, the second sample image in the second image sample set is obtained, and then a weak enhancement process is performed on the second sample image, specifically by performing a color gamma transformation. The specific transformation formula is as follows:

[0098] X w =X gamma

[0099] Where X is the pixel value of each pixel in the second sample image, X w denoted as the pixel value of each pixel in the second sample image after color gamma transformation. When gamma < 1, the image becomes darker, and when gamma > 1, the image becomes brighter. In this embodiment, gamma can take any value in the range [0.8, 1.2].

[0100] Next, the second sample image after color gamma transformation is input into the reference label generation network corresponding to the currently trained image recognition network, thereby obtaining the reference instance label corresponding to the second sample image.

[0101] It is understandable that the reference label generation network can have the same network structure as the image recognition network mentioned above, and thus output a classification score S based on the input second sample image. c The average score S of the segmented mask mask .

[0102] Next, the reference instance label corresponding to the second sample image is determined using the following formula:

[0103] S = S c *S mask

[0104] In the above formula, S indicates the score of a candidate instance object. When S is greater than 0.8, the candidate instance object corresponding to the above score S is determined as the second instance object included in the second sample image.

[0105] The following are the classification scores S above. c The average score S of the above segmented mask mask Further explanation is provided. In this embodiment, after the reference label generation network identifies the second sample image, it can predict the instance category of the instance objects included in the second sample image and the score of the segmentation mask in different image regions.

[0106] Specifically, assuming the classification scores for the recognition results in the second sample image are as follows: the score for the instance category predicted as "zebra" is 0.9, and the score for the instance category predicted as "sheep" is 0.1;

[0107] Each pixel in the first region described above has a corresponding segmentation mask score, which indicates the probability that the current pixel is the pixel corresponding to the instance object. For example, for pixel (100pt, 110pt), its mask score is 0.9, indicating that the probability that the current pixel is a pixel in the image where the instance object is located is 90%. Further, assuming that a first region exists in the second sample image, and the average segmentation mask score of each pixel in this region is 0.9, then according to the above formula:

[0108] S = S c *S mask

[0109] It can be determined that the score of the "zebra" instance object in the first region is 0.9*0.9, or 0.81; the score of the "sheep" instance object in the first region is 0.1*0.9, or 0.09. Since the score of the "zebra" instance object is greater than the threshold of 0.8, the instance category "zebra" and the image range where the first region is located are used as the reference instance label of the second sample image.

[0110] Using the above method, a reference instance label can be generated for a second sample image that does not carry instance labels, using a reference label generation network corresponding to the current image recognition network.

[0111] Next, since the reference label generation network may not obtain correct instance results for unlabeled samples during the initial training phase due to insufficient training, this embodiment further employs a strong enhancement method based on instance fusion. The first sample image carrying instance labels is fused with the second sample image carrying reference instance labels to obtain the third sample image. Specifically, half of the instance objects are randomly extracted from the first sample image and fused to the corresponding positions in the second sample image. Since the first sample image contains labeled samples, the fused instances have true labels. For instances in the second sample image, weak enhancement processing can be applied as described above, and the result obtained by the reference label generation network is used as a pseudo-label. Therefore, the third sample image can be input into the image recognition network for supervised training.

[0112] Through the above-described embodiments of this application, an instance image corresponding to a first instance object in a first sample image is obtained; an instance image is added to a target position in a second sample image; the second sample image carrying the instance image is determined as a third sample image; if the second sample image has a corresponding reference instance label, the instance label corresponding to the third sample image is determined based on the reference instance label corresponding to the second sample image and the instance label corresponding to the first sample image; if the instance object in the second sample image does not have a corresponding reference instance label, the instance label corresponding to the third sample image is determined based on the instance label corresponding to the first sample image. Thus, a labeled third sample image is generated based on the first sample image carrying the label and the second sample image without the label, thereby expanding the number of samples that can be used for training. Furthermore, when only a small number of labeled samples can be obtained, the image recognition network is trained by combining labeled samples and unlabeled samples, thereby obtaining a more robust model and solving the technical problem of overfitting in existing training methods due to insufficient actual labeled samples.

[0113] As an optional implementation, the above-mentioned determination of the recognition loss value based on the prediction result of the image recognition network in training state and the instance label corresponding to the third sample image includes:

[0114] S1, obtain the first recognition result of the first instance object and the second recognition result of the second instance object included in the third sample image by the image recognition network in the training state;

[0115] S2, determine a first loss value based on the first identification result and the instance label, and determine a second loss value based on the second identification result and the reference instance label;

[0116] S3, determine the identification loss value based on the product of the loss confidence of the second loss value and the first loss value, wherein the loss confidence is determined based on the label confidence of the reference instance label.

[0117] It should be noted that some of the instances in the third sample image after the above fusion process are obtained by generating a reference label network after weak enhancement, and are used as reference instance labels, i.e., pseudo-labels. This process may produce false positive samples. Therefore, when training the image recognition network using the third sample image after the above fusion process, the loss weight of false instances and their corresponding pseudo-labels is reduced to minimize the negative impact of erroneous instances on training.

[0118] Therefore, the instances in the fused third sample image have a loss weight of 1 for the instances from the first sample image, while the pseudo-labels in the samples from the second sample image are weighted using the score (i.e., the score S mentioned above) predicted by the reference label generation network for the weakly enhanced samples. Thus, the pseudo-label loss in the sample loss for training the image recognition network using the aforementioned third sample image can be obtained using the following formula:

[0119] L u =s(L c +L m )

[0120] Where s = S c *S mask S c To score by category, The average score of the segmented mask.

[0121] Therefore, the total loss of the above training process can be obtained as follows:

[0122] L = L u +L l

[0123] Among them, L l This is the loss value corresponding to the instance of the first sample image.

[0124] Through the above-described embodiments of this application, a first recognition result of a first instance object and a second recognition result of a second instance object included in a third sample image are obtained by an image recognition network in training state; a first loss value is determined based on the first recognition result and instance label, and a second loss value is determined based on the second recognition result and reference instance label; a recognition loss value is determined based on the product of the loss confidence of the second loss value and the first loss value, wherein the loss confidence is determined based on the label confidence of the reference instance label. Thus, even when only a small number of labeled samples are available, the image recognition network is trained by combining labeled and unlabeled samples, thereby obtaining a more robust model and solving the overfitting problem in existing training methods due to insufficient actual labeled samples. Furthermore, by assigning a corresponding confidence value to the loss value corresponding to the reference instance label, the impact of noise that may be carried in the reference instance label on the training results is reduced.

[0125] As an optional implementation, the above-described method of determining the reference label generation network using adjusted network parameters includes:

[0126] S1, Obtain the generation parameters of the reference label generation network corresponding to the image recognition network before parameter adjustment;

[0127] S2, Obtain the network parameters of the image recognition network after parameter adjustment;

[0128] S3, determine the reference labels and generate the network based on the generation parameters and network parameters.

[0129] It is understood that, in this embodiment, the current reference label generation network can be determined based on the generation parameters of the reference label generation network corresponding to the image recognition network before parameter adjustment and the network parameters of the image recognition network.

[0130] In this embodiment, the parameter-adjusted image recognition network can be used to determine the aforementioned reference label generation network. Furthermore, the determined reference label generation network can be used to further determine the image samples used to train the image recognition network in the next round of training.

[0131] The following further explains the above correspondence. For example, in the current training process, it is determined that image recognition network A has not reached the convergence condition. Then, based on image recognition network A, image recognition network B is obtained by adjusting the parameters. Then, based on image recognition network B, a corresponding reference label generation network B is determined. Based on the reference label generation network B, a sample set for training image recognition network B is determined, and image recognition network B is trained using the determined sample set.

[0132] If image recognition network B fails to converge, image recognition network C is obtained by adjusting the parameters of image recognition network B. Then, a corresponding reference label generation network C is determined based on image recognition network C. Based on the reference label generation network C, a sample set for training image recognition network C is determined, and image recognition network C is trained using the determined sample set...

[0133] According to the above embodiments, the image recognition network B and the reference label generation network B are corresponding networks, and the image recognition network C and the reference label generation network C are corresponding networks.

[0134] As an optional implementation, the above-described method of determining the reference label generation network based on the generation parameters and network parameters includes:

[0135] S1, obtain the first update coefficient;

[0136] S2, determine the first product result of the generated parameters and the first update coefficient, and the second product result of the network parameters and the second update coefficient, wherein the sum of the first update coefficient and the second update coefficient is 1;

[0137] S3 will determine the reference label generation network based on the first product result and the second product result.

[0138] The following explains the confirmation relationship between the image recognition network and the reference label generation network.

[0139] Assuming the first update coefficient is α, the corresponding second update coefficient is 1-α;

[0140] The confirmation relationship between the aforementioned image recognition network and the reference label generation network can be confirmed in the following way:

[0141] T(t+1)=αT(t)+(1-α)*S(t+1)

[0142] In the above formula, T represents the generation parameters of the reference label generation network, S represents the network parameters of the image recognition network, T(t+1) represents the generation parameters of the reference label generation network corresponding to the image recognition network after parameter adjustment, S(t+1) represents the network parameters of the image recognition network after parameter adjustment, and T(t) represents the generation parameters of the reference label generation network corresponding to the image recognition network before parameter adjustment.

[0143] Through the above-described embodiments of this application, a first update coefficient is obtained; a first product result of the generation parameter and the first update coefficient, and a second product result of the network parameter and the second update coefficient are determined, wherein the sum of the first update coefficient and the second update coefficient is 1; a reference label generation network is determined based on the first product result and the second product result, thereby using the reference label generation network that is continuously iteratively optimized during the training process to generate pseudo-labels for training the image recognition network, thereby achieving rapid training of the image recognition network and improving training efficiency.

[0144] The following combination Figure 7 , Figure 8 A complete implementation process of this application is described below:

[0145] like Figure 7 As shown, the multiple data flow processes in this embodiment are explained. Student(S) and Teacher(T) are single-stage instance segmentation algorithms with the same network structure (this embodiment uses the SOLO model). Figure 7 As shown in the figure, there are three data streams in this embodiment:

[0146] The first data stream is D1(x, y), which includes images with labels, where x represents the image and y represents the corresponding label. The first data stream D1 is input into S, and the S model is trained using supervised loss, with the training loss being Ls.

[0147] Further, as shown in the figure, the second data stream is unlabeled data Du(x). After weak enhancement (color gamma transformation is used in this embodiment), x is input into T, and the result obtained by T is used as pseudo-label y.

[0148] In addition, strong enhancement processing can be applied to the unlabeled data Du(x), and pseudo-labels can be obtained using (2) for supervised learning, with a loss of Lu.

[0149] Here, T does not participate in gradient backpropagation training; it is only used to train model S. The model parameters in T are updated using EMA (moving exponential moving average) through S, as shown in the following formula:

[0150] T(t+1)=αT(t)+(1-α)*S(t+1)

[0151] Where t is the current iteration step. As training progresses, the Teacher updates the model parameters, making the segmentation pseudo-labels obtained by the Teacher better and better, and thus the training results of S based on the segmentation pseudo-labels obtained by the Teacher also become better and better.

[0152] The following combination Figure 8 A complete process of this application is described as follows:

[0153] S802 uses a reference label generation network to predict pseudo-labels for second image samples that do not carry labeled labels;

[0154] Specifically, in this embodiment, the second image sample without labels is first subjected to weak enhancement processing. This is specifically achieved by performing a color gamma transformation on the second sample image. The specific transformation formula is as follows:

[0155] X w =X gamma

[0156] In this embodiment, gamma can take any value in the range [0.8, 1.2].

[0157] Next, the second sample image after color gamma transformation is input into the reference label generation network corresponding to the currently trained image recognition network, thereby obtaining the reference instance label corresponding to the second sample image.

[0158] Next, the reference instance label corresponding to the second sample image is determined using the following formula:

[0159] S = S c *S mask

[0160] In the above formula, S indicates the score of a candidate instance object. When S is greater than 0.8, the candidate instance object corresponding to the above score S is determined as the second instance object included in the second sample image.

[0161] S804, determine the third image sample using the second image sample and the first image sample;

[0162] S806 uses third image samples to train the image recognition network;

[0163] Since the network may not obtain correct instance results for unlabeled samples during the initial training phase, a strong augmentation method based on instance fusion is adopted. This method fuses images X1 (the first sample image) and Xu (the second sample image) from the same training set to obtain X (the third sample image). Half of the instances in X1 are randomly extracted and fused to the corresponding positions in Xu. Because X1 is a labeled sample, the fused instances have true labels. For instances in Xu, the results obtained through weak augmentation via network T can be used as pseudo-labels. Therefore, X can be input into T for supervised training.

[0164] Next, since the instances in the strongly enhanced X are used as pseudo-labels based on the results obtained from the weak enhancement through the T network, this process may result in false positives. Therefore, the weight of false instances is reduced to mitigate the negative impact of erroneous instances on training. Thus, in the fused third sample image, the instances from the first sample image have a loss weight of 1, while the pseudo-labels from the samples in the second sample image use the Score (i.e., the score S mentioned above) predicted by the reference label generation network for the weakly enhanced samples as weight. Therefore, the pseudo-label loss in the sample loss for training the image recognition network using the aforementioned third sample image can be obtained using the following formula:

[0165] L u =s(L c +L m )

[0166] Where s = S c *S mask S c To score by category, The average score of the segmented mask.

[0167] Therefore, the total loss of the above training process can be obtained as follows:

[0168] L = L u +L l

[0169] Among them, Ll This is the loss value corresponding to the instance of the first sample image.

[0170] S808, Determine if the current image recognition network has reached the convergence condition? If the convergence condition is met, proceed to step S814, identify the current image recognition network as the target image recognition network, and terminate; if the convergence condition is not met, proceed to step S810, update the network parameters of the current image recognition network; specifically, gradient backpropagation can be used to update the parameters.

[0171] S812, Update the generation parameters of the reference label generation network;

[0172] Assuming the first update coefficient is α, the corresponding second update coefficient is 1-α;

[0173] The confirmation relationship between the aforementioned image recognition network and the reference label generation network can be confirmed in the following way:

[0174] T(t+1)=αT(t)+S(t+1)

[0175] In the above formula, T represents the generation parameters of the reference label generation network, S represents the network parameters of the image recognition network, T(t+1) represents the generation parameters of the reference label generation network corresponding to the image recognition network after parameter adjustment, S(t+1) represents the network parameters of the image recognition network after parameter adjustment, and T(t) represents the generation parameters of the reference label generation network corresponding to the image recognition network before parameter adjustment.

[0176] Understandably, after performing step S812, the reference label generation network can be returned to step S802 based on the updated generation parameters to generate new pseudo-labels based on the second image samples, so as to enter the next round of training.

[0177] Through the above-described embodiments of this application, when only a small number of labeled samples are available, end-to-end training with unlabeled samples can yield a more robust model that is less prone to overfitting. Furthermore, the mean-teacher structure used during training dynamically updates pseudo-labels, increasing their accuracy as training progresses and further impacting the Student model's recognition performance. Additionally, pseudo-labels are generated in real-time during each training round and used directly in the current training phase, while newly generated pseudo-labels are used in the next round, eliminating the need for storing pseudo-labels. Moreover, the image recognition network used in this embodiment differs from the semi-supervised method of two-stage networks; it does not focus on detection boxes, predicts masks more globally, and is more suitable for semi-supervised instance segmentation, generally being faster than two-stage recognition networks.

[0178] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0179] According to another aspect of the present invention, a training apparatus for an image recognition network for implementing the above-described image recognition network training method is also provided. For example... Figure 9 As shown, the device includes:

[0180] The acquisition unit 902 is used to acquire a first image sample set and a second image sample set. The first image sample set includes multiple first samples, each of which includes a first sample image carrying an instance label. The instance label is used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image. The second image sample set includes multiple second samples, each of which includes a second sample image without an instance label.

[0181] The training unit 904 is used to input the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set into the image recognition network in the training state to obtain the recognition loss value. The image recognition network is used to identify the instance category of the instance object in the sample image and the image position of the instance object in the sample image.

[0182] The update unit 906 is used to adjust the network parameters in the image recognition network when the recognition loss value has not reached the threshold condition, and to determine the reference label generation network using the adjusted network parameters. The reference label generation network is used to generate reference instance labels for the input image. The second sample image is input into the reference label generation network to obtain the reference instance labels generated for the second sample image. The second image sample set is updated using the second sample image carrying the reference instance labels.

[0183] The determining unit 908 is used to determine the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition when the recognition loss value reaches the threshold condition.

[0184] Optionally, in this embodiment, the implementation of each of the above-mentioned unit modules can be referred to the above-mentioned method embodiments, which will not be repeated here.

[0185] According to another aspect of the present invention, an electronic device for implementing the above-described image recognition network training method is also provided. This electronic device may be... Figure 10 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps of any of the above method embodiments via the computer program.

[0186] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0187] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0188] S1, obtain a first image sample set and a second image sample set, wherein the first image sample set includes multiple first samples, each first sample includes a first sample image carrying an instance label, the instance label is used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image, and the second image sample set includes multiple second samples, each second sample includes a second sample image without an instance label;

[0189] S2, the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set are input into the image recognition network in the training state to obtain the recognition loss value. The image recognition network is used to identify the instance category of the instance object in the sample image and the image position of the instance object in the sample image.

[0190] S3, if the recognition loss value does not reach the threshold condition, adjust the network parameters in the image recognition network, and use the adjusted network parameters to determine the reference label generation network, wherein the reference label generation network is used to generate reference instance labels for the input image; input the second sample image into the reference label generation network to obtain the reference instance labels generated for the second sample image; update the second image sample set using the second sample image carrying the reference instance labels;

[0191] S4. If the recognition loss value reaches the threshold condition, determine the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition.

[0192] Alternatively, as those skilled in the art will understand, Figure 10The structure shown is for illustrative purposes only. Electronic devices can also be in-vehicle terminals, smartphones (such as Android phones, iOS phones, etc.), tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 10 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 10 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 10 The different configurations shown.

[0193] The memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the image recognition network training method and apparatus in this embodiment of the invention. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, thereby realizing the above-mentioned image recognition network training method. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include memory remotely located relative to the processor 1004, and these remote memories can be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1002 may be used, but is not limited to, to store information such as various elements in the viewing angle image and training information of the image recognition network. As an example, such as Figure 10 As shown, the memory 1002 may include, but is not limited to, the acquisition unit 902, training unit 904, update unit 906, and determination unit 908 from the training device for the image recognition network. Furthermore, it may include, but is not limited to, other module units from the training device for the image recognition network, which will not be elaborated upon in this example.

[0194] Optionally, the transmission device 1006 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1006 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0195] In addition, the aforementioned electronic device also includes a display 1008 and a connection bus 1010 for connecting various module components in the aforementioned electronic device.

[0196] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0197] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0198] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0199] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the training method of the image recognition network described above.

[0200] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0201] S1, obtain a first image sample set and a second image sample set, wherein the first image sample set includes multiple first samples, each first sample includes a first sample image carrying an instance label, the instance label is used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image, and the second image sample set includes multiple second samples, each second sample includes a second sample image without an instance label;

[0202] S2, the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set are input into the image recognition network in the training state to obtain the recognition loss value. The image recognition network is used to identify the instance category of the instance object in the sample image and the image position of the instance object in the sample image.

[0203] S3, if the recognition loss value does not reach the threshold condition, adjust the network parameters in the image recognition network, and use the adjusted network parameters to determine the reference label generation network, wherein the reference label generation network is used to generate reference instance labels for the input image; input the second sample image into the reference label generation network to obtain the reference instance labels generated for the second sample image; update the second image sample set using the second sample image carrying the reference instance labels;

[0204] S4. If the recognition loss value reaches the threshold condition, determine the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition.

[0205] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0206] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0207] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0208] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0209] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0210] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0211] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A training method for an image recognition network, characterized in that, include: A first image sample set and a second image sample set are obtained. The first image sample set includes a plurality of first samples, each of which includes a first sample image carrying an instance label. The instance label is used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image. The second image sample set includes a plurality of second samples, each of which includes a second sample image without an instance label. A third sample image is generated based on the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set, wherein the third sample image includes a first instance object in the first sample image and a second instance object in the second sample image; image features of the third sample image are extracted based on an image recognition network in training state, and the predicted instance category of the instance object included in the third sample image and the predicted image position of the instance object in the third sample image are identified based on the image features; a recognition loss value is determined based on the prediction result of the image recognition network in training state and the instance label corresponding to the third sample image, wherein the image recognition network is used to identify the instance category of the instance object in the sample image and the image position of the instance object in the sample image; If the recognition loss value does not reach the threshold condition, the network parameters in the image recognition network are adjusted, and the reference label generation network is determined using the adjusted network parameters. The reference label generation network is used to generate reference instance labels for the input image. The second sample image is input into the reference label generation network to obtain the reference instance labels generated for the second sample image. The second image sample set is updated using the second sample image carrying the reference instance labels. If the recognition loss value reaches the threshold condition, the image recognition network containing the current network parameters is determined as the target image recognition network that has reached the convergence condition.

2. The method according to claim 1, characterized in that, Generating a third sample image based on the first sample image obtained from the first image sample set and the second sample image obtained from the second image sample set includes: Obtain the instance image corresponding to the first instance object in the first sample image; Add the instance image to the target location in the second sample image; The second sample image carrying the instance image is determined as the third sample image; If the second sample image has a corresponding reference instance label, the instance label corresponding to the third sample image is determined based on the reference instance label corresponding to the second sample image and the instance label corresponding to the first sample image. If the instance object in the second sample image does not have a corresponding reference instance label, the instance label corresponding to the third sample image is determined based on the instance label corresponding to the first sample image.

3. The method according to claim 2, characterized in that, The recognition loss value is determined based on the prediction results of the image recognition network in training state and the instance label corresponding to the third sample image, including: Obtain the first recognition result of the first instance object and the second recognition result of the second instance object included in the third sample image by the image recognition network in the training state; A first loss value is determined based on the first identification result and the instance label, and a second loss value is determined based on the second identification result and the reference instance label; The identification loss value is determined based on the product of the loss confidence of the second loss value and the second loss value, and the first loss value, wherein the loss confidence is determined based on the label confidence of the reference instance label.

4. The method according to claim 2, characterized in that, Before determining the second sample image carrying the instance image as the third sample image, the method further includes: The second sample image is subjected to image enhancement processing; The reference label generation network corresponding to the image recognition network before parameter adjustment is used to identify the second sample image after image enhancement processing, and the classification confidence and region confidence of each of the multiple candidate instance objects are obtained. Obtain the product of the classification confidence and region confidence corresponding to each of the multiple candidate instance objects, and determine the product corresponding to each of the multiple candidate instance objects as the label confidence corresponding to each of the multiple candidate instance objects; Candidate instances with label confidence scores greater than or equal to a confidence threshold are identified as second instance objects included in the second sample image. Reference instance labels corresponding to the second sample image are determined based on the classification confidence and region confidence scores of the second instance objects.

5. The method according to claim 1, characterized in that, Determining the reference label generation network using the adjusted network parameters includes: Obtain the generation parameters of the reference label generation network corresponding to the image recognition network before parameter adjustment; Obtain the network parameters of the image recognition network after parameter adjustment; The reference label generation network is determined based on the generation parameters and the network parameters.

6. The method according to claim 5, characterized in that, Determining the reference label generation network based on the generation parameters and the network parameters includes: Obtain the first update coefficient; Determine the first product result of the generated parameters and the first update coefficient, and the second product result of the network parameters and the second update coefficient, wherein the sum of the first update coefficient and the second update coefficient is 1; The reference label generation network will be determined based on the first product result and the second product result.

7. The method according to claim 1, characterized in that, After determining that the image recognition network containing the current network parameters has reached the convergence condition when the recognition loss value reaches the threshold condition, the method further includes: Acquire the target image to be identified; Image features are extracted from the target image based on the feature extraction layer of the target image recognition network; The image features are input into the category recognition layer of the target image recognition network to determine the target instance category of the instance objects included in the target image; The image features are input into the location recognition layer of the target image recognition network to determine the target image location of the instance object included in the target image. The target instance category and the target image location are determined as the recognition result of the target image.

8. A training device for an image recognition network, characterized in that, include: The acquisition unit is used to acquire a first image sample set and a second image sample set. The first image sample set includes a plurality of first samples, each of which includes a first sample image carrying an instance label. The instance label is used to indicate the instance category of the instance object in the first sample image and the image position of the instance object in the first sample image. The second image sample set includes a plurality of second samples, each of which includes a second sample image without an instance label. A training unit is configured to generate a third sample image based on a first sample image obtained from a first image sample set and a second sample image obtained from a second image sample set, wherein the third sample image includes a first instance object in the first sample image and a second instance object in the second sample image; extract image features of the third sample image based on an image recognition network in a training state, and identify the predicted instance category of the instance object included in the third sample image and the predicted image position of the instance object in the third sample image based on the image features; and determine a recognition loss value based on the prediction result of the image recognition network in a training state and the instance label corresponding to the third sample image, wherein the image recognition network is used to identify the instance category of the instance object in the sample image and the image position of the instance object in the sample image; An update unit is configured to adjust the network parameters in the image recognition network when the recognition loss value does not reach a threshold condition, and determine a reference label generation network using the adjusted network parameters, wherein the reference label generation network is used to generate reference instance labels for the input image; the second sample image is input into the reference label generation network to obtain reference instance labels generated for the second sample image; and the second image sample set is updated using the second sample image carrying the reference instance labels. The determining unit is used to determine, when the recognition loss value reaches a threshold condition, the image recognition network containing the current network parameters as the target image recognition network that has reached the convergence condition.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 7.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.