Method, device, equipment, storage medium and program product for generating training sample

Abstract

The present disclosure provides a training sample generation method and device, equipment, a storage medium and a program product, relates to the field of artificial intelligence, in particular to computer vision and deep learning technology, and can be applied to intelligent traffic and smart parking scenes. A specific embodiment of the method comprises: obtaining an image to be recognized, inputting the image to be recognized into a teacher network and a student network respectively to obtain a first recognition result and a second recognition result; determining whether the first recognition result and the second recognition result match; in response to determining that the first recognition result and the second recognition result do not match, labeling an image region in the image to be recognized where the first recognition result and the second recognition result do not match as a pre-labeling result; using the pre-labeling result to generate a correct recognition result of the image to be recognized, and determining the image to be recognized and the generated correct recognition result as training samples of the teacher network and the student network. The embodiment helps to save image labeling costs.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of artificial intelligence, in particular to computer vision and deep learning techniques, which can be applied to intelligent transportation and smart parking scenarios. BACKGROUND

[0002] With the rapid development of technologies such as the Internet, cloud computing, and the Internet of Things, various fields such as social networks, e-commerce platforms, and sensor networks are generating massive amounts of data every moment. Through data mining and analysis, many valuable knowledge or information can be obtained, but the massive amounts of data also pose challenges to data processing. How to reasonably utilize massive amounts of data and form a data closed loop is one of the problems that attracts much attention.

[0003] For example, in the scenario of smart parking, how to realize a data closed loop of data collection, data labeling, model training, model testing, model deployment, and data feedback is one of the problems worth studying. Massive amounts of data generated every day usually need to be manually labeled by labeling personnel, and a test set suitable for the model also needs to be selected from the massive amounts of data. Therefore, the massive amounts of data pose great challenges to the research and implementation of the data closed loop. SUMMARY

[0004] Embodiments of the present disclosure provide a method and apparatus for generating training samples, a device, a storage medium, and a program product.

[0005] In a first aspect, embodiments of the present disclosure provide a method for generating training samples, the method comprising: obtaining an image to be recognized, and inputting the image to be recognized into a teacher network and a student network for image recognition to obtain a first recognition result and a second recognition result; determining whether the first recognition result and the second recognition result match; in response to determining that the first recognition result and the second recognition result do not match, labeling an image region in the image to be recognized where the first recognition result and the second recognition result do not match as a pre-labeling result; generating a correct recognition result of the image to be recognized using the pre-labeling result, and determining the image to be recognized and the generated correct recognition result as training samples of the teacher network and the student network.

[0006] In a second aspect, embodiments of the present disclosure provide a device for generating training samples, the device comprising: an identification module configured to obtain an image to be identified, and input the image to be identified into a teacher network and a student network for image identification respectively, to obtain a first identification result and a second identification result; a matching module configured to determine whether the first identification result and the second identification result match; a pre-labeling module configured to, in response to determining that the first identification result and the second identification result do not match, label an image region in the image to be identified in which the first identification result and the second identification result do not match as a pre-labeling result; and a generation module configured to generate a correct identification result of the image to be identified using the pre-labeling result, and determine the image to be identified and the generated correct identification result as training samples of the teacher network and the student network.

[0007] In a third aspect, embodiments of the present disclosure provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any implementation manner of the first aspect.

[0008] In a fourth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to perform the method described in any implementation manner of the first aspect.

[0009] In a fifth aspect, embodiments of the present disclosure provide a computer program product, comprising a computer program, the computer program being used to implement the method described in any implementation manner of the first aspect when executed by a processor.

[0010] The method, device, equipment, storage medium, and program product for generating training samples provided by the embodiments of the present disclosure use a teacher network and a student network to respectively perform image identification on an image to be identified to obtain a first identification result and a second identification result, and then label an image region in the image to be identified in which the first identification result and the second identification result do not match as a pre-labeling result, so as to generate a correct identification result of the image to be identified using the pre-labeling result, and then use the image to be identified and the correct identification result as training samples of the teacher network and the student network to further improve the image identification capabilities of the teacher network and the student network. Since the pre-labeling result presents the part in which the identification results of the teacher network and the student network are inconsistent, i.e., the part in which the identification effect is poor, the part in which the identification results of the teacher network and the student network are poor can be quickly found based on the pre-labeling result to label the correct identification result, so that the case of carefully labeling all regions of the image to be identified can be avoided, which helps to save the image labeling cost and improve the generation efficiency of the training samples.

[0011] It should be understood that nothing in this section is intended to limit the scope of the embodiments of the present disclosure. Other aspects of the present disclosure will become apparent to those of ordinary skill in the art upon reviewing the description below in conjunction with the accompanying figures. BRIEF DESCRIPTION OF DRAWINGS

[0012] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings:

[0013] Figure 1 is an exemplary system architecture diagram in which the present disclosure can be applied;

[0014] Figure 2 is a flowchart of one embodiment of a method for generating training samples of the present disclosure;

[0015] Figure 3a is a schematic diagram of a case where the first recognition result and the second recognition result do not match;

[0016] Figure 3b is a schematic diagram of a case where the first recognition result and the second recognition result match;

[0017] Figure 4 is a flowchart of another embodiment of a method for generating training samples of the present disclosure;

[0018] Figure 5 is a schematic diagram of one application scenario of the method for generating training samples of the present disclosure;

[0019] Figure 6 is a structural schematic diagram of one embodiment of a training sample generation apparatus of the present disclosure;

[0020] Figure 7 is a structural schematic diagram of an electronic device suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help the understanding of the present disclosure. These should be considered in their context only. Thus, those of ordinary skill in the art will recognize the various changes and modifications of the embodiments described herein, without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and structures are omitted in the following description for the sake of clarity and conciseness.

[0022] It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0023] Figure 1An exemplary system architecture 100 is shown to which embodiments of the training sample generation method or the training sample generation apparatus of the present disclosure can be applied.

[0024] As shown in Figure 1 , the system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0025] The terminal devices 101, 102, 103 can interact with the server 105 through the network 104 to receive or send various information (such as images, videos, etc.), etc. The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices. When the terminal devices 101, 102, 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made herein.

[0026] The server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (such as multiple software or software modules used to provide distributed services), or as a single software or software module. No specific limitation is made herein.

[0027] It should be noted that the training sample generation method provided by the embodiments of the present disclosure is generally executed by the server 105, and correspondingly, the training sample generation apparatus is generally provided in the server 105. In some cases, the exemplary system architecture 100 can not include the terminal devices 101, 102, 103 and the network 104.

[0028] It should be noted that in some cases, the terminal devices 101, 102, 103 can have image recognition and model training functions, at which time the terminal devices 101, 102, 103 can acquire training samples for the teacher network and the student network by using the to-be-recognized images, and complete network training. At this time, the exemplary system architecture 100 can not include the network 104 and the server 105.

[0029] It should be understood that Figure 1 the number of terminal devices, networks, and servers in is merely illustrative. According to the needs of implementation, there can be any number of terminal devices, networks, and servers.

[0030] With reference to both figures Figure 2 which shows a flow 200 of one embodiment of a method for generating a training sample according to the present disclosure. The generation of the training sample comprises the following steps:

[0031] Step 201, obtaining an image to be recognized, and inputting the image to be recognized into a teacher network and a student network for image recognition respectively, to obtain a first recognition result and a second recognition result.

[0032] In this embodiment, the image to be recognized can be an image of any type and content, which can be determined according to the actual application scenario. For example, the image to be recognized can be an image of a certain lane, or an image collected by a preset image collection device (such as a camera, etc.), and so on. The execution subject (such as a server 105, etc.) can obtain the image to be recognized from the local, other storage devices or third-party data platforms, etc. In some scenarios, the image to be recognized can be an image in a large number of images to be labeled. Figure 1

[0033] The teacher network and the student network can be various neural networks for image recognition constructed in advance based on knowledge distillation and other technologies. Generally, the teacher network has a larger size, more complex network parameters, and higher accuracy of the output recognition result. Conversely, the student network has a smaller size, simpler network parameters, and lower accuracy of the output recognition result. For example, the teacher network can be a large model used on the server side, and the student network can be a front-end model used on the mobile side.

[0034] Specifically, the image to be recognized can be input into the teacher network to obtain the first recognition result, and the image to be recognized can be input into the student network to obtain the second recognition result. The content indicated by the recognition results output by the teacher network and the student network can be flexibly set according to the actual application scenario. For example, the recognition result can indicate the positions of each object segmented from the image to be recognized. For another example, the recognition result can indicate the categories and positions of each object segmented from the image to be recognized.

[0035] Step 202, determining whether the first recognition result and the second recognition result match.

[0036] In this embodiment, after obtaining the first recognition result output by the teacher network for the image to be recognized and the second recognition result output by the student network for the image to be recognized, since both of them represent the recognition result of the image to be recognized, it can be further determined whether the first recognition result and the second recognition result match.

[0037] ​The specific conditions used to determine whether the first and second recognition results match can be flexibly set according to the actual application scenario and requirements. For example, when the recognition results indicate the locations of each object segmented from the image to be recognized, the matching conditions can include that the number of segmented objects is the same, and the difference in the location of the same object indicated in the two recognition results is less than a preset threshold. In this case, if the first and second recognition results indicate that the segmented objects are one-to-one, and the IOU (Intersection Over Union) of the corresponding object's detection box is greater than the preset threshold, then the first and second recognition results can be considered to match; otherwise, they can be considered not to match.

[0038] Step 203: In response to determining that the first recognition result and the second recognition result do not match, the image regions in the image to be recognized where the first recognition result and the second recognition result do not match are labeled as pre-labeled results.

[0039] If the first and second recognition results do not match, the mismatched parts can be labeled as image regions in the image to be recognized, serving as pre-labeled results. That is, image regions in the image to be recognized that yield different recognition results using the teacher network and the student network are labeled.

[0040] The annotation method can be flexibly set according to actual application needs. For example, the mismatched parts can be highlighted in the image area of ​​the image to be identified (e.g., by setting a specified color light) to achieve annotation.

[0041] Generally, mismatches between the first and second identification results can include cases where the teacher network or student network misses or makes multiple detections, or cases where the positions of the objects identified by the teacher network and student network differ significantly, and so on.

[0042] As an example, when the first recognition result and the second recognition result include detection boxes and categories of each object segmented from the image to be recognized, the image regions indicated by the detection boxes that cannot be matched in the first recognition result and the second recognition result can be determined as the image regions where the first recognition result and the second recognition result do not match by cyclically comparing the categories and IOUs indicated by each detection box in the first recognition result and the second recognition result.

[0043] like Figure 3a As shown, it illustrates a schematic diagram where the first and second recognition results do not match. Figure 3aAs shown, the solid-line bounding boxes in the figure are the first recognition results output by the teacher network, and the dashed-line bounding boxes in the figure are the second recognition results output by the student network. The first recognition results include two pedestrians and three vehicles shown by the solid-line bounding boxes in the figure. The second recognition results include two pedestrians and two vehicles shown by the dashed-line bounding boxes in the figure. Moreover, the two pedestrians and the two vehicles in the first recognition results and the second recognition results can be matched, but the vehicle located near the center of the image (as shown by the shaded part in the figure) included in the first recognition results is not included in the second recognition results, that is, the student network can miss the vehicle located near the center of the image, or the teacher network can over-detect the vehicle located near the center of the image. At this time, the image region where the bounding box of the vehicle located near the center of the image is located can be taken as the image region where the first recognition results and the second recognition results do not match, and then the bounding box of the image region can be highlighted or the like to realize pre-labeling.

[0044] In step 204, a correct recognition result of the to-be-recognized image is generated by using the pre-labeling result, and the to-be-recognized image and the generated correct recognition result are determined as training samples of the teacher network and the student network.

[0045] In the embodiment, after obtaining the pre-labeling result, the correct recognition result of the to-be-recognized image can be generated by using the pre-labeling result, so that the to-be-recognized image and its corresponding correct recognition result can be combined to form a training sample for training of the teacher network and the student network, so that the teacher network and the student network can improve the recognition ability for the to-be-recognized image.

[0046] Since the pre-labeling result highlights the image region where the recognition results of the teacher network and the student network do not match, various methods can be used to label the correct recognition result of the image region, and the recognition results of other image regions output by the teacher network and the student network are combined to form the correct recognition result of the to-be-recognized image.

[0047] For example, other trained image recognition models with high accuracy can be used to perform image recognition on the image region where the first recognition results and the second recognition results do not match, and the obtained recognition result can be taken as the correct recognition result of the image region. For another example, the pre-labeling result can be displayed to a labeling personnel, and the correct recognition result of the image region where the first recognition results and the second recognition results do not match labeled by the labeling personnel can be received.

[0048] For each image, based on the image recognition results obtained by the teacher network and the student network respectively, the image regions in the image where the two image recognition results are inconsistent are pre-labeled as pre-labeled results. Based on this, the locations in the image that always need to be specially labeled with the correct recognition results can be quickly found and labeled with the correct recognition results. This allows labelers to pay more attention to the image regions where the recognition results of the current teacher network and student network are inconsistent, reducing the time spent by labelers on image regions where the recognition results of the current teacher network and student network are good, improving labeling efficiency and reducing the workload of labelers.

[0049] In some optional implementations of this embodiment, in response to determining that the first recognition result and the second recognition result match, the image to be recognized is deleted from the training samples of the teacher network and the student network.

[0050] If the first recognition result and the second recognition result match, it means that the recognition results of the teacher network and the student network for the image to be recognized are consistent. Therefore, the image to be recognized can be considered as an image that both the teacher network and the student network can accurately recognize, which is a simple sample. Thus, the image to be recognized can be deleted from the training samples of the teacher network and the student network to avoid using the image to be recognized for training the teacher network and the student network again.

[0051] As an example, such as Figure 3b As shown, it illustrates a schematic diagram of the matching of the first and second recognition results. Figure 3b The solid-line detection boxes represent the recognition results output by the teacher network, specifically including two vehicle detection boxes. The dashed-line detection boxes represent the recognition results output by the student network, specifically including two vehicle detection boxes. Furthermore, there is a one-to-one correspondence between the two vehicles detected by the teacher network and the two vehicles detected by the student network, and the Intersection over Union (IoU) of the corresponding detection boxes is high. Therefore, it can be considered that the first recognition result matches the second recognition result, meaning that both the teacher network and the student network achieve relatively good recognition results for this image.

[0052] Since simple samples have little effect on improving the recognition ability of both teacher and student networks, filtering out images that can be recognized by both teacher and student networks can not only avoid useless training, but also save the process of correctly labeling the recognition results of such images.

[0053] Current image annotation typically involves an image recognition model processing the image to obtain a recognition result, followed by annotation by personnel who then check and label the correctly identified results. These results are then used as training samples to further train the image recognition model. This approach usually requires annotators to carefully review both the image to be recognized and the recognition result output by the image recognition model. If the image to be recognized contains multiple objects, the annotators also need to check and adjust each object individually.

[0054] The method provided by the above embodiments of the present disclosure simultaneously utilizes the teacher network and the student network to respectively process the to-be-identified image, obtains the corresponding first identification result and second identification result, then compares and analyzes the first identification result and the second identification result, determines the to-be-identified image whose first identification result and second identification result match as a simple sample, and filters out the simple sample, thereby avoiding the process of spending time cost and the like of the labeling personnel to label the simple sample and the process of utilizing the labeled simple sample to train the network and the like. In addition, the image region whose first identification result and second identification result do not match is highlighted, so as to facilitate the labeling personnel to quickly find the image region that needs to be highlighted labeled to be correctly labeled, thereby avoiding the process of the labeling personnel needing to carefully examine all image regions, and thus helps to save the workload of the labeling personnel.

[0055] Further reference is made to Figure 4 which shows a flow 400 of yet another embodiment of the method of generating training samples. The method 400 of generating training samples comprises the following steps:

[0056] Step 401, obtaining a to-be-identified image, and inputting the to-be-identified image into a teacher network and a student network for image identification respectively to obtain a first identification result and a second identification result.

[0057] Step 402, determining whether the first identification result and the second identification result match.

[0058] Step 403, in response to determining that the first identification result and the second identification result do not match, labeling the image region in the to-be-identified image whose first identification result and second identification result do not match as a pre-labeling result.

[0059] Step 404, utilizing the pre-labeling result to generate a correct identification result of the to-be-identified image, and determining the to-be-identified image and the generated correct identification result as training samples of the teacher network and the student network.

[0060] Step 405, comparing the first identification result, the second identification result and the correct identification result to determine whether the identification result of the target image region indicated by the first identification result and the second identification result is consistent with the correct identification result of the target image region.

[0061] In this embodiment, the target image region can be the image region whose first identification result and second identification result match, i.e., the image region whose identification results obtained by the teacher network and the student network are consistent.

[0062] At this time, it can be determined whether the recognition results of the target image region obtained by the teacher network and the student network are consistent with the correct recognition result of the target image region. If consistent, it can be indicated that the teacher network and the student network correctly recognize the target image region, and if inconsistent, it can be indicated that the teacher network and the student network both incorrectly recognize the target image region.

[0063] Step 406, in response to determining that the recognition results of the target image region indicated by the first recognition result and the second recognition result are inconsistent with the correct recognition result of the target image region, adding the to-be-recognized image and the correct recognition result to the test set of the teacher network and the student network.

[0064] In the embodiment, if the recognition results of the target image region indicated by the first recognition result and the second recognition result are inconsistent with the correct recognition result of the target image region, that is, the teacher network and the student network both incorrectly recognize the target image region. In other words, the target image region is a sample that is more likely to be incorrectly recognized by the teacher network and the student network. At this time, the to-be-recognized image and the correct recognition result can be added to the test set of the teacher network and the student network, so that the training effect of the teacher network and the student network can be more accurately evaluated.

[0065] In the scenario of massive data continuously generated, the annotation work also needs to continue, and cannot stop, so as the data volume increases, the workload of the annotation personnel also increases, which may produce a long tail effect, increase a lot of annotation cost, and cannot reasonably screen network test samples, which makes the long tail effect more obvious.

[0066] Therefore, by comparing whether the correct recognition result is consistent with the recognition results of the teacher network and the student network in the region modified, the image incorrectly recognized by the teacher network and the student network can be screened out as the image to be focused on, such as effectively evaluating the training effect of the teacher network and the student network as the test set.

[0067] Referring back to Figure 5 , Figure 5 is one illustrative application scenario 500 of the training sample generation method according to the present disclosure. In the Figure 5 application scenario, the intelligent camera of the parking lot can continuously collect image data of the parking lot, and send the collected massive parking lot image data to the server. For each parking lot image, the server can input the parking lot image into the teacher network and the student network respectively to obtain the first recognition result and the second recognition result.

[0068] As shown in the table, the first and second recognition results are analyzed based on the following approach: If both the teacher network and the student network obtain correct recognition results, the parking lot image can be considered a simple sample, which does not significantly improve the capabilities of the teacher and student networks. If either the teacher network or the student network obtains an error, the parking lot image can be considered a difficult sample, which can be used for further iterative training of the teacher and student networks to improve their image recognition capabilities for difficult samples. If both the teacher network and the student network obtain incorrect recognition results, the parking lot image can be used as a test set to test the training effect of the teacher and student networks.

[0069] Specifically, for difficult samples, the incorrectly identified parts can be highlighted in the parking lot image or its recognition results, so that the parts that need to be highlighted can be quickly identified and correctly labeled during subsequent manual annotation, thus reducing the amount of manual annotation processing.

[0070] After obtaining the correct recognition result, the teacher network and student network can be iteratively trained by combining parking lot images as training samples to further improve the image recognition capabilities of the teacher network and student network. Then, the newly trained student network can be used to update the student network used by the front end. At the same time, the front end can continue to collect parking lot data and process it in the same way, and so on.

[0071] In some optional implementations of this embodiment, the number of student networks can be two or more. In this case, the second recognition result can include the recognition results output by each student network. For example, in some scenarios, the student networks can be front-end models for various scenarios.

[0072] In this case, the conditions for determining whether the first and second recognition results match can be flexibly set according to actual application needs. For example, the matching condition can be that the first recognition result and the recognition results output by each student network are consistent.

[0073] This allows for the cross-matching and verification of recognition results from multiple networks on the image to be recognized. The recognition results are then distributed to each network, and the image to be recognized is filtered based on these results to determine whether it is a simple or difficult sample. This saves annotators the work of annotating simple samples and allows them to focus on correctly annotating difficult samples, thus reducing the workload of manual annotation in many ways. Furthermore, the filtered samples can be used to further train the network, which can improve the network's image recognition capabilities in a targeted manner.

[0074] It should be noted that any content not specifically described in this embodiment can be found by referring to [the relevant documentation]. Figure 2 The relevant steps in the embodiments will not be repeated here.

[0075] Further referring to Figure 6 , as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of a training sample generation device, which corresponds to the method embodiment shown in Figure 2 , and the device can be applied in various electronic devices.

[0076] As shown in Figure 6 , the training sample generation device 600 provided in the embodiment includes an identification module 601, a matching module 602, a pre-labeling module 603, and a generation module 604. The identification module 601 is configured to obtain a to-be-identified image, and input the to-be-identified image into a teacher network and a student network for image identification to obtain a first identification result and a second identification result. The matching module 602 is configured to determine whether the first identification result and the second identification result match. The pre-labeling module 603 is configured to, in response to a determination that the first identification result and the second identification result do not match, label an image region in the to-be-identified image for which the first identification result and the second identification result do not match as a pre-labeling result. The generation module 604 is configured to generate a correct identification result of the to-be-identified image by using the pre-labeling result, and determine the to-be-identified image and the generated correct identification result as training samples of the teacher network and the student network.

[0077] In the embodiment, the specific processing of the identification module 601, the matching module 602, the pre-labeling module 603, and the generation module 604 in the training sample generation device 600 and the technical effects brought by the specific processing can be respectively referred to the related descriptions of steps 201-204 in the corresponding embodiments, which will not be described here in detail. Figure 2

[0078] In some optional implementations of the embodiment, the training sample generation device 600 described above further includes a deletion module (not shown in the figure) configured to, in response to a determination that the first identification result and the second identification result match, delete the to-be-identified image from the training samples of the teacher network and the student network.

[0079] In some optional implementations of the embodiment, the training sample generation device 600 described above further includes a comparison module (not shown in the figure) configured to compare the first identification result, the second identification result, and the correct identification result to determine whether the identification result of a target image region indicated by the first identification result and the second identification result is consistent with the correct identification result of the target image region, where the target image region is an image region for which the first identification result and the second identification result match. An addition module (not shown in the figure) is configured to, in response to a determination that the identification result of the target image region indicated by the first identification result and the second identification result is inconsistent with the correct identification result of the target image region, add the to-be-identified image and the correct identification result to a test set of the teacher network and the student network.​

[0080] In some optional implementations of the present embodiment, the number of student networks is more than two; and the second recognition result comprises a recognition result output by each student network.

[0081] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0082] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0083] As shown in Figure 7 The device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0084] Various components in the device 700 are connected to the I / O interface 705, including an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, a speaker, etc.; the storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0085] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the method of generating training samples. For example, in some embodiments, the method of generating training samples can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the method of generating training samples described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the method of generating training samples by any other suitable means, such as by means of firmware.

[0086] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0087] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0088] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0089] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0090] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0091] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0092] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions provided by the present disclosure can be achieved, which is not limited herein.

[0093] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating training samples, comprising: obtaining a to-be-recognized image, and inputting the to-be-recognized image into a teacher network and a student network for image recognition respectively to obtain a first recognition result and a second recognition result; determining whether the first recognition result and the second recognition result match; in response to determining that the first recognition result and the second recognition result do not match, including at least one of: missing detection and multiple detection of the teacher network or the student network, a position difference of an object recognized by the teacher network and the student network being greater than a preset value, and labeling an image region in the to-be-recognized image in which the first recognition result and the second recognition result do not match as a pre-labeling result; generating a correct recognition result of the to-be-recognized image by using the pre-labeling result, and determining the to-be-recognized image and the generated correct recognition result as training samples of the teacher network and the student network; in response to determining that the first recognition result and the second recognition result match, deleting the to-be-recognized image from the training samples of the teacher network and the student network.

2. The method of claim 1, wherein, The method further comprises: comparing the first recognition result, the second recognition result and the correct recognition result to determine whether recognition results of a target image region indicated by the first recognition result and the second recognition result are consistent with a correct recognition result of the target image region, wherein the target image region is an image region in which the first recognition result and the second recognition result match; in response to determining that the recognition results of the target image region indicated by the first recognition result and the second recognition result are inconsistent with the correct recognition result of the target image region, adding the to-be-recognized image and the correct recognition result to a test set of the teacher network and the student network.

3. The method according to one of claims 1-2, wherein, The number of student networks is more than two; and The second recognition result includes recognition results output by each student network respectively. 4.An apparatus for generating training samples, comprising: an identification module configured to obtain a to-be-recognized image, and input the to-be-recognized image into a teacher network and a student network for image recognition respectively to obtain a first recognition result and a second recognition result; a matching module configured to determine whether the first recognition result and the second recognition result match; a pre-labeling module configured to, in response to determining that the first recognition result and the second recognition result do not match, include at least one of: missing detection and multiple detection of the teacher network or the student network, a position difference of an object recognized by the teacher network and the student network being greater than a preset value, and labeling an image region in the to-be-recognized image in which the first recognition result and the second recognition result do not match as a pre-labeling result; a generation module configured to generate a correct recognition result of the to-be-recognized image by using the pre-labeling result, and determine the to-be-recognized image and the generated correct recognition result as training samples of the teacher network and the student network; a deletion module configured to, in response to determining that the first recognition result and the second recognition result match, delete the to-be-recognized image from the training samples of the teacher network and the student network.

5. The apparatus of claim 4, wherein, The apparatus further comprises: The comparison module is configured to compare the first recognition result, the second recognition result and the correct recognition result to determine whether the recognition result of the target image region indicated by the first recognition result and the second recognition result is consistent with the correct recognition result of the target image region, wherein the target image region is an image region matched by the first recognition result and the second recognition result. The adding module is configured to, in response to determining that the recognition result of the target image region indicated by the first recognition result and the second recognition result is inconsistent with the correct recognition result of the target image region, add the to-be-recognized image and the correct recognition result to a test set of the teacher network and the student network.

6. The apparatus of one of claims 4-5, wherein, The number of the student networks is more than two; and The second recognition result includes recognition results respectively output by the student networks. 7.An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3. 8.A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-3. 9.A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-3.

Images