Image sample labeling method and device, storage medium and electronic device

Abstract

The present disclosure relates to an image sample labeling method and device, a storage medium and an electronic device, and relates to the technical field of computers. The method displays a target image pair, and in response to a selection operation on a target image region of any image in the target image pair, synchronously determines a to-be-labeled image block corresponding to the target image region in the first image and the second image, then labels the to-be-labeled image block, and obtains an image labeling result of the target image pair. The method can not only improve the labeling efficiency of image samples, but also make the sample labeling result more accurate by comparing the first image and the second image for labeling.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to an image sample annotation method, apparatus, storage medium, and electronic device. Background Technology

[0002] Generally, a large amount of training sample data is needed before model training to adjust the model parameters. However, training data samples are usually labeled manually. Especially for image semantic segmentation datasets, manual labeling requires a large amount of repetitive image semantic annotation, which reduces the efficiency of sample labeling. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] Firstly, this disclosure provides an image sample annotation method, including:

[0005] Display a pair of target images, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods;

[0006] In response to a selection operation of a target image region for any image in the target image pair, an image block to be labeled corresponding to the target image region is simultaneously determined in the first image and the second image;

[0007] The image blocks to be labeled are labeled to obtain the image labeling results of the target image pairs.

[0008] Secondly, this disclosure provides an image sample annotation apparatus, comprising:

[0009] The display module is configured to display a pair of target images, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods;

[0010] The selection module is configured to simultaneously determine the image block to be labeled corresponding to the target image region in the first image and the second image in response to a selection operation of a target image region in any image of the target image pair.

[0011] The annotation module is configured to annotate the image block to be annotated and obtain the image annotation results of the target image pair.

[0012] Thirdly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processing device, implements the steps of the method described in the first aspect.

[0013] Fourthly, this disclosure provides an electronic device, comprising:

[0014] A storage device on which computer programs are stored;

[0015] A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0016] Based on the above technical solution, by displaying the target image pair and responding to the selection operation of the target image region of any image in the target image pair, the image block to be labeled corresponding to the target image region is simultaneously determined in the first image and the second image, and then the image block to be labeled is labeled to obtain the image labeling result of the target image pair. This not only improves the labeling efficiency of image samples, but also makes the sample labeling result more accurate by comparing the first image and the second image.

[0017] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 This is a flowchart illustrating an image sample annotation method according to an exemplary embodiment.

[0020] Figure 2 This is a schematic diagram of the user interface of an annotation tool according to an exemplary embodiment.

[0021] Figure 3 This is a schematic diagram of the user interface of an annotation tool according to another exemplary embodiment.

[0022] Figure 4 This is a schematic diagram illustrating the annotation of an image block to be annotated according to an exemplary embodiment.

[0023] Figure 5 This is a schematic diagram illustrating the image annotation results according to an exemplary embodiment.

[0024] Figure 6This is a flowchart illustrating an image sample annotation method according to another exemplary embodiment.

[0025] Figure 7 This is a flowchart illustrating a first image block according to an exemplary embodiment.

[0026] Figure 8 This is a flowchart illustrating the determination of a target color according to an exemplary embodiment.

[0027] Figure 9 This is a flowchart illustrating an image sample annotation method according to yet another exemplary embodiment.

[0028] Figure 10 This is a schematic diagram of the module connections of an image sample annotation device according to an exemplary embodiment.

[0029] Figure 11 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0037] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0038] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0039] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0040] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0041] Figure 1 This is a flowchart illustrating an image sample annotation method according to an exemplary embodiment. For example... Figure 1 As shown, this disclosure provides an image sample annotation method, which can be executed by an electronic device, specifically by an image sample annotation device. This device can be implemented in software and / or hardware and configured within the electronic device. Figure 1 As shown, the method may include the following steps.

[0042] In step 110, a pair of target images is displayed, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods.

[0043] Here, the target image pair includes a first image and a second image obtained by capturing the same scene using different shooting methods. For example, the first image could be an RGB image obtained by capturing the scene using a visible light camera, and the second image could be a thermal image (T image) obtained by capturing the scene using a thermal imaging camera; then the target image pair would be an RGB image followed by a thermal image. Alternatively, the second image could also be a depth image (D image) obtained by capturing the scene using a depth camera; then the image pair would be an RGB image followed by a depth image. Or, the second image could also be a point cloud image of the scene obtained using a LiDAR scanner; then the image pair would be an RGB image followed by a point cloud image.

[0044] The annotation tool can display the first and second images of the target image pair in different display areas. It should be understood that the annotation tool refers to an application program that implements the image sample annotation method proposed in this disclosure. When annotating image samples, the annotation tool can load the target image pair and simultaneously display the first and second images in different image areas.

[0045] Figure 2 This is a schematic diagram of the user interface of an annotation tool according to an exemplary embodiment. Figure 2 As shown, the annotation tool 200 includes a toolbar 201, a first display area 202 and a second display area 203. The first display area 202 displays a first image 204 and the second display area 203 displays a second image 205.

[0046] In step 120, in response to the selection operation of a target image region for any image in the target image pair, an image block to be labeled corresponding to the target image region is simultaneously determined in the first image and the second image.

[0047] Here, the selection operation can be done using, for example... Figure 2 The selection can be performed using specific tools in the toolbar 201 shown, or it can be a click operation, touch operation, or gesture operation, etc. The target image area can be the image to be labeled, such as "car", "motorcycle", "bicycle", "person", "lane line" and "traffic light" in the target image pair, etc.

[0048] Figure 3 This is a schematic diagram of the user interface of an annotation tool according to another exemplary embodiment. Figure 3As shown, the annotation tool 200 includes a toolbar 201, a first display area 202, and a second display area 203. The first display area 202 displays a first image 204, and the second display area 203 displays a second image 205. When an annotator needs to annotate a target image region in the target image pair, the annotator can select the first target image region 206 in the second image 205. Simultaneously, the second target image region 207 will also be selected in the first image 204. The first target image region 206 and the second target image region 207 refer to the same pixel region in the first image 204 and the second image 205. It is worth noting that this pixel region does not change with image scaling; it is determined by inherent image factors such as image size and resolution. Since the target image pair consists of images of the same scene, the image coordinates of the same object in the first and second images are consistent.

[0049] It should be understood that during an annotation process, the annotator can select target image regions on either the first image 204 or the second image 205. The selection operation on either image 204 or the second image 205 will be synchronized to the other image. Therefore, the annotator can repeatedly compare the first image 204 and the second image 205 to determine the completed image block to be annotated. For example, if a part of an object is unclear in the first image 204, the annotator can select the unclear part in the second image 205.

[0050] It is worth noting that the selection operations for the first image 204 or the second image 205 are synchronous. However, other operations on the first image 204 or the second image 205 can be independent. That is, other operations on the first display area 202 or the second display area 203 can be performed independently on either display area 202 or the second display area 203. For example, when it is necessary to magnify the first image 204 in the first display area 202, the first image 204 in the first display area 202 can be magnified only, while the second image in the second display area 203 is not magnified.

[0051] In step 130, the image blocks to be labeled are labeled to obtain the image labeling results of the target image pairs.

[0052] Here, image annotation results can be obtained by using different colors to label image blocks with different semantic categories. For example, red can be used to label image blocks with the semantic category of "car", while green can be used to label image blocks with the semantic category of "person".

[0053] Figure 4 This is a schematic diagram illustrating the annotation of an image block to be annotated according to an exemplary embodiment. For example... Figure 4 As shown, the annotation tool 200 includes a toolbar 201, a first display area 202, and a second display area 203. The first display area 202 displays a first image 204, and the second display area 203 displays a second image 205. When an annotator needs to annotate a target image region within a target image pair, the annotator can select the first target image region 206 in the second image 205, and simultaneously, the second target image region 207 will also be selected in the first image 204. Then, the annotator can choose a preset color to annotate either the first target image region 206 or the second target image region 207. It should be understood that the annotation color is also synchronized; only one of the first target image region 206 or the second target image region 207 needs to be annotated.

[0054] Figure 5 This is a schematic diagram illustrating image annotation results according to an exemplary embodiment. For example... Figure 5 As shown, different colors are used to label objects of different semantic categories in the target image pair, such as "car", "motorcycle", "bicycle", "person", "lane line" and "traffic light".

[0055] Of course, in other embodiments, text can also be used to annotate image blocks with different image semantic categories to obtain image annotation results.

[0056] Therefore, by displaying the target image pair and responding to the selection operation of the target image region for any image in the target image pair, the image block to be labeled corresponding to the target image region is simultaneously determined in the first and second images, and then the image block to be labeled is labeled to obtain the image labeling result of the target image pair. This not only improves the labeling efficiency of image samples, but also makes the sample labeling result more accurate by comparing the first and second images.

[0057] Figure 6 This is a flowchart illustrating an image sample annotation method according to another exemplary embodiment. For example... Figure 6 As shown, after step 120, the image sample annotation method may further include the following steps:

[0058] In step 601, in response to the automatic selection instruction, the first pixel similarity between the image block to be labeled in the first image and the adjacent images is determined.

[0059] Here, the automatic selection command can be triggered by a preset operation, which may include, but is not limited to, clicking a preset button, a preset operation trajectory, etc. After detecting the automatic selection command, the electronic device, in response to the automatic selection command, calculates the first pixel similarity between the image patch to be labeled in the first image and the adjacent images.

[0060] The adjacent pixels can be pixels that are adjacent to the edge region of the image to be labeled in the first image. Of course, the adjacent pixels can also be pixels that are a preset pixel distance from the edge region of the image to be labeled.

[0061] It is worth noting that the first pixel similarity can be obtained by calculating the grayscale values ​​between pixels.

[0062] In step 602, a first image block is determined in the first image based on the first pixel similarity, wherein the first image block includes the image block to be labeled in the first image and pixels whose first pixel similarity is greater than a first preset threshold.

[0063] Here, after calculating the first pixel similarity, the image block to be labeled and the pixels with a first pixel similarity greater than a first preset threshold in the first image are determined as the first image block. Here, pixels with a first pixel similarity greater than the first preset threshold refer to pixels similar to pixels in the edge region of the image to be labeled.

[0064] The first preset threshold can be set according to the actual situation, and is not specifically limited in this embodiment.

[0065] Figure 7 This is a flowchart illustrating a first image block according to an exemplary embodiment. (e.g.) Figure 7 As shown, in the first image 701, the pixels located within the first image block 703 and outside the image block 702 to be labeled are pixels with a first pixel similarity greater than a first preset threshold.

[0066] In step 603, the second pixel similarity between the image block to be labeled in the second image and the adjacent images is determined.

[0067] Here, step 603 is the same as step 601. The execution logic of step 603 can be found in the relevant description of step 601, and will not be repeated here.

[0068] In step 604, a second image block is determined in the second image based on the second pixel similarity, wherein the second image block includes the image block to be labeled in the second image and pixels whose second pixel similarity is greater than a second preset threshold.

[0069] Here, step 604 is the same as step 602. The execution logic of step 604 can be found in the relevant description of step 602, and will not be repeated here.

[0070] It is worth noting that steps 603 to 604 are executed synchronously with steps 601 to 602. That is, when an automatic selection command is detected, the electronic device simultaneously begins to execute steps 601 and 603.

[0071] In step 605, based on the first image block and the second image block, a new image block to be labeled is determined in the first image and the second image.

[0072] Here, the first image block and the second image block may be identical or different, depending on the specific image pair. In this case, the first image block and the second image block can be displayed on the first image and the second image respectively. Then, based on the annotator's selection instruction for the first or second image block, a new image block to be annotated can be determined. For example, the annotator can select one image from the first and second image blocks as the new image block to be annotated by comparing them.

[0073] Therefore, in the above embodiments, by first selecting, the image blocks to be labeled can be initially determined in the first and second images. Then, by using the automatic selection command, more detailed image blocks to be labeled can be determined. Furthermore, the automatic selection command can greatly improve the labeling speed of image samples.

[0074] In some feasible implementations, in step 130, the image block to be labeled can be labeled based on the target color that matches the image semantic category to which the image block to be labeled belongs, so as to obtain the image labeling result of the target image pair.

[0075] Here, image annotation results can be obtained by using different colors to label image blocks with different semantic categories. For example, red can be used to label image blocks with the semantic category of "car", while green can be used to label image blocks with the semantic category of "person".

[0076] It is worth noting that annotators can trigger electronic devices to annotate image blocks using target colors via annotation commands, without requiring annotators to select a color for each image block.

[0077] Figure 8 This is a flowchart illustrating the determination of a target color according to an exemplary embodiment. For example... Figure 8 As shown, in some feasible implementations, the target color can be determined through the following steps:

[0078] In step 801, a first similarity is determined between the image block to be labeled in the first image and a preset image in the database.

[0079] Here, the preset images in the database refer to the images stored in the database. The database can store images with different semantic categories, such as images with different semantic categories like "car", "motorcycle", "bicycle", "person", "lane line" and "traffic light".

[0080] The electronic device can respond to a labeling instruction by calculating a first similarity between the image patch to be labeled in the first image and a preset image in the database. This first similarity can be obtained by calculating the grayscale value, color, etc., of pixels.

[0081] In step 802, a second similarity is determined between the image block to be labeled in the second image and a preset image in the database.

[0082] Here, the electronic device can respond to the annotation instruction and calculate a second similarity between the image patch to be annotated in the second image and a preset image in the database. The second similarity can be obtained by calculating the grayscale value, color, etc., of pixels. It is worth noting that steps 802 and 801 can be performed simultaneously.

[0083] In step 803, the image semantic category to which the image patch to be labeled belongs is determined based on the first similarity and the second similarity.

[0084] Here, after calculating the first similarity and the second similarity, the image semantic category to which the preset image corresponding to the maximum value of the first similarity and the second similarity belongs can be determined as the image semantic category to which the labeled image block belongs.

[0085] In step 804, the target color is determined based on the image semantic category.

[0086] Here, the target color can be determined based on the mapping relationship between different image semantic categories and different colors, so that different objects in the target image pair can be labeled using different colors.

[0087] Therefore, by determining the target color through similarity, there is no need for annotators to select a color for each image block to be labeled, which can greatly improve the labeling efficiency of image samples.

[0088] Figure 9 This is a flowchart illustrating an image sample annotation method according to yet another exemplary embodiment. For example... Figure 9As shown, in some feasible implementations, the image sample annotation method may further include the following steps:

[0089] In step 910, in response to the verification command, the image annotation results of the target image pair are sent to the annotation verification terminal so that the annotation verification terminal can verify the image annotation results of the target image pair.

[0090] Here, after completing the annotation of the target image pair, the annotation tool can respond to the verification command and send the image annotation results of the target image pair to the annotation verification terminal. The annotation verification terminal is used to verify the image annotation results of the target image pair.

[0091] As examples, the annotation verification terminal can display image annotation results and then receive verification results from verification personnel for the image annotation results of the target image pair. Verification personnel can obtain the verification result based on whether the labels in the image annotation results displayed by the annotation verification terminal match the images corresponding to those labels.

[0092] As another example, the annotation verification end can use image recognition algorithms to classify the objects included in the target image pair, and then determine whether the semantic categories of the images to which the objects included in the target image pair belong are consistent based on the obtained classification results to obtain the verification result.

[0093] In step 920, in response to the instruction sent by the annotation verification terminal indicating that the image annotation result of the target image pair has passed, the image annotation result is stored in the image sample library.

[0094] Here, when the annotation verification end determines that the image annotation result of the target image pair is consistent with the object category included in the target image pair, it sends an instruction to the electronic device to indicate that the image annotation result of the target image pair has passed. The electronic device then responds to the instruction and stores the annotation result of the target image pair in the image sample library.

[0095] If the image annotation result of the target image pair fails, the annotation personnel will be prompted to re-annotate the target image pair in response to the instruction sent by the annotation verification terminal indicating that the image annotation result of the target image pair fails.

[0096] Therefore, by verifying the image annotation results of the target image pairs through the annotation verification terminal, it can be ensured that the annotated image sample data is accurate.

[0097] Figure 10 This is a schematic diagram of the module connections of an image sample annotation apparatus according to an exemplary embodiment. Figure 10 As shown, this disclosure provides an image sample annotation apparatus, the apparatus 1000 comprising:

[0098] The display module 1010 is configured to display a pair of target images, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods;

[0099] Selection module 1020 is configured to simultaneously determine the image block to be labeled corresponding to the target image region in the first image and the second image in response to a selection operation of a target image region in any image of the target image pair.

[0100] The annotation module 1030 is configured to annotate the image block to be annotated and obtain the image annotation result of the target image pair.

[0101] Optionally, the device 1000 further includes:

[0102] The first determining module is configured to determine the first pixel similarity between the image block to be labeled in the first image and the adjacent images in response to an automatic selection instruction;

[0103] The second determining module is configured to determine a first image block in the first image based on the first pixel similarity, wherein the first image block includes the image block to be labeled in the first image and pixels whose first pixel similarity is greater than a first preset threshold;

[0104] The third determining module is configured to determine the second pixel similarity between the image block to be labeled in the second image and adjacent images;

[0105] The fourth determining module is configured to determine a second image block in the second image based on the second pixel similarity, wherein the second image block includes the image block to be labeled in the second image and pixels whose second pixel similarity is greater than a second preset threshold;

[0106] The fifth determining module is configured to determine a new image block to be labeled in the first image and the second image based on the first image block and the second image block.

[0107] Optionally, the annotation module 1030 includes:

[0108] The color annotation unit is configured to annotate the image block to be annotated based on the target color that matches the image semantic category to which the image block to be annotated belongs, thereby obtaining the image annotation result of the target image pair.

[0109] Optionally, the color marking unit is specifically configured as follows:

[0110] Determine the first similarity between the image block to be labeled in the first image and a preset image in the database;

[0111] Determine the second similarity between the image patch to be labeled in the second image and a preset image in the database;

[0112] Based on the first similarity and the second similarity, the image semantic category to which the image patch to be labeled belongs is determined;

[0113] The target color is determined based on the image semantic category.

[0114] Optionally, the device 1000 further includes:

[0115] The sending module is configured to send the image annotation results of the target image pair to the annotation verification terminal in response to the verification command, so that the annotation verification terminal can verify the image annotation results of the target image pair.

[0116] The storage module is configured to store the image annotation results in an image sample library in response to an instruction sent by the annotation verification terminal indicating that the image annotation results of the target image pair have passed.

[0117] Regarding the apparatus 1000 in the above embodiments, the method logic executed by each functional module has been described in detail in the section on methods, and will not be repeated here.

[0118] The following is for reference. Figure 11 The diagram illustrates a structural schematic of an electronic device 1100 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0119] like Figure 11 As shown, electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from storage device 1108 into random access memory (RAM) 1103. The RAM 1103 also stores various programs and data required for the operation of electronic device 1100. The processing device 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0120] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1109. Communication device 1109 allows electronic device 1100 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 An electronic device 1100 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0121] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1109, or installed from storage device 1108, or installed from ROM 1102. When the computer program is executed by processing device 1101, it performs the functions defined in the methods of embodiments of this disclosure.

[0122] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some implementations, electronic devices can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0124] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0125] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: display a pair of target images, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods; in response to a selection operation of a target image region in either image of the pair of target images, synchronously determine an image block to be labeled corresponding to the target image region in the first image and the second image; and label the image block to be labeled to obtain an image labeling result for the pair of target images.

[0126] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0129] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0130] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0132] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0133] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. An image sample annotation method, characterized in that, include: Display a pair of target images, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods; In response to a selection operation of a target image region for any image in the target image pair, an image block to be labeled corresponding to the target image region is simultaneously determined in the first image and the second image; The image blocks to be labeled are labeled to obtain the image labeling results of the target image pairs; After the synchronous determination of the image patch to be labeled corresponding to the target image region in the first image and the second image, the method further includes: In response to an automatic selection instruction, the first pixel similarity between the image block to be labeled in the first image and adjacent images is determined; Based on the first pixel similarity, a first image block is determined in the first image, wherein the first image block includes the image block to be labeled in the first image and pixels whose first pixel similarity is greater than a first preset threshold; Determine the second pixel similarity between the image patch to be labeled in the second image and its neighboring images; Based on the second pixel similarity, a second image block is determined in the second image, wherein the second image block includes the image block to be labeled in the second image and pixels whose second pixel similarity is greater than a second preset threshold; Based on the first image block and the second image block, a new image block to be labeled is determined in the first image and the second image.

2. The method according to claim 1, characterized in that, The step of annotating the image patch to be annotated to obtain the image annotation result of the target image pair includes: Based on the target color that matches the image semantic category to which the image block to be labeled belongs, the image block to be labeled is labeled to obtain the image labeling result of the target image pair.

3. The method according to claim 2, characterized in that, The target color is determined through the following steps: Determine the first similarity between the image block to be labeled in the first image and a preset image in the database; Determine the second similarity between the image patch to be labeled in the second image and a preset image in the database; Based on the first similarity and the second similarity, the image semantic category to which the image patch to be labeled belongs is determined; The target color is determined based on the image semantic category.

4. The method according to claim 1, characterized in that, The method further includes: In response to a verification command, the image annotation results of the target image pair are sent to the annotation verification terminal so that the annotation verification terminal can verify the image annotation results of the target image pair. In response to the instruction sent by the annotation verification terminal indicating that the image annotation result of the target image pair has passed, the image annotation result is stored in the image sample library.

5. An image sample annotation device, characterized in that, include: The display module is configured to display a pair of target images, wherein the pair of target images includes a first image and a second image obtained by shooting the same scene using different shooting methods; The selection module is configured to simultaneously determine the image block to be labeled corresponding to the target image region in the first image and the second image in response to a selection operation of a target image region in any image of the target image pair. The annotation module is configured to annotate the image blocks to be annotated, and obtain the image annotation results of the target image pairs; The device further includes: The first determining module is configured to determine the first pixel similarity between the image block to be labeled in the first image and the adjacent images in response to an automatic selection instruction; The second determining module is configured to determine a first image block in the first image based on the first pixel similarity, wherein the first image block includes the image block to be labeled in the first image and pixels whose first pixel similarity is greater than a first preset threshold. The third determining module is configured to determine the second pixel similarity between the image block to be labeled in the second image and adjacent images; The fourth determining module is configured to determine a second image block in the second image based on the second pixel similarity, wherein the second image block includes the image block to be labeled in the second image and pixels whose second pixel similarity is greater than a second preset threshold; The fifth determining module is configured to determine a new image block to be labeled in the first image and the second image based on the first image block and the second image block.

6. The apparatus according to claim 5, characterized in that, The annotation module includes: The color annotation unit is configured to annotate the image block to be annotated based on the target color that matches the image semantic category to which the image block to be annotated belongs, thereby obtaining the image annotation result of the target image pair.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processing device, the program implements the steps of the method according to any one of claims 1 to 4.

8. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 4.

Images