Data processing method and device, electronic device, and storage medium
By acquiring and fusion of pre-stored annotated image data sets, the problem of time-consuming and labor-intensive and poor accuracy in the image annotation process is solved, and an efficient and accurate annotation process is achieved, combining the data set to ensure the comprehensiveness and effectiveness of the annotation.
Patent Information
- Application Number
- CN202010100012.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-02-18
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2040-02-18
AI Technical Summary
The prior art has problems such as time-consuming and labor-intensive, manual participation, equipment errors and algorithm errors in the image labeling process, resulting in low labeling efficiency and poor accuracy.
By obtaining the pre-stored annotated image dataset, selecting the target image and pasting it in a specific area of the image to be marked, performing fusion processing to generate annotated images, avoiding the dependence of manual annotation and high-precision devices, and combining the first and second datasets to ensure pictures authenticity.
It has achieved manpower saving, improved labeling efficiency and accuracy, reduced labeling errors, and ensured the comprehensiveness and effectiveness of labeling.
Smart Images
Figure CN113342236B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of Internet technology, and in particular to a data processing method and device, an electronic device, and a storage medium. Background Art
[0002] With the rapid development and popularization of computer and Internet technologies, there is a huge amount of data on the Internet. These data are mainly in the form of pictures and other information carriers. Labeling these information carriers has become an important basis for computer data training.
[0003] In the existing technical solutions, when annotating pictures, it is mainly achieved through the following methods: a. Manually depicting the edges of objects of each category with polygons; b. Annotating through semi-manual methods, such as training partially annotated data samples to obtain a model, predicting the picture to be annotated through the model, obtaining the prediction result, and correcting the picture to be annotated with incorrect prediction result through manual correction; c. Annotating the picture to be annotated by processing depth map data, clustering and manual methods; d. Transferring the style of the target scene to the existing annotated picture through the style transfer method.
[0004] However, in the process of implementing the present disclosure, the inventors found that there are at least the following problems: it takes about 2 hours to annotate a picture using method a, and manual participation is required, so there is a problem of being time-consuming and labor-intensive; it takes about 1 hour to annotate a picture using method b, although there is an improvement compared to method a, it is still time-consuming and requires manual participation, which is labor-intensive; when method c is used, the acquisition of depth maps requires high-precision equipment, and it is difficult to avoid errors caused by the equipment itself during the acquisition process. Although a certain amount of time can be saved by combining clustering and manual work, there are still problems of labeling errors or incomplete labeling caused by the algorithm itself; when method d is used, since the acquired pictures are very different from the real pictures, it is likely to cause a negative impact on training. Summary of the invention
[0005] The present disclosure provides a data processing method and device, an electronic device, and a storage medium to solve at least one of the above technical problems.
[0006] In one aspect, an embodiment of the present disclosure provides a data processing method, the method comprising:
[0007] In response to receiving a request to annotate the first data set, obtaining a pre-stored second data set, wherein the second data set includes at least one annotated picture;
[0008] Selecting a target picture from the at least one picture according to the first data set;
[0009] Selecting a to-be-annotated picture from the first data set, and determining an annotated area of the to-be-annotated picture;
[0010] Pasting the target image according to the marked area;
[0011] The pasted pictures are fused to generate a marked picture so as to display the marked picture.
[0012] In some embodiments, before acquiring the pre-stored second data set, the method further includes:
[0013] Acquire a pre-stored third data set, wherein the third data set includes labeled initial images, and each of the initial images includes an object of at least one category;
[0014] Performing cropping processing on the objects corresponding to each category in each of the initial images to obtain cropped images;
[0015] The second data set is constructed according to the cropped images.
[0016] In some embodiments, the cropping of objects corresponding to each category in each of the initial images includes:
[0017] For an object corresponding to any category in each of the initial images, setting a pixel value of the object corresponding to any category to a first pixel value, and setting a pixel value of an object corresponding to another category to a second pixel value;
[0018] Determine a connected area corresponding to the first pixel value or the second pixel value;
[0019] The objects in the connected area are cropped.
[0020] In some embodiments, determining the annotated area of the picture to be annotated includes:
[0021] Dividing the image to be labeled into regions of preset sizes;
[0022] Obtaining location information of the target image in each image in the first data set;
[0023] A marked area is selected from the area according to the position information.
[0024] In some embodiments, pasting the target image according to the marked area includes:
[0025] Obtaining the center point of the target image;
[0026] The target image is pasted to the target area according to the center point.
[0027] In some embodiments, the target area includes a height range corresponding to the target image determined based on the position information, and pasting the target image to the target area according to the center point includes:
[0028] Selecting a positioning point in the target area;
[0029] Paste the target image to the target area, and place the center point and the positioning point at the same position;
[0030] Selecting a target height from the range of heights;
[0031] The target image is scaled according to the target height.
[0032] In some embodiments, the fusion processing of the pasted pictures includes:
[0033] The pasted images are fused according to Poisson fusion;
[0034] And, after generating the annotated image, the method further includes:
[0035] Gaussian blur processing is performed on the edge of the target image in the labeled image to obtain the target labeled image, so as to display the target labeled image.
[0036] In some embodiments, the at least one picture, the target picture, the initial picture and each of the cropped pictures respectively include a label picture and an RGB picture corresponding to the label picture.
[0037] On the other hand, an embodiment of the present disclosure further provides a data processing device, the device comprising:
[0038] A first acquisition module, configured to acquire a pre-stored second data set in response to receiving a request to annotate the first data set, wherein the second data set includes at least one annotated picture;
[0039] A selection module, configured to select a target image from the at least one image according to the first data set;
[0040] A determination module, configured to select a to-be-annotated picture from the first data set and determine an annotated area of the to-be-annotated picture;
[0041] A pasting module, used for pasting the target image according to the marked area;
[0042] The fusion module is used to perform fusion processing on the pasted pictures to generate annotated pictures so as to display the annotated pictures.
[0043] In some embodiments, the apparatus further comprises:
[0044] A second acquisition module is used to acquire a pre-stored third data set, wherein the third data set includes labeled initial images, and each of the initial images includes an object of at least one category;
[0045] A cropping module, used to crop objects corresponding to each category in each of the initial images to obtain cropped images;
[0046] A construction module is used to construct the second data set according to the cropped pictures.
[0047] In some embodiments, the cropping module is used to, for each object corresponding to any category in the initial image, set the pixel value of the object corresponding to any category to a first pixel value, and set the pixel value of the object corresponding to other categories to a second pixel value, determine the connected area corresponding to the first pixel value or the second pixel value, and crop the objects in the connected area.
[0048] In some embodiments, the determination module is used to divide the to-be-annotated image into regions of preset sizes, obtain position information of the target image in each image in the first data set, and select an annotated region from the regions according to the position information.
[0049] In some embodiments, the pasting module is used to obtain the center point of the target image and paste the target image to the target area according to the center point.
[0050] In some embodiments, the target area includes a height range corresponding to the target image determined based on the position information, and the pasting module is used to select a positioning point in the target area, paste the target image to the target area, and locate the center point and the positioning point at the same position, select a target height from the height range, and scale the target image according to the target height.
[0051] In some embodiments, the apparatus further comprises:
[0052] The fusion module is used to perform fusion processing on the pasted pictures according to Poisson fusion;
[0053] The processing module is used to perform Gaussian blur processing on the edge of the target image in the labeled image to obtain the target labeled image so as to display the target labeled image.
[0054] In some embodiments, the at least one picture, the target picture, the initial picture and each of the cropped pictures respectively include a label picture and an RGB picture corresponding to the label picture.
[0055] On the other hand, an embodiment of the present disclosure further provides an electronic device, including: a memory, a processor;
[0056] The memory is a memory for storing instructions executable by the processor;
[0057] When executing the instructions in the memory, the processor is configured to implement the method described in any of the above embodiments.
[0058] On the other hand, an embodiment of the present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in any of the above embodiments.
[0059] The present disclosure provides a method for obtaining a pre-stored second data set in response to receiving a request for annotating a first data set, wherein the second data set includes at least one annotated picture, selecting a target picture from at least one picture according to the first data set, selecting a picture to be annotated from the first data set, and determining an annotation area of the picture to be annotated, pasting the target picture according to the annotation area, and performing a fusion process on the pasted pictures to generate an annotated picture so as to display the annotated picture, and selecting a target picture from the annotated pictures so as to paste the target picture into the annotation area of the picture to be annotated, and performing a fusion process on the pasted pictures to obtain the annotated picture. On the one hand, the above-mentioned problems caused by manual annotation are avoided (for details, please refer to the background technology, which will not be repeated here), thereby achieving the technical effect of saving manpower and improving annotation efficiency; on the other hand, by pasting and fusion, there is no need to use high-precision acquisition equipment, thereby avoiding the problem of annotation errors, thereby achieving the technical effect of improving the accuracy and reliability of annotation; on the other hand, by combining the first data set and the second data set, it can be ensured that the difference between the acquired picture and the real picture is small, thereby achieving the technical effects of comprehensiveness and effectiveness of annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0061] Figure 1 A schematic diagram of a scenario of a data processing method according to an embodiment of the present disclosure;
[0062] Figure 2 A flowchart of a data processing method according to an embodiment of the present disclosure;
[0063] Figure 3 A schematic diagram of a picture in a first data set according to an embodiment of the present disclosure;
[0064] Figure 4 is a schematic diagram of a target image according to an embodiment of the present disclosure;
[0065] Figure 5 A schematic diagram of annotated pictures according to an embodiment of the present disclosure;
[0066] Figure 6 A schematic diagram of a display interface of a terminal according to an embodiment of the present disclosure;
[0067] Figure 7 A flowchart of a data processing method according to another embodiment of the present disclosure;
[0068] Figure 8 It is a module schematic diagram of a data processing device according to an embodiment of the present disclosure;
[0069] Fig. 9 A schematic diagram of a module of a data processing device according to another embodiment of the present disclosure;
[0070] Fig.10 A schematic diagram of a module of a data processing device according to another embodiment of the present disclosure;
[0071] Figure numerals: 10. user, 20. terminal, 30. server, 1. first acquisition module, 2. selection module, 3. determination module, 4. pasting module, 5. fusion module, 6. second acquisition module, 7. cropping module, 8. construction module, 9. processing module.
[0072] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0073] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0074] The data processing method provided in the embodiment of the present disclosure can be applied to Figure 1The scene shown.
[0075] Among them, the executor of the data processing method of the embodiment of the present disclosure can be a data processing device, and the data processing device can be a terminal, such as a computer, iPad, and a car terminal, or a server, such as a cloud server and a local server.
[0076] In such Figure 1 In the application scenario shown, the data processing device is a server.
[0077] For example, a user (such as an operator) 10 inputs a first data set into the terminal 20 , and sends a request for labeling the first data set to the terminal 20 .
[0078] The user terminal 20 sends the first data set and the request to the server 30 .
[0079] The server 30 executes the data processing method of the embodiment of the present disclosure to implement labeling of the first data set and obtain a labeled image.
[0080] In some embodiments, the server 30 feeds back the annotated image to the terminal 20 .
[0081] The terminal 20 displays the annotated image.
[0082] In some embodiments, the first data set may be acquired by an image acquisition device and the first data set may be transmitted to the terminal 20 .
[0083] Among them, the image acquisition device includes a device with photo or video recording functions, and also includes a laser radar, etc.
[0084] In some embodiments, the data processing method of the embodiments of the present disclosure can also be used in application scenarios such as autonomous driving, robotics, human-computer interaction, remote sensing, etc., which will not be described in detail here.
[0085] The following specific embodiments are used to describe in detail the technical solution of the present invention and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.
[0086] In one aspect, an embodiment of the present disclosure provides a data processing method suitable for the above scenario.
[0087] See also Figure 2 , Figure 2 The figure is a flowchart of the data processing method according to the embodiment of the present disclosure.
[0088] like Figure 2 As shown, the method includes:
[0089] S101: In response to receiving a request to annotate a first data set, obtaining a pre-stored second data set, wherein the second data set includes at least one annotated picture.
[0090] Based on the above examples, it can be seen that the subject that executes the data processing method of the embodiment of the present disclosure can be a terminal or a server. In the above examples, the exemplary description is made using the server as an example. In the embodiment of the present disclosure, the execution subject will be explained as a terminal.
[0091] The first data set includes multiple pictures.
[0092] The second data set may be a set of annotated pictures stored in a memory of the terminal, or may be a set of annotated pictures stored in a cloud server or other device.
[0093] If the second data set is a set of marked pictures stored in the memory of the terminal, when the terminal receives a request initiated by the user or an external device, the second data set is extracted from the memory for subsequent processing.
[0094] If the second data set is a collection of annotated images stored in a cloud server or other device, when the terminal receives a request initiated by a user or an external device, an acquisition instruction is initiated to the cloud server or other device, and the acquisition instruction carries an identifier of the second data set. After receiving the acquisition instruction, the cloud server or other device determines the storage location of the second data set in the cloud server or other device based on the identifier, acquires the second data set from the determined storage location, and sends the second data set to the terminal for subsequent processing by the terminal.
[0095] That is, if the second data set is a collection of labeled images stored in a cloud server or other device, a communication link between the cloud server or other device and the terminal is established in advance, and considering that the cloud server or other device may also store other data sets, each data set (including the second data set) is identified in advance, and a mapping relationship between the identification and the storage location can be constructed, so that after the cloud server or other device determines the identification, it can quickly determine the storage location based on the identification and the mapping relationship, thereby quickly obtaining the second data set from the storage location and feeding it back to the terminal. The identification can be an ID.
[0096] S102: Select a target picture from at least one picture according to the first data set.
[0097] In some embodiments, the first data set includes a plurality of pictures, each picture including at least one object. Figure 3 , Figure 3 This is a schematic diagram of a picture in the first data set of an embodiment of the present disclosure, where the picture includes multiple objects, such as n cars, where one car is one object; m trees, where one tree is one object; a house, where one house is one object, and so on.
[0098] In some embodiments, the objects included in the first data set include at least an object corresponding to the target image.
[0099] In some embodiments, this step may include: performing a category traversal process on each image in the first data set to determine all categories of the first data set, and selecting an image corresponding to any category in all categories from at least one image to determine a target image. Figure 4 , Figure 4 A schematic diagram of a target image according to an embodiment of the present disclosure.
[0100] For example, a first data set includes a pictures, and a category traversal process is performed on each of the a pictures, that is, the category corresponding to each of the a pictures is determined to obtain all categories of the first data set, and a picture corresponding to any category of all categories is randomly selected from at least one picture.
[0101] In some embodiments, the same categories may exist in all the obtained categories. In this case, before selecting the target image, the same categories in all the categories are filtered to reduce the amount of calculation.
[0102] S103: Select a picture to be labeled from the first data set, and determine a labeling area of the picture to be labeled.
[0103] In this step, the image to be annotated is determined, and the annotated area is determined on the image to be annotated.
[0104] The images to be labeled may be randomly determined, or the images in the first data set may be sorted (e.g., sorted based on the number of objects in each image, and the sorting may include sorting in descending order based on the number from large to small, or sorting in ascending order based on the number from small to large), to obtain a sorting result, and the images to be labeled are selected in sequence based on the sorting result. In some embodiments, the images to be labeled may refer to Figure 3 .
[0105] The image to be labeled may include the target image or may not include the target image.
[0106] S104: Paste the target image according to the marked area.
[0107] In this step, the target image is pasted into the annotation area of the image to be annotated.
[0108] Based on the above example, Figure 4 Paste the picture shown in Figure 3 In the picture shown, and paste it into Figure 3 The marked area in the picture shown (not shown in the figure).
[0109] S105: performing fusion processing on the pasted pictures to generate an annotated picture so as to display the annotated picture.
[0110] In this step, after the target is pasted, the pasted image is fused so that the target image is fused with the image to be labeled to obtain a labeled image.
[0111] Based on the above example, Figure 4 Paste the picture shown in Figure 3 After the picture shown in the figure, the pasted picture is fused to generate the following Figure 5 The marked picture shown in the figure is displayed. The display interface diagram of the terminal can be seen in Figure 6 .
[0112] In an embodiment of the present disclosure, a new data processing method is provided, the method comprising: in response to receiving a request to annotate a first data set, obtaining a pre-stored second data set, wherein the second data set includes at least one annotated picture, selecting a target picture from at least one picture according to the first data set, selecting a picture to be annotated from the first data set, and determining an annotated area of the picture to be annotated, pasting the target picture according to the annotated area, performing a fusion process on the pasted pictures, and generating an annotated picture so as to display the annotated picture, selecting a target picture from the annotated pictures so as to paste the target picture into the annotated area of the picture to be annotated, and performing a fusion process on the pasted pictures to obtain the annotated picture, on the one hand, avoiding the above-mentioned problems caused by manual annotation (for details, please refer to the background technology, which will not be repeated here), thereby achieving the technical effect of saving manpower and improving annotation efficiency; on the other hand, by pasting and fusion, there is no need to use a high-precision acquisition device, thereby avoiding the problem of annotation errors, thereby achieving the technical effect of improving the accuracy and reliability of annotation; on the other hand, by combining the first data set and the second data set, it can be ensured that the difference between the acquired picture and the real picture is small, thereby achieving the technical effects of comprehensiveness and effectiveness of annotation.
[0113] See also Figure 7 , Figure 7 The figure is a flowchart of a data processing method according to another embodiment of the present disclosure.
[0114] Combination Figure 7 It can be seen that the method includes:
[0115] S01: Acquire a pre-stored third data set, wherein the third data set includes labeled initial pictures, and each initial picture includes an object of at least one category.
[0116] Similarly, the third data set may be a set of labeled pictures stored in the memory of the terminal, or may be a set of labeled initial pictures stored in a cloud server or other device.
[0117] The third data includes a plurality of initial images, each of which includes objects of one or more categories. Figure 3 As shown, the objects include every car, every tree and every house, so the categories can be divided into three, namely vehicle category, tree category and house category.
[0118] Of course, the categories can also be adaptively divided or adjusted based on demand, such as dividing trees and grass (not shown in the figure) into one category; or dividing vehicles of different brands into different categories, and so on.
[0119] S02: Perform cropping processing on the objects corresponding to each category in each initial image to obtain cropped images.
[0120] In some embodiments, for each initial image and for a certain category in the initial image, each object of the category is cropped respectively.
[0121] For example, for the initial image a, the initial image a includes 3 categories, one of which is the vehicle category, and the initial image a includes 3 vehicles. That is to say, for the vehicle category in the initial image a, there are 3 objects in total, and the 3 objects are cropped respectively, that is, 3 images are obtained after cropping.
[0122] In some other embodiments, for each initial image and for a certain category in the initial image, all objects of the category are cropped as a whole.
[0123] For example, for the initial image a, the initial image a includes 3 categories, one of which is the vehicle category, and the initial image a includes 3 vehicles. That is to say, for the vehicle category in the initial image a, there are 3 objects in total, and the 3 objects are cropped as a whole, that is, after cropping, a picture including the 3 objects is obtained.
[0124] S03: Construct a second data set according to the cropped images.
[0125] Based on the above examples, it can be known that the second data set includes multiple cropped pictures, and each picture includes an object of one category. The number of objects can be multiple or one.
[0126] In some embodiments, S03 may include: performing deduplication processing on each cropped image and constructing a second data set. In this embodiment, by performing deduplication processing on each cropped image, the amount of subsequent calculations can be reduced and storage space can be released.
[0127] S201: In response to receiving a request to annotate a first data set, obtaining a pre-stored second data set, wherein the second data set includes at least one annotated picture.
[0128] The description of S201 may refer to S101, and the at least one picture is each cropped picture in the above example, or each picture after deduplication processing.
[0129] S202: Select a target picture from at least one picture according to the first data set.
[0130] The description of S202 may refer to S102.
[0131] S203: Select a picture to be labeled from the first data set, and determine a labeling area of the picture to be labeled.
[0132] The description of S203 may refer to S103.
[0133] S204: Paste the target image according to the marked area.
[0134] The description of S204 may refer to S104.
[0135] S205: performing fusion processing on the pasted pictures to generate annotated pictures so as to display the annotated pictures.
[0136] The description of S205 may refer to S105.
[0137] In some embodiments, S02 includes:
[0138] S021: For an object corresponding to any category in each initial image, set a pixel value of the object corresponding to any category to a first pixel value, and set pixel values of objects corresponding to other categories to a second pixel value.
[0139] Based on the above example, for the initial picture a, the initial picture a includes three categories, namely vehicles, trees and houses. For the vehicle category, the pixel value of the car corresponding to the vehicle category is set to the first pixel value, and the pixel values of the tree category and the house category objects are set to the second pixel value.
[0140] The first pixel value and the second pixel value are different pixel values.
[0141] In some embodiments, the first pixel value is 255 and the second pixel value is 0, so as to quickly and accurately distinguish objects of different categories.
[0142] S022: Determine a connected area corresponding to the first pixel value or the second pixel value.
[0143] The connected areas refer to all areas corresponding to the first pixel value.
[0144] For example: there are 3 vehicles in the initial picture a, then there are 3 objects for the vehicle category, and the pixel values of the 3 objects are all set to the first pixel value. The area of the first pixel value is the sum of the areas to which each of the 3 objects belongs, and the total area of the areas to which the 3 objects belong is the connected area.
[0145] For another example: there is 1 car in the initial image a, then for the vehicle category, there is one object, the pixel value of the object is set to the first pixel value, and the area of the first pixel value is the area to which the object belongs, that is, the area to which the object belongs is the connected area.
[0146] It is worth noting that if the first pixel value is set to 255 and the second pixel value is 0, then based on the display characteristics of the pixel value (existing technology, no longer repeated here), in this step, it is preferred to determine the connected area corresponding to the first pixel value in order to reduce the subsequent conversion calculation amount and time.
[0147] S023: Crop the objects in the connected area.
[0148] In the embodiments of the present disclosure, by setting pixel values to crop corresponding objects, the technical effects of convenience, efficiency and accuracy of cropping can be achieved.
[0149] In some embodiments, determining the annotation area of the image to be annotated includes:
[0150] S1031: Divide the image to be annotated into regions of preset sizes.
[0151] In some embodiments, this step may include: dividing the image to be labeled into grids of a preset size.
[0152] For example: the size of the image to be labeled is H*W, where H is the length of the image to be labeled (the unit can be centimeters, etc.), and W is the width of the image to be labeled (similarly, the unit can also be centimeters, etc.). The image to be labeled is divided into n*n squares, and the number of squares is N, then N=(H / n)*(W / n).
[0153] S1032: Obtain location information of the target image in each image in the first data set.
[0154] In some embodiments, this step may include: determining a picture including an object corresponding to the target image from the first data set, and acquiring position information of the object corresponding to the target image from the determined picture.
[0155] Based on the above example, it can be known that at least some of the images in the first data set include objects corresponding to the target image. If the target image is a motorcycle, then at least some of the images in the first data set also include the motorcycle, and the location information of the motorcycle is determined based on the images in the first data set that include the motorcycle.
[0156] For example, if the target image is a motorcycle, and there are x images in the first data set, including y images of motorcycles, the position information of the motorcycle in each of the y images is obtained. The position information is the position information of the motorcycle relative to the image to which it belongs, including the height, width and other information of the motorcycle, as well as the position information between the motorcycle and other objects. In other words, the position information includes the position information of the object itself and the relative position information.
[0157] S1033: Select a marked area from the area according to the location information.
[0158] After determining the location information, the annotation area can be selected based on the location information. Since the location information is determined based on the real picture, that is, the location information has high authenticity and reliability, therefore, by selecting the annotation area based on the location information, the technical effect of improving the authenticity and reliability of the annotated picture can be achieved.
[0159] In some embodiments, S104 includes:
[0160] S1041: Obtain the center point of the target image.
[0161] S1042: Paste the target image to the target area according to the center point.
[0162] In the disclosed embodiment, by pasting the target image through the determined center point, deformation of the target image during the pasting process can be avoided, so as to ensure the reliability of the pasting process, thereby achieving the technical effect of subsequently obtaining a highly reliable annotated image.
[0163] In some embodiments, the target area includes a height range corresponding to the target image determined based on the position information, and S1042 includes:
[0164] S10421: Select a positioning point in the target area.
[0165] It is understandable that in different pictures, the height information of the same object may be inconsistent due to reasons such as the distance at which the picture is acquired or the parameters of the picture acquisition device, and the height range of the object can be determined based on the inconsistent height information.
[0166] Therefore, based on the above example, the target area is the determined area for pasting the target image, and the height range of the motorcycle can be determined based on the position information, and the height range can be determined based on the height information of the motorcycle in each of the y images.
[0167] In some embodiments, any point in the target area may be determined as a positioning point; in other embodiments, the positioning point may also be determined based on the position where the motorcycle appears most frequently in each of the y pictures.
[0168] S10422: Paste the target image to the target area, and place the center point and the positioning point at the same position.
[0169] In this step, the target image is pasted to the target area according to the center point and the positioning point, and the center point is aligned with the positioning point, that is, the center point of the pasted target image is the positioning point.
[0170] S10423: Select target altitude from altitude range.
[0171] In some embodiments, any height in the height range may be selected as the target height; in other embodiments, the height with the highest frequency may be selected as the target height.
[0172] S10424: Scale the target image according to the target height.
[0173] In some embodiments, to ensure that the target image is not distorted during the scaling process, the target image is scaled while maintaining the aspect ratio of the target image until the target image is scaled to a target height.
[0174] In some embodiments, the center point is kept unchanged during the scaling process, that is, the center point after the scaling process is the same as the center point before the scaling process.
[0175] In some embodiments, performing fusion processing on the pasted image includes:
[0176] S1051: Perform fusion processing on the pasted images according to Poisson fusion.
[0177] In the disclosed embodiment, the pasted pictures are fused through Poisson fusion, so that the target picture can be embedded into the picture to be annotated, and a seamlessly combined annotated picture can be generated to improve the authenticity of the annotated picture.
[0178] And, after S1051, the method further includes:
[0179] S1052: Perform Gaussian blur processing on the edge of the target image in the annotated image to obtain the target annotated image, so as to display the target annotated image.
[0180] In order to improve the realism of the annotated image, Gaussian blur processing is performed on the edge of the target image in the annotated image, and based on the Gaussian blur processing on the edge of the target image in the annotated image, the fit between the annotated image and the image to be annotated can be improved.
[0181] In some embodiments, at least one picture, the target picture, the initial picture, and each of the cropped pictures respectively include a label picture and an RGB picture corresponding to the label picture.
[0182] It can be understood that when annotating pictures, generally label pictures and RGB pictures are included, and the label pictures and RGB pictures are in a one-to-one correspondence, that is, one label picture corresponds to one RGB picture, and the annotation process also includes labeling of label pictures and RGB pictures.
[0183] In the disclosed embodiment, when cropping the objects corresponding to each category in each initial image, the label image and the RGB image can be cropped respectively through the above example. However, since the label image and the RGB image are in a one-to-one correspondence, in order to improve efficiency and save costs, the label image can be cropped based on the above example, and the RGB image can be cropped based on the correspondence between the label image and the RGB image.
[0184] For example, through the above example, an object in the label image is cropped, the position of the object in the RGB image is determined, and the object in the RGB image is cropped based on the determined position.
[0185] Among them, the pasting and scaling of label images and RGB images can be found in the cropping process, which will not be repeated here.
[0186] According to another aspect of an embodiment of the present disclosure, an embodiment of the present disclosure further provides a data processing device.
[0187] See also Figure 8 , Figure 8 It is a module schematic diagram of the data processing device of an embodiment of the present disclosure.
[0188] like Figure 8 As shown, the device comprises:
[0189] A first acquisition module 1 is configured to acquire a pre-stored second data set in response to receiving a request to annotate a first data set, wherein the second data set includes at least one annotated picture;
[0190] A selection module 2, configured to select a target image from the at least one image according to the first data set;
[0191] A determination module 3 is used to select a picture to be labeled from the first data set and determine a labeling area of the picture to be labeled;
[0192] Pasting module 4, used for pasting the target image according to the marked area;
[0193] The fusion module 5 is used to perform fusion processing on the pasted pictures to generate annotated pictures so as to display the annotated pictures.
[0194] Combination Fig. 9 It can be seen that, in some embodiments, the device further includes:
[0195] A second acquisition module 6 is used to acquire a pre-stored third data set, wherein the third data set includes labeled initial images, and each of the initial images includes an object of at least one category;
[0196] A cropping module 7, used for cropping the objects corresponding to each category in each of the initial images to obtain cropped images;
[0197] The construction module 8 is used to construct the second data set according to the cropped pictures.
[0198] In some embodiments, the cropping module 7 is used to, for each object corresponding to any category in the initial image, set the pixel value of the object corresponding to any category to a first pixel value, and set the pixel value of the object corresponding to other categories to a second pixel value, determine the connected area corresponding to the first pixel value or the second pixel value, and crop the objects in the connected area.
[0199] In some embodiments, the determination module 3 is used to divide the to-be-annotated picture into regions of preset sizes, obtain position information of the target picture in each picture in the first data set, and select an annotated region from the regions according to the position information.
[0200] In some embodiments, the pasting module 4 is used to obtain the center point of the target image and paste the target image to the target area according to the center point.
[0201] In some embodiments, the target area includes a height range corresponding to the target image determined based on the position information, and the pasting module 4 is used to select a positioning point in the target area, paste the target image to the target area, and locate the center point and the positioning point at the same position, select a target height from the height range, and scale the target image according to the target height.
[0202] Combination Fig.10 It can be seen that, in some embodiments, the device further includes:
[0203] The fusion module 5 is used to perform fusion processing on the pasted pictures according to Poisson fusion;
[0204] The processing module 9 is used to perform Gaussian blur processing on the edge of the target image in the annotated image to obtain the target annotated image, so as to display the target annotated image.
[0205] In some embodiments, the at least one picture, the target picture, the initial picture and each of the cropped pictures respectively include a label picture and an RGB picture corresponding to the label picture.
[0206] According to another aspect of the embodiments of the present disclosure, the embodiments of the present disclosure further provide an electronic device, including: a memory, a processor;
[0207] The memory is a memory for storing instructions executable by the processor;
[0208] When executing the instructions in the memory, the processor is configured to implement the method described in any of the above embodiments.
[0209] According to another aspect of an embodiment of the present disclosure, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in any of the above embodiments.
[0210] The reader should understand that in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.
[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0212] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0213] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.
[0214] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0215] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0216] It should also be understood that in the embodiments of the present disclosure, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.
[0217] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A data processing method, characterized in that: The method comprises: In response to receiving a request to annotate the first data set, obtaining a pre-stored second data set, wherein the second data set includes at least one annotated picture; Processing each picture in the first data set by category traversal, determining all categories of the first data set, and selecting a target picture corresponding to any category among all categories from the at least one picture; Selecting a to-be-annotated picture from the first data set, and determining an annotated area of the to-be-annotated picture; Paste the target image into the annotation area of the image to be annotated; The pasted picture is fused to fuse the target picture with the picture to be annotated to generate an annotated picture, so as to display the annotated picture.
2. The method according to claim 1, characterized in that Before acquiring the pre-stored second data set, the method further includes: Acquire a pre-stored third data set, wherein the third data set includes labeled initial images, and each of the initial images includes an object of at least one category; Performing cropping processing on the objects corresponding to each category in each of the initial images to obtain cropped images; The second data set is constructed according to the cropped images.
3. The method according to claim 2, characterized in that The cropping of the objects corresponding to each category in each of the initial images comprises: For an object corresponding to any category in each of the initial images, setting a pixel value of the object corresponding to any category to a first pixel value, and setting a pixel value of an object corresponding to another category to a second pixel value; Determine a connected area corresponding to the first pixel value or the second pixel value; The objects in the connected area are cropped.
4. The method according to any one of claims 1 to 3, characterized in that Determining the annotation area of the image to be annotated includes: Dividing the image to be labeled into regions of preset sizes; Obtaining location information of the target image in each image in the first data set; A marked area is selected from the area according to the position information.
5. The method according to claim 4, characterized in that The step of pasting the target image to the annotation area of the image to be annotated includes: Obtaining the center point of the target image; The target image is pasted to the marked area according to the center point.
6. The method according to claim 5, characterized in that The marked area includes a height range corresponding to the target image determined based on the position information, and pasting the target image to the marked area according to the center point includes: Selecting a positioning point in the marked area; Paste the target image into the marked area, and place the center point and the positioning point at the same position; Selecting a target height from the range of heights; The target image is scaled according to the target height.
7. The method according to any one of claims 1 to 3, characterized in that The fusion processing of the pasted pictures includes: The pasted images are fused according to Poisson fusion; And, after generating the annotated image, the method further includes: Gaussian blur processing is performed on the edge of the target image in the labeled image to obtain the target labeled image, so as to display the target labeled image.
8. A data processing device, characterized in that: The device comprises: A first acquisition module, configured to acquire a pre-stored second data set in response to receiving a request to annotate the first data set, wherein the second data set includes at least one annotated picture; A selection module, configured to traverse and process each image in the first data set by category, determine all categories of the first data set, and select a target image corresponding to any category among all categories from the at least one image; A determination module, configured to select a to-be-annotated picture from the first data set and determine an annotated area of the to-be-annotated picture; A pasting module, used for pasting the target image into the annotation area of the image to be annotated; The fusion module is used to perform fusion processing on the pasted picture, fuse the target picture with the picture to be annotated, and generate an annotated picture so as to display the annotated picture.
9. An electronic device, comprising: Memory, processor; The memory is a memory for storing instructions executable by the processor; Wherein, when executing the instructions in the memory, the processor is configured to implement the method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.