Image annotation method and device

Abstract

An embodiment of the present application provides an image annotation method and apparatus, the method comprising: obtaining multiple first images containing a target object, wherein the multiple first images are at least shot from different orientations; extracting multiple target images from the multiple first images, wherein the multiple target images contain two-dimensional annotation information for the target object; determining three-dimensional annotation information for the target object based at least on the two-dimensional annotation information in the multiple target images; and determining two-dimensional annotation information corresponding to the target object in each second image based at least on the three-dimensional annotation information, wherein the second image is an unannotated first image. The image annotation method and apparatus of the present application can simply and quickly perform two-dimensional annotation on target objects in multiple images.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to an image annotation method and apparatus. Background Art

[0002] The development of AI technology has entered a new phase. Focus is not only on improving the effectiveness and efficiency of AI algorithms, but also on their engineering deployment and resolving various challenges inherent in practical deployment. AI training requires a large amount of labeled training data. However, labeling training data is labor-intensive and prone to errors, making it a major pain point in the engineering deployment of AI algorithms. Summary of the Invention

[0003] The present application provides an image annotation method and apparatus that can simply and quickly perform two-dimensional annotation on target objects in multiple images.

[0004] In order to solve the above technical problems, an embodiment of the present application provides an image annotation method, which includes:

[0005] Obtaining a plurality of first images containing a target object, wherein the plurality of first images are at least shot in different orientations;

[0006] Extracting a plurality of target images from the plurality of first images, wherein the plurality of target images have two-dimensional annotation information for the target object;

[0007] determining three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the plurality of target images;

[0008] Two-dimensional annotation information corresponding to the target object in each second image is determined based at least on the three-dimensional annotation information, where the second image is the unannotated first image.

[0009] As an optional embodiment, the two-dimensional annotation information is a 2D bounding box, and the three-dimensional annotation information is a 3D bounding box, wherein determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes:

[0010] determining, based at least on the 3D bounding box, a candidate 2D bounding box corresponding to each second image;

[0011] The shapes of the candidate 2D bounding boxes are adjusted based on the image area enclosed by each candidate 2D bounding box, so as to obtain a target 2D bounding box corresponding to each second image.

[0012] As an optional embodiment, adjusting the shape of each candidate 2D bounding box based on the image area enclosed by each candidate 2D bounding box includes:

[0013] The image area enclosed by each candidate 2D bounding box is input into the trained target network to adjust the shape of each candidate 2D bounding box accordingly.

[0014] As an optional embodiment, the method further includes:

[0015] receiving a labeling instruction, where the labeling instruction is used to label at least one of the plurality of first images;

[0016] The marked first image is determined as the target image.

[0017] As an optional embodiment, determining the three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the multiple target images includes:

[0018] Acquiring first shooting orientation information of each target image relative to the target object;

[0019] The three-dimensional annotation information of the target object is determined based on the two-dimensional annotation information in the multiple target images and the first shooting orientation information.

[0020] As an optional embodiment, determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes:

[0021] determining at least second shooting orientation information of each of the second images relative to the target object;

[0022] Two-dimensional annotation information corresponding to each second image is determined based on the second shooting orientation information and the three-dimensional annotation information.

[0023] As an optional embodiment, determining the two-dimensional annotation information corresponding to each second image based on the second shooting orientation information and the three-dimensional annotation information includes:

[0024] determining a projection relationship between each of the second images and the target object based at least on the second shooting orientation information;

[0025] determining a projection of the three-dimensional annotation information on each of the second images based on each of the projection relationships;

[0026] Two-dimensional annotation information corresponding to each second image is determined based on each of the projections.

[0027] Another embodiment of the present application provides an image annotation device, comprising:

[0028] An acquisition module, configured to acquire a plurality of first images containing a target object, wherein the plurality of first images are at least shot in different orientations;

[0029] an extraction module, configured to extract a plurality of target images from the plurality of first images, wherein the plurality of target images have two-dimensional annotation information for the target object;

[0030] A first determining module, configured to determine three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the plurality of target images;

[0031] The second determining module is configured to determine, based at least on the three-dimensional annotation information, the two-dimensional annotation information corresponding to the target object in each second image, where the second image is the unannotated first image.

[0032] As an optional embodiment, the two-dimensional annotation information is a 2D bounding box, and the three-dimensional annotation information is a 3D bounding box, wherein determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes:

[0033] determining, based at least on the 3D bounding box, a candidate 2D bounding box corresponding to each second image;

[0034] The shapes of the candidate 2D bounding boxes are adjusted based on the image area enclosed by each candidate 2D bounding box, so as to obtain a target 2D bounding box corresponding to each second image.

[0035] As an optional embodiment, adjusting the shape of each candidate 2D bounding box based on the image area enclosed by each candidate 2D bounding box includes:

[0036] The image area enclosed by each candidate 2D bounding box is input into the trained target network to adjust the shape of each candidate 2D bounding box accordingly.

[0037] Based on the disclosure of the above embodiments, it can be known that the beneficial effects of the embodiments of the present application include obtaining multiple first images containing target objects and shot in different orientations, then extracting a small number of target images with two-dimensional annotation information corresponding to the target objects from the first images, and then using the two-dimensional annotation information on the small number of target images to determine the three-dimensional annotation information corresponding to the target objects, and finally directly processing the second image that is not marked with the two-dimensional annotation information of the target object based on the three-dimensional annotation information so that the second image contains the two-dimensional annotation information. The method based on the present application can speed up the efficiency of image annotation, reduce the amount of manual annotation and the difficulty of annotation, and at the same time unify the annotation standards and reduce the errors caused by manual annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart of the image annotation method in an embodiment of the present application.

[0039] Figure 2 This is a flowchart of an image annotation method in another embodiment of the present application.

[0040] Figure 3 This is a flowchart of an image annotation method in another embodiment of the present application.

[0041] Figure 4 Schematic diagram of the method for determining three-dimensional annotation information based on two-dimensional annotation information in this application.

[0042] Figure 5 2 is a structural block diagram of an image annotation device in an embodiment of the present application. DETAILED DESCRIPTION

[0043] Below, specific embodiments of the present application are described in detail with reference to the accompanying drawings, but are not intended to limit the present application.

[0044] It should be understood that various modifications may be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope and spirit of the present disclosure will occur to those skilled in the art.

[0045] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0046] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0047] It should also be understood that although the present application has been described with reference to certain specific examples, those skilled in the art will be able to implement many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0048] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0049] Specific embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant detail. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but rather serve merely as a basis and representative basis for the claims to teach those skilled in the art to variously employ the present disclosure with substantially any suitable detailed structure.

[0050] This description may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," each of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0051] Below, the embodiments of the present application are described in detail with reference to the accompanying drawings.

[0052] like Figure 1 As shown, an embodiment of the present application provides an image annotation method, which includes:

[0053] S101: Acquire a plurality of first images containing a target object, wherein the plurality of first images are at least shot in different orientations;

[0054] S201: extracting a plurality of target images from a plurality of first images, wherein the plurality of target images have two-dimensional annotation information for a target object;

[0055] S301: Determine three-dimensional annotation information of a target object based on at least two-dimensional annotation information in a plurality of target images;

[0056] S401: Determine two-dimensional annotation information corresponding to a target object in each second image based at least on the three-dimensional annotation information, where the second image is the unannotated first image.

[0057] For example, a user can use a device with an acquisition function, such as a mobile phone, tablet, camera, smart glasses, AR device, etc., to shoot the target object at different shooting directions, and obtain multiple first images that all contain the target object. Alternatively, the user can directly obtain the acquired first image based on other methods. Then, the user selects and extracts a small number of target images from the multiple first images, for example, 3, 4, 5, etc., whichever are greater. The shooting directions of the multiple target images are different, and preferably the angle difference between the shooting directions is large, so that the images of the target object at different angles can be reflected based on the multiple target images, for example, with the center of the target object, the angle between two adjacent shooting directions is an obtuse angle, etc. If the angle difference between the shooting directions is small, it may cause the error of the two-dimensional annotation information to be too large, thereby causing the subsequent three-dimensional annotation information to also have large errors. When selecting the target image, the shooting direction information of the specific image obtained by the device can be combined, or the internal and external parameters of the acquisition device can be combined.

[0058] Furthermore, in this embodiment, the target image records two-dimensional annotation information of the target object. This two-dimensional annotation information can be manually annotated or annotated by other devices or applications. This two-dimensional annotation information can be detailed information of the target object as a whole, or rough information of the target object as a whole, such as two-dimensional annotation information of the target object's external graphics, or information about the length, width, and height of the target object as a whole. It can also be annotation information of a local area of ​​the target object, such as annotation information of a region of interest such as a defect area. After the device determines the two-dimensional annotation information of the target image, it can determine three-dimensional annotation information corresponding to the target object as a whole or a part based on the two-dimensional annotation information. Based on this three-dimensional annotation information, the overall three-dimensional structure of the target object can be obtained, or a three-dimensional structure covering the entire target object can be obtained, or the three-dimensional structure of the local area of ​​the target object or the three-dimensional structure covering the local area can be obtained, etc. The specific details are not unique and are determined based on the specific two-dimensional annotation information. After obtaining the three-dimensional annotation information of the target object, the device can at least combine the three-dimensional annotation information to quickly annotate the second image that is not annotated with the two-dimensional annotation information in the first image, so that the second image and the target image's area annotated with the second annotation information also have the second annotation information. If the entire or partial target image is annotated with secondary annotation information, the device can automatically perform adaptive 2D annotation for the entire or partial target object in the second image based on the aforementioned 3D annotation information. Because different first images have different 2D annotation information due to different shooting orientations, the device will overcome this issue and complete the 2D annotation.

[0059] Based on the disclosure of the above embodiment, it can be known that the beneficial effects of this embodiment include obtaining multiple first images containing target objects and shot in different orientations, then extracting a small number of target images with two-dimensional annotation information corresponding to the target objects from the first images, and then using the two-dimensional annotation information on the small number of target images to determine the three-dimensional annotation information corresponding to the target objects, and finally directly processing the second image that is not marked with the two-dimensional annotation information of the target object based on the three-dimensional annotation information so that the second image contains the two-dimensional annotation information. The above process is simple and fast. Based on the above method, the efficiency of image annotation can be accelerated, the amount of manual annotation and the difficulty of annotation can be reduced, and the annotation standards can be unified, reducing the errors caused by manual annotation.

[0060] Furthermore, the method in this embodiment also includes:

[0061] S501: Receive a labeling instruction, where the labeling instruction is used to label at least one of a plurality of first images;

[0062] S502: Determine the marked first image as the target image.

[0063] For example, the device may have a labeling function, such as one implemented using labeling software. A user can input labeling instructions into the device to inform the device which first images require two-dimensional labeling. Specifically, the device may number the first images, and the user can use labeling instructions to inform the device that certain numbered first images require two-dimensional labeling. Alternatively, the user can enter characteristic information of the first images to be labeled into the labeling instructions, causing the device to identify, analyze, and filter multiple first images based on the characteristic information, ultimately determining which first images meet the characteristic information for two-dimensional labeling. This characteristic information may include, for example, one or more parameters such as the image's shooting orientation, the angle and positional difference between the shooting orientations, the angle of the target object relative to the image in the image, and the content characteristics of the area to be labeled. For example, when labeling a local area, the image content and content characteristics of the local area can be input into the device. The device then selects the first image containing the area based on the local area characteristics and then determines the target image from the identified first images based on other information in the labeling instructions, such as information about the shooting angle. Alternatively, the user can directly input the first image to be labeled into the device and request the device to label it. Alternatively, the device may include a model trained based on historical first images and historical target images. Once the device has acquired multiple first images, it may process the first images based on the model, ultimately outputting a first image to be annotated. The device may then annotate the first image determined. Once the device has completed annotating the extracted first image, it may directly determine that the first image containing the two-dimensional annotation information is the target image.

[0064] Optionally, for example, when the device is able to obtain the posture information of the acquisition device, such as the internal parameters, external parameters and other data of the acquisition device, or when the device is able to know the specific shooting orientation of each image, the farthest point sampling method can be used to extract the target image from multiple first images. Specifically, the device can select the image acquisition position of the acquisition device based on the farthest point sampling method, and the first image whose image shooting orientation point is farthest away is used as the target image. First, the device can regard each acquisition position / image shooting orientation as a point, and then select one or more points from all the "points" to form a selected point set, and the remaining points form an unselected point set. Then, a point farthest from the selected point set is found in the unselected point set, and the point is added to the selected point set. At the same time, the point is deleted from the unselected point set, and the process is iterated until the number of points in the selected point set reaches the number specified by the user. The first image corresponding to the point in the selected point set is the target image that needs to be labeled.

[0065] Among them, when marking, in this embodiment, the target object is roughly marked. Taking the marking information of the target object as a whole as an example, it is sufficient to mark an area that roughly covers the target object. For example, the information of the circumscribed rectangle, circumscribed circle, circumscribed triangle, etc. of the target object is marked. The shape of the circumscribed figure is determined according to the actual shape of the target object. It is also possible to mark a bounding box that is similar to the shape of the target object, or directly mark a rectangular box that can frame the target object, etc., to reduce the difficulty of marking and increase the marking speed. Of course, detailed marking can also be performed. In this embodiment, the choice of rough marking is only as an example.

[0066] Furthermore, if Figure 2 As shown, determining the three-dimensional annotation information of the target object based on at least the two-dimensional annotation information in the plurality of target images includes:

[0067] S302: Acquire first shooting orientation information of each target image relative to the target object;

[0068] S303: Determine three-dimensional annotation information of the target object based on the two-dimensional annotation information in the multiple target images and the first shooting orientation information.

[0069] In this embodiment, when determining the three-dimensional annotation information based on the two-dimensional annotation information, the device needs to first obtain the first shooting orientation information of each target image, that is, the shooting orientation when the acquisition device takes each target image. This information can be automatically carried by each first image, or it can be obtained by the device through interaction with the acquisition device, or the device can have a reference image and corresponding reference shooting orientation information, and then analyze and process each target image based on the reference image and reference shooting orientation information, such as processing the target image based on the coordinate system of the reference image, and then calculating and comparing it with the reference image and its shooting orientation information, etc. The specific acquisition method is not unique. After the device obtains the first shooting orientation information of each target image, it can calculate and determine the three-dimensional annotation information of the target object based on the first shooting orientation information and the two-dimensional annotation information in the target image. Based on the three-dimensional annotation information, the device can obtain a three-dimensional structure, and the target object is at least located in the space enclosed by the three-dimensional structure, and can also be the same as the shape of the three-dimensional structure.

[0070] For example, combined with Figure 3 As shown in the figure, the positional relationship between the target object and different first shooting directions is illustrated from a top-down perspective. The shoes in the figure are the target object, and their specific size can be determined based on the two-dimensional annotation information. The points and triangles in the figure represent the acquisition position and posture of the target image, which are determined by the first shooting direction information. From each point, a dotted line is projected to the target object so that the target object is located between the projected dotted lines. Only two dotted lines are shown in the figure as an example. When a dotted line is drawn for each point, the intersection area of ​​all dotted lines can be obtained. Specifically, you can refer to the area outlined by the solid line in the figure. This area is actually a three-dimensional area, and the three-dimensional area limits the spatial range of the target object. Based on this method, a three-dimensional polyhedron structure covering the target object can be obtained. The three-dimensional polyhedron structure is the three-dimensional structure formed by the three-dimensional annotation information in this embodiment.

[0071] It should be noted that the projection of the above three-dimensional annotation information on each target image coincides with the two-dimensional annotation information on each target image.

[0072] The more target images there are, or the more accurate the two-dimensional annotation information, the more precisely the spatial range defined by the three-dimensional polyhedron structure will be. However, to avoid increasing the annotation workload, this embodiment considers various factors and sets the number of target images to 3-5. Of course, more target images can be extracted based on actual conditions, and the specific number is not fixed.

[0073] Furthermore, if Figure 4 As shown, when determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information, the method includes:

[0074] S402: Determine at least second shooting orientation information of each second image relative to the target object;

[0075] S403: Determine two-dimensional annotation information corresponding to each second image based on the second shooting orientation information and the three-dimensional annotation information.

[0076] For example, since the shooting orientation of each second image is also different, when the device determines the two-dimensional annotation information of each second image, it is necessary to first determine the second shooting orientation information of each second image. Then, based on the above steps, the device has obtained the three-dimensional annotation information containing the target object, that is, it has obtained a three-dimensional structure corresponding to the two-dimensional annotation information in the target image. Therefore, the device can combine the three-dimensional structure and the second shooting orientation information to comprehensively determine the orientation information of the three-dimensional structure in each second image, that is, at this time, the orientation of the three-dimensional structure relative to the second image is consistent with the orientation of the target object relative to the second image. The device can automatically annotate the target object in each second image based on the positional relationship of the three-dimensional structure relative to the second image and the shape data of the three-dimensional structure.

[0077] Specifically, determining the two-dimensional annotation information corresponding to each second image based on the second shooting orientation information and the three-dimensional annotation information includes:

[0078] S404: Determine a projection relationship between each second image and the target object based at least on the second shooting orientation information;

[0079] S405: Determine the projection of the three-dimensional annotation information on each second image based on each projection relationship;

[0080] S406: Determine the two-dimensional annotation information corresponding to each second image based on each projection.

[0081] In this embodiment, as described above, the positional relationship of the three-dimensional structure relative to each second image is determined based on each second shooting orientation information, that is, the positional relationship of the target object relative to each second image is determined. The device can then determine the projective relationship between the second image and the target object based on this positional relationship, such as determining the projection direction and projection position. After determining the projection relationship, the device projects the three-dimensional structure relative to the second image based on the projection relationship. This specifically includes fitting the coordinate system of the three-dimensional structure to the coordinate system of the second image based on the determined projection direction and projection position, thereby determining the shape data of the coordinate system of the three-dimensional structure on the second image, that is, determining the projection of the three-dimensional structure on the second image. The target object on the second image may completely overlap with the projection or be located within the area of ​​the projection. After determining the projection corresponding to each second image, the device can automatically annotate the corresponding second image with two-dimensional information based on each projection, that is, annotate the projection on each second image with two-dimensional information.

[0082] Alternatively, the three-dimensional structure can be projected based on a unified projection direction and projection position. In this case, the position of the projection on the second image is different from the position of the target object in the image and needs to be adjusted. When making adjustments, the device can adjust the position of the projection on each image based on the second shooting orientation information of each second image, thereby making the projection coincide with the area where the target object is located, that is, even if the projection covers the area where the target object is located.

[0083] Optionally, the two-dimensional annotation information in this embodiment may be the size information of the 2D bounding box of the target object, and the three-dimensional annotation information may be the size information of the 3D bounding box of the target object, wherein the projection of the 3D bounding box on the second image has the same shape as the 2D bounding box, for example, both are the same rectangular frame.

[0084] The step of determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes:

[0085] S407: Determine a candidate 2D bounding box corresponding to each second image based at least on the 3D bounding box;

[0086] S408: Adjusting the shapes of the candidate 2D bounding boxes based on the image area enclosed by each candidate 2D bounding box to obtain a target 2D bounding box corresponding to each second image.

[0087] For example, based on the above method, a 3D bounding box can be projected onto the second image, and based on the projection on each second image, a candidate 2D bounding box on each second image can be determined, and the 2D bounding box can surround and cover the target object on the second image. However, if the two-dimensional annotation information corresponding to the target object on the previous target image is a rough annotation, then the projection and 2D bounding box obtained at this time cannot exactly surround and cover the target object on the corresponding image, that is, the area enclosed by the 2D bounding box includes not only the target object, but also the remaining image content on the second image. Therefore, in order to obtain a more accurate target 2D bounding box that can exactly surround and cover the target object, the device in this embodiment will determine the image area enclosed by each candidate 2D bounding box, and perform processing based on each image area, such as image recognition, to determine the specific area of ​​the target object in the projection or in the 2D bounding box, and then adjust the size of the 2D bounding box based on the specific area, that is, adjust the two-dimensional annotation information, so that the adjusted 2D bounding box (two-dimensional annotation information) is more accurate and closer to the size information of the target object. The above adjustment not only adjusts the two-dimensional annotation information of the second image, but also adjusts the two-dimensional annotation information of the target image, so that all first images have two-dimensional annotation information with a size closer to the actual size of the target object.

[0088] Specifically, in order to more conveniently and quickly adjust the size of the candidate 2D bounding box to make it a target 2D bounding box with higher precision, this embodiment adjusts the shape of each candidate 2D bounding box based on the image area enclosed by each candidate 2D bounding box, including:

[0089] S409: Input the image area enclosed by each candidate 2D bounding box into the trained target network to adjust the shape of each candidate 2D bounding box accordingly.

[0090] For example, the AI ​​function of the device can be used in practical applications. Specifically, the trained target network can be, for example, a semantic segmentation network that can realize the size adjustment of the 2D bounding box of this embodiment. The semantic segmentation network itself has the ability to process the image to obtain pixel-level accuracy, so after training, the technical effect of this embodiment can be efficiently achieved. Alternatively, the target network is selected as a salient target detection network, which is similar to the semantic segmentation network. The salient target detection network also has pixel-level image processing accuracy. At the same time, the network can also divide the image content into two parts: background and foreground with significance. Therefore, the trained network is trained based on the salient target detection network, and the generalization ability of the trained network is better than that of the semantic segmentation network. Because the target object in this embodiment accounts for a large proportion of the image enclosed by the candidate 2D bounding box and is significant, the network can more easily and quickly determine the target object and background in the image area enclosed by the candidate 2D bounding box after training.

[0091] Therefore, in this embodiment, it is preferred to train the salient target network as the target network so that it can process the image area enclosed by the candidate 2D bounding box to obtain an accurate target 2D bounding box. In actual application, the device can input each target image and the image area determined based on the candidate 2D bounding box (two-dimensional annotation information) in the second image into the trained salient target network. The salient target network processes the received image data to obtain the specific area of ​​the target object in each image, and then accurately determines the target 2D bounding box based on the specific area.

[0092] like Figure 5 As shown, another embodiment of the present application also provides an image annotation device, which includes:

[0093] An acquisition module, configured to acquire a plurality of first images containing a target object, wherein the plurality of first images are at least shot in different orientations;

[0094] an extraction module, configured to extract a plurality of target images from the plurality of first images, wherein the plurality of target images have two-dimensional annotation information for the target object;

[0095] A first determining module, configured to determine three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the plurality of target images;

[0096] The second determining module is configured to determine, based at least on the three-dimensional annotation information, the two-dimensional annotation information corresponding to the target object in each second image, where the second image is the unannotated first image.

[0097] As an optional embodiment, the two-dimensional annotation information is a 2D bounding box, and the three-dimensional annotation information is a 3D bounding box, wherein determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes:

[0098] determining, based at least on the 3D bounding box, a candidate 2D bounding box corresponding to each second image;

[0099] The shapes of the candidate 2D bounding boxes are adjusted based on the image area enclosed by each candidate 2D bounding box, so as to obtain a target 2D bounding box corresponding to each second image.

[0100] As an optional embodiment, adjusting the shape of each candidate 2D bounding box based on the image area enclosed by each candidate 2D bounding box includes:

[0101] The image area enclosed by each candidate 2D bounding box is input into the trained target network to adjust the shape of each candidate 2D bounding box accordingly.

[0102] As an optional embodiment, the device further includes:

[0103] a receiving module, configured to receive a labeling instruction, wherein the labeling instruction is used to label at least one of the plurality of first images;

[0104] A determination module is configured to determine the marked first image as the target image.

[0105] As an optional embodiment, determining the three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the multiple target images includes:

[0106] Acquiring first shooting orientation information of each target image relative to the target object;

[0107] The three-dimensional annotation information of the target object is determined based on the two-dimensional annotation information in the multiple target images and the first shooting orientation information.

[0108] As an optional embodiment, determining the two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes:

[0109] determining at least second shooting orientation information of each of the second images relative to the target object;

[0110] Two-dimensional annotation information corresponding to each second image is determined based on the second shooting orientation information and the three-dimensional annotation information.

[0111] As an optional embodiment, determining the two-dimensional annotation information corresponding to each second image based on the second shooting orientation information and the three-dimensional annotation information includes:

[0112] determining a projection relationship between each of the second images and the target object based at least on the second shooting orientation information;

[0113] determining a projection of the three-dimensional annotation information on each of the second images based on each of the projection relationships;

[0114] Two-dimensional annotation information corresponding to each second image is determined based on each of the projections.

[0115] Another embodiment of the present application further provides an electronic device, comprising:

[0116] one or more processors;

[0117] a memory configured to store one or more programs;

[0118] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned image annotation method.

[0119] One embodiment of the present application further provides a storage medium storing a computer program that, when executed by a processor, implements the image annotation method described above. It should be understood that each solution in this embodiment has the corresponding technical effects of the above method embodiments, and will not be further described here.

[0120] The present application also provides a computer program product tangibly stored on a computer-readable medium and comprising computer-readable instructions that, when executed, cause at least one processor to perform an image annotation method such as that described in the above embodiments. It should be understood that each solution in this embodiment has the corresponding technical effects of the above-described method embodiments and will not be further elaborated here.

[0121] It should be noted that the computer storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program configured for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, antenna, optical cable, RF, or any suitable combination thereof.

[0122] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A method for image annotation, wherein: include: Obtaining a plurality of first images containing a target object, wherein the plurality of first images are at least shot in different orientations; Extracting a plurality of target images from the plurality of first images, wherein the plurality of target images have two-dimensional annotation information for the target object; determining three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the plurality of target images; determining, based at least on the three-dimensional annotation information, two-dimensional annotation information corresponding to the target object in each second image, where the second image is an unannotated version of the first image; The two-dimensional annotation information is a 2D bounding box, and the three-dimensional annotation information is a 3D bounding box.

2. The method according to claim 1, wherein Determining two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes: determining, based at least on the 3D bounding box, a candidate 2D bounding box corresponding to each second image; The shapes of the candidate 2D bounding boxes are adjusted based on the image area enclosed by each candidate 2D bounding box, so as to obtain a target 2D bounding box corresponding to each second image.

3. The method according to claim 2, wherein: The adjusting the shape of each candidate 2D bounding box based on the image area enclosed by each candidate 2D bounding box includes: The image area enclosed by each candidate 2D bounding box is input into the trained target network to adjust the shape of each candidate 2D bounding box accordingly.

4. The method according to claim 3, wherein: The method further comprises: receiving a labeling instruction, where the labeling instruction is used to label at least one of the plurality of first images; The marked first image is determined as the target image.

5. The method according to claim 1, wherein The determining of the three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the plurality of target images includes: Acquiring first shooting orientation information of each target image relative to the target object; The three-dimensional annotation information of the target object is determined based on the two-dimensional annotation information in the multiple target images and the first shooting orientation information.

6. The method according to claim 1, wherein Determining two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes: determining at least second shooting orientation information of each of the second images relative to the target object; Two-dimensional annotation information corresponding to each second image is determined based on the second shooting orientation information and the three-dimensional annotation information.

7. The method according to claim 6, wherein: The determining, based on the second shooting orientation information and the three-dimensional annotation information, the two-dimensional annotation information corresponding to each second image includes: determining a projection relationship between each of the second images and the target object based at least on the second shooting orientation information; determining a projection of the three-dimensional annotation information on each of the second images based on each of the projection relationships; Two-dimensional annotation information corresponding to each second image is determined based on each of the projections.

8. An image annotation device, wherein: include: An acquisition module, configured to acquire a plurality of first images containing a target object, wherein the plurality of first images are at least shot in different orientations; an extraction module, configured to extract a plurality of target images from the plurality of first images, wherein the plurality of target images have two-dimensional annotation information for the target object; A first determining module, configured to determine three-dimensional annotation information of the target object based at least on the two-dimensional annotation information in the plurality of target images; a second determining module, configured to determine, based at least on the three-dimensional annotation information, two-dimensional annotation information corresponding to the target object in each second image, where the second image is the unannotated first image; The two-dimensional annotation information is a 2D bounding box, and the three-dimensional annotation information is a 3D bounding box.

9. The device according to claim 8, wherein The two-dimensional annotation information is a 2D bounding box, and the three-dimensional annotation information is a 3D bounding box; Determining two-dimensional annotation information corresponding to each second image based at least on the three-dimensional annotation information includes: determining, based at least on the 3D bounding box, a candidate 2D bounding box corresponding to each second image; The shapes of the candidate 2D bounding boxes are adjusted based on the image area enclosed by each candidate 2D bounding box, so as to obtain a target 2D bounding box corresponding to each second image.

10. The device according to claim 9, wherein The adjusting the shape of each candidate 2D bounding box based on the image area enclosed by each candidate 2D bounding box includes: The image area enclosed by each candidate 2D bounding box is input into the trained target network to adjust the shape of each candidate 2D bounding box accordingly.

Images