A three-dimensional annotation method and device for two-dimensional images

Abstract

The present application provides a method and device for three-dimensional annotation of a two-dimensional image, wherein the method includes: obtaining two-dimensional annotation information of a target object in a two-dimensional image, wherein the two-dimensional annotation information includes at least one of the following: a first contour frame for annotating the overall contour of the target object; two grounding points for annotating the contact position between the target object and the ground; a second contour frame for annotating the width of the target object; obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information; and generating three-dimensional annotation information of the target object according to the coordinate information of the preset annotation points. By obtaining the two-dimensional annotation information of the target object in the two-dimensional image and automatically generating the three-dimensional annotation information of the target object based on the two-dimensional annotation information, the three-dimensional annotation information can be obtained by only manually performing two-dimensional annotation, which further reduces the manual workload and reduces errors, and is also conducive to further improving the accuracy of the three-dimensional annotation.

Description

Technical Field

[0001] The present application relates to the technical field of object annotation, and in particular to a method and device for three-dimensional annotation of a two-dimensional image. Background Art

[0002] Currently, when annotating objects in a two-dimensional image in three dimensions, a three-dimensional frame that includes the object is constructed manually based on the shape of the object in the image, and the three-dimensional annotation of the object is completed by obtaining the coordinates of multiple points on the three-dimensional frame. However, in the annotation process, in order to improve the accuracy of the annotation, it is necessary to construct a three-dimensional frame to fit the object. At this time, it will be necessary to rotate or move the three-dimensional frame multiple times to change the size and position of the three-dimensional frame. This annotation method is not only cumbersome and time-consuming, but also requires a high level of spatial perception from the annotator because all the annotations need to be completed manually. It is easy to fail to accurately obtain the boundary between the object and the ground, which leads to large errors in the obtained three-dimensional frame and coordinates, making the accuracy of the three-dimensional annotation low. Summary of the invention

[0003] The technical purpose to be achieved by the embodiments of the present application is to provide a three-dimensional annotation method for a two-dimensional image, so as to solve the problem that when three-dimensional annotation is currently performed on a two-dimensional image, high requirements on manpower are imposed due to the manual operation, which easily leads to low accuracy of the obtained three-dimensional annotation.

[0004] To solve the above technical problems, the present application provides a method for three-dimensional annotation of a two-dimensional image, including:

[0005] Acquire two-dimensional annotation information of a target object in a two-dimensional image, where the two-dimensional annotation information includes at least one of the following:

[0006] A first contour frame for marking the overall contour of the target object;

[0007] Two grounding points used to mark the contact position between the target object and the ground;

[0008] A second contour frame for marking the width of the target object;

[0009] According to the two-dimensional annotation information, coordinate information of at least two preset annotation points is obtained;

[0010] Generate three-dimensional annotation information of the target object based on the coordinate information of the preset annotation points.

[0011] Preferably, in the above-mentioned three-dimensional annotation method of a two-dimensional image, the preset annotation points at least include:

[0012] A first vertex and a second vertex on the first outline frame, the first vertex and the second vertex are arranged diagonally, and the first vertex is the upper left vertex of the first outline frame;

[0013] Two grounding points; and

[0014] The third vertex and the fourth vertex when the second contour frame is tangent to the straight line where the two grounding points are located, the third vertex corresponds to the first vertex, and the fourth vertex corresponds to the second vertex.

[0015] Furthermore, the three-dimensional annotation method of the two-dimensional image as described above, obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information, includes:

[0016] According to the first contour frame, respectively determine the coordinates of the first vertex and the coordinates of the second vertex;

[0017] According to the two grounding points, the grounding point coordinates corresponding to the two grounding points are determined respectively;

[0018] Determine the initial coordinates of the third vertex and the initial coordinates of the fourth vertex according to the second contour frame;

[0019] The calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex are determined according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, and the coordinates of the two grounding points.

[0020] Specifically, the three-dimensional annotation method of the two-dimensional image as described above determines the calculated coordinates of the third vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points, including:

[0021] The ordinate in the coordinates of the first vertex is determined as the ordinate of the calculated coordinates of the third vertex, and the abscissa in the initial coordinates of the third vertex is determined as the abscissa of the calculated coordinates of the third vertex.

[0022] Specifically, the three-dimensional annotation method of the two-dimensional image as described above determines the calculated coordinates of the fourth vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points, including:

[0023] According to the coordinates of the two grounding points, a first distance between the two grounding points is obtained;

[0024] Determine the coordinates of two extended points on the straight line where the two grounding points are located according to the coordinates of the two grounding points and the first distance, wherein the two extended points are respectively located on one side of one grounding point away from the other grounding point, and the second distance between each extended point and an adjacent grounding point is a preset multiple of the first distance;

[0025] According to the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, the calculated coordinates of the fourth vertex are obtained, wherein the target grounding point is the grounding point adjacent to the fourth vertex, and the target extension point is the extension point away from the fourth vertex.

[0026] Furthermore, the three-dimensional annotation method of the two-dimensional image as described above obtains the calculated coordinates of the fourth vertex according to the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, including:

[0027] Determine a lateral distance from the third vertex to the fourth vertex according to the initial coordinates of the third vertex and the initial coordinates of the fourth vertex;

[0028] Determine the ordinate in the calculated coordinates of the fourth vertex according to the lateral distance, the coordinates of the target grounding point and the coordinates of the target extension point;

[0029] The abscissa in the initial coordinates of the fourth vertex is determined as the abscissa in the calculated coordinates of the fourth vertex.

[0030] Preferably, the three-dimensional annotation method of the two-dimensional image as described above, obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information, includes:

[0031] According to the two-dimensional annotation information, the coordinates of the first vertex and the coordinates of the second vertex on the first contour frame are obtained, and the first vertex and the second vertex are set diagonally;

[0032] And / or, according to the two-dimensional annotation information, the coordinates of the third vertex and the coordinates of the fourth vertex on the second contour frame are obtained, and the third vertex and the fourth vertex are set diagonally.

[0033] Preferably, the three-dimensional annotation method of the two-dimensional image as described above further includes:

[0034] When a two-dimensional annotation change signal is received, updated two-dimensional annotation information is obtained according to the two-dimensional annotation change signal, and a step of obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information is performed.

[0035] Another embodiment of the present application further provides a control device, including:

[0036] The first processing module is used to obtain two-dimensional annotation information of the target object in the two-dimensional image, where the two-dimensional annotation information includes at least one of the following:

[0037] A first contour frame for marking the overall contour of the target object;

[0038] Two grounding points used to mark the contact position between the target object and the ground;

[0039] A second contour frame for marking the width of the target object;

[0040] A second processing module, used to obtain coordinate information of at least two preset annotation points according to the two-dimensional annotation information;

[0041] The third processing module is used to generate three-dimensional annotation information of the target object according to the coordinate information of the preset annotation points.

[0042] Preferably, in the control device as described above, the preset marking points at least include:

[0043] A first vertex and a second vertex on the first outline frame, the first vertex and the second vertex are arranged diagonally, and the first vertex is the upper left vertex of the first outline frame;

[0044] Two grounding points; and

[0045] The third vertex and the fourth vertex when the second contour frame is tangent to the straight line where the two grounding points are located, the third vertex corresponds to the first vertex, and the fourth vertex corresponds to the second vertex.

[0046] Furthermore, the control device as described above, the second processing module includes:

[0047] A first processing submodule, used to determine the coordinates of the first vertex and the coordinates of the second vertex respectively according to the first contour frame;

[0048] The second processing submodule is used to determine the grounding point coordinates corresponding to the two grounding points respectively according to the two grounding points;

[0049] A third processing submodule, used for determining the initial coordinates of the third vertex and the initial coordinates of the fourth vertex according to the second contour frame;

[0050] The fourth processing submodule is used to determine the calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points.

[0051] Specifically, the fourth processing submodule of the control device as described above includes:

[0052] The first processing unit is used to determine the ordinate in the coordinates of the first vertex as the ordinate of the calculated coordinates of the third vertex, and determine the abscissa in the initial coordinates of the third vertex as the abscissa of the calculated coordinates of the third vertex.

[0053] Specifically, the fourth processing submodule of the control device as described above includes:

[0054] A second processing unit, configured to obtain a first distance between the two grounding points according to the coordinates of the two grounding points;

[0055] A third processing unit is used to determine the coordinates of two extended points on the straight line where the two grounding points are located according to the coordinates of the two grounding points and the first distance, wherein the two extended points are respectively located on a side of one grounding point away from the other grounding point, and the second distance between each extended point and an adjacent grounding point is a preset multiple of the first distance;

[0056] The fourth processing unit is used to obtain the calculated coordinates of the fourth vertex based on the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, wherein the target grounding point is the grounding point adjacent to the fourth vertex, and the target extension point is the extension point away from the fourth vertex.

[0057] Furthermore, the control device as described above, the fourth processing unit, includes:

[0058] A first processing subunit, configured to determine a lateral distance from the third vertex to the fourth vertex according to the initial coordinates of the third vertex and the initial coordinates of the fourth vertex;

[0059] A second processing subunit is used to determine the ordinate in the calculated coordinates of the fourth vertex according to the lateral distance, the coordinates of the target grounding point and the coordinates of the target extension point;

[0060] The third processing subunit is used to determine the horizontal coordinate in the initial coordinates of the fourth vertex as the horizontal coordinate in the calculated coordinates of the fourth vertex.

[0061] Preferably, the control device as described above, the second processing module comprises:

[0062] A fifth processing submodule is used to obtain the coordinates of the first vertex and the coordinates of the second vertex on the first contour frame according to the two-dimensional annotation information, and the first vertex and the second vertex are set diagonally;

[0063] And / or, a sixth processing submodule is used to obtain the coordinates of the third vertex and the coordinates of the fourth vertex on the second contour frame according to the two-dimensional annotation information, and the third vertex and the fourth vertex are set diagonally.

[0064] Preferably, the control device as described above further comprises:

[0065] The fourth processing module is used to obtain updated two-dimensional annotation information according to the two-dimensional annotation change signal when receiving the two-dimensional annotation change signal, and execute the step of obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information.

[0066] Yet another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the three-dimensional annotation method of a two-dimensional image as described above are implemented.

[0067] Compared with the prior art, the method and device for three-dimensional annotation of a two-dimensional image provided in the embodiment of the present application have at least the following beneficial effects:

[0068] The present application obtains the two-dimensional annotation information of the target object in the two-dimensional image, and automatically generates the three-dimensional annotation information of the target object based on this, so that the three-dimensional annotation information can be obtained only by manual two-dimensional annotation, and only the overall contour, partial contour and at least one of the grounding points of the target object need to be two-dimensionally annotated. While further reducing the manual workload and reducing errors, it is also beneficial to further improve the accuracy of the three-dimensional annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is one of the flowcharts of the three-dimensional annotation method of a two-dimensional image of the present application;

[0070] Figure 2 This is the second flow chart of the three-dimensional annotation method of a two-dimensional image of the present application;

[0071] Figure 3 This is the third flow chart of the three-dimensional annotation method of a two-dimensional image of the present application;

[0072] Figure 4 This is a fourth flow chart of the three-dimensional annotation method of a two-dimensional image of the present application;

[0073] Figure 5 It is a structural schematic diagram of the control device of the present application;

[0074] Figure 6 This is one of the structural schematic diagrams of the present application for three-dimensionally marking target objects of different shapes in a two-dimensional image;

[0075] Figure 7 This is the second structural schematic diagram of the present application for three-dimensionally labeling target objects of different shapes in a two-dimensional image;

[0076] Figure 8 This is the third structural schematic diagram of the present application for three-dimensionally labeling target objects of different shapes in a two-dimensional image;

[0077] Fig. 9 This is the fourth structural schematic diagram of the present application for three-dimensionally labeling target objects of different shapes in a two-dimensional image. DETAILED DESCRIPTION

[0078] In order to make the technical problems, technical solutions and advantages to be solved by the application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.

[0079] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0080] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0081] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0082] In the embodiments provided in this application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, but B can also be determined according to A and / or other information.

[0083] See also Figure 1 A preferred embodiment of the present application provides a three-dimensional annotation method for a two-dimensional image, comprising:

[0084] Step S101, obtaining two-dimensional annotation information of a target object in a two-dimensional image, wherein the two-dimensional annotation information includes at least one of the following:

[0085] A first contour frame for marking the overall contour of the target object;

[0086] Two grounding points used to mark the contact position between the target object and the ground;

[0087] A second contour frame for marking the width of the target object;

[0088] Step S102, obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information;

[0089] Step S103: Generate three-dimensional annotation information of the target object according to the coordinate information of the preset annotation points.

[0090] In one embodiment of the present application, a method for three-dimensional annotation of a two-dimensional image is provided, specifically, three-dimensional annotation is performed on the objects to be annotated on the two-dimensional image by determining the coordinates of key points, etc. Among them, taking the target object as a hexahedron such as a car as an example, when performing three-dimensional annotation, an object to be three-dimensionally annotated on the two-dimensional image is first taken as the target object, and the two-dimensional annotation information of the target object previously annotated manually or by a machine is obtained. The two-dimensional annotation information includes at least one of a first contour box, two grounding points, and a second contour box, wherein the first contour box is used to annotate the overall contour of the target object, wherein the four sides of the first contour box are used to completely surround the length, width, and height of the target object displayed in the two-dimensional image, and are tangent to the target object; the second contour box is mainly used to annotate the width of the target object, and no special requirements are made on its width. Specifically, the length of the second contour frame is the width of the target object displayed in the two-dimensional image, and the height of the second contour frame can be a fixed value, corresponding to the target object or an arbitrary height; the grounding point is used to indicate the position where the target object contacts the ground, and the number of grounding points is at least two, which is convenient for forming a boundary line between an object and the ground according to the at least two grounding points, and separating the target object from the ground. By marking in the form of points, compared with marking in the form of lines, it is helpful to reduce the workload. The number of grounding points is preferably two, which is conducive to reducing the workload of marking grounding points while ensuring that the boundary line can be determined. After obtaining the above-mentioned two-dimensional annotation information, coordinate acquisition and / or calculation operations can be automatically performed according to the two-dimensional annotation information, and the coordinate information of at least two preset annotation points used for three-dimensional frame construction can be obtained, and then the three-dimensional annotation information of the target object is generated according to the information of the preset annotation points. Preferably, the three-dimensional annotation information is a data structure including the coordinate information of the above-mentioned preset annotation points, so as to facilitate the operation of subsequent algorithms and / or the display of three-dimensional graphics according to the three-dimensional annotation information.

[0091] To summarize, the present application obtains the two-dimensional annotation information of the target object in the two-dimensional image, and automatically generates the three-dimensional annotation information of the target object based on this, so that the three-dimensional annotation information can be obtained by only manual two-dimensional annotation, and only two-dimensional annotation is required for at least one of the overall contour, partial contour and grounding point of the target object. While further reducing the manual workload and reducing errors, the three-dimensional annotation information obtained by marking the grounding points and contour points is also conducive to improving the accuracy of the three-dimensional annotation.

[0092] Preferably, in the above-mentioned three-dimensional annotation method of a two-dimensional image, the preset annotation points at least include:

[0093] A first vertex and a second vertex on the first outline frame, the first vertex and the second vertex are arranged diagonally, and the first vertex is the upper left vertex of the first outline frame;

[0094] Two grounding points; and

[0095] The third and fourth vertices when the reference vertex on the second contour box is tangent to the straight line where the two grounding points are located, the reference vertex does not coincide with the first and second vertices in the horizontal coordinate, the third vertex corresponds to the first vertex, and the fourth vertex corresponds to the second vertex.

[0096] In a specific implementation of the present application, taking the target object as a hexahedron such as a car as an example, when the three faces of the target object appear on the two-dimensional image, the length, width and height of the target object need to be marked to ensure that the target object can be correctly annotated in three dimensions. Therefore, it is necessary to obtain points related to the length, width and height as preset marking points. In this embodiment, the preset marking points are selected from the first vertex and the second vertex diagonally set on the first contour frame. The contour length and contour height displayed on the two-dimensional image of the target object can be known through the first vertex and the second vertex. Preferably, the first vertex is the upper left vertex; the preset marking points also select two grounding points to separate the target object and the ground, so as to avoid considering the ground as part of the target object when generating the three-dimensional annotation information of the target object, which affects the accuracy of the three-dimensional annotation information; the preset marking points also select the third vertex and the fourth vertex when the reference point of the second contour frame is tangent to the straight line where the two grounding points are located, wherein, when the second contour frame is tangent to the straight line where the two grounding points are located, the third vertex and the fourth vertex can be used to determine the ground height and width of the target object, wherein the reference vertex is a vertex that does not share a horizontal coordinate with the first vertex and the second vertex. Then, by combining the coordinates of the above-mentioned preset marking points for calculation, the length, width, height and coordinates of each vertex of the target object in the two-dimensional image can be determined.

[0097] It should be noted that, when the target object is a vehicle, the two ground contact points are preferably contact points between two tires on the same side of the vehicle and the ground.

[0098] See also Figure 2 Further, the three-dimensional annotation method of the two-dimensional image as described above, obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information, includes:

[0099] Step S201, determining the coordinates of the first vertex and the coordinates of the second vertex respectively according to the first outline frame;

[0100] Step S202, according to the two grounding points, respectively determining the grounding point coordinates corresponding to the two grounding points;

[0101] Step S203, determining the initial coordinates of the third vertex and the initial coordinates of the fourth vertex according to the second outline frame;

[0102] Step S204, determining the calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points.

[0103] In a specific embodiment of the present application, when obtaining the coordinate information of at least two preset annotation points according to the two-dimensional annotation information, the coordinates of each preset annotation point are obtained based on the coordinate system of the two-dimensional image, wherein the coordinates of the first vertex and the second vertex, and the grounding point coordinates corresponding to the two grounding points can be determined by calculating the distance from the point to the coordinate origin in the two coordinate axis directions; and the third and fourth vertices have lower requirements for the second contour box. Therefore, when obtaining the required coordinates, the initial coordinates of the annotation in the two-dimensional annotation information are first determined, and then the coordinates after the second contour box is switched are calculated based on the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the two grounding point coordinates, so that the calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex can be obtained, and the calculated coordinates are the coordinates corresponding to the third vertex and the fourth vertex when they are preset annotation points.

[0104] Specifically, the three-dimensional annotation method of the two-dimensional image as described above determines the calculated coordinates of the third vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points, including:

[0105] The ordinate in the coordinates of the first vertex is determined as the ordinate of the calculated coordinates of the third vertex, and the abscissa in the initial coordinates of the third vertex is determined as the abscissa of the calculated coordinates of the third vertex.

[0106] In a specific embodiment of the present application, when the second contour frame switches from an initial state to a state where the reference vertex is tangent to the straight line where the two ground coordinates are located, the coordinates of the third vertex will not change in the horizontal direction, but need to be aligned with the first contour frame in the vertical direction. Therefore, it can be determined that the vertical coordinate in the coordinates of the first vertex is the vertical coordinate of the calculated coordinates of the third vertex, and the horizontal coordinate in the initial coordinates of the third vertex is the horizontal coordinate of the calculated coordinates of the third vertex.

[0107] See also Figure 3 Specifically, the three-dimensional annotation method of the two-dimensional image as described above, according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points, determines the calculated coordinates of the fourth vertex, including:

[0108] Step S301, obtaining a first distance between two grounding points according to the coordinates of the two grounding points;

[0109] Step S302, determining the coordinates of two extended points on the straight line where the two grounding points are located according to the coordinates of the two grounding points and the first distance, wherein the two extended points are respectively located on a side of one grounding point away from the other grounding point, and a second distance between each extended point and an adjacent grounding point is a preset multiple of the first distance;

[0110] Step S303, obtaining the calculated coordinates of the fourth vertex according to the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, wherein the target grounding point is the grounding point adjacent to the fourth vertex, and the target extension point is the extension point away from the fourth vertex.

[0111] In another specific embodiment of the present application, before updating the coordinates of the third and fourth vertices, i.e. obtaining their calculated coordinates, the second contour frame needs to be switched from the initial state to the state where the reference vertex is tangent to the straight line where the two grounding coordinates are located. At this time, the two grounding points are connected into a straight line. To avoid the straight line from extending infinitely and to facilitate subsequent calculations, the points on both sides of the straight line extending a certain distance are determined as extension points. At this time, the coordinates of the two extension points can be determined according to the straight line equations where the two grounding points are located. During the switching process, the coordinates of the fourth vertex will not change in the horizontal direction, but in the vertical direction, the reference vertex needs to be tangent to the straight line where the two grounding points are located. According to the orientation of the target object, the reference vertex can be the fourth vertex or another vertex with the same vertical coordinate as the fourth vertex, so the vertical coordinate of the fourth vertex needs to be calculated. Preferably, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point of the present application are used to obtain the calculated coordinates of the fourth vertex.

[0112] See also Figure 4 Furthermore, the three-dimensional annotation method of the two-dimensional image as described above obtains the calculated coordinates of the fourth vertex according to the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, including:

[0113] Step S401, determining a lateral distance from the third vertex to the fourth vertex according to the initial coordinates of the third vertex and the initial coordinates of the fourth vertex;

[0114] Step S402, determining the ordinate in the calculated coordinates of the fourth vertex according to the lateral distance, the coordinates of the target grounding point and the coordinates of the target extension point;

[0115] The abscissa in the initial coordinates of the fourth vertex is determined as the abscissa in the calculated coordinates of the fourth vertex.

[0116] In a specific embodiment, when obtaining the calculated coordinates of the fourth vertex, the abscissa of the fourth vertex does not change, so the abscissa in the initial coordinates of the fourth vertex is determined as the abscissa in the calculated coordinates of the fourth vertex; and when calculating the ordinate in the calculated coordinates of the fourth vertex, the following calculation formula is preferably used:

[0117] k=(E y -F y )*1.0 / (E x -F x )

[0118] b=E y -k*E x

[0119] y=kX hk +b

[0120] Among them, k and b are related parameters used to calculate the vertical coordinate;

[0121] (E x , E y ) are the coordinates of the target grounding point;

[0122] (F x , F y ) are the coordinates of the target grounding point;

[0123] F x is the lateral distance from the third vertex to the fourth vertex.

[0124] Preferably, the three-dimensional annotation method of the two-dimensional image as described above, obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information, includes:

[0125] According to the two-dimensional annotation information, the coordinates of the first vertex and the coordinates of the second vertex on the first contour frame are obtained, and the first vertex and the second vertex are set diagonally;

[0126] And / or, according to the two-dimensional annotation information, the coordinates of the third vertex and the coordinates of the fourth vertex on the second contour frame are obtained, and the third vertex and the fourth vertex are set diagonally.

[0127] In a specific embodiment of the present application, when only one face of a target object is displayed on a two-dimensional image, the three-dimensional structure of the target object cannot be known due to occlusion and other factors, and one of the dimensions of the length, width and height cannot be known. In this case, it can be understood that the target object is a two-dimensional structure, so it is only necessary to mark a first outline box or a second outline box in the two-dimensional image in advance. At this time, the marked outline box can be used as a three-dimensional annotation of the target object.

[0128] Preferably, the three-dimensional annotation method of the two-dimensional image as described above further includes:

[0129] When a two-dimensional annotation change signal is received, updated two-dimensional annotation information is obtained according to the two-dimensional annotation change signal, and a step of obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information is performed.

[0130] In another specific implementation of the present application, after the three-dimensional annotation information of the target object has been obtained, if a two-dimensional annotation change signal corresponding to the target object is received, the updated two-dimensional annotation information will be obtained according to the two-dimensional annotation change signal, and the step of obtaining the coordinate information of at least two preset annotation points according to the updated two-dimensional annotation information will be performed again. In order to modify the three-dimensional annotation information of the target object in a timely manner to ensure the accuracy of the modified three-dimensional annotation information. Optionally, after obtaining the two-dimensional annotation information, the coordinates of the preset annotation points can be directly modified according to the relationship between the coordinates to further obtain the modified three-dimensional annotation information.

[0131] It should be noted that Figure 6-9 A common schematic diagram of labeling a target object 602 (taking a vehicle as an example) is given, wherein other objects in the two-dimensional image 601 are omitted for the sake of clarity, a first contour box 603 frames the overall frame of the target object 602, an initial second contour box 604 is used to label the width of the target object 602, and an extension point 607 on the straight line where the two grounding points 606 are located can be used to calculate the coordinates of the third vertex and the fourth vertex when the reference point on the updated second contour box 605 is tangent to the straight line.

[0132] See also Figure 5 Another embodiment of the present application further provides a control device, comprising:

[0133] The first processing module 501 is used to obtain two-dimensional annotation information of a target object in a two-dimensional image. The two-dimensional annotation information includes at least one of the following:

[0134] A first contour frame for marking the overall contour of the target object;

[0135] Two grounding points used to mark the contact position between the target object and the ground;

[0136] A second contour frame for marking the width of the target object;

[0137] The second processing module 502 is used to obtain coordinate information of at least two preset annotation points according to the two-dimensional annotation information;

[0138] The third processing module 503 is used to generate three-dimensional annotation information of the target object according to the coordinate information of the preset annotation points.

[0139] Preferably, in the control device as described above, the preset marking points at least include:

[0140] A first vertex and a second vertex on the first outline frame, the first vertex and the second vertex are arranged diagonally, and the first vertex is the upper left vertex of the first outline frame;

[0141] Two grounding points; and

[0142] The third vertex and the fourth vertex when the second contour frame is tangent to the straight line where the two grounding points are located, the third vertex corresponds to the first vertex, and the fourth vertex corresponds to the second vertex.

[0143] Furthermore, the control device as described above, the second processing module includes:

[0144] A first processing submodule, used to determine the coordinates of the first vertex and the coordinates of the second vertex respectively according to the first contour frame;

[0145] The second processing submodule is used to determine the grounding point coordinates corresponding to the two grounding points respectively according to the two grounding points;

[0146] A third processing submodule, used for determining the initial coordinates of the third vertex and the initial coordinates of the fourth vertex according to the second contour frame;

[0147] The fourth processing submodule is used to determine the calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points.

[0148] Specifically, the fourth processing submodule of the control device as described above includes:

[0149] The first processing unit is used to determine the ordinate in the coordinates of the first vertex as the ordinate of the calculated coordinates of the third vertex, and determine the abscissa in the initial coordinates of the third vertex as the abscissa of the calculated coordinates of the third vertex.

[0150] Specifically, the fourth processing submodule of the control device as described above includes:

[0151] A second processing unit, configured to obtain a first distance between the two grounding points according to the coordinates of the two grounding points;

[0152] A third processing unit is used to determine the coordinates of two extended points on the straight line where the two grounding points are located according to the coordinates of the two grounding points and the first distance, wherein the two extended points are respectively located on a side of one grounding point away from the other grounding point, and the second distance between each extended point and an adjacent grounding point is a preset multiple of the first distance;

[0153] The fourth processing unit is used to obtain the calculated coordinates of the fourth vertex based on the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, wherein the target grounding point is the grounding point adjacent to the fourth vertex, and the target extension point is the extension point away from the fourth vertex.

[0154] Furthermore, the control device as described above, the fourth processing unit, includes:

[0155] a first processing subunit, configured to determine a lateral distance from the third vertex to the fourth vertex according to the initial coordinates of the third vertex and the initial coordinates of the fourth vertex;

[0156] A second processing subunit is used to determine the ordinate in the calculated coordinates of the fourth vertex according to the lateral distance, the coordinates of the target grounding point and the coordinates of the target extension point;

[0157] The third processing subunit is used to determine the horizontal coordinate in the initial coordinates of the fourth vertex as the horizontal coordinate in the calculated coordinates of the fourth vertex.

[0158] Preferably, the control device as described above, the second processing module comprises:

[0159] A fifth processing submodule, configured to obtain the coordinates of a first vertex and a second vertex on the first contour frame according to the two-dimensional annotation information, wherein the first vertex and the second vertex are arranged diagonally;

[0160] And / or, a sixth processing submodule is used to obtain the coordinates of the third vertex and the coordinates of the fourth vertex on the second contour frame according to the two-dimensional annotation information, and the third vertex and the fourth vertex are set diagonally.

[0161] Preferably, the control device as described above further comprises:

[0162] The fourth processing module is used to obtain updated two-dimensional annotation information according to the two-dimensional annotation change signal when receiving the two-dimensional annotation change signal, and execute the step of obtaining coordinate information of at least two preset annotation points according to the two-dimensional annotation information.

[0163] An embodiment of the control device of the present application is a control device corresponding to the embodiment of the method for determining the utilization rate of resource elements described above. All implementation means in the above method embodiment are applicable to the embodiment of the control device and can achieve the same technical effect.

[0164] Yet another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the three-dimensional annotation method of a two-dimensional image as described above are implemented.

[0165] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity, and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0166] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusions.

[0167] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A three-dimensional annotation method for a two-dimensional image, characterized in that: include: Acquire two-dimensional annotation information of a target object in a two-dimensional image, wherein the two-dimensional annotation information includes at least one of the following: A first contour frame for marking the overall contour of the target object; Two grounding points for marking the contact position between the target object and the ground; A second outline frame for marking the width of the target object; According to the two-dimensional annotation information, obtaining coordinate information of at least two preset annotation points; Generating three-dimensional annotation information of the target object according to the coordinate information of the preset annotation points; The preset marking points include at least: A first vertex and a second vertex on the first outline frame, the first vertex and the second vertex are arranged diagonally, and the first vertex is the upper left vertex of the first outline frame; two said grounding points; and a third vertex and a fourth vertex when the second contour frame is tangent to the straight line where the two grounding points are located, the third vertex corresponds to the first vertex, and the fourth vertex corresponds to the second vertex; The acquiring coordinate information of at least two preset annotation points according to the two-dimensional annotation information includes: According to the first contour frame, respectively determine the coordinates of the first vertex and the coordinates of the second vertex; According to the two grounding points, respectively determine the grounding point coordinates corresponding to the two grounding points; Determine the initial coordinates of the third vertex and the initial coordinates of the fourth vertex according to the second contour frame; The calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex are determined according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, and the two ground point coordinates.

2. The three-dimensional annotation method of a two-dimensional image according to claim 1, characterized in that: Determining the calculated coordinates of the third vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, and the two ground point coordinates includes: The ordinate in the coordinates of the first vertex is determined as the ordinate of the calculated coordinates of the third vertex, and the abscissa in the initial coordinates of the third vertex is determined as the abscissa of the calculated coordinates of the third vertex.

3. The three-dimensional annotation method of a two-dimensional image according to claim 1, characterized in that: Determining the calculated coordinates of the fourth vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, and the two ground point coordinates includes: Acquire a first distance between the two grounding points according to the coordinates of the two grounding points; Determine the coordinates of two extended points on the straight line where the two grounding points are located according to the coordinates of the two grounding points and the first distance, wherein the two extended points are respectively located on a side of one grounding point away from the other grounding point, and the second distance between each extended point and an adjacent grounding point is a preset multiple of the first distance; The calculated coordinates of the fourth vertex are obtained according to the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point and the coordinates of the target extension point, wherein the target grounding point is the grounding point adjacent to the fourth vertex, and the target extension point is the extension point away from the fourth vertex.

4. The three-dimensional annotation method of a two-dimensional image according to claim 3, characterized in that: The step of obtaining the calculated coordinates of the fourth vertex according to the initial coordinates of the third vertex, the initial coordinates of the fourth vertex, the coordinates of the target grounding point, and the coordinates of the target extension point includes: Determine a lateral distance from the third vertex to the fourth vertex according to the initial coordinates of the third vertex and the initial coordinates of the fourth vertex; Determine the ordinate in the calculated coordinates of the fourth vertex according to the lateral distance, the coordinates of the target grounding point and the coordinates of the target extension point; The abscissa in the initial coordinates of the fourth vertex is determined as the abscissa in the calculated coordinates of the fourth vertex.

5. The three-dimensional annotation method of a two-dimensional image according to claim 1, characterized in that: The step of acquiring coordinate information of at least two preset annotation points according to the two-dimensional annotation information includes: According to the two-dimensional annotation information, the coordinates of a first vertex and a second vertex on the first contour frame are acquired, wherein the first vertex and the second vertex are arranged diagonally; And / or, according to the two-dimensional annotation information, the coordinates of the third vertex and the coordinates of the fourth vertex on the second contour frame are obtained, and the third vertex and the fourth vertex are arranged diagonally.

6. The three-dimensional annotation method of a two-dimensional image according to claim 1, characterized in that: Also includes: When a two-dimensional annotation change signal is received, the updated two-dimensional annotation information is obtained according to the two-dimensional annotation change signal, and the step of acquiring coordinate information of at least two preset annotation points according to the two-dimensional annotation information is performed.

7. A control device, characterized in that: include: The first processing module is used to obtain two-dimensional annotation information of the target object in the two-dimensional image, and the two-dimensional annotation information includes at least one of the following: A first contour frame for marking the overall contour of the target object; Two grounding points for marking the contact position between the target object and the ground; A second outline frame for marking the width of the target object; A second processing module, used to obtain coordinate information of at least two preset annotation points according to the two-dimensional annotation information; A third processing module, used to generate three-dimensional annotation information of the target object according to the coordinate information of the preset annotation points; The preset marking points include at least: A first vertex and a second vertex on the first outline frame, the first vertex and the second vertex are arranged diagonally, and the first vertex is the upper left vertex of the first outline frame; Two grounding points; and The third vertex and the fourth vertex when the second contour frame is tangent to the straight line where the two grounding points are located, the third vertex corresponds to the first vertex, and the fourth vertex corresponds to the second vertex; The second processing module comprises: A first processing submodule, used to determine the coordinates of the first vertex and the coordinates of the second vertex respectively according to the first contour frame; The second processing submodule is used to determine the grounding point coordinates corresponding to the two grounding points respectively according to the two grounding points; A third processing submodule, used for determining the initial coordinates of the third vertex and the initial coordinates of the fourth vertex according to the second contour frame; The fourth processing submodule is used to determine the calculated coordinates of the third vertex and the calculated coordinates of the fourth vertex according to the coordinates of the first vertex, the initial coordinates of the third vertex, the initial coordinates of the fourth vertex and the coordinates of the two grounding points.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the three-dimensional annotation method of a two-dimensional image are implemented as described in any one of claims 1 to 6.

Images