A panoramic image annotation method and device
By converting line position information into a panoramic spherical coordinate system and determining pixel coordinates, the problem of low efficiency in manual labeling of line position information in a panoramic system is solved, automatic labeling of panoramic images is realized, and operation efficiency is improved.
Patent Information
- Application Number
- CN202411303600.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-19
AI Technical Summary
The existing panoramic system lacks automatic labeling function, and requires manual addition of line bit information to the panoramic image, especially when the data volume is large, the operation efficiency is inefficient.
By obtaining line position information, panoramic image and shooting information, after preprocessing, the position parameters of line position are converted into position coordinates in the panoramic spherical coordinate system, and the pixels are determined based on the equal ratio relationship between the panoramic spherical coordinate system and the panoramic image. Coordinates, automatic labeling.
The function of automatically labeling line bit information in panoramic images is realized, which improves operation efficiency and reduces the time and cost of manual labeling.
Smart Images

Figure CN118823173B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method and device for annotating a panoramic image. Background Art
[0002] The panoramic system can fully display all objects within a certain range, giving people an immersive feeling by displaying the real scene to the greatest extent. By taking aerial photos of all the objects in the current scene, the captured images are obtained, and then the captured images are processed to obtain a panoramic image showing the real scene.
[0003] In engineering applications, it is necessary to add additional line information to panoramic images to present specific geographic locations or objects. Current panoramic systems do not have automatic annotation capabilities, and line information needs to be manually added at the target location of the panoramic image, which is time-consuming and labor-intensive, especially when the amount of data to be added is large, and the operation efficiency is low. Summary of the invention
[0004] In view of this, an object of the embodiments of the present application is to provide a method and device for annotating a panoramic image, so as to solve the problem of annotating line position information of a panoramic image.
[0005] Based on the above purpose, the embodiment of the present application provides a panoramic image annotation method, including:
[0006] Acquire line position information, a panoramic image, and shooting information of the panoramic image; wherein the shooting information includes position parameters of the shooting position;
[0007] Preprocessing the line position information to obtain preprocessed line position information; wherein the preprocessed line position information includes position parameters of line position points;
[0008] Determine the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the preset position correction parameters; wherein the unfolding plane of the panoramic spherical coordinate system is in geometric proportion to the panoramic image;
[0009] Determine the pixel coordinates of the line point in the panoramic image based on the geometric relationship according to the position coordinates of the line point in the panoramic spherical coordinate system;
[0010] The line point is marked in the panoramic image according to the pixel coordinates of the line point in the panoramic image.
[0011] Optionally, the line position information includes the longitude and latitude of at least one single-point line position and / or a curve data set, wherein the curve data set includes a plurality of line positions constituting the curve and position parameters of each line position; preprocessing the line position information to obtain the preprocessed line position information includes:
[0012] Converting the latitude and longitude into plane coordinates in a spatial rectangular coordinate system by projection calculation; and / or,
[0013] Based on two adjacent line points in the curve data set, construct a new line point and corresponding position parameters;
[0014] The new line point and the corresponding position parameters are added to the curve data set to generate a new curve data set.
[0015] Optionally, the position correction parameter includes a rotation correction parameter and an altitude correction parameter; determining the position coordinates of the line position point in a pre-established panoramic spherical coordinate system according to the position parameter of the line position point, the position parameter of the shooting position and a preset position correction parameter, including:
[0016] Determine the deflection angle of the line point in the panoramic spherical coordinate system according to the position parameter of the line point, the position parameter of the shooting position and the rotation correction parameter;
[0017] The pitch angle of the line point in the panoramic spherical coordinate system is determined according to the position parameter of the line point, the position parameter of the shooting position and the height correction parameter.
[0018] Optionally, the position parameters of the line point include the easting and northing of the line point in the spatial rectangular coordinate system, and the position parameters of the shooting position include the easting and northing of the shooting position in the spatial rectangular coordinate system; determining the deflection angle of the line point in the panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the rotation correction parameter includes:
[0019] Determine the easting and northing of the line point relative to the shooting position according to the easting and northing of the line point and the easting and northing of the shooting position;
[0020] Performing rotation correction on the easting and northing of the line position point relative to the shooting position according to the rotation correction parameter to obtain the easting and northing after rotation correction;
[0021] The deflection angle of the line point in the panoramic spherical coordinate system is determined according to the rotation-corrected easting and northing.
[0022] Optionally, the position parameter of the line point includes the height of the line point in the spatial rectangular coordinate system, and the position parameter of the shooting position includes the camera height of the shooting position in the spatial rectangular coordinate system; determining the pitch angle of the line point in the panoramic spherical coordinate system according to the position parameter of the line point, the position parameter of the shooting position and the height correction parameter includes:
[0023] Correcting the height of the line point according to the height correction parameter to obtain the height of the line point after height correction;
[0024] Determine the vertical height of the line point relative to the shooting position according to the height of the line point after the height correction and the camera height of the shooting position;
[0025] Determine the horizontal distance of the line point relative to the shooting position according to the easting and northing corrected by the rotation;
[0026] The elevation angle of the line point in the panoramic spherical coordinate system is determined according to the vertical height and the horizontal distance.
[0027] Optionally, the deflection angle of the line point in the panoramic spherical coordinate system is:
[0028] (8)
[0029] in, is the rotation-corrected easting, is the rotation-corrected northing;
[0030] The pitch angle of the line point in the panoramic spherical coordinate system is:
[0031] (12)
[0032] Among them, horizontal is the horizontal distance, is the vertical height.
[0033] Optionally, the method for determining the unfolding plane of the panoramic spherical coordinate system includes:
[0034] Determine a first point, a second point, and a third point on the panoramic spherical coordinate system;
[0035] The panoramic spherical coordinate system is unfolded according to the rectangle using a line connecting the first point, the second point and the third point as the center line of the rectangle in the long side direction to obtain the unfolded plane; wherein the wide side of the unfolded plane corresponds to the deflection angle in the panoramic spherical coordinate system, and the high side of the unfolded plane corresponds to the pitch angle in the panoramic spherical coordinate system; the deflection angle corresponding to the wide side of the unfolded plane is in geometric proportion to the width of the panoramic image, and the pitch angle corresponding to the high side of the unfolded plane is in geometric proportion to the height of the panoramic image.
[0036] Optionally, determining the pixel coordinates of the line point in the panoramic image based on the geometric relationship according to the position coordinates of the line point in the panoramic spherical coordinate system includes:
[0037] The pixel coordinates of the line point in the width direction of the panoramic image are:
[0038] (18)
[0039] in, is the deflection angle of the line point in the panoramic spherical coordinate system, is the width of the panoramic image, and n is the width of the panoramic image divided into n parts;
[0040] The pixel coordinates of the line point in the height direction of the panoramic image are:
[0041] (19)
[0042] in, is the elevation angle of the line point in the panoramic spherical coordinate system, is the height of the panoramic image, and m is the height of the panoramic image divided into m parts.
[0043] Optionally, marking the line points in the panoramic image according to the pixel coordinates of the line points in the panoramic image includes:
[0044] For the single point line location, the position of the pixel coordinate corresponding to the single point line location in the panoramic image is marked with the single point line location in a specific form;
[0045] For the new curve data set, the line segment formed by two adjacent line points in the new curve data set is determined according to the pixel coordinates of the two line points, and the line segment is drawn at the corresponding position in the panoramic image using a preset line segment drawing function.
[0046] The present application also provides a panoramic image annotation device, including:
[0047] An acquisition module, used to acquire line position information, a panoramic image, and shooting information of the panoramic image; wherein the shooting information includes position parameters of the shooting position;
[0048] A preprocessing module, used for preprocessing the line position information to obtain preprocessed line position information; wherein the preprocessed line position information includes position parameters of line position points;
[0049] a position determination module, configured to determine the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and a preset position correction parameter; wherein the unfolding plane of the panoramic spherical coordinate system is in geometric proportion to the panoramic image;
[0050] A pixel determination module, configured to determine the pixel coordinates of the line point in the panoramic image based on the geometric relationship according to the position coordinates of the line point in the panoramic spherical coordinate system;
[0051] The marking module is used to mark the line point in the panoramic image according to the pixel coordinates of the line point in the panoramic image.
[0052] From the above description, it can be seen that the panoramic image annotation method and device provided in the embodiment of the present application obtain line position information, panoramic images and shooting information of the panoramic images, pre-process the line position information to obtain the pre-processed line position information, and determine the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the preset position correction parameters; determine the pixel coordinates of the line point in the panoramic image based on the geometric relationship according to the position coordinates of the line point in the panoramic spherical coordinate system, and annotate the line point in the panoramic image according to the pixel coordinates of the line point in the panoramic image, thereby realizing the function of automatically annotating the line position information in the panoramic image, and the operation is convenient and efficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0054] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;
[0055] Figure 2 A schematic diagram of a coordinate system for an embodiment of the present application;
[0056] Figure 3is a schematic diagram of a coordinate system of a panoramic image according to an embodiment of the present application;
[0057] Figure 4 A schematic diagram of a panoramic spherical coordinate system according to an embodiment of the present application;
[0058] Figure 5 A schematic diagram of an unfolded plane of an embodiment of the present application;
[0059] Figure 6 A schematic diagram of coordinates of a panoramic image according to an embodiment of the present application;
[0060] Figure 7 A schematic diagram of line position information according to an embodiment of the present application;
[0061] Figure 8 A schematic diagram of a panoramic image with marked line position information according to an embodiment of the present application;
[0062] Fig. 9 This is a block diagram of the device structure of an embodiment of the present application;
[0063] Fig.10 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0064] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0065] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0066] As described in the background technology section, in some application scenarios, it is necessary to add additional line information to the panoramic image. The line information can represent a specific geographical location, irregular curves such as railways and roads, etc. The current panoramic system does not have the function of automatically marking line information. The manual marking method is time-consuming and labor-intensive, and the efficiency is very low. It is difficult to add curves. Using third-party software to assist in marking is cumbersome and costly.
[0067] In view of this, an embodiment of the present application provides a method for annotating a panoramic image, by converting the position of a line point in a spatial rectangular coordinate system into a corresponding position in a panoramic spherical coordinate system, and then determining the pixel position of the line point in the panoramic image according to the correspondence between the panoramic spherical coordinate system and the panoramic image, thereby marking the line point in the panoramic image, thereby realizing the function of automatically annotating line position information in the panoramic image.
[0068] The technical solution of the present application is further described in detail below through specific embodiments.
[0069] like Figure 1 As shown, the embodiment of the present application provides a panoramic image annotation method, including:
[0070] S101: Acquire line position information, a panoramic image, and shooting information of the panoramic image; wherein the shooting information includes position parameters of the shooting position;
[0071] In this embodiment, in order to realize the marking of line information in the panoramic image, the panoramic image and the shooting information of the panoramic image are first obtained, and the line information to be added to the panoramic image is obtained. The line information includes at least one line point representing a specific geographical location and the position information of the line point; in some methods, the line information includes at least one single-point line point, which is an object located at a specific geographical location, and the corresponding position information is the longitude and latitude of the geographical location of the object, for example, the longitude and latitude of a specific building; in other methods, the line information can be a curve data set, which includes a plurality of line points and the position parameters of each line point, and each line point is connected in sequence to form a curve representing a specific object, such as a road, a railway, a mountain, a river, etc., and the curve can be used to represent the route or outline of the corresponding object. The position parameters of the line point include the easting, northing and height of the line point, and the shooting information of the panoramic image includes the position parameters of the shooting location such as the easting, northing and camera height of the shooting location for shooting the panoramic image.
[0072] In some implementations, the position parameters of the line point and the position parameters of the panoramic image refer to the position parameters of the line point and the shooting position in a spatial rectangular coordinate system. Figure 2As shown, the spatial rectangular coordinate system is defined as, with the shooting position O as the center, when viewed from above, the X axis of the coordinate system is perpendicular to the earth's meridian, and the east direction is the positive direction of the X axis; the Y axis of the coordinate system is perpendicular to the earth's latitude, and the north direction is the positive direction of the Y axis; the vertical ground line is upward as the positive direction of the Z axis. To simplify the calculation, in this embodiment, the earth is simplified into a plane, and the (x, y, z) coordinates in the spatial rectangular coordinate system correspond to the position parameters (easting, northing, height).
[0073] Optionally, the way to obtain each information may be to input each information into the panoramic system, or to load each information from a predetermined storage path, or to obtain each information from a received annotation request, etc., without specific limitation.
[0074] S102: preprocessing the line position information to obtain preprocessed line position information; wherein the preprocessed line position information includes position parameters of the line position points;
[0075] In this embodiment, after the original line position information is obtained, the line position information needs to be pre-processed to facilitate subsequent coordinate transformation and labeling processing.
[0076] In some embodiments, preprocessing the line position information to obtain the preprocessed line position information includes:
[0077] Convert the longitude and latitude into plane coordinates in a spatial rectangular coordinate system by projection calculation; and / or,
[0078] Based on two adjacent line points in the curve data set, construct new line points and corresponding position parameters;
[0079] Add the new line points and corresponding position parameters to the curve dataset to generate a new curve dataset.
[0080] In this embodiment, if the line position information includes the longitude and latitude of the line point, the longitude and latitude need to be converted into plane coordinates in the spatial rectangular coordinate system through projection calculation. Among them, the longitude and latitude are spatial coordinates in the spatial geodetic coordinate system. According to the relationship between the spatial geodetic coordinate system and the spatial rectangular coordinate system, the longitude and latitude are converted into corresponding plane coordinates in the spatial rectangular coordinate system using a specific projection algorithm, and the unit of the plane coordinate is meter. This embodiment does not explain in detail the relationship between the spatial geodetic coordinate system and the spatial rectangular coordinate system, as well as the specific projection algorithm and conversion process.
[0081] If the line position information includes a curve data set, a new line position is constructed based on two adjacent line positions in the curve data set, and the position parameters of the new line position are determined. The new line position and the corresponding position parameters are added to the curve data set to generate a new curve data set. By adding more and denser line positions to the existing line positions in the curve data set, the distance between the line segments of two adjacent line positions can be reduced, and the number of line segments required to be annotated can be increased, so that the curves annotated on the panoramic image are smoother and more natural.
[0082] In some methods, a line segment bisection algorithm is used to add new line points based on the existing line points in the curve data set. Specifically, suppose two adjacent line points in the curve data set , , the distance between points AB is: (1)
[0083] Assume that the interval between equal divisions is , then the number of equally divided points is: (2)
[0084] Coordinates C of the equally divided point i , when the domain i When , it means that the distance between points AB is small, the number of equally divided points is less than 1, and no new points need to be added between the line segments formed by points AB. , indicating that the number of equally divided points between two points is at least 2, and a new point C is added between the line segment formed by the two points AB :
[0085] (3)
[0086] The easting distance of the new point C is , the north distance is , the height is the ground height.
[0087] S103: determining the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the preset position correction parameters; wherein the unfolding plane of the panoramic spherical coordinate system is in geometric proportion to the panoramic image;
[0088] like Figure 2 As shown, in this embodiment, the panoramic spherical coordinate system (polar coordinates) is defined as: the shooting position O is the origin, the spherical radius of the coordinate system is radius, and the north direction is the direction deflection angle , the deflection angle to the east is positive, the deflection angle to the west is negative, the deflection angle to the south is 180° or -180°, and the pitch angle in the horizontal direction is , the pitch angle vertically upward (positive direction of Z axis in the spatial rectangular coordinate system) is , the vertical downward pitch angle is .
[0089] In some embodiments, determining the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position, and the preset position correction parameters includes:
[0090] Determine the deflection angle of the line point in the panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the rotation correction parameters;
[0091] The pitch angle of the line point in the panoramic spherical coordinate system is determined according to the position parameters of the line point, the position parameters of the shooting position and the height correction parameters.
[0092] In this embodiment, the position coordinates of the line point in the panoramic spherical coordinate system are determined according to the position parameters of the line point in the spatial rectangular coordinate system, the position parameters of the shooting position in the spatial rectangular coordinate system, and the position correction parameters used to correct the position conversion between the spatial rectangular coordinate system and the panoramic spherical coordinate system, and the position of the line point in the spatial rectangular coordinate system is converted to the corresponding position in the panoramic spherical coordinate system. Among them, the position correction parameters include a rotation correction parameter for correcting rotation and a height correction parameter for correcting height. The rotation correction parameter and the height correction parameter are obtained by adjusting the superposition effect according to visual observation, or calculated according to the azimuth angle of the photo recorded during shooting. The specific method is not limited. On the one hand, the deflection angle of the line point in the panoramic spherical coordinate system is determined according to the position parameters of the line point, the position parameters of the shooting position, and the rotation correction parameters. On the other hand, the pitch angle of the line point in the panoramic spherical coordinate system is determined according to the position parameters of the line point, the position parameters of the shooting position, and the height correction parameters.
[0093] In some embodiments, determining the deflection angle of the line point in the panoramic spherical coordinate system according to the position parameter of the line point, the position parameter of the shooting position, and the rotation correction parameter includes:
[0094] Determine the easting and northing of the line point relative to the shooting position based on the easting and northing of the line point and the easting and northing of the shooting position;
[0095] According to the rotation correction parameters, the easting and northing of the line position relative to the shooting position are rotated and corrected to obtain the easting and northing after rotation correction;
[0096] According to the rotation-corrected easting and northing, the deflection angle of the line point in the panoramic spherical coordinate system is determined.
[0097] Specifically, first convert the rotation correction parameter (i.e., rotation correction angle) into radians , and the rotation correction parameters after rotation are obtained, which is expressed as:
[0098] (4)
[0099] in, is the rotation correction parameter, is the rotation correction parameter after rotation.
[0100] According to the easting and northing of the line point and the easting and northing of the shooting position, the easting tempX and northing tempY of the line point relative to the shooting position O are determined, which can be expressed as:
[0101] (5)
[0102] (6)
[0103] in, is the easting of the line point, is the northing of the line point, is the easting of the shooting location, is the northing of the shooting location.
[0104] With the shooting position O as the rotation center and the Z axis as the rotation axis, the line position is rotated and corrected according to the rotation correction parameter angleModified1 after rotation, and the easting tempX1 and northing tempY1 after rotation correction are obtained, which are expressed as:
[0105] (7)
[0106] Define the shooting position O as the center, and the deflection angle in the north Y-axis direction is , according to the left-hand rule, clockwise is positive and counterclockwise is negative, and the deflection angle in the north direction is , the deflection angle in the east direction is , the deflection angle in the south direction is or , the deflection angle in the west direction is .
[0107] According to the easting tempX1 and northing tempY1 of the line point after rotation correction, the deflection angle of the line point in the spatial rectangular coordinate system is determined as:
[0108] (8)
[0109] The deflection angle range of the line point is Since the center of the panoramic spherical coordinate system and the center of the spatial rectangular coordinate system are both the shooting position O, the deflection angle of the line point in the spatial rectangular coordinate system is the same as the deflection angle in the panoramic spherical coordinate system, thereby obtaining the deflection angle of the line point in the panoramic spherical coordinate system.
[0110] In some embodiments, determining the pitch angle of the line point in the panoramic spherical coordinate system according to the position parameter of the line point, the position parameter of the shooting position and the height correction parameter includes:
[0111] Correct the height of the line point according to the height correction parameter to obtain the height of the line point after height correction;
[0112] Determine the vertical height of the line point relative to the shooting position according to the height of the line point after height correction and the camera height at the shooting position;
[0113] According to the rotation-corrected easting and northing, determine the horizontal distance of the line point relative to the shooting position;
[0114] According to the vertical height and horizontal distance, the pitch angle of the line point in the panoramic spherical coordinate system is determined.
[0115] Specifically, the height of the line point is corrected according to the height correction parameter to obtain the height of the line point after height correction, which is expressed as:
[0116] (9)
[0117] in, is the height of the line point, is the height correction parameter, is the height of the line point after height correction.
[0118] According to the height of the line point after height correction and the camera height at the shooting position, the vertical height of the line point relative to the shooting position is determined, which is expressed as:
[0119] (10)
[0120] Among them, heightCamera is the camera height, is the height of the line point after height correction, vertical is the vertical height, a positive vertical height indicates that the camera height at the shooting position is lower than the height of the line point, and a negative vertical height indicates that the camera height at the shooting position is higher than the height of the line point.
[0121] According to the easting and northing of the corrected and rotated line position, the horizontal distance of the line position relative to the shooting position is calculated as the horizontal distance of the line position information relative to the camera position. The method is: (11)
[0122] Define the plane formed by the X-axis and the Y-axis with the shooting position O as the center as the pitch angle , the pitch angle in the positive direction of the Z axis is positive, and the pitch angle in the vertical direction upward is , the pitch angle in the negative direction of the Z axis is negative, and the pitch angle in the downward direction perpendicular to the ground is .
[0123] According to the vertical height and horizontal distance of the line point relative to the shooting position, the pitch angle of the line point in the spatial rectangular coordinate system is determined as:
[0124] (12)
[0125] The pitch angle range of the line position is Since the center of the panoramic spherical coordinate system and the center of the spatial rectangular coordinate system are both the shooting position O, the pitch angle of the line point in the spatial rectangular coordinate system is the same as the pitch angle in the panoramic spherical coordinate system, thereby obtaining the pitch angle of the line point in the panoramic spherical coordinate system.
[0126] S104: determining the pixel coordinates of the line point in the panoramic image according to the position coordinates of the line point in the panoramic spherical coordinate system based on a geometric relationship;
[0127] like Figure 3 As shown, the panoramic image coordinate system is defined as: the upper left corner of the panoramic image is the starting point, the image width imageWidth is the direction of the U axis, and the right is the positive direction of the U axis, and the image height imageHeight is the direction of the V axis, and the downward direction is the positive direction of the V axis.
[0128] like Figure 4-6 As shown, the radius of the panoramic sphere is equal to the camera height. The first point C, the second point D and the third point E are determined on the panoramic sphere, where the deflection angle of point C is , the pitch angle is , denoted as C( , ), the deflection angle of point D is , the pitch angle is , expressed as D( , ), the deflection angle of point E is , the pitch angle is , denoted as C( , ).
[0129] The line connecting the first point C, the second point D and the third point E is used as the center line of the rectangle in the long side direction, and the panoramic spherical surface corresponding to the panoramic spherical coordinate system is unfolded according to the rectangle to obtain an unfolded plane corresponding to the panoramic spherical surface, the unfolded plane is rectangular, the long side of the rectangle is equal to the perimeter of the panoramic spherical surface, and the wide side of the rectangle is equal to the half perimeter of the panoramic spherical surface; wherein, the wide side of the unfolded plane (i.e., the long side of the rectangle) corresponds to the deflection angle in the panoramic spherical coordinate system, and the high side of the unfolded plane (i.e., the wide side of the rectangle) corresponds to the pitch angle in the panoramic spherical coordinate system; the deflection angle corresponding to the wide side of the unfolded plane is in geometric proportion to the width of the panoramic image, and the pitch angle corresponding to the high side of the unfolded plane is in geometric proportion to the height of the panoramic image.
[0130] The width direction of the unfolded plane is 0, which is the 0 point of the width axis, indicating that the deflection angle is - Position, width direction The deflection angle is The position of the wide side width indicates the deflection angle is The high side direction 0 of the unfolded plane is the 0 point of the height axis, which means the pitch angle is Position, high side direction The pitch angle is The position of the high side height indicates the pitch angle is location.
[0131] Among them, the width and height of the unfolded plane are expressed as:
[0132] (13)
[0133] (14)
[0134] The aspect ratio of the unfolded plane is:
[0135] (15)
[0136] Since the aspect ratio of the panoramic image is:
[0137] (16)
[0138] Therefore, the aspect ratio of the unfolded plane is proportional to the aspect ratio of the panoramic image, that is:
[0139] (17)
[0140] Based on the proportional relationship between the unfolded plane of the panoramic sphere and the panoramic image, the one-to-one correspondence between the width and height of the panoramic image and the deflection angle and pitch angle of the unfolded plane can be obtained. , ] are mapped one by one in geometric relationship, and the length range of the panoramic image is [0, ] and pitch angle range[ , ] are mapped one to one in geometric proportion.
[0141] In some methods, the pixel position of the line point in the panoramic image is determined based on the deflection angle, pitch angle and width and height of the panoramic image of the line point in a panoramic spherical coordinate system.
[0142] Among them, the pixel coordinates of the line point in the width direction of the panoramic image are:
[0143] (18)
[0144] in, is the deflection angle of the line point in the panoramic spherical coordinate system, is the width of the panoramic image, This is a rounding operation. Divide the width of the panoramic image imageWidth into n parts, multiply it by the deflection angle, and add the image width , and finally rounded to the integer value, which is the position of the line point in the width direction of the panoramic image, imageU. Optionally, n can be 360.
[0145] The pixel coordinates of the line point in the height direction of the panoramic image are:
[0146] (19)
[0147] in, is the pitch angle of the line point in the panoramic spherical coordinate system, is the height of the panoramic image, where the subtraction means that the height direction of the pixel coordinate is opposite to the direction of the pitch angle. m is the height of the panoramic image imageHeight divided into m parts, and optionally, m can be 180.
[0148] S105: Marking the line points in the panoramic image according to the pixel coordinates of the line points in the panoramic image.
[0149] In this embodiment, after determining the pixel coordinates of the line point in the panoramic image, the line point can be marked in the panoramic image. For a single-point line point representing a specific object, the line point can be directly marked in a specific form at the position of the pixel coordinates corresponding to the single-point line point in the panoramic image; Figure 7 ,8 As shown, for a curve data set representing a specific object, the line segment formed by the two line points is determined according to the pixel coordinates of the two adjacent line points, and the line segment is drawn at the corresponding position in the panoramic image using a preset line segment drawing function. When all the line segments formed by the two adjacent line points in the curve data set are drawn, the automatic annotation of the curve in the panoramic image is completed.
[0150] The panoramic image annotation method provided in the embodiment of the present application determines the conversion of the position of the line point in the spatial rectangular coordinate system into the position coordinate in the panoramic spherical coordinate system according to the position parameters of the line point to be annotated, the position parameters of the shooting position of the panoramic image and the set position correction parameters, and then determines the pixel coordinates of the line point in the panoramic image according to the geometric relationship between the unfolded plane of the panoramic spherical coordinate system and the panoramic image, and annotates the line point at the corresponding pixel position of the panoramic image, thereby realizing the function of automatically annotating the line position information in the panoramic image, which is convenient to operate and efficient in annotation.
[0151] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.
[0152] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0153] like Fig. 9 As shown, the embodiment of the present application also provides a panoramic image annotation device, including:
[0154] An acquisition module, used to acquire line position information, a panoramic image and shooting information of the panoramic image; wherein the shooting information includes position parameters of the shooting position;
[0155] A preprocessing module is used to preprocess the line position information to obtain preprocessed line position information; wherein the preprocessed line position information includes position parameters of the line position points;
[0156] A position determination module, used to determine the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the preset position correction parameters; wherein the unfolding plane of the panoramic spherical coordinate system is in geometric proportion to the panoramic image;
[0157] A pixel determination module, used to determine the pixel coordinates of the line point in the panoramic image based on the geometric relationship according to the position coordinates of the line point in the panoramic spherical coordinate system;
[0158] The marking module is used to mark the line points in the panoramic image according to the pixel coordinates of the line points in the panoramic image.
[0159] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0160] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0161] Fig.10 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0162] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0163] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0164] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0165] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0166] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).
[0167] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0168] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0169] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0170] A person of ordinary skill in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.
[0171] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device may be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0172] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.
[0173] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.
Claims
1. A panoramic image annotation method, It is characterized in that include: Acquire line position information, a panoramic image, and shooting information of the panoramic image; wherein the shooting information includes position parameters of the shooting position in a spatial rectangular coordinate system established with the shooting position as the origin; Preprocessing the line position information to obtain preprocessed line position information; wherein the preprocessed line position information includes position parameters of the line position points in the spatial rectangular coordinate system; Determine the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position, and the preset rotation correction parameters and height correction parameters, including: determining the deflection angle of the line point in the panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position, and the rotation correction parameters; determine the pitch angle of the line point in the panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position, and the height correction parameters; wherein the unfolding plane of the panoramic spherical coordinate system is in geometric proportion to the panoramic image, and the panoramic spherical coordinate system is a polar coordinate system established with the shooting position as the origin and the spherical radius as the camera height of the shooting position; According to the position coordinates of the line point in the panoramic spherical coordinate system, based on the geometric relationship, the pixel coordinates of the line point in the panoramic image are determined, including: the pixel coordinates of the line point in the width direction of the panoramic image are: (18) in, is the deflection angle of the line point in the panoramic spherical coordinate system, is the width of the panoramic image, and n is the width of the panoramic image divided into n parts; The pixel coordinates of the line point in the height direction of the panoramic image are: (19) in is the elevation angle of the line point in the panoramic spherical coordinate system, is the height of the panoramic image, and m is the height of the panoramic image divided into m parts; The line point is marked in the panoramic image according to the pixel coordinates of the line point in the panoramic image.
2. The method according to claim 1, It is characterized in that The line position information includes the longitude and latitude of at least one single-point line position and / or a curve data set, wherein the curve data set includes a plurality of line positions constituting the curve and position parameters of each line position; Preprocessing the line position information to obtain preprocessed line position information includes: Convert the longitude and latitude into plane coordinates in a spatial rectangular coordinate system through projection calculation; and / or, Based on two adjacent line points in the curve data set, construct a new line point and corresponding position parameters; The new line point and the corresponding position parameters are added to the curve data set to generate a new curve data set.
3. The method according to claim 2, It is characterized in that The position parameters of the line point include the easting and northing of the line point in the spatial rectangular coordinate system, and the position parameters of the shooting position include the easting and northing of the shooting position in the spatial rectangular coordinate system; determining the deflection angle of the line point in the panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and the rotation correction parameter, includes: Determine the easting and northing of the line point relative to the shooting position according to the easting and northing of the line point and the easting and northing of the shooting position; Performing rotation correction on the easting and northing of the line position point relative to the shooting position according to the rotation correction parameter to obtain the easting and northing after rotation correction; The deflection angle of the line point in the panoramic spherical coordinate system is determined according to the rotation-corrected easting and northing.
4. The method according to claim 3, It is characterized in that The position parameter of the line point comprises the height of the line point in the spatial rectangular coordinate system, and the position parameter of the shooting position comprises the camera height of the shooting position in the spatial rectangular coordinate system; determining the pitch angle of the line point in the panoramic spherical coordinate system according to the position parameter of the line point, the position parameter of the shooting position and the height correction parameter, comprises: Correcting the height of the line point according to the height correction parameter to obtain the height of the line point after height correction; Determine the vertical height of the line point relative to the shooting position according to the height of the line point after the height correction and the camera height of the shooting position; Determine the horizontal distance of the line point relative to the shooting position according to the easting and northing corrected by the rotation; The elevation angle of the line point in the panoramic spherical coordinate system is determined according to the vertical height and the horizontal distance.
5. The method according to claim 4, It is characterized in that The deflection angle of the line point in the panoramic spherical coordinate system is: (8) in, is the rotation-corrected easting, is the rotation-corrected northing; The pitch angle of the line point in the panoramic spherical coordinate system is: (12) Among them, horizontal is the horizontal distance, is the vertical height.
6. The method according to any one of claims 1 to 5, It is characterized in that The method for determining the unfolding plane of the panoramic spherical coordinate system includes: Determine a first point, a second point, and a third point on the panoramic spherical coordinate system; The panoramic spherical coordinate system is unfolded according to the rectangle using a line connecting the first point, the second point and the third point as the center line of the rectangle in the long side direction to obtain the unfolded plane; wherein the wide side of the unfolded plane corresponds to the deflection angle in the panoramic spherical coordinate system, and the high side of the unfolded plane corresponds to the pitch angle in the panoramic spherical coordinate system; the deflection angle corresponding to the wide side of the unfolded plane is in geometric proportion to the width of the panoramic image, and the pitch angle corresponding to the high side of the unfolded plane is in geometric proportion to the height of the panoramic image.
7. The method according to claim 2, It is characterized in that According to the pixel coordinates of the line points in the panoramic image, the line points are marked in the panoramic image, including: For the single point line location, the position of the pixel coordinate corresponding to the single point line location in the panoramic image is marked with the single point line location in a specific form; For the new curve data set, the line segment formed by two adjacent line points in the new curve data set is determined according to the pixel coordinates of the two line points, and the line segment is drawn at the corresponding position in the panoramic image using a preset line segment drawing function.
8. A panoramic image annotation device, It is characterized in that For implementing the method according to claim 1, comprising: An acquisition module, used to acquire line position information, a panoramic image, and shooting information of the panoramic image; wherein the shooting information includes position parameters of the shooting position; A preprocessing module, used for preprocessing the line position information to obtain preprocessed line position information; wherein the preprocessed line position information includes position parameters of line position points; a position determination module, configured to determine the position coordinates of the line point in a pre-established panoramic spherical coordinate system according to the position parameters of the line point, the position parameters of the shooting position and a preset position correction parameter; wherein the unfolding plane of the panoramic spherical coordinate system is in geometric proportion to the panoramic image; A pixel determination module, configured to determine the pixel coordinates of the line point in the panoramic image based on the geometric relationship according to the position coordinates of the line point in the panoramic spherical coordinate system; The marking module is used to mark the line point in the panoramic image according to the pixel coordinates of the line point in the panoramic image.
Citation Information
Patent Citations
Element labeling method, device and equipment, storage medium and computer program product
CN112509135A
Position labeling method and device in panorama, electronic equipment and storage medium
CN115937494A
Labeling method and system for 360-degree panoramic image
CN116091595A
Panoramic image data storage method
CN116739885A
GNSS / IMU surveying and mapping system and method
US20210364654A1
