A tag bidirectional synchronization method, device, electronic device and storage medium

Abstract

The present application discloses a method, device, electronic device and storage medium for bidirectional tag synchronization; the method includes: receiving status information of a drone at the current moment sent by a tag communication server and a video screen sent by a streaming media server; calculating the horizontal field of view angle and vertical field of view angle of the camera based on the status information and determining the position of the video screen in the station center rectangular coordinate system; obtaining the position of the video tag and the position of the three-dimensional scene tag in the video screen and the three-dimensional scene model; synchronizing the video tag to the three-dimensional scene model and synchronizing the three-dimensional scene tag to the video screen based on the position of the video tag, the position of the three-dimensional scene tag, the horizontal field of view angle and vertical field of view angle of the camera and the position of the video screen. The present application can achieve bidirectional synchronization of video tags and three-dimensional scene tags, avoid a large amount of calibration work, expand the monitoring field of view, meet the application requirements of a large scene range, and enhance the user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a tag bidirectional synchronization method, device, electronic device, and storage medium. Background Art

[0002] Bidirectional synchronization of video footage and 3D scene labels leverages the capabilities of a 3D engine, combined with a 3D model. Labels added to the video can be synchronized to the 3D scene, and vice versa. Label positions can also be updated synchronously when the monitoring video position changes. Synchronizing drone video labels with 3D scene labels aims to provide more comprehensive and accurate image information and scene understanding. The main reasons for this are: 1) Providing more precise positioning and navigation: Leveraging the drone's positioning capabilities, synchronizing drone video with 3D scene labels. 3D scene labels provide information such as map layout, building locations, and terrain, helping the drone more accurately determine its position and flight path. 2) Enhancing environmental perception: A drone's camera captures video of its surroundings. Synchronizing this with 3D scene labels enables a more detailed interpretation and understanding of the camera footage. For example, the 3D scene label information can be used to identify the types of objects in the camera's field of view, such as buildings, roads, and people, further enhancing environmental perception. 3) Optimizing target recognition and tracking: Drone video labels are used to identify and classify video content. Synchronizing these with 3D scene labels provides more accurate target recognition and tracking. 3D scene labels allow for the positioning of targets in space, enabling more precise tracking of targets. 4) Improving map annotation accuracy: By synchronizing drone video labels with 3D scene labels, the accuracy of map annotation can be improved. By identifying and classifying targets in drone videos and combining them with information from 3D scene labels, the target location or attributes can be more accurately annotated on the map, providing more reliable map data. Therefore, synchronizing drone video labels with 3D scene labels can provide more comprehensive and accurate image information and environmental understanding, thereby enhancing the drone's performance in positioning, navigation, environmental perception, and target recognition.

[0003] Traditional tag synchronization solutions are based on fixed-position gun cameras and dome cameras. These solutions require manual measurement of the camera's position, posture, and field of view parameters to ensure that the camera's projection in the three-dimensional scene is consistent with the physical world. This requires a significant amount of time for measurement and calibration. Furthermore, fixed-position gun cameras and dome cameras have a limited field of view and cannot cover the entire scene. Summary of the Invention

[0004] The present application provides a bidirectional tag synchronization method, device, electronic device and storage medium, which can realize bidirectional synchronization of video tags and three-dimensional scene tags. It can not only avoid a large amount of calibration work, but also expand the monitoring field of view, meet the application requirements of a large scene range, and enhance the user experience.

[0005] In a first aspect, an embodiment of the present application provides a tag bidirectional synchronization method, applied to a client device, the method comprising:

[0006] Receive the current state information of the drone sent by the mobile device through the tag communication server, and receive the video images sent by the mobile device through the streaming media server;

[0007] Calculating the horizontal and vertical field of view angles of the camera in the drone based on the current state information of the drone, and determining the position of the video image in a station-centered geodesic rectangular coordinate system based on the current state information of the drone; wherein the station-centered geodesic rectangular coordinate system has the position of the camera as its origin and the distance from the position of the camera to the midpoint of the image of the video image as its radius;

[0008] Obtaining the position of the video tag and the position of the three-dimensional scene tag in the video image and the pre-built three-dimensional scene model respectively;

[0009] According to the position of the video tag, the horizontal field of view and vertical field of view of the camera, and the position of the video screen in the station center rectangular coordinate system, the video tag is synchronized to the three-dimensional scene model, and according to the position of the three-dimensional scene tag, the horizontal field of view and vertical field of view of the camera, and the position of the video screen in the station center rectangular coordinate system, the three-dimensional scene tag is synchronized to the video screen.

[0010] In a second aspect, an embodiment of the present application further provides a tag bidirectional synchronization device, the device comprising: a receiving module, a calculating module, an acquiring module and a synchronization module; wherein,

[0011] The receiving module is used to receive the status information of the drone at the current moment sent by the mobile device through the tag communication server, and receive the video images sent by the mobile device through the streaming media server;

[0012] The calculation module is configured to calculate the horizontal field of view angle and the vertical field of view angle of the camera in the drone based on the current state information of the drone, and determine the position of the video image in a station-centered earth-plane rectangular coordinate system based on the current state information of the drone; wherein the station-centered earth-plane rectangular coordinate system has the position of the camera as its origin and the distance from the position of the camera to the position of the midpoint of the image of the video image as its radius;

[0013] The acquisition module is used to respectively acquire the position of the video tag and the position of the three-dimensional scene tag in the video image and the pre-built three-dimensional scene model;

[0014] The synchronization module is used to synchronize the video tag to the three-dimensional scene model according to the position of the video tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video screen in the station's geocentric rectangular coordinate system, and synchronize the three-dimensional scene tag to the video screen according to the position of the three-dimensional scene tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video screen in the station's geocentric rectangular coordinate system.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0016] one or more processors;

[0017] a memory for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the tag bidirectional synchronization method described in any embodiment of the present application.

[0019] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the program is executed by a processor, the tag bidirectional synchronization method described in any embodiment of the present application is implemented.

[0020] The embodiments of the present application propose a tag bidirectional synchronization method, device, electronic device and storage medium, wherein the client device can receive the status information of the drone at the current moment sent by the mobile device through the tag communication server, and receive the video screen sent by the mobile device through the streaming media server; then the horizontal field of view angle and the vertical field of view angle of the camera in the drone are calculated according to the status information of the drone at the current moment, and the position of the video screen in the station-centered rectangular coordinate system is determined according to the status information of the drone at the current moment; then the position of the video tag and the position of the three-dimensional scene tag are obtained in the video screen and the pre-constructed three-dimensional scene model respectively; then the video tag is synchronized to the three-dimensional scene model according to the position of the video tag, the horizontal field of view angle and the vertical field of view angle of the camera and the position of the video screen in the station-centered rectangular coordinate system, and the three-dimensional scene tag is synchronized to the video screen according to the position of the three-dimensional scene tag, the horizontal field of view angle and the vertical field of view angle of the camera and the position of the video screen in the station-centered rectangular coordinate system. That is, in the technical solution of the present application, based on the real-time status information obtained by the drone during flight, video tags can be automatically synchronized to the three-dimensional scene model, and three-dimensional scene tags can also be synchronized to the video image. In the prior art, traditional tag synchronization solutions are based on fixed-position mounted gun cameras and dome cameras. They require manual measurement of the camera's position, posture, and field of view parameters to ensure that the camera's projection in the three-dimensional scene is consistent with the physical world. This requires a lot of time for measurement and calibration. At the same time, gun cameras and dome cameras based on fixed positions have a limited field of view and cannot cover the entire scene. Therefore, compared with the prior art, the bidirectional tag synchronization method, device, electronic device, and storage medium proposed in the embodiments of the present application can achieve bidirectional synchronization between video tags and three-dimensional scene tags. This not only avoids a lot of calibration work, but also expands the monitoring field of view, meets the application requirements of a wide range of scenes, and improves the user experience. In addition, the technical solution of the embodiments of the present application is simple and convenient to implement, easy to popularize, and has a wider range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the first flow chart of the tag bidirectional synchronization method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the structure of a tag bidirectional synchronization system provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a second flow chart of the tag bidirectional synchronization method provided in an embodiment of the present application;

[0024] Figure 4 A third flow chart of the tag bidirectional synchronization method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of the projection of the video screen provided in the embodiment of the present application in the station center rectangular coordinate system;

[0026] Figure 6 A schematic diagram of the principle of video image offset provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a tag bidirectional synchronization device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the present application, not all of the structures.

[0030] Example 1

[0031] Figure 1 This is a first flow chart of the tag bidirectional synchronization method provided in an embodiment of the present application. The method can be executed by a tag bidirectional synchronization device or a client device. The device or client device can be implemented by software and / or hardware. The device or client device can be integrated into any smart device with network communication function. Figure 1 As shown, the tag bidirectional synchronization method may include the following steps:

[0032] S101, receiving the status information of the drone at the current moment sent by the mobile terminal device through the tag communication server, and receiving the video picture sent by the mobile terminal device through the streaming media server.

[0033] Figure 2 This is a schematic diagram of the structure of the tag bidirectional synchronization system provided in the embodiment of the present application. Figure 2As shown, the system may include: a drone, a mobile device, a tag communication server, and a streaming media server. First, when the drone is flying, the mobile device can obtain the drone's current state information and send the above state information to the client device through the tag communication server. The drone's current state information may include: the drone's position, camera attitude, and camera zoom. The drone's position includes: the drone's longitude, latitude, and altitude; the camera's attitude includes: the camera's pitch angle, yaw angle, and roll angle; the camera's zoom includes: the camera's focal length focalLength and zoom step focalStep. Then, the mobile device can calculate the camera's horizontal field of view and vertical field of view angle based on the zoom focal length focalLength and zoom step focalStep and predetermined camera specifications. Specifically, the client device can first calculate the camera's focal length coefficient based on pre-determined camera specifications. It can then calculate the camera's actual focal length based on the camera's focal length coefficient, as well as the camera's zoom focal length and zoom step size. Finally, based on the camera's actual focal length and focal length coefficient, it can calculate the camera's horizontal and vertical field of view angles. Furthermore, the mobile device can also transmit the real-time video stream captured by the drone to the client device via a streaming server. Thus, the client device can receive the video images transmitted by the mobile device via the streaming server.

[0034] S102. Calculate the horizontal and vertical field of view angles of the camera in the drone based on the current state information of the drone, and determine the position of the video image in the station-centered rectangular coordinate system based on the current state information of the drone; wherein the station-centered rectangular coordinate system uses the position of the camera as its origin and the distance from the position of the camera to the midpoint of the video image as its radius.

[0035] In one embodiment, calculating the camera's horizontal and vertical field of view angles can be performed as follows: 1) Query the camera's COMS dimensions from the drone's specifications, where width is w and height is h. 2) Calculate the camera's focal length factor using the following formula: focalFactor = sqrt(36×36+24×24) / sqrt(w×w+h×h). 3) Calculate the camera's actual focal length using the following formula: f = focalLength / focalStep / focalFactor. 4) Calculate the camera's horizontal and vertical field of view angles (HFov) and (VFov) using the following formulas: HFov = 2×atan(w / 2f); VFov = 2×atan(h / 2f).

[0036] In one embodiment, determining the position of the video image in the station-centered geocentric rectangular coordinate system can be performed according to the following steps: 1) establishing a station-centered geocentric rectangular coordinate system based on the status information of the drone at the current moment; wherein the status information of the drone at the current moment includes: the position of the drone, the attitude of the camera, and the zoom of the camera; the position of the drone includes: the longitude, latitude and altitude of the drone; the attitude of the camera includes: the yaw angle, pitch angle, and roll angle of the camera; the zoom of the camera includes: the zoom focal length and zoom step size of the camera; 2) determining the position of the video image in the station-centered geocentric rectangular coordinate system.

[0037] S103: Obtain the position of the video tag and the position of the 3D scene tag in the video image and the pre-built 3D scene model respectively.

[0038] In this step, when synchronizing the video tag to the 3D scene model, the client device can obtain the location of the video tag within the video frame. Specifically, the client device can obtain the location of the tag added by the mouse within the video frame, which is the location of the video tag. When synchronizing the 3D scene tag to the video frame, the client device can obtain the location of the 3D scene tag within the 3D scene model. Specifically, the client device can use the 3D engine to send a ray to the position of the mouse on the screen, and the hit position is the location of the 3D scene tag.

[0039] S104. Synchronize the video tag to the three-dimensional scene model according to the position of the video tag, the horizontal and vertical field of view angles of the camera, and the position of the video screen in the station's earth-centered rectangular coordinate system, and synchronize the three-dimensional scene tag to the video screen according to the position of the three-dimensional scene tag, the horizontal and vertical field of view angles of the camera, and the position of the video screen in the station's earth-centered rectangular coordinate system.

[0040] In this step, the client device can synchronize the video tag with the three-dimensional scene model based on the position of the video tag, the horizontal and vertical field of view angles of the camera, and the position of the video image in the station-centered Cartesian coordinate system. Specifically, the client device can first determine the horizontal rotation angle and vertical rotation angle of the midpoint of the video image based on the current state information of the drone; wherein the horizontal rotation angle of the midpoint of the image is the horizontal rotation angle of the midpoint of the image in the station-centered Cartesian coordinate system, and the vertical rotation angle of the midpoint of the image is the vertical rotation angle of the midpoint of the image in the station-centered Cartesian coordinate system; then, based on the position of the video tag, the horizontal and vertical rotation angles of the midpoint of the image, the horizontal and vertical field of view angles of the camera, and the predetermined camera specifications, the video tag is synchronized with the three-dimensional scene model.

[0041] In this step, the client device can also synchronize the 3D scene tag to the video screen based on the position of the 3D scene tag, the horizontal and vertical field of view angles of the camera, and the position of the video screen in the station's centerline rectangular coordinate system. Specifically, the client device can first send a ray from the position of the camera to the position of the three-dimensional scene tag in the station-centered Cartesian coordinate system to obtain the horizontal rotation angle and vertical rotation angle of the three-dimensional scene tag; wherein, the horizontal rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the horizontal direction in the station-centered Cartesian coordinate system, and the vertical rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the vertical direction in the station-centered Cartesian coordinate system; then, based on the horizontal rotation angle and vertical rotation angle of the three-dimensional scene tag, the position of the video screen in the station-centered Cartesian coordinate system, and the horizontal field of view and vertical field of view of the camera, calculate the horizontal offset ratio and vertical offset ratio of the three-dimensional scene tag in the video screen to the midpoint of the picture; then, based on the horizontal offset ratio and vertical offset ratio of the three-dimensional scene tag in the video screen to the midpoint of the picture, calculate the position of the three-dimensional scene tag synchronized to the video screen, so that the three-dimensional scene tag can be synchronized to the video screen.

[0042] In the bidirectional tag synchronization method proposed in the embodiment of the present application, a client device can receive the current state information of the drone sent by the mobile device through the tag communication server, and receive the video screen sent by the mobile device through the streaming media server; then calculate the horizontal field of view angle and vertical field of view angle of the camera in the drone based on the current state information of the drone, and determine the position of the video screen in the station-centered Cartesian coordinate system based on the current state information of the drone; then obtain the position of the video tag and the position of the 3D scene tag in the video screen and the pre-built 3D scene model respectively; then synchronize the video tag to the 3D scene model based on the position of the video tag, the horizontal field of view angle and vertical field of view angle of the camera, and the position of the video screen in the station-centered Cartesian coordinate system, and synchronize the 3D scene tag to the video screen based on the position of the 3D scene tag, the horizontal field of view angle and vertical field of view angle of the camera, and the position of the video screen in the station-centered Cartesian coordinate system. In other words, in the technical solution of the present application, based on the real-time state information obtained by the drone during flight, the video tag can be automatically synchronized to the 3D scene model, and the 3D scene tag can also be synchronized to the video screen. In the prior art, traditional tag synchronization solutions are based on fixed-position mounted gun cameras and dome cameras. These solutions require manual measurement of the camera's position, posture, and field of view parameters to ensure that the camera's projection in the three-dimensional scene is consistent with the physical world. This requires a significant amount of time for measurement and calibration. Furthermore, fixed-position mounted gun cameras and dome cameras have a limited field of view and cannot cover the entire scene. Therefore, compared to the prior art, the bidirectional tag synchronization method proposed in the present embodiment can achieve bidirectional synchronization between video tags and three-dimensional scene tags. This not only avoids extensive calibration work, but also expands the monitoring field of view, meeting the application requirements of a wide range of scenes and enhancing the user experience. Furthermore, the technical solution in the present embodiment is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0043] Example 2

[0044] Figure 3 This is a second flow chart of the tag bidirectional synchronization method provided in the embodiment of the present application. It is further optimized and expanded based on the above technical solution and can be combined with the above optional implementation methods. Figure 3 As shown, the tag bidirectional synchronization method may include the following steps:

[0045] S301: Receive the status information of the drone at the current moment sent by the mobile device through the tag communication server, and receive the video image sent by the mobile device through the streaming media server.

[0046] S302. Calculate the horizontal and vertical field of view angles of the camera in the drone based on the current state information of the drone, and determine the position of the video image in the station-centered rectangular coordinate system based on the current state information of the drone; wherein the station-centered rectangular coordinate system uses the position of the camera as its origin and the distance from the position of the camera to the midpoint of the video image as its radius.

[0047] S303: Obtain the position of the video tag and the position of the 3D scene tag in the video image and the pre-built 3D scene model respectively.

[0048] S304. Determine the horizontal rotation angle and vertical rotation angle of the midpoint of the image of the video image based on the status information of the drone at the current moment; wherein the horizontal rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the vertical direction in the station-centered earth-plane rectangular coordinate system.

[0049] In this step, if the difference in the 3D engine coordinate system is not considered, the horizontal rotation angle of the image center point is the yaw angle obtained by the drone, the vertical rotation angle of the image center point is the pitch angle obtained by the drone, and the roll angle is generally 0 and can be ignored.

[0050] S305 , synchronizing the video tag to the 3D scene model according to the position of the video tag, the horizontal rotation angle and the vertical rotation angle of the midpoint of the picture, the horizontal field of view angle and the vertical field of view angle of the camera, and predetermined camera specifications.

[0051] In this step, the client device can synchronize the video tag to the three-dimensional scene model based on the position of the video tag, the horizontal rotation angle and vertical rotation angle of the midpoint of the image, the horizontal field of view angle and vertical field of view angle of the camera, and predetermined camera specifications. Specifically, the client device can first calculate the horizontal rotation angle and vertical rotation angle of the video tag based on the position of the video tag, the horizontal rotation angle and vertical rotation angle of the midpoint of the image, the horizontal field of view angle and vertical field of view angle of the camera, and predetermined camera specifications; wherein the horizontal rotation angle of the video tag is the horizontal rotation angle of the video tag in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the video tag is the vertical rotation angle of the video tag in the station-centered earth-plane rectangular coordinate system; then, at the position of the camera in the three-dimensional scene model, rays are sent to the horizontal rotation angle and the vertical rotation angle of the video tag respectively, and the position hit in the three-dimensional scene model is used as the position of the video tag synchronized to the three-dimensional scene model.

[0052] S306. Send a ray from the position of the camera to the position of the three-dimensional scene tag in the station-centered geodesic rectangular coordinate system to obtain the horizontal rotation angle and vertical rotation angle of the three-dimensional scene tag; wherein, the horizontal rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the horizontal direction in the station-centered geodesic rectangular coordinate system, and the vertical rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the vertical direction in the station-centered geodesic rectangular coordinate system.

[0053] S307. Calculate the horizontal and vertical offset ratios of the 3D scene label in the video image relative to the midpoint of the image based on the horizontal rotation angle and vertical rotation angle of the 3D scene label, the position of the video image in the station's centerline rectangular coordinate system, and the horizontal and vertical field of view angles of the camera.

[0054] In this step, the client device can calculate the horizontal and vertical offset ratios of the three-dimensional scene label in the video screen relative to the midpoint of the image based on the horizontal rotation angle and vertical rotation angle of the three-dimensional scene label, the position of the video screen in the station's centerline rectangular coordinate system, and the horizontal and vertical field of view angles of the camera. Specifically, the client device can first calculate the first parameter, the second parameter and the third parameter according to the horizontal rotation angle and the vertical rotation angle of the three-dimensional scene label respectively; then calculate the horizontal rotation angle and the vertical rotation angle of the midpoint of the image according to the position of the video screen in the station-centered earth-plane rectangular coordinate system; wherein the horizontal rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the vertical direction in the station-centered earth-plane rectangular coordinate system; then calculate the horizontal offset of the three-dimensional scene label in the projection plane and the vertical offset of the projection plane according to the first parameter, the second parameter, the third parameter and the horizontal rotation angle and the vertical rotation angle of the midpoint of the image; then calculate the horizontal offset ratio and the vertical offset ratio of the three-dimensional scene label in the video screen to the midpoint of the image according to the horizontal offset and the vertical offset of the projection plane of the three-dimensional scene label and the horizontal field of view and the vertical field of view of the camera.

[0055] S308: Calculate the position of the 3D scene tag synchronized to the video screen according to the horizontal and vertical offset ratios between the 3D scene tag and the midpoint of the image in the video screen.

[0056] In this step, the client device can calculate the position of the 3D scene tag synchronized to the video screen based on the horizontal and vertical offset ratios of the 3D scene tag in the video screen relative to the midpoint of the image. Specifically, the client device can calculate the position of the 3D scene tag synchronized to the video screen according to the following formula: PX = xRadio × 2 / w + 2 / w, PY = yRadio × 2 / h + 2 / h; where PX is the horizontal coordinate of the 3D scene tag synchronized to the video screen, PY is the vertical coordinate of the 3D scene tag synchronized to the video screen, xRadio is the horizontal offset ratio of the 3D scene tag in the video screen relative to the midpoint of the image, yRadio is the vertical offset ratio of the 3D scene tag in the video screen relative to the midpoint of the image, w is the width of the video screen, and h is the height of the video screen.

[0057] In the bidirectional tag synchronization method proposed in the embodiment of the present application, a client device can receive the current state information of the drone sent by the mobile device through the tag communication server, and receive the video screen sent by the mobile device through the streaming media server; then calculate the horizontal field of view angle and vertical field of view angle of the camera in the drone based on the current state information of the drone, and determine the position of the video screen in the station-centered Cartesian coordinate system based on the current state information of the drone; then obtain the position of the video tag and the position of the 3D scene tag in the video screen and the pre-built 3D scene model respectively; then synchronize the video tag to the 3D scene model based on the position of the video tag, the horizontal field of view angle and vertical field of view angle of the camera, and the position of the video screen in the station-centered Cartesian coordinate system, and synchronize the 3D scene tag to the video screen based on the position of the 3D scene tag, the horizontal field of view angle and vertical field of view angle of the camera, and the position of the video screen in the station-centered Cartesian coordinate system. In other words, in the technical solution of the present application, based on the real-time state information obtained by the drone during flight, the video tag can be automatically synchronized to the 3D scene model, and the 3D scene tag can also be synchronized to the video screen. In the prior art, traditional tag synchronization solutions are based on fixed-position mounted gun cameras and dome cameras. These solutions require manual measurement of the camera's position, posture, and field of view parameters to ensure that the camera's projection in the three-dimensional scene is consistent with the physical world. This requires a significant amount of time for measurement and calibration. Furthermore, fixed-position mounted gun cameras and dome cameras have a limited field of view and cannot cover the entire scene. Therefore, compared to the prior art, the bidirectional tag synchronization method proposed in the present embodiment can achieve bidirectional synchronization between video tags and three-dimensional scene tags. This not only avoids extensive calibration work, but also expands the monitoring field of view, meeting the application requirements of a wide range of scenes and enhancing the user experience. Furthermore, the technical solution in the present embodiment is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0058] Example 3

[0059] Figure 4 This is a third flow chart of the tag bidirectional synchronization method provided in the embodiment of the present application. It is further optimized and expanded based on the above technical solution and can be combined with the above optional implementation methods. Figure 4 As shown, the tag bidirectional synchronization method may include the following steps:

[0060] S401: Receive the status information of the drone at the current moment sent by the mobile device through the tag communication server, and receive the video image sent by the mobile device through the streaming media server.

[0061] S402. Calculate the horizontal and vertical field of view angles of the camera in the drone based on the current state information of the drone, and determine the position of the video image in the station-centered rectangular coordinate system based on the current state information of the drone; wherein the station-centered rectangular coordinate system uses the position of the camera as its origin and the distance from the position of the camera to the midpoint of the video image as its radius.

[0062] S403: Obtain the position of the video tag and the position of the 3D scene tag in the video image and the pre-built 3D scene model respectively.

[0063] S404. Determine the horizontal rotation angle and vertical rotation angle of the midpoint of the image of the video image based on the status information of the drone at the current moment; wherein the horizontal rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the vertical direction in the station-centered earth-plane rectangular coordinate system.

[0064] Figure 5 This is a schematic diagram of the structure of the projection of the video screen in the station center rectangular coordinate system provided by the embodiment of the present application. Figure 5 As shown, the centroidal rectangular coordinate system takes the position of the camera as the origin and the distance from the position of the camera to the position of the midpoint of the image of the video screen as the radius. 1) Point A is the midpoint of the image of the video screen, point B is the intersection of point A perpendicular to the xy plane, and line segment OB is the projection line of OA on the xy plane. 2) The horizontal rotation angle is defined as the angle between the positive y-axis and the line segment OB, that is, ∠SOB (S is the point on the positive y-axis), and the rotation in the positive direction of the x-axis is a positive angle. 3) The vertical rotation angle is defined as the angle between the projection line OB and the line segment OA, that is, ∠BOA, and the rotation in the positive direction of the z-axis is a positive angle.

[0065] 4) Plane p, consisting of points P, M, N, and Q, is perpendicular to vector OA and represents the projection range of the video image. 5) Point G is the projection point of the coordinates of a point △P on the video image onto plane p. The length of line segment AD is the vertical projection distance between △P and the midpoint of the image, and the length of line segment DG is the horizontal projection distance. 6) Point C is on perpendicular line AB and has the same z-axis coordinate as point D. ∠ACD = 90°. 7) Since vector OA is perpendicular to AD, ∠DAC = ∠BOA, which is the vertical rotation angle of point A. 8) Point F is the intersection of a line through point D and the plane with the x-axis and a line through point G and parallel to the y-axis. ∠GFD = 90°. Point I is the intersection of a line through point C and parallel to the y-axis with line DF. ∠DIC = 90°, so ∠DCI = ∠SOB, which is the horizontal rotation angle of point A.

[0066] S405. Calculate the horizontal rotation angle and vertical rotation angle of the video tag based on the position of the video tag, the horizontal rotation angle and vertical rotation angle of the midpoint of the image, the horizontal field of view angle and vertical field of view angle of the camera, and predetermined camera specifications; wherein the horizontal rotation angle of the video tag is the rotation angle of the video tag in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the video tag is the rotation angle of the video tag in the vertical direction in the station-centered earth-plane rectangular coordinate system.

[0067] Figure 6 The schematic diagram of the principle of video image offset provided in the embodiment of the present application. Figure 6As shown, 1) the horizontal offset ratio is: xRadio = (PX-W / 2) / (W / 2). 2) the vertical offset ratio is yRadio = (PY-H / 2) / (H / 2). 3) the horizontal offset length (line segment DG) is: xxx = xRadio×tan(HFov / 2×π / 180). 4) the vertical offset length (line segment AD) is: yyy = yRadio×tan(VFov / 2×π / 180). 5) let sinPhi = sin(HorizontalRotation×π / 180), where HorizontalRotation is the horizontal rotation angle of point A. 6) let cosPhi = cos(HorizontalRotation×π / 180). 7) let sinTheta = sin(VerticalRotation×π / 180), where VerticalRotation is the vertical rotation angle of point A. 8) Let cosTheta = cos(VerticalRotation × π / 180). 9) The coordinates of point A are: (cosTheta × sinPhi, cosTheta × cosPhi, sinTheta). 10) The x-axis offset between point A and point G is line segment DG - line segment DI, i.e., line segment IF. The x-axis coordinate of point G is calculated as: xCoord = cosTheta × sinPhi + xxx × cosPhi - yyy × sinTheta × sinPhi. 11) The y-axis offset between point A and point G is: line segment GF + line segment IC. The y-axis coordinate of point G is calculated as: yCoord = cosTheta × cosPhi - xxx × sinPhi - yyy × sinTheta × cosPhi. 12) The z-axis offset between points A and G is line segment AD. The z-axis coordinate of point G is calculated as: zCoord = sinTheta + yyy × cosTheta. 13) The length of line segment OG is: r = sqrt(xCoord × xCoord + yCoord × yCoord + zCoord × zCoord). 14) The horizontal rotation angle of point G, calculated using the spherical polar coordinate system, is: phi = atan²(xCoord, yCoord) × 180 / π. 15) The vertical rotation angle of point G, calculated using the spherical polar coordinate system, is: theta = asin(zCoord / r) × 180 / π.

[0068] S406: Send rays at the camera position in the 3D scene model to the horizontal rotation angle and the vertical rotation angle of the video tag, respectively, and synchronize the hit position in the 3D scene model as the video tag position in the 3D scene model.

[0069] S407. Send a ray from the position of the camera to the position of the three-dimensional scene tag in the station-centered geodesic rectangular coordinate system to obtain the horizontal rotation angle and vertical rotation angle of the three-dimensional scene tag; wherein, the horizontal rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the horizontal direction in the station-centered geodesic rectangular coordinate system, and the vertical rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the vertical direction in the station-centered geodesic rectangular coordinate system.

[0070] S408: Calculate a first parameter, a second parameter, and a third parameter respectively according to the horizontal rotation angle and the vertical rotation angle of the three-dimensional scene label.

[0071] In this step, the client device can calculate the first parameter, the second parameter, and the third parameter based on the horizontal rotation angle and the vertical rotation angle of the 3D scene label. Specifically, 1) the first parameter m = cos(theta×π / 180)×sin(phi×π / 180). 2) the second parameter n = cos(theta×π / 180)×cos(phi×π / 180). 3) the third parameter o = sin(theta×π / 180).

[0072] S409. Calculate the horizontal rotation angle and vertical rotation angle of the midpoint of the image according to the position of the video image in the station-centered geodesic rectangular coordinate system; wherein the horizontal rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the horizontal direction in the station-centered geodesic rectangular coordinate system, and the vertical rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the vertical direction in the station-centered geodesic rectangular coordinate system.

[0073] S410: Calculate the horizontal offset and the vertical offset of the three-dimensional scene label in the projection plane according to the first parameter, the second parameter, the third parameter, and the horizontal rotation angle and the vertical rotation angle of the midpoint of the image.

[0074] In this step, the client device can calculate the horizontal offset and vertical offset of the three-dimensional scene label in the projection plane based on the first parameter, the second parameter, the third parameter, and the horizontal rotation angle and vertical rotation angle of the point in the image. Specifically, 1) let sinPhi = sin(HorizontalRotation×π / 180), where HorizontalRotation is the horizontal rotation angle of point A. 2) let cosPhi = cos(HorizontalRotation×π / 180). 3) let sinTheta = sin(VerticalRotation×π / 180), where VerticalRotation is the vertical rotation angle of point A. 4) let cosTheta = cos(VerticalRotation×π / 180). 5) let t1 = cosPhi×o. 6) let t2 = -sinTheta×sinPhi×o-cosTheta×m. 7) Let t3 = sinTheta × m - cosTheta × sinPhi × o. 8) Let t4 = -sinPhi × o. 9) Let t5 = -sinTheta × cosPhi × o - cosTheta × n. 10) Let t6 = sinTheta × n - cosTheta × cosPhi × o. 11) The horizontal offset of the label in the projection plane xxx = (t3 × t5 - t2 × t6) / (t1 × t5 - t2 × t4). 12) The vertical offset of the label in the projection plane yyy = (t3 × t4 - t1 × t6) / (t2 × t4 - t1 × t5).

[0075] S411. Calculate the horizontal and vertical offset ratios of the 3D scene label relative to the midpoint of the image in the video image based on the horizontal and vertical offsets of the 3D scene label in the projection plane and the horizontal and vertical field of view angles of the camera.

[0076] In this step, the client device can calculate the horizontal offset ratio and vertical offset ratio of the three-dimensional scene label in the video screen relative to the midpoint of the image based on the horizontal offset and vertical offset of the three-dimensional scene label in the projection plane and the horizontal field of view and vertical field of view of the camera. Specifically, the client device can calculate the horizontal offset ratio and vertical offset ratio of the three-dimensional scene label in the video screen to the midpoint of the image according to the following formula: xRadio = xxx / tan(HFov / 2×π / 180), yRadio = yyy / tan(VFov / 2×π / 180); wherein, xRadio is the horizontal offset ratio of the three-dimensional scene label in the video screen to the midpoint of the image, yRadio is the vertical offset ratio of the three-dimensional scene label in the video screen to the midpoint of the image, HFov is the horizontal field of view of the camera, VFov is the vertical field of view of the camera, xxx is the horizontal offset ratio of the three-dimensional scene label in the video screen to the midpoint of the image, and yyy is the vertical offset ratio of the three-dimensional scene label in the video screen to the midpoint of the image.

[0077] S412: Calculate the position of the 3D scene tag synchronized to the video screen according to the horizontal and vertical offset ratios of the 3D scene tag and the midpoint of the image in the video screen.

[0078] In this step, the client device can calculate the position of the 3D scene tag synchronized to the video screen based on the horizontal and vertical offset ratios of the 3D scene tag in the video screen relative to the midpoint of the image. Specifically, the client device can calculate the position of the 3D scene tag synchronized to the video screen according to the following formula: PX = xRadio × 2 / w + 2 / w, PY = yRadio × 2 / h + 2 / h; where PX is the horizontal coordinate of the 3D scene tag synchronized to the video screen, PY is the vertical coordinate of the 3D scene tag synchronized to the video screen, xRadio is the horizontal offset ratio of the 3D scene tag in the video screen relative to the midpoint of the image, yRadio is the vertical offset ratio of the 3D scene tag in the video screen relative to the midpoint of the image, w is the width of the video screen, and h is the height of the video screen.

[0079] In the bidirectional tag synchronization method proposed in the embodiment of the present application, a client device can receive the current state information of the drone sent by the mobile device through the tag communication server, and receive the video screen sent by the mobile device through the streaming media server; then calculate the horizontal field of view angle and vertical field of view angle of the camera in the drone based on the current state information of the drone, and determine the position of the video screen in the station-centered Cartesian coordinate system based on the current state information of the drone; then obtain the position of the video tag and the position of the 3D scene tag in the video screen and the pre-built 3D scene model respectively; then synchronize the video tag to the 3D scene model based on the position of the video tag, the horizontal field of view angle and vertical field of view angle of the camera, and the position of the video screen in the station-centered Cartesian coordinate system, and synchronize the 3D scene tag to the video screen based on the position of the 3D scene tag, the horizontal field of view angle and vertical field of view angle of the camera, and the position of the video screen in the station-centered Cartesian coordinate system. In other words, in the technical solution of the present application, based on the real-time state information obtained by the drone during flight, the video tag can be automatically synchronized to the 3D scene model, and the 3D scene tag can also be synchronized to the video screen. In the prior art, traditional tag synchronization solutions are based on fixed-position mounted gun cameras and dome cameras. These solutions require manual measurement of the camera's position, posture, and field of view parameters to ensure that the camera's projection in the three-dimensional scene is consistent with the physical world. This requires a significant amount of time for measurement and calibration. Furthermore, fixed-position mounted gun cameras and dome cameras have a limited field of view and cannot cover the entire scene. Therefore, compared to the prior art, the bidirectional tag synchronization method proposed in the present embodiment can achieve bidirectional synchronization between video tags and three-dimensional scene tags. This not only avoids extensive calibration work, but also expands the monitoring field of view, meeting the application requirements of a wide range of scenes and enhancing the user experience. Furthermore, the technical solution in the present embodiment is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0080] Example 4

[0081] Figure 7 This is a schematic diagram of the structure of the tag bidirectional synchronization device provided in the embodiment of the present application. Figure 7 As shown, the device includes: a receiving module 701, a calculating module 702, an acquiring module 703 and a synchronizing module 704; wherein,

[0082] The receiving module 701 is used to receive the current state information of the drone sent by the mobile device via the tag communication server, and receive the video images sent by the mobile device via the streaming media server;

[0083] The calculation module 702 is configured to calculate the horizontal and vertical field of view angles of the camera in the drone based on the current state information of the drone, and determine the position of the video image in a station-centered geodesic rectangular coordinate system based on the current state information of the drone; wherein the station-centered geodesic rectangular coordinate system has the position of the camera as its origin and the distance from the position of the camera to the midpoint of the image of the video image as its radius;

[0084] The acquisition module 703 is used to respectively acquire the position of the video tag and the position of the 3D scene tag in the video image and the pre-built 3D scene model;

[0085] The synchronization module 704 is used to synchronize the video tag to the three-dimensional scene model according to the position of the video tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video screen in the station center rectangular coordinate system, and synchronize the three-dimensional scene tag to the video screen according to the position of the three-dimensional scene tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video screen in the station center rectangular coordinate system.

[0086] The above-mentioned tag bidirectional synchronization device can execute the method provided by any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the tag bidirectional synchronization method provided by any embodiment of the present application.

[0087] Example 5

[0088] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present application is shown. Figure 8 The electronic device 12 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present application. The electronic device in the embodiments of the present application can be a mobile device, a client device, a tag communication server, or a streaming media server.

[0089] like Figure 8 As shown, electronic device 12 is implemented as a general-purpose computing device. Components of electronic device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0090] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0091] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0092] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 8 Not shown, often called a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present application.

[0093] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0094] The electronic device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the electronic device 12 via the bus 18. It should be understood that although Figure 8 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0095] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the tag bidirectional synchronization method provided in the embodiment of the present application.

[0096] Example 6

[0097] An embodiment of the present application provides a computer storage medium.

[0098] The computer-readable storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media.Computer-readable media can be computer-readable signal media or computer-readable storage media.Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof.More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination thereof.In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0099] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0100] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0101] The computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0102] Note that the above are only preferred embodiments of the present application and the technical principles employed. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the appended claims.

Claims

1. A tag bidirectional synchronization method, characterized in that: Applied to a client device, the method includes: Receive the current state information of the drone sent by the mobile device through the tag communication server, and receive the video images sent by the mobile device through the streaming media server; Calculating the horizontal and vertical field of view angles of the camera in the drone based on the current state information of the drone, and determining the position of the video image in a station-centered geodesic rectangular coordinate system based on the current state information of the drone; wherein the station-centered geodesic rectangular coordinate system has the position of the camera as its origin and the distance from the position of the camera to the midpoint of the image of the video image as its radius; Obtaining the position of the video tag and the position of the three-dimensional scene tag in the video image and the pre-built three-dimensional scene model respectively; According to the position of the video tag, the horizontal field of view and vertical field of view of the camera, and the position of the video screen in the station center rectangular coordinate system, the video tag is synchronized to the three-dimensional scene model, and according to the position of the three-dimensional scene tag, the horizontal field of view and vertical field of view of the camera, and the position of the video screen in the station center rectangular coordinate system, the three-dimensional scene tag is synchronized to the video screen.

2. The method according to claim 1, characterized in that The status information of the drone at the current moment includes: the position of the drone, the attitude of the camera and the zoom of the camera; the position of the drone includes: the longitude, latitude and altitude of the drone; the attitude of the camera includes: the yaw angle, pitch angle and roll angle of the camera; the zoom of the camera includes: the zoom focal length and zoom step of the camera.

3. The method according to claim 2, characterized in that Synchronizing the video tag to the three-dimensional scene model according to the position of the video tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video image in the station center rectangular coordinate system includes: Determining a horizontal rotation angle and a vertical rotation angle of a midpoint of a picture of the video image according to the state information of the drone at a current moment; wherein the horizontal rotation angle of the midpoint of the picture is the rotation angle of the midpoint of the picture in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the midpoint of the picture is the rotation angle of the midpoint of the picture in the vertical direction in the station-centered earth-plane rectangular coordinate system; The video tag is synchronized to the three-dimensional scene model according to the position of the video tag, the horizontal rotation angle and the vertical rotation angle of the midpoint of the picture, the horizontal field of view angle and the vertical field of view angle of the camera, and the predetermined specification parameters of the camera.

4. The method according to claim 3, characterized in that Synchronizing the video tag into the three-dimensional scene model according to the position of the video tag, the horizontal rotation angle and the vertical rotation angle of the midpoint of the image, the horizontal field of view angle and the vertical field of view angle of the camera, and predetermined specifications of the camera, including: Calculate the horizontal rotation angle and vertical rotation angle of the video tag according to the position of the video tag, the horizontal rotation angle and vertical rotation angle of the midpoint of the picture, the horizontal field of view angle and vertical field of view angle of the camera, and predetermined specifications of the camera; wherein the horizontal rotation angle of the video tag is the rotation angle of the video tag in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the video tag is the rotation angle of the video tag in the vertical direction in the station-centered earth-plane rectangular coordinate system; In the three-dimensional scene model, rays are sent to the horizontal rotation angle and the vertical rotation angle of the video tag at the position of the camera, and the hit position in the three-dimensional scene model is used as the position of the video tag synchronized to the three-dimensional scene model.

5. The method according to claim 4, characterized in that Calculating the horizontal rotation angle and the vertical rotation angle of the video tag according to the position of the video tag, the horizontal rotation angle and the vertical rotation angle of the midpoint of the picture, the horizontal field of view angle and the vertical field of view angle of the camera, and predetermined specifications of the camera includes: Calculating the coordinates of the video tag on the X axis, the Y axis, and the Z axis according to the position of the video tag, the horizontal rotation angle and the vertical rotation angle of the midpoint of the picture, the horizontal field of view angle and the vertical field of view angle of the camera, and predetermined specifications of the camera; Calculate the horizontal rotation angle and the vertical rotation angle of the video tag according to the coordinates of the video tag on the X axis, the coordinates on the Y axis, and the coordinates on the Z axis.

6. The method according to claim 1, characterized in that Synchronizing the 3D scene tag to the video picture according to the position of the 3D scene tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video picture in the station center rectangular coordinate system, including: Sending rays from the position of the camera to the position of the three-dimensional scene tag in the station-centered earth-plane rectangular coordinate system to obtain a horizontal rotation angle and a vertical rotation angle of the three-dimensional scene tag; wherein the horizontal rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the horizontal direction in the station-centered earth-plane rectangular coordinate system, and the vertical rotation angle of the three-dimensional scene tag is the rotation angle of the three-dimensional scene tag in the vertical direction in the station-centered earth-plane rectangular coordinate system; Calculate the horizontal and vertical offset ratios of the 3D scene label in the video image relative to the midpoint of the image based on the horizontal rotation angle and vertical rotation angle of the 3D scene label, the position of the video image in the station center rectangular coordinate system, and the horizontal and vertical field of view angles of the camera; The position where the 3D scene tag is synchronized to the video screen is calculated according to the horizontal and vertical offset ratios of the 3D scene tag in the video screen and the midpoint of the picture.

7. The method according to claim 6, characterized in that Calculating the horizontal and vertical offset ratios of the three-dimensional scene label in the video image relative to the midpoint of the image based on the horizontal rotation angle and the vertical rotation angle of the three-dimensional scene label, the position of the video image in the station center rectangular coordinate system, and the horizontal and vertical field of view angles of the camera, including: Calculate a first parameter, a second parameter, and a third parameter respectively according to the horizontal rotation angle and the vertical rotation angle of the three-dimensional scene label; Calculate the horizontal rotation angle and vertical rotation angle of the midpoint of the image according to the position of the video image in the station center geocentric rectangular coordinate system; wherein the horizontal rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the horizontal direction in the station center geocentric rectangular coordinate system, and the vertical rotation angle of the midpoint of the image is the rotation angle of the midpoint of the image in the vertical direction in the station center geocentric rectangular coordinate system; Calculate the offset of the three-dimensional scene label in the horizontal direction and the offset in the vertical direction of the projection plane according to the first parameter, the second parameter, the third parameter, and the horizontal rotation angle and the vertical rotation angle of the midpoint of the image; Based on the horizontal offset and vertical offset of the three-dimensional scene label in the projection plane and the horizontal field of view and vertical field of view of the camera, the horizontal offset ratio and vertical offset ratio of the three-dimensional scene label in the video image to the midpoint of the image are calculated.

8. The method according to claim 7, characterized in that Calculating the offset of the three-dimensional scene label in the horizontal direction and the offset in the vertical direction of the projection plane according to the first parameter, the second parameter, the third parameter, and the horizontal rotation angle and the vertical rotation angle of the midpoint of the image, including: Calculating the sine and cosine of the horizontal rotation angle of the three-dimensional scene label and the sine and cosine of the vertical rotation angle of the three-dimensional scene label according to the first parameter, the second parameter, the third parameter, and the horizontal rotation angle and the vertical rotation angle of the midpoint of the image; According to the sine value and cosine value of the horizontal rotation angle of the three-dimensional scene label, the sine value and cosine value of the vertical rotation angle of the three-dimensional scene label, and the first parameter, the second parameter and the third parameter, the offset of the three-dimensional scene label in the horizontal direction and the offset in the vertical direction of the projection plane are calculated.

9. A tag bidirectional synchronization device, characterized in that: The device includes: a receiving module, a calculating module, an acquiring module and a synchronizing module; wherein, The receiving module is used to receive the status information of the drone at the current moment sent by the mobile device through the tag communication server, and receive the video images sent by the mobile device through the streaming media server; The calculation module is configured to calculate the horizontal field of view angle and the vertical field of view angle of the camera in the drone based on the current state information of the drone, and determine the position of the video image in a station-centered earth-plane rectangular coordinate system based on the current state information of the drone; wherein the station-centered earth-plane rectangular coordinate system has the position of the camera as its origin and the distance from the position of the camera to the position of the midpoint of the image of the video image as its radius; The acquisition module is used to respectively acquire the position of the video tag and the position of the three-dimensional scene tag in the video image and the pre-built three-dimensional scene model; The synchronization module is used to synchronize the video tag to the three-dimensional scene model according to the position of the video tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video screen in the station's geocentric rectangular coordinate system, and synchronize the three-dimensional scene tag to the video screen according to the position of the three-dimensional scene tag, the horizontal field of view angle and the vertical field of view angle of the camera, and the position of the video screen in the station's geocentric rectangular coordinate system.

10. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the tag bidirectional synchronization method according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the tag bidirectional synchronization method according to any one of claims 1 to 8 is implemented.

Images