Target Tracking Method, Device, System, Electronic Device and Storage Medium

Abstract

The present application relates to a target tracking method, device, system, electronic device, and storage medium. Among them, the target tracking method includes: obtaining a video stream of a video surveillance device, and obtaining a conversion relationship between a camera coordinate system and a world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device; obtaining a flight positioning message; generating an aircraft AR target box according to the conversion relationship, the flight positioning message, and the video stream, and sending the aircraft AR target box to a user terminal for display; and obtaining a first tracking result for an aircraft dynamic target when receiving a tracking instruction for the aircraft AR target box from the user terminal. Through the present application, the problem of low target tracking accuracy in an airport surveillance scenario is solved, and high-precision airport target tracking based on video surveillance and AR technology is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of video surveillance, and particularly to a target tracking method, device, system, electronic device, and storage medium. Background Art

[0002] In the airport surveillance system, the guarantee activities of a single aircraft are in a competitive relationship in terms of resource allocation. Activities such as planning decisions, aircraft selection, and resource allocation are all carried out before the guarantee activities start, and it is very difficult to make purposeful adjustments after the guarantee activities start. The main reason for this situation is that the guarantee personnel of a single aircraft know very little about the purpose of aircraft use and the overall state of the aircraft fleet. Although the fleet manager knows the purpose of aircraft use, due to the low efficiency and lack of real-time nature of summarizing status information, the adjustment cannot keep up with the changes and is difficult to achieve, resulting in low accuracy of target tracking in the airport surveillance scenario.

[0003] Currently, for the problem of low accuracy of target tracking applied to the airport surveillance scenario in the related art, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of the present application provide a target tracking method, device, system, electronic device, and storage medium to at least solve the problem of low accuracy of target tracking in the airport surveillance scenario in the related art.

[0005] In a first aspect, embodiments of the present application provide a method, which includes:

[0006] Obtain the video stream of the video surveillance device, and obtain the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device;

[0007] Obtain the flight positioning message;

[0008] Generate an aircraft Augmented Reality (AR) target box according to the conversion relationship, the flight positioning message, and the video stream, and send the aircraft AR target box to the user terminal for display;

[0009] In the case of receiving the tracking instruction for the aircraft AR target box from the user terminal, obtain the first tracking result for the aircraft dynamic target.

[0010] In one embodiment, after obtaining the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream, the method further includes:

[0011] Obtain the vehicle positioning message of the airport vehicle system;

[0012] Generate a vehicle AR target box based on the conversion relationship, the vehicle positioning message, and the video stream, and send the vehicle AR target box to the user terminal for display;

[0013] When receiving a tracking instruction for the vehicle AR target box from the user terminal, obtain a second tracking result for the vehicle dynamic target.

[0014] In one embodiment, after obtaining the vehicle positioning message of the airport vehicle system, the method further includes:

[0015] Push the vehicle positioning message to the user terminal; wherein, the user terminal displays the license plate number in the vehicle positioning message in the vehicle AR target box.

[0016] In one embodiment, the method further includes:

[0017] Push the flight status message of the airport flight guarantee system to the user terminal;

[0018] Wherein, when the user terminal receives a viewing instruction for the aircraft AR target box, it displays the flight status message and displays the video stream matching the flight status message.

[0019] In one embodiment, the obtaining the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream includes:

[0020] Obtain multiple points on the video stream through the user terminal, and obtain the point pixel coordinates of the points on the video stream;

[0021] Obtain the corresponding geographical coordinates of the points through a positioning device;

[0022] Obtain the conversion relationship according to the point pixel coordinates and the geographical coordinates.

[0023] In one embodiment, after obtaining the flight positioning message, the method further includes:

[0024] Obtain the flight positioning message through a broadcast monitoring system, and store the flight positioning message in a message queue;

[0025] Take out the flight positioning message from the message queue in sequence and push it to the user terminal; or store the flight positioning message in a database through the message queue.

[0026] In one embodiment, the obtaining the video stream of the video monitoring device includes:

[0027] Obtain the identity information of the user uploaded by the user terminal, and verify the identity information;

[0028] In the case of successful verification, send the obtained video stream to the user terminal for display; or, send the video stream to a storage device for storage.

[0029] In one embodiment, after sending the aircraft AR target frame to the user terminal for display, the method further includes:

[0030] Push the flight positioning message to the user terminal; wherein, the user terminal displays the flight information in the flight positioning message in the aircraft AR target frame.

[0031] In a second aspect, an embodiment of the present application provides a target tracking device, and the device includes: a conversion module, an acquisition module, a generation module, and a tracking module;

[0032] The conversion module is used to obtain the video stream of the video monitoring device, and obtain the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video monitoring device;

[0033] The acquisition module is used to obtain the flight positioning message;

[0034] The generation module is used to generate an aircraft AR target frame according to the conversion relationship, the flight positioning message, and the video stream, and send the aircraft AR target frame to the user terminal for display;

[0035] The tracking module is used to obtain a first tracking result for the aircraft dynamic target in the case of receiving a tracking instruction for the aircraft AR target frame from the user terminal.

[0036] In a third aspect, an embodiment of the present application provides a target tracking system, and the system includes: a server, a video monitoring device, and a user terminal; wherein, the server is respectively connected to the video monitoring device and the user terminal;

[0037] The video monitoring device is used to obtain a video stream and send the video stream to the server;

[0038] The user terminal is used to receive the video stream from the server for display;

[0039] The server is used to execute the target tracking method as described in the first aspect above.

[0040] Fourthly, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the target tracking method described in the first aspect above is implemented.

[0041] Fifthly, an embodiment of the present application provides a storage medium, on which a computer program is stored. When the program is executed by a processor, the target tracking method described in the first aspect above is implemented.

[0042] Compared with the related art, the target tracking method, device, system, electronic device, and storage medium provided by the embodiments of the present application obtain a video stream of a video monitoring device, and obtain a conversion relationship between a camera coordinate system and a world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video monitoring device; obtain flight positioning messages; generate an aircraft AR target box according to the conversion relationship, the flight positioning messages, and the video stream, and send the aircraft AR target box to a user terminal for display; in the case of receiving a tracking instruction for the aircraft AR target box from the user terminal, obtain a first tracking result for the aircraft dynamic target, solve the problem of low target tracking accuracy in the airport monitoring scenario, and achieve high-precision airport target tracking based on video monitoring and AR technology. Description of the Drawings

[0043] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0044] Figure 1 is a schematic diagram of an application scenario of a target tracking method according to an embodiment of the present application;

[0045] Figure 2 is a flowchart of a target tracking method according to an embodiment of the present application;

[0046] Figure 3 is a flowchart of another target tracking method according to an embodiment of the present application;

[0047] Figure 4 is a flowchart of yet another target tracking method according to an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of a video frame according to an embodiment of the present application;

[0049] Figure 6 is a schematic diagram of a target tracking architecture according to an embodiment of the present application;

[0050] Figure 7It is a schematic diagram of the effect of a target tracking method according to an embodiment of the present application;

[0051] Figure 8 It is a structural block diagram of a target tracking device according to an embodiment of the present application;

[0052] Figure 9 It is a structural block diagram of a target tracking system according to an embodiment of the present application;

[0053] Figure 10 It is a structural diagram inside a computer device according to an embodiment of the present application. Detailed implementation manners

[0054] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be described and explained below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided in the present application without making creative efforts belong to the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood that the content disclosed in the present application is insufficient.

[0055] Referring to "embodiment" in the present application means that the specific features, structures or characteristics described in combination with the embodiment may be included in at least one embodiment of the present application. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those of ordinary skill in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0056] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the ordinary meanings understood by those with ordinary skills in the technical field to which this application belongs. The words such as "a", "an", "one kind", "the" and the like involved in this application do not indicate a quantity limitation and may represent a singular or plural number. The terms "including", "containing", "having" and any variations thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but may further include unlisted steps or units, or may further include other steps or units inherent to these processes, methods, products or devices. The words such as "connected", "linked", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. The terms "first", "second", "third" and the like involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0057] In this embodiment, an application scenario of a target tracking method is provided. Figure 1 It is a schematic diagram of an application scenario of a target tracking method according to an embodiment of the present application, as Figure 1 shown. In this application environment, it includes a user terminal 12 and a server 14. The user terminal 12 communicates with the server 14 through a network; the server 14 obtains a video stream of a video monitoring device and obtains a conversion relationship between a camera coordinate system and a world coordinate system based on the video stream; the server 14 obtains a flight positioning message, generates an aircraft AR target box according to the conversion relationship, the flight positioning message and the video stream, and sends the aircraft AR target box to the user terminal 12 for display, and finally obtains a first tracking result for the dynamic target of the aircraft. The user terminal 12 may be, but is not limited to, various smart phones, personal computers, laptop computers and tablet computers, and the server 14 may be implemented by an independent server or a server cluster composed of multiple servers.

[0058] This embodiment provides a target tracking method, and this target tracking method is applied to an airport management system. Figure 2 It is a flowchart of a target tracking method according to an embodiment of the present application, as Figure 2 shown. This process includes the following steps:

[0059] Step S202: Obtain the video stream of the video surveillance device, and based on this video stream, obtain the conversion relationship between the camera coordinate system and the world coordinate system; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device.

[0060] Among them, the above video surveillance device can be a panoramic bullet camera, a panoramic camera, a binocular camera, a video recorder or other devices; the video surveillance device can be deployed in scenarios such as airports. Through the video stream of the airport captured by the video surveillance device, calibrate the video surveillance device, and thus obtain the conversion relationship between the coordinates of the camera coordinate system and the world coordinate system based on the calibration result.

[0061] Step S204: Obtain the flight positioning message.

[0062] Among them, the above flight positioning message refers to the positioning information corresponding to each flight aircraft, and the flight positioning message includes: flight information, flight position longitude, flight position latitude, flight position altitude, etc. Specifically, the above airport management system obtains flight messages from an external system connected to it, such as an Automatic Dependent Surveillance - Broadcast (ADS - B) system, filters out flight messages related to the airport, and then assembles and generates flight positioning messages.

[0063] Step S206: Generate an aircraft AR target box according to the conversion relationship, the flight positioning message, and the video stream, and send the aircraft AR target box to the user terminal for display.

[0064] Specifically, the above airport management system uses the above conversion relationship to convert the geographical coordinates of the aircraft in the above flight positioning message, that is, the flight position longitude, flight position latitude, and flight position altitude, to the camera coordinate system of the above video surveillance device, and then obtains the aircraft pixel coordinates in the camera coordinate system. Then, based on the aircraft pixel coordinates, draw and generate the above aircraft AR target box at the corresponding position in the picture of the above video stream. Among them, the aircraft AR target box is sent to the above user terminal, and the user terminal can display the video stream picture with the aircraft AR target box superimposed to the user, that is, airport staff.

[0065] In one embodiment, after sending the aircraft AR target box to the user terminal for display, the target tracking method further includes the following steps: Push the flight positioning message to the user terminal; wherein, the user terminal displays the flight information in the flight positioning message in the aircraft AR target box. That is, when drawing and generating the aircraft AR target box at the specified position in the picture of the above video stream, the flight positioning message can be pushed to the user terminal, and the flight positioning message and the aircraft AR target box are synchronously displayed for the user to view.

[0066] Step S208, when receiving a tracking instruction for the aircraft AR target box from the user terminal, obtain a first tracking result for the aircraft dynamic target.

[0067] Specifically, the user can interact with the above-mentioned user terminal, and the user terminal sends the above-mentioned tracking instruction for the aircraft AR target box to the above-mentioned airport management system. For example, the user can click on the above-mentioned aircraft AR target box in the display screen of the user terminal. At this time, when the above-mentioned airport management system receives the tracking instruction for the aircraft AR target box, it can obtain the above-mentioned first tracking result from the external monitoring device based on the tracking instruction for the aircraft AR target box. It should be noted that the external monitoring device can be a dome camera device, a bullet camera device, a maglev camera device, or other devices used to monitor and shoot detailed scenes and capable of adjusting the shooting angle. The external monitoring device can be bound to the above-mentioned video monitoring device, and the airport management system controls the external monitoring device to rotate the angle and monitor the aircraft dynamic target indicated by the tracking instruction by controlling the video monitoring device; or, the external monitoring device can also be set on a pan-tilt, and the airport management system instructs the pan-tilt to drive the external monitoring device to rotate, which will not be elaborated here. Among them, taking the external monitoring device as a dome camera device as an example, the airport management system synchronizes the vehicle geographical coordinates in the above-mentioned flight positioning message to the dome camera device in real time, and then controls the dome camera device to continuously track the aircraft dynamic target, and uses the enlarged video picture of the continuous tracking of the aircraft dynamic target as the above-mentioned first tracking result. It can be understood that the above-mentioned user terminal can display the first tracking result at an appropriate position in the picture of the above-mentioned video stream. For example, it can display the enlarged video picture of the aircraft dynamic target in the lower right corner of the video stream picture for the user to preview.

[0068] In the related art, the efficiency and accuracy of summarizing the status information of the airport are relatively low, and it only supports obtaining the aircraft status by augmented reality for the aircraft in a stationary state, and cannot provide an intuitive and effective auxiliary decision-making means. However, in the embodiments of the present application, through the above-mentioned steps S202 to S208, by means of technologies such as video monitoring technology and AR technology, on the video picture of video monitoring devices such as panoramic bullet cameras, based on the obtained conversion relationship, flight positioning message and video stream, an AR target box is generated for stationary or dynamic targets, and at the same time, the target is video-tracked and enlarged, providing a very intuitive picture display for the user, being able to feedback the dynamic targets in the airport flight area to the airport staff in real time and comprehensively, upgrading the manual scheduling in the airport flight area to intelligent scheduling, and providing an intelligent auxiliary decision-making means, thereby effectively solving the problem of low accuracy of target tracking in the airport monitoring scenario and realizing high-precision airport target tracking based on video monitoring and AR technology.

[0069] In one embodiment, a target tracking method is provided.Figure 3 It is a flowchart of another target tracking method according to an embodiment of the present application. As Figure 3 shown, the process includes the following steps:

[0070] Step S302, obtain the vehicle positioning message of the airport vehicle system.

[0071] Among them, the above-mentioned airport vehicle system refers to an external system used to generate vehicle information in the airport scenario. Connecting the airport management system to which the target tracking method in the present application is applied to the airport vehicle system can obtain the real-time vehicle information stored in the airport vehicle system. Then, the airport management system assembles the real-time vehicle information into the above-mentioned vehicle positioning message. The vehicle positioning message includes the license plate number and the geographical coordinates of the vehicle, etc.; among them, the geographical coordinates of the vehicle include the vehicle position longitude, the vehicle position latitude, and the vehicle position altitude.

[0072] Step S304, generate a vehicle AR target box according to the conversion relationship, the vehicle positioning message, and the video stream, and send the vehicle AR target box to the user terminal for display.

[0073] Specifically, the above-mentioned airport management system uses the above-mentioned conversion relationship to convert the geographical coordinates of the vehicle in the above-mentioned vehicle positioning message, that is, the vehicle position longitude, the vehicle position latitude, and the vehicle position altitude, into the camera coordinate system of the above-mentioned video monitoring device, and then obtains the vehicle pixel coordinates in the camera coordinate system. Then, based on the vehicle pixel coordinates, the above-mentioned vehicle AR target box is drawn and generated at the corresponding position in the picture of the above-mentioned video stream. Among them, the vehicle AR target box is sent to the above-mentioned user terminal, and the user terminal can display the video stream picture with the vehicle AR target box superimposed thereon to the user, that is, the airport staff.

[0074] Step S306, when receiving the tracking instruction of the user terminal for the vehicle AR target box, obtain the second tracking result for the vehicle dynamic target.

[0075] Specifically, the user can interact with the above user terminal, and the user terminal can send the above tracking instruction for the vehicle AR target box to the above airport management system. For example, the user can click on the above vehicle AR target box in the display screen of the user terminal. Similar to the above step S208, when the above airport management system receives the tracking instruction for the vehicle AR target box, it can obtain the second tracking result for the vehicle dynamic target from the above external monitoring device based on the tracking instruction for the vehicle AR target box. For example, the vehicle geographical coordinates in the above vehicle positioning message are synchronously sent to the PTZ device bound to the above video monitoring device in real time, and then the PTZ device is controlled to continuously track the vehicle dynamic target, and the enlarged video image of the continuous tracking of the vehicle dynamic target is used as the above second tracking result. Among them, the above user terminal can display the second tracking result at an appropriate position in the picture of the above video stream. For example, the enlarged video image of the vehicle dynamic target can be displayed in the lower right corner of the video stream picture for the user to preview.

[0076] Through the above steps S302 to S306, based on the vehicle positioning message of the airport vehicle system, a vehicle AR target box is generated and displayed through the user terminal. Furthermore, when the tracking instruction of the user terminal is received, the vehicle dynamic target is specifically tracked according to the vehicle positioning message, thereby realizing the real-time and comprehensive feedback of the vehicle dynamic target in the airport apron to the airport staff, and effectively improving the accuracy of target tracking in the airport application scenario.

[0077] In one embodiment, after the above step S302, the target tracking method further includes the following steps: pushing the vehicle positioning message to the user terminal; wherein, the user terminal displays the license plate number in the vehicle positioning message in the vehicle AR target box.

[0078] Among them, after the above airport management system obtains the vehicle positioning message, the vehicle positioning message can be sent into the message queue deployed in the airport management system, and the vehicle positioning messages saved in the message queue are taken out in sequence, so as to store the vehicle positioning message in the database of the airport management system. In addition, the airport apron management service of the airport management system can also obtain the vehicle positioning message in the message queue and push the vehicle positioning message to the above user terminal through the message push service. When the user terminal receives the above pushed vehicle positioning message, the user terminal can display the license plate number of the corresponding vehicle in the vehicle AR target box drawn in the video stream.

[0079] Through the above embodiment, the vehicle positioning message is pushed to the user terminal, and the user terminal displays the license plate number of the corresponding vehicle in the vehicle AR target box based on the vehicle positioning message, so that the user can quickly and easily find the target dynamic vehicle in the video stream, and further provide real-time and comprehensive feedback of the airport flight zone dynamic targets to the airport staff, thereby improving the accuracy of airport management.

[0080] In one embodiment, a target tracking method is provided. Figure 4 is a flowchart of another target tracking method according to an embodiment of the present application. Figure 4 As shown, the process includes the following steps:

[0081] Step S402, pushing the flight status message of the airport flight support system to the user terminal; wherein, upon receiving a viewing instruction for the aircraft AR target frame, the user terminal displays the flight status message and displays the video stream matching the flight status message.

[0082] Among them, the above-mentioned airport management system obtains flight status messages from an external system connected to it, such as the airport flight support system; the flight status message includes the flight number, including the pre-takeoff status, stopover status, landing status, runway departure status, in-position status and boarding gate status. The above-mentioned airport management system can push the above-mentioned flight status message to the user terminal. If the user interacts with the user terminal, for example, the user clicks on the above-mentioned aircraft AR target box on the user terminal, the user terminal will receive a viewing instruction corresponding to the above-mentioned user operation for the aircraft AR target box; at this time, the above-mentioned received flight status message is displayed to the user through the user terminal, so as to view the details of the aircraft flight history status. At the same time, the airport management system can also support the playback of video streams corresponding to the time of the flight status of the flight history message.

[0083] Through the above step S402, based on the flight status message of the airport flight support system, the user terminal is instructed to display the flight history status and video stream, so that the user can trace the flight status, further improving the convenience of airport flight management.

[0084] In one embodiment, the above-mentioned obtaining the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream further includes the following steps:

[0085] Step S502: obtaining multiple points on the video stream through the user terminal, and obtaining the pixel coordinates of the points on the video stream corresponding to the points.

[0086] Specifically, taking the above video surveillance device using a panoramic camera device as an example; this panoramic camera device usually includes several lenses, and multiple lenses are spliced into a panoramic picture. Hereinafter, an example in which the panoramic camera device has 4 lenses will be used for illustration. Figure 5 is a schematic diagram of a video picture according to an embodiment of the present application. As Figure 5 shown, the real-time video picture of the above panoramic camera device is displayed on the above user terminal, the video picture is equally divided into 4 parts, at least 3 points can be selected in each equal division interval, and the user terminal parses these points through parsing software to obtain the point pixel coordinates of each point in the video picture based on the video picture resolution; the point pixel coordinates can be represented by (A, B), where A represents the distance length relative to the starting point in the pixel, and B represents the distance width relative to the starting point in the pixel. Among them, in the embodiment of the present application, the pixel coordinates of the starting point in the upper left corner of the above video picture are (0, 0).

[0087] Step S504, obtaining the geographical coordinates corresponding to the point through a positioning device.

[0088] Among them, manual intervention can be carried out to move to each point marked on the panoramic camera device on the airport ground, and then a high-precision positioning device is used to collect the geographical coordinates corresponding to these points; the geographical coordinates can be represented by (X, Y, H), where X represents the longitude of the point deployed in the airport, Y represents the latitude, and H represents the altitude.

[0089] Step S506, obtaining the conversion relationship according to the point pixel coordinates and the geographical coordinates.

[0090] Among them, the user can input the point pixel coordinates (A, B) and geographical coordinates (X, Y, H) of these points one by one through the user terminal, and then the user terminal calculates the conversion relationship between the pixel coordinates and geographical coordinates in the video picture. It should be noted that if there are multiple video surveillance devices in the airport, each video surveillance device repeats the above calibration process.

[0091] Through the above steps S502 to S506, each video surveillance device is calibrated based on the point pixel coordinates, and the conversion relationship between the camera coordinate system and the world coordinate system of each video surveillance device is obtained, which can effectively improve the calibration efficiency and accuracy of the video surveillance device, thereby further improving the efficiency and accuracy of target tracking.

[0092] In one embodiment, after the above step S204 is executed, the target tracking method further includes the following steps: obtaining the flight positioning message through a broadcast monitoring system, and storing the flight positioning message in a message queue; sequentially taking out the flight positioning message from the message queue and pushing it to the user terminal; or storing the flight positioning message in a database through the message queue.

[0093] Among them, the above broadcast monitoring system can be an ADS-B system. The above airport management system sequentially stores the flight positioning messages obtained through the broadcast monitoring system into the message queue, and sequentially retrieves the flight positioning messages saved in the message queue, so as to store the flight positioning messages into the database of the airport management system; in addition, the airport flight area management service of the airport management system can also obtain the flight positioning messages in the message queue, and push the flight positioning messages to the above user terminal through the message push service. When the user terminal receives the pushed flight positioning message, the flight details of the corresponding flight can be displayed in the flight AR target box drawn in the video stream by the user terminal. Through the above embodiments, storing the flight positioning messages in the message queue makes the access to the flight positioning messages more orderly and improves the efficiency of data processing in the target tracking process.

[0094] In one embodiment, the above obtaining the video stream of the video monitoring device further includes the following steps: obtaining the identity information of the user uploaded by the user terminal and verifying the identity information; in the case of successful verification, sending the obtained video stream to the user terminal for display; or sending the video stream to a storage device for storage.

[0095] Among them, the identity of the logged-in user can be verified through the above user terminal, that is, the obtained identity information of the user will be matched with the password information stored in the user terminal or the above airport management system. If the match is successful, it means that the identity verification is passed, and the above video stream can be sent to the user terminal so that the above qualified user can view the picture of the video stream; or, the above airport management system can also save the video stream picture data of the video monitoring device to the storage device through the streaming media service. The storage device can be a device deployed in the airport management system that can store picture or audio and video data.

[0096] The above identity verification process can be: setting the above password information as preset password data; the user terminal performs password recognition, taking the password entered by the user on the user terminal as user information, and comparing and authenticating the user information with the corresponding password data. Or, the above password information can also be pre-stored fingerprint information; the user terminal performs a fingerprint security recognition mode, taking the fingerprint entered by the user on the user terminal as user information, and comparing and authenticating the user information with the corresponding fingerprint information. It can be understood that the above password information can also be information such as smart card recognition information, face recognition data, etc., and the user terminal performs identity recognition corresponding to the password information, which will not be elaborated here.

[0097] Through the above embodiments, the identity information of the user uploaded by the user terminal is used to authenticate the user, and the pushed video stream is displayed through the user terminal only when the authentication is passed, thereby realizing the authentication of the user's identity, avoiding irrelevant personnel from querying and managing the airport flight status, and effectively improving the management security of target tracking in the airport scenario.

[0098] The embodiments of the present application will be described in detail below in combination with actual application scenarios. Figure 6 is a schematic diagram of a target tracking architecture according to an embodiment of the present application, as Figure 6 shown. The target tracking architecture includes a user terminal, an airport management system, and a video surveillance device. Among them, a storage device, a streaming media service, a message queue, a database, a message push service, an airport apron management service, and multiple docking services are deployed in the airport management system. The message queue in the airport management system docks with the corresponding external systems through each docking service, and then obtains the flight location messages, vehicle location messages, and flight status messages of each external system respectively. The message queue sends flight location information, etc. to the database for storage, and pushes each message to the user terminal through the airport apron management service and the message push service. The video surveillance device sends audio and video data to the streaming media service, and the streaming media service stores the audio and video data in the storage device, and sends the audio and video data to the user terminal for display. Finally, the user terminal displays a video surveillance screen with real-time dynamic AR tags for airplanes and vehicles.

[0099] Figure 7 is a schematic diagram of the effect of a target tracking method according to an embodiment of the present application, as Figure 7 shown. At this time, the above video surveillance device uses a panoramic camera device. Figure 7 The overall picture in is the video picture captured by the panoramic camera device; two airplane AR target frames and one vehicle AR target frame that have been recognized are distributed on the overall picture, and the flight numbers are displayed in each airplane AR target frame, which are QDA9794 and CSN6224 respectively; at this time, the dynamic target of the airplane with the flight number QDA9794 is in a selected state, and the airport management system controls the PTZ device to continuously track the dynamic target of the airplane and obtains a tracking video picture. Figure 7 The lower left corner is the tracking video picture of the PTZ device tracking the target clicked by the user and magnifying it.

[0100] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0101] This embodiment also provides a target tracking device, which is used to implement the above embodiments and preferred embodiments. Those that have been described will not be repeated. As used hereinafter, terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0102] Figure 8 is a structural block diagram of a target tracking device according to an embodiment of the present application. As Figure 8 shown, the device includes: a conversion module 82, an acquisition module 84, a generation module 86, and a tracking module 88.

[0103] The conversion module 82 is used to obtain the video stream of the video surveillance device and obtain the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device; the acquisition module 84 is used to obtain flight positioning messages; the generation module 86 is used to generate an aircraft AR target box according to the conversion relationship, the flight positioning message, and the video stream, and send the aircraft AR target box to the user terminal for display; the tracking module 88 is used to obtain a first tracking result for the aircraft dynamic target when receiving a tracking instruction for the aircraft AR target box from the user terminal.

[0104] Through the above embodiments, the tracking module 88, through technical means such as video surveillance technology and AR technology, generates an AR target box for stationary or dynamic targets on the video screen of video surveillance devices such as panoramic cameras based on the obtained conversion relationship, flight positioning messages, and video stream. At the same time, it video-tracks the target and magnifies the target, providing a very intuitive visual display for the user, and can effectively feedback the dynamic targets in the airport flight area to airport staff in real time and comprehensively, upgrade the manual scheduling in the airport flight area to intelligent scheduling, and provide intelligent auxiliary decision-making means, thereby effectively solving the problem of low accuracy of target tracking in airport monitoring scenarios and achieving high-precision airport target tracking based on video surveillance and AR technology.

[0105] In one embodiment, the acquisition module 84 is further used to obtain vehicle positioning messages of the airport vehicle system; the generation module 86 is further used to generate a vehicle AR target box according to the conversion relationship, the vehicle positioning messages, and the video stream, and send the vehicle AR target box to the user terminal for display; the tracking module 88 is further used to obtain a second tracking result for the vehicle dynamic target when receiving a tracking instruction for the vehicle AR target box from the user terminal.

[0106] In one embodiment, the target tracking device further includes a first push module; the first push module is configured to push the vehicle positioning message to the user terminal; wherein, the user terminal displays the license plate number in the vehicle positioning message in the vehicle AR target box.

[0107] In one embodiment, the target tracking device further includes a second push module; the second push module is configured to push the flight status message of the airport flight guarantee system to the user terminal; wherein, when the user terminal receives a viewing instruction for the aircraft AR target box, it displays the flight status message and displays the video stream that matches the flight status message.

[0108] In one embodiment, the conversion module 82 is further configured to obtain multiple points on the video stream through the user terminal, and obtain the point pixel coordinates of the points on the video stream; the conversion module obtains the geographical coordinates corresponding to the points through the positioning device; the conversion module obtains the conversion relationship according to the point pixel coordinates and the geographical coordinates.

[0109] In one embodiment, the target tracking device further includes a queue module; the queue module is configured to obtain the flight positioning message through the broadcast monitoring system, and store the flight positioning message in the message queue; the queue module sequentially retrieves the flight positioning message from the message queue and pushes it to the user terminal; alternatively, the queue module stores the flight positioning message in the database through the message queue.

[0110] In one embodiment, the conversion module 82 is further configured to obtain the identity information of the user uploaded by the user terminal, and verify the identity information; when the verification is passed, the conversion module sends the obtained video stream to the user terminal for display; or sends the video stream to the storage device for storage.

[0111] In one embodiment, the target tracking device further includes a third push module; the third push module is configured to push the flight positioning message to the user terminal; wherein, the user terminal displays the flight information in the flight positioning message in the aircraft AR target box.

[0112] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combination form.

[0113] This embodiment also provides a target tracking system, Figure 9 is a structural block diagram of a target tracking system according to an embodiment of the present application, as Figure 9As shown in the figure, the system includes: a server 14, a video surveillance device 92, and a user terminal 12; wherein, the server 14 is respectively connected to the video surveillance device 92 and the user terminal 12.

[0114] The video surveillance device 92 is used to obtain a video stream and send the video stream to the server 14; the user terminal 12 is used to receive the video stream from the server 14 for display.

[0115] The server 14 is used to obtain the video stream of the video surveillance device and obtain the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device; obtain the flight positioning message; the server 14 generates an aircraft AR target box according to the conversion relationship, the flight positioning message, and the video stream, and sends the aircraft AR target box to the user terminal for display; when the server 14 receives a tracking instruction from the user terminal 12 for the aircraft AR target box, it obtains a first tracking result for the dynamic target of the aircraft.

[0116] Through the above embodiments, the server 14, by means of technologies such as video surveillance technology and AR technology, generates an AR target box for static or dynamic targets on the video screen of video surveillance devices 92 such as panoramic cameras, and at the same time tracks and magnifies the target, providing a very intuitive picture display for users. It can real-time and comprehensively feedback the dynamic targets in the airport flight area to airport staff, upgrade the manual scheduling of the airport flight area to intelligent scheduling, and provide intelligent auxiliary decision-making means, thus effectively solving the problem of low accuracy of target tracking in airport monitoring scenarios and realizing high-precision airport target tracking based on video surveillance and AR technology.

[0117] In this embodiment, a computer device is provided. The computer device can be a server. Figure 10 It is a structural diagram inside a computer device according to an embodiment of the present application, as Figure 10 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video streams. The network interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements the above target tracking method.

[0118] Those skilled in the art can understand.Figure 10 The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0119] This embodiment also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0120] Optionally, the above electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0121] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0122] S1, obtain the video stream of the video surveillance device, and obtain the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device.

[0123] S2, obtain the flight positioning message.

[0124] S3, generate an aircraft AR target box according to the conversion relationship, the flight positioning message and the video stream, and send the aircraft AR target box to the user terminal for display.

[0125] S4, when receiving the tracking instruction of the user terminal for the aircraft AR target box, control to obtain the first tracking result for the dynamic target of the aircraft.

[0126] It should be noted that the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be repeated here.

[0127] In addition, in combination with the target tracking method in the above embodiments, an embodiment of the present application can be implemented by providing a storage medium. A computer program is stored on the storage medium; when the computer program is executed by a processor, it implements any one of the above target tracking methods.

[0128] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0129] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0130] The above embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A target tracking method, characterized in that, The method includes: Obtaining a video stream of a video surveillance device, and obtaining a conversion relationship between a camera coordinate system and a world coordinate system based on the video stream, including: Obtaining multiple points on the video stream through a user terminal, and obtaining the point pixel coordinates of the points on the video stream; Obtaining the geographical coordinates corresponding to the points through a positioning device; Obtaining the conversion relationship according to the point pixel coordinates and the geographical coordinates; wherein, the camera coordinate system refers to the coordinate system of the video surveillance device; Obtaining a flight positioning message; Generating an aircraft AR target box according to the conversion relationship, the flight positioning message and the video stream, and sending the aircraft AR target box to the user terminal for display; When receiving a tracking instruction of the user terminal for the aircraft AR target box, obtaining a first tracking result for an aircraft dynamic target, including: Based on the tracking instruction for the aircraft AR target box, obtaining the first tracking result from an external monitoring device; wherein, the external monitoring device is bound to the video surveillance device, and the rotation angle of the external monitoring device is controlled by controlling the video surveillance device to monitor the aircraft dynamic target indicated by the tracking instruction; the first tracking result includes an enlarged video picture for continuously tracking the aircraft dynamic target.

2. The target tracking method according to claim 1, wherein After obtaining the conversion relationship between the camera coordinate system and the world coordinate system based on the video stream, the method further includes: Obtaining a vehicle positioning message of an airport vehicle system; Generating a vehicle AR target box according to the conversion relationship, the vehicle positioning message and the video stream, and sending the vehicle AR target box to the user terminal for display; When receiving a tracking instruction of the user terminal for the vehicle AR target box, obtaining a second tracking result for a vehicle dynamic target.

3. The target tracking method according to claim 2, wherein After obtaining the vehicle positioning message of the airport vehicle system, the method further includes: Pushing the vehicle positioning message to the user terminal; wherein, the user terminal displays the license plate number in the vehicle positioning message in the vehicle AR target box.

4. The target tracking method according to claim 1, characterized in that The method further includes: Pushing a flight status message of an airport flight guarantee system to the user terminal; Wherein, when the user terminal receives a viewing instruction for the aircraft AR target box, the flight status message is displayed, and the video stream matching the flight status message is displayed.

5. The target tracking method according to claim 1, characterized in that After obtaining the flight positioning message, the method further includes: Obtaining the flight positioning message through a broadcast monitoring system, and storing the flight positioning message in a message queue; Sequentially taking out the flight positioning message from the message queue and pushing it to the user terminal; or storing the flight positioning message in a database through the message queue.

6. The target tracking method according to claim 1, characterized in that, The obtaining of the video stream of the video surveillance device includes: Obtaining the identity information of the user uploaded by the user terminal, and verifying the identity information; In the case of successful verification, the obtained video stream is sent to the user terminal for display; or, the video stream is sent to a storage device for storage.

7. The target tracking method according to any one of claims 1 to 6, characterized in that After sending the aircraft AR target frame to the user terminal for display, the method further includes: Pushing the flight positioning message to the user terminal; wherein, the user terminal displays the flight information in the flight positioning message in the aircraft AR target frame.

8. A target tracking device, characterized in that, The device includes: a conversion module, an acquisition module, a generation module, and a tracking module; The conversion module is configured to obtain a video stream of a video monitoring device and obtain a conversion relationship between a camera coordinate system and a world coordinate system based on the video stream; wherein, the camera coordinate system refers to the coordinate system of the video monitoring device. The conversion module is further configured to obtain multiple points on the video stream through the user terminal and obtain the point pixel coordinates of the points on the video stream; obtain the geographical coordinates corresponding to the points through a positioning device; and obtain the conversion relationship according to the point pixel coordinates and the geographical coordinates. The acquisition module is configured to obtain a flight positioning message. The generation module is configured to generate an aircraft AR target frame according to the conversion relationship, the flight positioning message, and the video stream, and send the aircraft AR target frame to the user terminal for display. The tracking module is configured to obtain a first tracking result for an aircraft dynamic target when receiving a tracking instruction of the user terminal for the aircraft AR target frame. The tracking module is further configured to obtain the first tracking result from an external monitoring device based on the tracking instruction for the aircraft AR target frame; wherein, the external monitoring device is bound to the video monitoring device, and the rotation angle of the external monitoring device is controlled by controlling the video monitoring device to monitor the aircraft dynamic target indicated by the tracking instruction; the first tracking result includes an enlarged video picture for continuously tracking the aircraft dynamic target.

9. A target tracking system, characterized in that, The system includes: a server, a video monitoring device, and a user terminal; wherein, the server is respectively connected to the video monitoring device and the user terminal. The video monitoring device is configured to obtain a video stream and send the video stream to the server. The user terminal is configured to receive the video stream from the server for display. The server is configured to execute the target tracking method according to any one of claims 1 to 7.

10. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the target tracking method according to any one of claims 1 to 7.

11. A storage medium, characterized in that, A computer program is stored in the storage medium, wherein the computer program is configured to execute the target tracking method according to any one of claims 1 to 7 when running.

Images