Airport panoramic monitoring system and airport panoramic monitoring method

Abstract

The invention discloses an airport panoramic monitoring system and an airport panoramic monitoring method, and relates to the technical field of monitoring. The system comprises a radar module, a panoramic video monitoring module and a central control module. The central control module is in communication connection with the radar module and the panoramic video monitoring module. The radar module is used for acquiring radar data of a current airport, performing data processing on the radar data to obtain position data of each target object in the airport, and sending the position data of each target object to the central control module; the central control module is used for performing data conversion on the position data of each target object and synchronously outputting the converted position data of each target object to the panoramic video monitoring module; and the panoramic video monitoring module is used for acquiring a video image of the current airport and marking each target object in the video image according to the converted position data of each target object. Therefore, the airport can be accurately and comprehensively monitored under the weather conditions of cloudy and rainy days, heavy fog and the like.

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to an airport panoramic monitoring system and an airport panoramic monitoring method. Background Art

[0002] The airport panoramic monitoring system is a platform that can provide continuous, all-round and real-time airport monitoring. It installs cameras or sensors at various locations in the airport to collect real-time data such as images, sounds, temperature, etc. of the airport, and then processes and analyzes these data to provide airport control personnel with complete airport information, thereby facilitating decision-making and management for airport control personnel.

[0003] However, in rainy, foggy and other weather conditions, the camera's line of sight may be obstructed, resulting in visual blind spots, which limits the visible range and image clarity of the airport's panoramic monitoring system. In addition, fog, raindrops, lighting, etc. will also affect the images captured by the camera, making it impossible for airport control personnel to obtain accurate and comprehensive image information. Summary of the invention

[0004] The main purpose of this application is to provide an airport panoramic monitoring system and an airport panoramic monitoring method, so as to achieve accurate and comprehensive monitoring of the airport under rainy, foggy and other weather conditions.

[0005] To achieve the above-mentioned purpose, the present application provides an airport panoramic monitoring system, comprising a radar module, a panoramic video monitoring module and a central control module, wherein the central control module is respectively communicated with the radar module and the panoramic video monitoring module; The radar module is used to obtain radar data of the current airport, perform data processing on the radar data, obtain position data of each target object in the airport, and send the position data of each target object to the central control module; The central control module is used to convert the position data of each target object and synchronously output the converted position data of each target object to the panoramic video monitoring module; The panoramic video monitoring module is used to obtain the current video image of the airport and mark each target object in the video image according to the converted position data of each target object.

[0006] Optionally, the radar module is specifically used to: convert the radar data into a radar image, and obtain the grayscale value change rate of each pixel in the radar image; when the grayscale value change rate of the pixel is less than a preset threshold, determine the pixel as a noise point; and denoise the noise point according to the sign of the grayscale value change rate of the noise point.

[0007] Optionally, the position data of the target object includes polar coordinate data of the target object, and the central control module includes middleware, and the central control module is specifically used to: perform coordinate conversion on the polar coordinate data of each target object to obtain intermediate coordinate data of each target object in the coordinate system of the panoramic video surveillance module; use the middleware to convert the data format of the intermediate coordinate data of each target object into a target data format, and the target data format is the data format corresponding to the panoramic video surveillance module; and synchronously transmit the intermediate coordinate data of each target object after the data format conversion to the panoramic video surveillance module.

[0008] Optionally, the panoramic video surveillance module is specifically used to: determine the geographic coordinate data of each target object based on the intermediate coordinate data of each target object; perform coordinate conversion on the geographic coordinate data of each target object to obtain the plane coordinate data of each target object, the plane coordinate data of the target object being the position coordinates of the target object in the coordinate system of the video image; and mark each target object in the video image based on the plane coordinate data of each target object.

[0009] Optionally, the radar module is also specifically used to obtain basic information of each target object and send the basic information of each target object to the central control module; the central control module is also specifically used to synchronously send the basic information of each target object to the panoramic video monitoring module; the panoramic video monitoring module is also specifically used to use the circle function and according to the plane coordinate data of each target object, to perform graphical annotations on each target object in the video image, and use the putText function and according to the basic information of each target object, to perform text annotations on each target object in the video image.

[0010] Optionally, the radar module includes at least one radar, and the panoramic video surveillance module includes at least one camera device; each radar is used to obtain the radar data of the current airport; each camera device is used to obtain the video image of the current airport; the central control module is also used to align the time starting point and perform data frame synchronization processing on each radar and each camera device.

[0011] Optionally, the panoramic video monitoring module further includes a display, and the display is used to display the video image after the graphic annotation and the text annotation.

[0012] In addition, to achieve the above-mentioned purpose, the present application also provides an airport panoramic monitoring method, which is applied to the airport panoramic monitoring system as described above, and the method includes: a panoramic video monitoring module obtains a video image of the current airport; a radar module obtains radar data of the current airport, and processes the radar data to obtain position data of each target object in the airport, and sends the position data of each target object to a central control module; the central control module converts the position data of each target object, and synchronously outputs the converted position data of each target object to the panoramic video monitoring module; the panoramic video monitoring module marks each target object in the video image according to the converted position data of each target object.

[0013] Optionally, the radar data is processed to obtain the position data of each target object in the airport, including: converting the radar data into a radar image, and obtaining the grayscale value change rate of each pixel in the radar image; when the grayscale value change rate of the pixel is less than a preset threshold, determining the pixel as a noise point; and denoising the noise point according to the sign of the grayscale value change rate of the noise point.

[0014] Optionally, the position data of the target object includes polar coordinate data of the target object; the data conversion of the position data of each target object includes: coordinate conversion of the polar coordinate data of each target object to obtain intermediate coordinate data of each target object in the coordinate system of the panoramic video surveillance module; converting the data format of the intermediate coordinate data of each target object into a target data format, wherein the target data format is a data format corresponding to the panoramic video surveillance module; and synchronously transmitting the intermediate coordinate data of each target object after the data format conversion to the panoramic video surveillance module.

[0015] Optionally, the target objects are marked in the video image based on the converted position data of each target object, including: determining the geographic coordinate data of each target object based on the intermediate coordinate data of each target object; performing coordinate conversion on the geographic coordinate data of each target object to obtain plane coordinate data of each target object, the plane coordinate data of the target object being the position coordinates of the target object in the coordinate system of the video image; and marking each target object in the video image based on the plane coordinate data of each target object.

[0016] Optionally, the panoramic video monitoring module converts the geographic coordinate data of the target object into the plane coordinate data of the target object by the following formula:

[0017] in, is the plane coordinate data of the target object, R is the radius of the earth, λ is the longitude in the geographic coordinate data of the target object, and φ is the latitude in the geographic coordinate data of the target object.

[0018] Optionally, after the radar module processes the radar data to obtain the position data of each target object in the airport, the method further includes: the radar module acquires basic information of each target object, and sends the basic information of each target object to the central control module; the central control module synchronously sends the basic information of each target object to the panoramic video monitoring module; and according to the plane coordinate data of each target object, each target object is marked in the video image, including: using the circle function and according to the plane coordinate data of each target object, graphically marking each target object in the video image, and using the putText function and according to the basic information of each target object, textually marking each target object in the video image.

[0019] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the airport panoramic monitoring method as described in any one of the above is implemented.

[0020] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described airport panoramic monitoring methods. The airport panoramic monitoring system of the embodiment of the present application performs real-time scanning of the airport through a radar module to obtain radar data, and performs data processing on the radar data to obtain position data of each target object in the airport; the central control module performs data conversion on the position data of each target object, and synchronously outputs the converted position data of each target object to the panoramic video monitoring module; the video image of the current airport is obtained through the panoramic monitoring module, and each target object is marked in the video image according to the converted position data of each target object. Therefore, in rainy, foggy and other weather conditions, the radar module is used to accurately detect the target objects at the airport, the radar data is combined with the video monitoring data, and the target objects monitored by the radar are visually marked in the video image, so that airport staff can obtain airport monitoring information more comprehensively and accurately, thereby realizing accurate and comprehensive monitoring of the airport. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a scenario example of the airport panoramic monitoring system provided by this application; Figure 2is a schematic diagram of the structure of the airport panoramic monitoring system according to an embodiment of the present application; Figure 3 This is one of the flow charts of the airport panoramic monitoring method according to an embodiment of the present application; Figure 4 is a schematic diagram of an echo of a specific embodiment of the present application; Figure 5 This is the second flowchart of the airport panoramic monitoring method according to an embodiment of the present application; Figure 6 This is the third flow chart of the airport panoramic monitoring method according to an embodiment of the present application; Figure 7 This is the fourth flow chart of the airport panoramic monitoring method according to an embodiment of the present application; In the figure: 110, radar; 120, radar server; 130, camera device; 140, panoramic video monitoring server; 150, central control server; 160, user terminal; 200, airport panoramic monitoring system; 210, radar module; 220, central control module; 230, panoramic video monitoring module.

[0022] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In modern society, with the rapid development of the aviation industry, the importance of airports has become increasingly prominent. As an important transportation hub connecting cities, the safety and operation efficiency of airports are of vital importance. In order to ensure that the safety management and monitoring of airport flight areas can be effectively implemented, the airport panoramic video surveillance system has become an indispensable solution.

[0025] At present, the existing airport panoramic video surveillance system mainly collects the airport's image, sound, temperature and other data in real time by installing cameras or sensors at various locations in the airport, and then processes and analyzes these data to provide airport control personnel with complete airport information so that airport control personnel can make decisions and management.

[0026] However, in rainy, foggy and other weather conditions, the camera's line of sight may be obstructed, for example, by haze, raindrops, sunlight, etc. These factors will affect the camera's image acquisition process and reduce the quality of the captured image. In addition, rainy, foggy and other weather conditions will also limit the visible range of the panoramic video surveillance system, making it impossible for airport control personnel to obtain accurate and comprehensive airport image information.

[0027] Radar is a radio device used to detect and observe targets. Radar can work in any weather conditions. Radar uses radio waves or microwaves as a transmission medium to send pulses to the target, and then receives the reflected waves from the target to estimate the direction, distance and motion state of the target. Radar systems can be widely used in weather forecasting, aircraft navigation, ship navigation, satellite navigation, military reconnaissance and other fields.

[0028] To this end, the embodiments of the present application provide an airport panoramic monitoring system and an airport panoramic monitoring method, which combine a radar system and a panoramic video monitoring system. In rainy, foggy and other weather conditions or when visibility is very low, the radar system is used to detect flying objects at the airport to obtain accurate location information of the flying objects. The panoramic video monitoring system is used to obtain all-round and clear video images of the airport in real time, and the information of the flying objects monitored by the radar system is visualized and marked in the video image, so as to achieve more accurate and comprehensive monitoring of the airport, thereby improving the work efficiency of the airport tower staff and the flight safety of the airport aircraft.

[0029] For ease of understanding, this specification provides an example scenario of an airport panoramic monitoring system. The example scenario is applied in Figure 1 In the application environment shown, there are several radars 110, radar servers 120, several cameras 130, panoramic video monitoring servers 140, central control servers 150 and user terminals 160. Each radar 110 and radar server 120 can establish a communication connection through wired or wireless means; each camera 130 can also establish a communication connection with the panoramic video monitoring server 140 through wired or wireless means; the radar server 120, panoramic video monitoring server 140, central control server 150 and user terminal 160 can establish a communication connection through the Internet and other means.

[0030] In this scenario example, all radars 110 can be distributed and installed at various locations in the airport, so that the radars 110 can scan and cover all aprons, runways, taxiways and other areas that need to be monitored on the airside of the airport. In addition, the user can also set the scanning parameters of each radar 110 on the radar server 120 through the user terminal 160. The scanning parameters include (but are not limited to) scanning range, scanning frequency and scanning sensitivity. Specifically, the scanning range of the radar 110 needs to cover all possible flying objects around the airport, and the scanning frequency and scanning sensitivity also need to be set to a suitable value so that the radar 110 can effectively detect flying objects in any weather conditions.

[0031] Likewise, all the camera devices 130 may be evenly distributed and installed at various locations in the airport, so that the camera devices 130 can monitor and cover all aprons, runways, taxiways and other areas that need to be monitored on the airside of the airport.

[0032] When monitoring an airport, all radars 110 start scanning at the same time. The radars 110 can capture the location information of all large objects or small targets, which is radar data. Each radar 110 will send the radar data in real time to the radar server 120. In the airport monitoring scenario, the main focus is on monitoring large objects in the airport. Therefore, the radar server 120 will process the radar data (for example, denoising), thereby retaining the location information of large objects (i.e., target objects) in the radar data and removing the location information of small targets.

[0033] Furthermore, the radar server 120 can send the processed position data of all target objects to the central control server 150; the central control server 150 can first convert the position data of each target object into data that can be recognized by the panoramic video surveillance server 140, and then synchronously transmit the converted position data of each target object to the panoramic video surveillance server 140.

[0034] In this scenario example, when all radars 110 start scanning, all cameras 130 can also start shooting the airport synchronously, and the video images collected by each camera 130 are sent to the panoramic video surveillance server 140 in real time; the panoramic video surveillance server 140 stitches these video images to obtain a panoramic video image of the airport.

[0035] After receiving the converted position data of each target object, the panoramic video monitoring server 140 can mark the position data of the target object detected by the radar 110 in the panoramic video image according to the converted position data of each target object, or directly mark the target object in the panoramic video image in the form of a graphic.

[0036] In this scenario example, after obtaining the annotated panoramic video, the panoramic video monitoring server 140 can transmit the annotated panoramic video to the user terminal 160, and the display of the user terminal 160 can display the annotated panoramic video for the user. In this scenario example, the user terminal 160 can include (but not limited to): mobile terminals such as mobile phones, laptops, PADs (tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc., and the user terminal 160 is not specifically limited here.

[0037] Therefore, when the tower monitoring personnel cannot see the flight status of the aircraft or flying objects with the naked eye in rainy weather, they can clearly see the relevant information of the flying objects through the user terminal 160, which effectively improves the efficiency and safety of airport operations.

[0038] With reference to the scenario example of the airport panoramic monitoring system in the aforementioned embodiment, the airport panoramic monitoring system in the embodiment of the present application is described in detail below.

[0039] Figure 2 Schematic diagram of the structure of the airport panoramic monitoring system of the embodiment of the present application. Figure 2 As shown, the airport panoramic monitoring system 200 includes a radar module 210, a panoramic video monitoring module 230 and a central control module 220. In this embodiment, the central control module 220 establishes communication connections with the radar module 210 and the panoramic video monitoring module 230 respectively, and the radar module 210 and the panoramic video monitoring module 230 exchange data through the central control module 220. In some embodiments, the radar module 210 may also directly establish a communication connection with the panoramic video monitoring module 230, and the radar module 210 and the panoramic video monitoring module 230 directly exchange data.

[0040] Among them, the radar module 210 is used to obtain radar data of the current airport, and process the radar data to obtain the position data of each target object in the airport, and send the position data of each target object to the central control module 220; the central control module 220 is used to convert the position data of each target object, and synchronously output the converted position data of each target object to the panoramic video monitoring module 230; the panoramic video monitoring module 230 is used to obtain the video image of the current airport, and mark each target object in the video image according to the converted position data of each target object.

[0041] In this embodiment, the radar module 210 may include multiple radars and a radar server capable of data processing. The radar module 210 can scan the airport in real time in rainy weather to capture the location information of all large objects including airplanes and the location information of small targets, which are the above-mentioned radar data. The small targets mentioned here may include raindrops, birds, insects, etc. In the airport monitoring scene, the main monitored targets are large objects in the airport, especially airplanes, vehicles, etc., and these small targets will cause redundant data interference, thereby affecting the accuracy of the radar module 210 in detecting flying objects, etc., so it is necessary to perform certain data processing on the radar data to remove the location information of these small targets and retain the location information of large objects that need to be monitored. Based on this, the target objects in this embodiment may include any objects that need to be monitored, such as airplanes and vehicles.

[0042] Specifically, the radar transmits radio waves and receives the signals reflected by the object to determine the object's direction, distance, speed, etc. Before the radar scans the airport, the radar's scanning parameters can be set according to the actual weather conditions and monitoring needs. The scanning parameters may include sensitivity, scanning range, scanning frequency, etc. Among them, the scanning range needs to cover all possible flying objects around the airport. In addition, in rainy, foggy and other weather conditions, especially rainy weather, the radar's radio waves may be scattered by raindrops, so the radar's sensitivity needs to be set higher. The volume of the target object is much larger than the raindrops, so the radar can still accurately detect the target object.

[0043] After the radar scans the current airport, the corresponding radar data can be obtained. Further, the radar transmits the radar data to the radar server, and the radar server can filter the radar data and retain the position data of each target object. As an example, the radar data can be denoised to eliminate the interference caused by raindrops, birds and other irrelevant objects.

[0044] After the radar module 210 obtains the position data of each target object, it needs to transmit this data to the panoramic video surveillance module 230 in real time and synchronously; in addition, since the data formats of the radar module 210 and the panoramic video surveillance module 230 are different, the data format transmitted by the radar module 210 to the panoramic video surveillance module 230 needs to be converted into a data format that the panoramic video surveillance module 230 can recognize.

[0045] In this embodiment, the position data of each target object acquired by the radar module 210 is converted into a data format that can be recognized by the panoramic video monitoring module 230 through the central control module 220. Specifically, the central control module 220 may include a central control server, and the central control server has a data processing unit (for example, a data processing chip such as a CPU), and the data processing unit performs data conversion on the position data of each target object.

[0046] It should be noted that the data conversion mentioned here mainly includes coordinate data conversion and data format conversion. As we all know, the radar data obtained by radar scanning is mainly the polar coordinate data (i.e., distance and angle) of the target object relative to the radar, while the panoramic video monitoring module 230 needs to use the Cartesian coordinate data of the target object. Therefore, the position data of each target object needs to be converted into coordinate data; in addition, in order to unify the data format into the data format corresponding to the panoramic video monitoring module 230, the position data of each target object also needs to be converted into a data format.

[0047] The central control module 220 converts the position data of each target object, and then synchronously transmits the converted position data of each target object to the panoramic video monitoring module 230 in real time.

[0048] In this embodiment, the panoramic video monitoring module 230 may include multiple cameras and a panoramic video monitoring server. The cameras may be evenly distributed and installed at various locations in the airport, so that the cameras can monitor and cover all aprons, runways, taxiways and other areas that need to be monitored on the airside of the airport. Each camera shoots the airport in real time, and each camera shoots the video images and sends them to the panoramic video monitoring server. The panoramic video monitoring server may have a built-in data processing unit, which may process the image data; after receiving the video images sent by each camera, the panoramic video monitoring server may splice all the video images to obtain a panoramic video image of the airport.

[0049] It should be noted that each camera device needs to collect data simultaneously with each radar to ensure that the radar data obtained by the radar module 210 is subsequently synchronized with the video image information obtained by the panoramic video monitoring module 230.

[0050] After obtaining the panoramic video image, the panoramic video monitoring module 230 may pre-process the panoramic video image; or after receiving the video images sent by each camera device, the panoramic video monitoring module 230 may pre-process each video image first, and then stitch the processed video images to obtain the panoramic video image. In this embodiment, the pre-processing of the video image may include denoising, brightness level adjustment, and image enhancement, so as to facilitate better subsequent display.

[0051] Further, after receiving the converted position data of each target object sent by the central control module 220, the panoramic video monitoring module 230 marks each target object in the panoramic video image according to the position data of each target object. In this embodiment, the marking form can be a graphic marking, a text marking, or a combination of graphics and text. The marking form is not specifically limited here.

[0052] As an example, when the target object is an airplane, the airplane can be directly marked in the panoramic video image with a dot, or with a label with relevant information of the target object, or with a dot and a label with information. It should be noted that the label can contain relevant information such as the flight altitude and flight number of the airplane.

[0053] Therefore, in rainy, foggy and other weather conditions, all flying objects at the airport can be detected by the radar module 210, and the radar data detected by the radar can be combined with the video image data obtained by the panoramic video monitoring module 230, effectively solving the problems of low visibility, incomplete and inaccurate monitoring of the panoramic video monitoring module 230 in such weather conditions. In addition, by visually marking the flying object position information obtained by the radar in the panoramic video image, the visual effect of the overall presentation of the panoramic video image is improved, and the tower staff can clearly and comprehensively understand the current situation of the airport through the marked panoramic video image, which is more conducive to the management of the airport and the command of the aircraft, improving work efficiency, and helping to avoid collisions and getting lost during take-off and landing, greatly improving flight safety.

[0054] Rain, snow, clouds, etc. in the atmosphere will attenuate radar waves and generate echoes, which seriously affect radar observation of target objects. The water content of rain and snow clouds is proportional to the intensity of the echoes they generate. For example, the radar echo of rain and snow clouds with less water content appears as loose cotton-like bright spots without obvious edges in the radar image; while the radar echo of rain and snow clouds with more water content appears as loose edges and continuous strong interference in the radar image.

[0055] In order to suppress the interference of rain, snow and fog on the radar module 210 obtaining radar data, in some embodiments, the radar module 210 can be specifically used to: convert radar data into a radar image, and obtain the grayscale value change rate of each pixel in the radar image; when the grayscale value change rate of the pixel is less than a preset threshold, the pixel is determined as a noise point; and the noise point is denoised according to the sign of the grayscale value change rate of the noise point.

[0056] It should be noted that radar images are a form of representation of radar data. Radar images are two-dimensional or three-dimensional images created based on radar data, which can help users understand radar data more intuitively.

[0057] It can be understood that each pixel in the radar image corresponds to a radar data point. If the radar data contains the location information of a large object (i.e., the target object), the radar data will be manifested in the radar image as follows: the grayscale value of the corresponding pixel is significantly different from the grayscale values ​​of the surrounding points. If the radar data point corresponds to the location information of rain and snow noise, the radar data will be manifested in the radar image as follows: the grayscale value of the corresponding pixel is slightly different from the grayscale values ​​of the surrounding points. This is because the radar signal reflected by rain and snow noise is usually weak, so it is manifested in the radar image as a small change in the grayscale value of the pixel.

[0058] Based on this, a preset threshold can be set, which can be used as a critical value for detecting the gray value change rate of the target object in the radar image, and the target object and rain and snow noise in the radar image can be distinguished by the preset threshold. It should be noted that the preset threshold can be artificially set to a suitable value through a large number of experiments, and the preset threshold is not specifically limited here.

[0059] Specifically, the grayscale value change rate of each pixel in the radar image can be determined first. The grayscale value change rate can be determined by the grayscale value of the pixel and the grayscale values ​​of the surrounding pixels. For details, reference can be made to the existing grayscale value change rate determination method, which will not be repeated here. Further, the grayscale value change rate of each pixel in the radar image is compared with a preset threshold value, and the grayscale of the pixel whose grayscale value change rate is greater than or equal to the preset threshold value is kept unchanged; the pixel whose grayscale value change rate is less than the preset threshold value is determined as a noise point, and the noise point is rain and snow noise.

[0060] After determining the noise point in the radar image, it is necessary to further determine the sign of the grayscale value change rate of the noise point to determine the denoising method of the noise point. It should be noted that the sign of the grayscale value change rate is simply the positive or negative of the grayscale value change rate, which represents the direction of the grayscale value change and can be used to determine whether the area is the leading edge or the trailing edge. For example, when the sign of the grayscale value change rate is negative, it means that the area is the trailing edge, indicating that the object has changed from existence to non-existence, so the grayscale value change rate is decreasing; when the sign of the grayscale value change rate is positive, it means that the area is the leading edge, indicating that the object has changed from non-existence to existence, so the grayscale value change rate is increasing. Based on this property, different processing methods can be used to process the rain and snow noise at the leading edge and the rain and snow noise at the trailing edge respectively.

[0061] One way to deal with noise points with a positive grayscale change rate (i.e., the leading edge) is to retain the grayscale of the noise point, because the change in this position may also represent the real target object. Specifically, only the grayscale of pixels with a grayscale change rate higher than a certain threshold can be retained, that is, only the grayscale of pixels with relatively fast grayscale changes can be retained, thereby reducing the impact of rain and snow noise.

[0062] One way to process noise points with a negative grayscale value change rate (i.e., trailing edge) is to remove rain and snow noise on the trailing edge to avoid misjudgment. Specifically, only the grayscale of pixels with a grayscale value change rate lower than a certain threshold can be removed, that is, the grayscale of pixels with a relatively slow grayscale value change can be removed.

[0063] Therefore, the above method can effectively eliminate the interference of rain and snow noise on radar data, retain the position data of each target object, and obtain more accurate position data of the target object. The radar module 210 can also process the radar data by other processing processes, which can be specifically referred to the data processing process in the following airport panoramic monitoring method embodiment, which will not be described in detail here.

[0064] It is understandable that the position data of the target object acquired by the radar module 210 is usually polar coordinate data (ie, distance and angle) of the object relative to the radar, so the position data of the target object includes the polar coordinate data of the target object.

[0065] In some embodiments, the central control module 220 may include middleware, and the central control module 220 may be specifically used to: perform coordinate conversion on the polar coordinate data of each target object to obtain the intermediate coordinate data of each target object in the coordinate system of the panoramic video surveillance module 230; use the middleware to convert the data format of the intermediate coordinate data of each target object into a target data format, and the target data format is the data format corresponding to the panoramic video surveillance module 230; and synchronously transmit the intermediate coordinate data of each target object after the data format conversion to the panoramic video surveillance module 230.

[0066] It should be noted that the intermediate coordinate data can be the coordinate data of the target object in the Cartesian coordinate system. Because the panoramic video monitoring module 230 usually uses the Cartesian coordinates of the object, it is necessary to convert the polar coordinate data of the target object into Cartesian coordinate data. The main purpose of doing this is to convert the data into more intuitive and easier to process Cartesian coordinate data, and to facilitate synchronous display in the panoramic video monitoring module 230, locate the target object identified by the radar in the panoramic video monitoring module 230, and reflect the real-time status of the target object. The specific conversion method of the central control module 220 can refer to the coordinate conversion process in the following airport panoramic monitoring method embodiment, which will not be described in detail here.

[0067] In addition, in order to communicate between different systems, a middleware is usually required to parse the data. The middleware can parse the data formats of different systems and perform necessary data conversion. Therefore, after converting the polar coordinate data of each target object into the intermediate coordinate data of each target object, the middleware in the central control module 220 can be used to convert the data format of each intermediate coordinate data into the data format corresponding to the panoramic video monitoring module 230. For example, if the data output by the radar module 210 is in XML format, and the panoramic video monitoring module 230 only receives data in JSON format, the middleware can convert the XML format data output by the radar module 210 into JSON format, and then transmit the JSON format data to the panoramic video monitoring module 230. In this way, the data format is unified, which facilitates the subsequent data processing by the panoramic video monitoring module 230.

[0068] The processing process of the target object's position data and the data synchronization transmission process by the central control module 220 can refer to the process in the following airport panoramic monitoring method embodiment, which will not be described in detail here.

[0069] In some embodiments, the panoramic video surveillance module 230 can be specifically used to: determine the geographic coordinate data of each target object based on the intermediate coordinate data of each target object; perform coordinate conversion on the geographic coordinate data of each target object to obtain the plane coordinate data of each target object, where the plane coordinate data of the target object is the position coordinates of the target object in the coordinate system of the video image; and mark each target object in the video image based on the plane coordinate data of each target object.

[0070] Specifically, after receiving the intermediate coordinate data of each target object sent by the central control module 220, the panoramic video monitoring module 230 can first convert the intermediate coordinate data of each target object into the geographic coordinate data of each target object. It can be understood that the position data of the target object obtained by the radar module 210 is the polar coordinate data of the target object relative to the radar. This is a kind of coordinate data related to the position of the radar device, and has nothing to do with the geographical location of the target object itself. Therefore, it is necessary to convert the intermediate coordinate data of each target object into the geographic coordinate data of each target object. The specific coordinate conversion process can refer to the coordinate conversion process in the following method embodiment, which will not be described in detail here.

[0071] Furthermore, after obtaining the geographic coordinate data of each target object, it is also necessary to perform coordinate conversion on the geographic coordinate data of each target object to obtain the plane coordinate data of each target object. It can be understood that in order to match the geographic coordinate data of the target object with the panoramic video image coordinate data obtained by the panoramic video monitoring module 230 to mark the target object in the panoramic video image, it is also necessary to convert the geographic coordinate data into plane coordinate data first. It should be noted here that the physical meanings represented by the geographic coordinate data and the plane coordinate data are different. The geographic coordinate data describes the position of the target object on the surface of the earth, while the plane coordinate data describes the position of the target object on the image array. The purpose of this step is to convert the position information of the target object in the geographic coordinate system into the position information in the image coordinate system, so as to accurately mark the actual position of the target object scanned by the radar on a certain frame of the panoramic video image. The specific coordinate conversion process can refer to the coordinate conversion process in the following method embodiment, which will not be described in detail here.

[0072] In some embodiments, the radar module 210 can also be specifically used to: obtain basic information of each target object, and send the basic information of each target object to the central control module 220; the central control module 220 can also be specifically used to: synchronously send the basic information of each target object to the panoramic video monitoring module 230; the panoramic video monitoring module 230 can also be specifically used to: use the circle function and according to the plane coordinate data of each target object, to graphically annotate each target object in the video image, and use the putText function and according to the basic information of each target object, to textually annotate each target object in the video image.

[0073] In this embodiment, according to the plane coordinate data of each target object, a method of visually marking each target object on the panoramic video image mainly uses a method of combining graphic marking and text marking.

[0074] Specifically, after detecting each target object, if the target object is an aircraft, the radar module 210 can link data based on the flight plan and the ATC (air traffic control) system to obtain basic information such as the flight number and expected landing time of the target object. Before the aircraft takes off, the flight plan has been submitted to the relevant agencies, and the flight plan includes information such as the aircraft's flight number, take-off time, and expected flight path. The radar module 210 can determine which flight plan matches the target object currently detected by the radar by comparing the real-time detection data with the flight plan, and then retrieve the matching flight plan to obtain the basic information of the target object in the flight plan.

[0075] In addition, the aircraft is usually equipped with a transponder, which can automatically respond after receiving the radar signal and send a signal containing the aircraft's identity information, real-time flight parameter coding and other data. After receiving this signal, the radar module 210 can directly obtain the basic information of the aircraft. This information exchange using a transponder is called a secondary radar or SSR (Secondary Surveillance Radar) system.

[0076] After the radar module 210 obtains the basic information of each target object through any of the above methods, it is also synchronously transmitted to the panoramic video monitoring module 230 through the central control module 220. It should be noted that the basic information of each target object can be sent to the panoramic video monitoring module 230 through the central control module 220 alone, or the basic information of each target object and the position data can be sent to the central control module 220 together, and then sent to the panoramic video monitoring module 230 synchronously after being uniformly processed by the central control module 220.

[0077] The panoramic video monitoring module 230 receives the basic information of each target object sent by the central control module 220. When the panoramic video monitoring module 230 labels each target object, the OpenCV algorithm can be used to implement the labeling process. Specifically, the circle function can be used to perform graphic labeling on each target object in the video image based on the plane coordinate data of each target object, and the putText function can be used to perform text labeling on each target object in the video image based on the basic information of each target object. The specific labeling process can refer to the labeling process in the following method embodiment, which will not be described in detail here.

[0078] In some embodiments, the radar module 210 includes at least one radar, and the panoramic video monitoring module 230 includes at least one camera device; each radar is used to obtain radar data of the current airport; each camera device is used to obtain video images of the current airport; the central control module 220 can also be used to align the time starting point and synchronize the data frames of each radar and each camera device.

[0079] It is understandable that in a system with multiple data acquisition devices, the data rates between different devices may be different, and the collected information is often based on different coordinate systems, and even the types of coordinates are different. For example, the data collected by the radar and camera device in this embodiment are data in different coordinate systems, so before the data is processed later, it is necessary to fuse the data. Before data fusion, the radar and camera devices need to be calibrated first. The calibration is mainly calibration on the timeline. The purpose of calibration on the timeline is to align the time starting points of these data and to correspond different types of data frames.

[0080] Based on this, it is necessary to align the time starting point and synchronize the data frames of all radars and all cameras to achieve the calibration on the above timeline. In this embodiment, the method for aligning the time starting point of the radar and the camera device can be: setting a working delay for the radar, or setting a working delay for the camera, so that the time starting point of the radar and the camera is the same. In addition, because the refresh rate of the radar output data is different from the frame rate of the camera, it is necessary to synchronize the data frames of the two so that the data used for fusion of the two devices describes the closest moment as possible. The specific time starting point alignment and data frame synchronization process can refer to the time starting point alignment and data frame synchronization process in the following method embodiment, which will not be described in detail here.

[0081] In some embodiments, the panoramic video monitoring module 230 may further include a display, and the display is used to display the video image after being annotated with graphics and text.

[0082] Specifically, the panoramic video monitoring module 230 may include a user terminal with a display, and the display can display the video image after graphic annotation and text annotation. The display process usually includes two sub-steps: image rendering and image playback. In addition, the display can also display a user interaction interface, and the user can control the playback of the annotated video image by operating the controls in the user interaction interface. The specific display process can refer to the display process in the following method embodiment, which will not be described in detail here.

[0083] Based on the above embodiments, the present application also provides an airport panoramic monitoring method, which can be applied to the airport panoramic monitoring system described in any of the above embodiments. The following takes the airport panoramic monitoring system executing the method as an example to exemplify the airport panoramic monitoring method.

[0084] Figure 3 This is one of the flow charts of the airport panoramic monitoring method according to the embodiment of the present application. Figure 3 As shown, the airport panoramic monitoring method may include the following steps: Step 310: The radar module obtains radar data of the current airport, and the panoramic video monitoring module obtains video images of the current airport.

[0085] Step 320: The radar module processes the radar data to obtain the position data of each target object in the airport, and sends the position data of each target object to the central control module.

[0086] Step 330: The central control module performs data conversion on the position data of each target object, and synchronously outputs the converted position data of each target object to the panoramic video monitoring module.

[0087] Step 340: The panoramic video monitoring module marks each target object in the video image according to the converted position data of each target object.

[0088] In this embodiment, the radar module includes multiple radars, and the panoramic video monitoring module includes multiple camera devices. Among them, the radar can detect the current airport and obtain corresponding radar data. The radar data can include the direction, distance and speed data of the target object. These data are data in the polar coordinate system with the radar as the reference point. The camera device can shoot the current airport and obtain the corresponding video image. The panoramic video monitoring module receives the video image output by each camera device in real time, and splices all the video images through the splicing technology to obtain a panoramic video image of an airport.

[0089] However, before the radar data is collected by the radar and the camera device collects the video image, it is necessary to initialize the radar module and the panoramic video monitoring module, and jointly calibrate all radars and all camera devices.

[0090] The process of initializing the radar module includes: after turning on the radar module, setting appropriate scanning parameters for each radar, which may include scanning range, frequency and sensitivity, etc. Among them, the scanning range should cover all possible flying objects around the airport, and the frequency and sensitivity need to be set moderately so that flying objects can be detected more effectively.

[0091] As an example, in rainy weather, the radar's scanning range can be 5 kilometers around the airport, the scanning frequency can be 10 times per second, and the sensitivity is set to be able to detect objects the size of a vehicle.

[0092] The process of initializing the panoramic video monitoring module includes: setting the shooting range and shooting angle of each camera device, etc. As an example, 4-12 cameras can be set at each monitoring point in the panoramic video monitoring module, and the viewing angle of each camera device is 72°, covering all areas of the airport apron and runway through one or more panoramic points.

[0093] It is understandable that, since radar and camera devices are different devices, the data rates between different devices may be different, and the information collected is often based on different coordinate systems, and even the types of coordinates are different. Usually, the operating frequency and timing of the radar module and the panoramic video surveillance module may be different, so certain calibration and synchronization operations are required to ensure that the radar data and the panoramic video surveillance data can be correctly matched in the subsequent data processing. Only after the timelines of the two modules are aligned and synchronized, the subsequent data acquisition and processing will be accurate and effective.

[0094] For example, the position data of the target object collected by the radar is the polar coordinate data relative to the radar; while the camera device collects image data, which mainly uses the Cartesian coordinate system. Therefore, it is necessary to convert the polar coordinate data of the target object collected by the radar into Cartesian coordinate data, and fuse the converted Cartesian coordinate data with the image data. When performing data fusion, it is necessary to ensure that the two data describe the airport-related data at the same time as much as possible. Therefore, before the radar and camera devices collect data, all radars and all camera devices need to be jointly calibrated.

[0095] The following is a detailed introduction to the timeline calibration process of the radar and camera devices.

[0096] First of all, it should be explained that the calibration of the radar and camera devices on the timeline mainly includes the alignment of the time start point and the synchronization of the data frame. In addition, the central control module can perform the process of calibrating the radar and camera devices on the timeline, that is, the central control module collects and integrates all relevant data of the radar and camera devices, and is responsible for ensuring the time alignment and data frame synchronization of the radar data and the panoramic video surveillance data.

[0097] For the process of aligning the time starting point of the radar and the camera device, a suitable time delay can be set for the camera device, or a suitable time delay can be set for the radar. Taking setting a suitable time delay for the camera device as an example, the time coordinates of the radar and the time coordinates of the camera device satisfy the relationship represented by the following formula:

[0098] In the formula, is the time coordinate of the radar, is the time coordinate of the camera device, It should be noted that the time delay can be set manually according to the hardware characteristics of the radar and camera devices. For example, it can be set according to the data processing speed and data transmission speed of the radar and camera devices respectively. In addition, according to the physical device installation distance and medium propagation speed of the radar and camera devices, the signal propagation delay also needs to be considered.

[0099] Specifically, after determining the time delay of the camera device, the time delay can be set in advance. Before the radar or camera device starts to collect data, the central control module first sends a synchronization signal to them, allowing the radar or camera device to start collecting data at the moment the signal is received. Since the time delay of the camera device has been set, the radar and the camera device will start working at the same time, thereby achieving synchronization of the time starting point and preventing data deviation caused by time asynchrony.

[0100] As an example, if the radar needs t1 time to start working after receiving the synchronization signal, and the camera device needs t2 time to start working after receiving the synchronization signal, then as long as the central control module sends synchronization signals to these two devices at the same time and sets the delay of the camera device to |t1-t2|, it can ensure that the radar and the camera device start collecting data at the same time.

[0101] For data frame synchronization of the data output by the radar and the camera, the data frame sequence number of the radar output data, the data frame sequence number of the camera output data, the radar data transmission cycle, and the camera data transmission cycle can meet the following relationship, so that the data output by the two devices can be synchronized:

[0102] In the formula, and represent functions of taking absolute values ​​and smaller values ​​respectively, Indicates the data frame number of the radar output data, Indicates the data frame number of the camera output data, Indicates the period of radar sending data. Indicates the period during which the camera sends data. , and radar data rate , video stream frame rate There is a reciprocal relationship between them.

[0103] In actual application, the radar data transmission cycle can use the fast and slow time dimension joint estimation algorithm to estimate the target parameters. The number of pulses used for one estimation is 128, and the single pulse period is 768us. Since the acquisition and processing are carried out simultaneously, the radar data refresh cycle is still 128x0.768=98ms (milliseconds), that is, the radar data transmission cycle Can be 98ms; video stream frame rate 30 frames per second.

[0104] In addition, it should be noted that the serial number of the data frame is usually automatically generated by the data acquisition device (i.e., the radar and camera device in this embodiment) during the data acquisition process and output together with the data. The serial number of the data frame is usually an increasing value used to identify the order of each frame of data. Therefore, although the radar and camera device have not yet started to collect data, once data collection begins, each frame of data will have its corresponding serial number. At this stage, the device configuration is usually set to start from serial number 0 or serial number 1. After acquiring the data, the serial number can be used to know when each frame of data in the data set was collected, which is helpful for subsequent time alignment and data frame synchronization.

[0105] After the radar and camera device are calibrated on the timeline in the above manner, the radar and camera device can be controlled to start data collection.

[0106] The following is a detailed description of the process in step 310 in which the radar module acquires the radar data of the airport and the panoramic video monitoring module acquires the video image of the airport.

[0107] First, all radars in the radar module transmit radio waves and receive signals reflected from different objects to obtain radar data such as the direction, distance and speed of different objects. It should be noted that in rainy weather, especially when it rains, the radio waves emitted by the radar may be scattered by raindrops, but since the volume of the target object is much larger than the raindrops, as long as the sensitivity is set high enough, the radar can still accurately detect the target object.

[0108] After the radar obtains the radar data, it transmits the radar data to the radar server in the radar module, and the radar server processes the radar data. In step 320, the data processing of the radar data mainly includes denoising and anti-interference processing, so as to eliminate the interference caused by other irrelevant objects such as raindrops and birds on the position data of the target object.

[0109] Specifically, a filter may be used to perform preliminary denoising on the radar data. In this embodiment, a moving average filter is used to perform denoising on the radar data. The moving average filter takes the data of the current point and the data of the previous M-1 points, calculates their average value, and uses the average value as the output value of the current point. Its mathematical formula can be expressed as:

[0110] in, is the radar data at the nth moment, M is the window length of the moving average filter, is the radar data output by the moving average filter at the nth moment. In this way, the impact of the sudden change point (ie, noise) on the output will be dispersed to multiple output points, thereby reducing the impact of the noise.

[0111] Furthermore, since the same-frequency radar working in the adjacent area will not only receive the pulse signal emitted by itself, but also receive the pulse signal of the same frequency band emitted by other radars, this will cause the same-frequency interference to the radar data. Based on this, in this embodiment, it is also necessary to remove the same-frequency interference from the radar data.

[0112] For the same target object, the same distance unit of adjacent scan lines emitted by the radar should have echoes reflected by the target object, while the same-frequency interference clutter does not have such correlation. Therefore, in the process of suppressing the same-frequency interference, echo correlation technology can be used to achieve it. Specifically, two or more adjacent scan lines can be detected. Taking the suppression of the same-frequency interference of two scan lines as an example, if there are the same or similar echoes at the same distance on two adjacent scan lines, it is considered to be the echo reflected by the target object and is retained; otherwise, it is considered to be echo interference and is removed.

[0113] Figure 4 FIG. 1 is a schematic diagram of an echo of a specific embodiment of the present application. Figure 4 As shown in the figure, as an example, when scan line 1 and scan line n have echoes at the same distance, the output retains the echo, and when only scan line 1 has an echo at a certain distance, or only scan line n has an echo at a certain distance, the output removes these echoes. It should be noted that the more scan lines involved in the correlation operation, the better the effect of suppressing the same-channel interference.

[0114] Figure 5 This is the second flowchart of the airport panoramic monitoring method according to an embodiment of the present application.

[0115] Furthermore, rain, snow, clouds, etc. in the atmosphere will attenuate radar waves and generate echoes, which will seriously affect the radar's observation of the target object. In order to suppress the interference of rain, snow, and fog on the radar module's acquisition of radar data, such as Figure 5 As shown, in some implementations, the radar module processes the radar data in step 320 to obtain the position data of each target object in the airport, and may further include the following steps: Step 510: convert the radar data into a radar image, and obtain the gray value change rate of each pixel in the radar image.

[0116] Step 520: When the grayscale value change rate of a pixel point is less than a preset threshold, the pixel point is determined as a noise point.

[0117] Step 530: De-noising the noise points according to the sign of the gray value change rate of the noise points.

[0118] It should be noted that radar images are a form of representation of radar data. Radar images are two-dimensional or three-dimensional images created based on radar data, which can help users understand radar data more intuitively. In addition, the preset threshold is a critical value for detecting the gray value change rate of the target object in the radar image. It can be artificially set to a suitable value through a large number of experiments, and the preset threshold is not specifically limited here.

[0119] It can be understood that each pixel in the radar image corresponds to a radar data point. If the radar data contains the location information of a large object (i.e., the target object), the radar data will be manifested in the radar image as follows: the grayscale value of the corresponding pixel is significantly different from the grayscale values ​​of the surrounding points. If the radar data point corresponds to the location information of rain and snow noise, the radar data will be manifested in the radar image as follows: the grayscale value of the corresponding pixel is slightly different from the grayscale values ​​of the surrounding points. This is because the radar signal reflected by rain and snow noise is usually weak, so it is manifested in the radar image as a small change in the grayscale value of the pixel.

[0120] Specifically, the grayscale value change rate of each pixel in the radar image can be determined first. The grayscale value change rate can be determined by the grayscale value of the pixel and the grayscale values ​​of the surrounding pixels. For details, reference can be made to the existing grayscale value change rate determination method, which will not be repeated here. Further, the grayscale value change rate of each pixel in the radar image is compared with a preset threshold value, and the grayscale of the pixel whose grayscale value change rate is greater than or equal to the preset threshold value is kept unchanged; the pixel whose grayscale value change rate is less than the preset threshold value is determined as a noise point, and the noise point is rain and snow noise.

[0121] After determining the noise point in the radar image, it is necessary to further determine the sign of the grayscale value change rate of the noise point to determine the denoising method of the noise point. It should be noted that the sign of the grayscale value change rate is simply the positive or negative of the grayscale value change rate, which represents the direction of the grayscale value change and can be used to determine whether the area is the leading edge or the trailing edge. For example, when the sign of the grayscale value change rate is negative, it means that the area is the trailing edge, indicating that the object has changed from existence to non-existence, so the grayscale value change rate is decreasing; when the sign of the grayscale value change rate is positive, it means that the area is the leading edge, indicating that the object has changed from non-existence to existence, so the grayscale value change rate is increasing. Based on this property, different processing methods can be used to process the rain and snow noise at the leading edge and the rain and snow noise at the trailing edge respectively.

[0122] One way to deal with noise points with a positive grayscale change rate (i.e., the leading edge) is to retain the grayscale of the noise point, because the change in this position may also represent the real target object. Specifically, only the grayscale of pixels with a grayscale change rate higher than a certain threshold can be retained, that is, only the grayscale of pixels with relatively fast grayscale changes can be retained, thereby reducing the impact of rain and snow noise.

[0123] One way to process noise points with a negative grayscale value change rate (i.e., trailing edge) is to remove rain and snow noise on the trailing edge to avoid misjudgment. Specifically, only the grayscale of pixels with a grayscale value change rate lower than a certain threshold can be removed, that is, the grayscale of pixels with a relatively slow grayscale value change can be removed.

[0124] Furthermore, the SPxFTCPro function and the SPxRunProcess function can be combined to adjust the technical parameters related to the radar equipment so that the radar echo can be displayed on the screen with clearer strong interference, thereby further suppressing the interference of rain and snow. Among them, the SPxFTCPro function can automatically detect the leading edge of the target object (such as an aircraft) from the input radar data, identify and remove interference information such as rain and snow noise. The SPxRunProcess function further processes the data processed by the SPxFTCPro function and outputs the processed radar data in a form suitable for the panoramic video monitoring module. Therefore, the above method can effectively eliminate the interference of rain and snow noise on radar data, retain the position data of each target object, and obtain more accurate position data of the target object.

[0125] In some implementations, the Laplace operator may be further used to perform edge detection on the radar echo image, and then a median filter algorithm may be used to effectively remove ray-shaped noise in the radar echo image.

[0126] It should be noted that ray-shaped noise may be caused by a fault in the radar itself or temporary external interference. This noise is generally manifested as ray-shaped or strip-shaped noise in the radar echo image. The median filter algorithm is an effective nonlinear filtering method, which is mainly used to eliminate salt and pepper noise, that is, black spots and white spots. Since ray-shaped noise is usually manifested as lines with extremely high or low brightness in the radar echo image, the median filter algorithm can effectively eliminate this type of noise. The specific edge detection process and the process of removing ray-shaped noise using the median filter algorithm can refer to the existing process, which will not be repeated here.

[0127] As an example, if the radar module detects an aircraft taxiing on the runway at (120.63°E, 31.14°N), its position data is (120.63°E, 31.141°N) and its speed is 30 km / h. The program code for the raw radar data obtained by the radar module can be: { "plane1":{"x":2000,"y":3000,"z":0}, "plane2":{"x":2500,"y":3500,"z":0}, "car1":{"x":2600,"y":3700,"z":0}, "truck1":{"x":2700,"y":3800,"z":0}, } Due to the interference caused by raindrops, birds and other irrelevant objects, the radar data is processed by the radar module to eliminate these noises and retain the information of the target object. The program code for the data obtained after the original radar data is processed and denoised can be: { "plane1":{"x":2000,"y":3000}, "plane2":{"x":2500,"y":3500} } These data represent the position data of the two aircraft. Therefore, after the radar module processes the radar data, the accurate position data of the target object can be obtained.

[0128] After the panoramic video monitoring module receives the video images sent by each camera device in real time, the panoramic video monitoring module can first pre-process these video images. In this embodiment, the pre-processing of the video images includes at least smoothing, histogram equalization, edge detection, and image enhancement, so as to facilitate better display of the video images.

[0129] First, in order to eliminate the noise in the video image, this embodiment uses a Gaussian filter to perform denoising on the video image, and its mathematical formula is:

[0130] In this formula, (x, y) is the position of the pixel in the video image, σ is the standard deviation of the Gaussian function, σ describes the degree of smoothness. The larger the σ value, the more obvious the smoothing effect. G(x, y) is the position of the pixel in the video image output by the Gaussian filter.

[0131] In order to enhance the contrast of the video image and make the brightness and darkness of the video image evenly distributed, this embodiment uses histogram equalization, and its mathematical formula is:

[0132] In the above formula, is the probability distribution of a single pixel value r appearing in the entire video image; is the frequency of occurrence of pixel value r, and N is the number of all pixels in the video image; is the cumulative distribution function.

[0133] In order to better display the objects in the video image, in this embodiment, edge detection can be performed on the video image, which can be implemented using the Sobel operator. The calculation formula of the Sobel operator is:

[0134] In these two 3x3 matrices, Gx represents the gradient in the horizontal direction, and Gy represents the gradient in the vertical direction. After convolving these two matrices with the pixel matrix of the video image to obtain two new matrices, and then taking the square root of the sum of the squares of these two matrices, a video image containing edge information can be obtained.

[0135] For the darker parts of the video image, its brightness can be enhanced. This can be achieved by using gamma correction. The corresponding mathematical formula is:

[0136] In this formula, S is the grayscale value of the output (processed) video image, R is the grayscale value of the input (original) video image, c and γ are both constants, c is the magnification, γ is the exponential value, and the larger the γ value, the brighter the video image as a whole.

[0137] The above is the process in which the radar module processes the radar data to obtain the position data of each target object, and the panoramic video monitoring module processes the video image. After obtaining the position data of each target object, the radar module needs to synchronously transmit the position data of each target object to the panoramic video monitoring module so that the panoramic video monitoring module can perform subsequent operations.

[0138] The process of synchronously transmitting the position data of each target object is described in detail below.

[0139] The data synchronization transmission between the radar module and the panoramic video surveillance module is mainly achieved through the central control module. Specifically, the radar module transmits the position data of each target object to the central control module in real time. The central control module needs to convert the position data of each target object and then synchronously transmit the converted position data of each target object to the panoramic video surveillance module.

[0140] It is understandable that the radar module usually outputs the polar coordinate data (i.e., distance and angle) of the target object relative to the radar, so the position data of the target object described in this embodiment includes the polar coordinate data of the target object; and the panoramic video monitoring module usually uses the Cartesian coordinates of the target object, so it is necessary to convert the polar coordinate data of the target object into Cartesian coordinate data. In addition, in order to communicate between different systems, a middleware is usually required to parse the data. Therefore, after converting the polar coordinate data of each target object into the intermediate coordinate data of each target object, the middleware in the central control module can be used to convert the data format of each intermediate coordinate data into the data format corresponding to the panoramic video monitoring module to ensure the uniformity of the data format and facilitate the subsequent data processing by the panoramic video monitoring module.

[0141] Figure 6This is the third flowchart of the airport panoramic monitoring method according to an embodiment of the present application.

[0142] Based on this, in some implementations, in step 330, the central control module performs data conversion on the position data of each target object, which may include the following steps: Step 610: performing coordinate conversion on the polar coordinate data of each target object to obtain the intermediate coordinate data of each target object in the coordinate system of the panoramic video monitoring module.

[0143] Step 620: Convert the data format of the intermediate coordinate data of each target object into a target data format, where the target data format is a data format corresponding to the panoramic video monitoring module.

[0144] Step 630: Synchronously transmit the intermediate coordinate data of each target object after data format conversion to the panoramic video monitoring module.

[0145] It should be noted that the coordinate system of the panoramic video monitoring module described in this embodiment is a Cartesian coordinate system.

[0146] Specifically, the polar coordinate data of the target object usually includes the distance l and angle θ of the target object relative to the radar (the angle is usually in degrees, but it needs to be converted into radians during calculation). The polar coordinate data of each target object can be converted into Cartesian coordinate data by the following formula to obtain the intermediate coordinate data of each target object:

[0147]

[0148] Where (X, Y) is the converted Cartesian coordinate of the target object, l is the distance of the target object relative to the radar, and θ is the angle of the target object relative to the radar. Further, assuming that the radar is located at the center of the panoramic image, the coordinate of the radar relative to the panoramic image is (X 0 ,Y 0 ), the position (X', Y') of the target object on the panoramic image can be calculated by the following formula: X'=X 0 +X Y'=Y 0 +Y It should be noted that the starting value of angle θ is determined by the setting of the radar module. For example, angle θ can be positive clockwise from the north direction, or counterclockwise from the east direction. Before conversion, the direction of angle θ needs to be determined and converted into radians, where radians = degrees*π / 180.

[0149] In this way, the central control module can convert the target object position obtained by radar scanning from polar coordinates to Cartesian coordinates in the panoramic video monitoring module, and obtain the intermediate coordinate data (X', Y') of each target object. Thus, these polar coordinate data are converted into more intuitive and easier to process Cartesian coordinate data, which is conducive to subsequent data synchronization and other processing.

[0150] Furthermore, after obtaining the intermediate coordinate data of each target object, if the intermediate coordinate data of each target object is inconsistent with the data format of the panoramic video surveillance module, the central control module also needs to convert the data format of the intermediate coordinate data of each target object to ensure the uniformity of the data format, which is convenient for the panoramic video surveillance module to process the data later.

[0151] As an example, if the data output by the radar module is in XML format, and the panoramic video surveillance module only receives data in JSON format, the middleware can convert the XML format data output by the radar module into JSON format, and then transmit the JSON format data to the panoramic video surveillance module.

[0152] In this embodiment, since the position of the target object may change in real time, the process of synchronously transmitting the intermediate coordinate data of each target object after data format conversion to the panoramic video surveillance module also involves a real-time update process, that is, it is necessary to obtain new position data from the radar module regularly or in real time, and synchronize this data to the panoramic video surveillance module to ensure that the video image acquired by the panoramic video surveillance module matches the radar data.

[0153] The following is a detailed introduction to the process of real-time updating of the target object's position data.

[0154] Specifically, an independent thread or process can be set in the radar module to continuously obtain real-time radar data from the radar. The acquired data will first be judged whether it is valid, and invalid data will be discarded. After the radar module processes the radar data and obtains the position data of each target object, the radar module can add a timestamp to the current position data of each target object, and the radar module can send the position data of each target object with the timestamp to the central control module in real time.

[0155] The central control module receives the position data of each target object with a timestamp, and after performing data conversion on the position data of each target object, the converted intermediate coordinate data of each target object with a timestamp can be uniformly stored in a buffer. It should be noted that the buffer can be an area in the memory or a special data structure, such as a ring buffer or a queue.

[0156] When the panoramic video monitoring module needs to obtain the intermediate coordinate data of each target object, it can directly read the required data from this buffer. In order to avoid data conflict or loss, a synchronization mechanism is generally used, such as a mutex, semaphore, or conditional variable. The data in the buffer is marked and stored with a timestamp to ensure the continuity and integrity of the data.

[0157] Alternatively, the middleware in the central control module can also broadcast the intermediate coordinate data of each target object to the panoramic video monitoring module in real time. In order to realize the broadcast function, the middleware generally uses a publish-subscribe model or a message queue.

[0158] At the same time, in the panoramic video monitoring module, one or more threads continuously read the intermediate coordinate data of each target object from the buffer. After data processing, the intermediate coordinate data of each target object is finally displayed in the video image screen.

[0159] When the amount of data is large or the reading and writing is frequent, the central control module and the panoramic video monitoring module will use the database or message queue to cache and store the data. For example, the time series database. The time series database is particularly suitable for storing and processing data with timestamps, which can facilitate data query and statistics.

[0160] During the entire real-time synchronization process, because it involves taking into account both real-time and concurrency, this embodiment can be implemented using technologies such as multi-threading and asynchronous I / O.

[0161] Through the above method, after the intermediate coordinate data of each target object is transmitted to the panoramic video monitoring module in real time, the panoramic video monitoring module will perform certain data processing on the intermediate coordinate data of each target object, and then mark each target object in the panoramic video image according to the processed data.

[0162] It is worth mentioning that TCP / IP protocol can be used for data transmission between radar module, central control module and panoramic video monitoring module. TCP / IP protocol can ensure the reliability and integrity of data. In practical applications, by using TCP protocol, real-time data delivery can be guaranteed. In addition, TCP / IP protocol has a confirmation reply mechanism. After sending data, the sender will wait for the confirmation reply from the receiver. If no reply is received within the specified time, the sender will send the data again.

[0163] The following is a detailed introduction to the process of labeling the panoramic video surveillance module.

[0164] Figure 7 This is the fourth flow chart of the airport panoramic monitoring method according to an embodiment of the present application.

[0165] like Figure 7 As shown, in some embodiments, the panoramic video monitoring module marks each target object in the video image according to the converted position data of each target object, which may include the following steps: Step 710: Determine the geographic coordinate data of each target object according to the intermediate coordinate data of each target object.

[0166] Step 720: coordinate conversion is performed on the geographic coordinate data of each target object to obtain the plane coordinate data of each target object. The plane coordinate data of the target object is the position coordinate of the target object in the coordinate system of the video image.

[0167] Step 730: Mark each target object in the video image according to the plane coordinate data of each target object.

[0168] It should be noted that the geographic coordinate data of the target object is the geographic coordinates of the target object relative to the earth, and the geographic coordinate data may include latitude, longitude and altitude information of the target object.

[0169] In step 710, the geographic coordinate data of each target object can be determined based on the intermediate coordinate data of each target object by taking the earth as a sphere and the earth as a reference point, and the intermediate coordinate data of the target object can be converted into geographic coordinate data through the following plane and spherical geometry conversion formula.

[0170] Among them, the conversion formula of longitude can be: Δλ = X' / (R * cosφ1) The conversion formula for latitude can be: Δφ = Y' / R Where Δλ is the longitude difference between the reference point and the target object, Δφ is the latitude difference between the reference point and the target object, φ1 is the latitude of the reference point, R is the radius of the earth, and (X', Y') is the intermediate coordinate data of the target object.

[0171] After obtaining the longitude difference and latitude difference between the reference point and the target object, the calculated longitude difference and latitude difference are added to the longitude and latitude of the reference point to obtain the geographic coordinate data of the target object.

[0172] Furthermore, because the geographic coordinate data of each target object needs to be matched with the video image coordinates obtained by the panoramic video surveillance module in order to mark the target object on the panoramic video image, it is also necessary to convert the geographic coordinate data of each target object into plane coordinate data in the video image coordinate system.

[0173] In some implementations, the conversion of the geographic coordinate data of the target object into the plane coordinate data may be achieved by the following formula:

[0174] in, is the plane coordinate data of the target object, R is the radius of the earth, λ is the longitude in the geographic coordinate data of the target object, and φ is the latitude in the geographic coordinate data of the target object.

[0175] It can be understood that the geographic coordinate data and plane coordinate data of the target object have different physical meanings. The geographic coordinate data describes the position of the target object on the surface of the earth, while the plane coordinate data describes the position of the target object on the image array. This conversion step is to accurately mark the actual position of the target object scanned by the radar on a certain frame of panoramic video image.

[0176] After obtaining the plane coordinate data of each target object, the panoramic video monitoring module can mark each target object on the video image according to the plane coordinate data of each target object. In this embodiment, each target object can be marked only in the form of graphic marking, or only in the form of text label, or in the form of combining graphic marking and text label marking to mark each target object.

[0177] The following takes the labeling of each target object in a combined manner of graphic labeling and text label labeling as an example to introduce the labeling process of the panoramic video surveillance module in detail.

[0178] In some embodiments, after the radar module processes the radar data to obtain the position data of each target object in the airport, the airport panoramic monitoring method may further include: the radar module obtains the basic information of each target object and sends the basic information of each target object to the central control module; the central control module synchronously sends the basic information of each target object to the panoramic video monitoring module. According to the plane coordinate data of each target object, each target object is marked in the video image, which may include: using the circle function and according to the plane coordinate data of each target object, each target object is graphically marked in the video image, and using the putText function and according to the basic information of each target object, each target object is textually marked in the video image.

[0179] Specifically, after the radar module detects each target object, if the target object is a large flying object aircraft, it can link data based on the flight plan and ATC (air traffic control) system to obtain basic information such as the target object's flight number and expected landing time. Before the plane takes off, the flight plan has been submitted to the relevant agencies, and the flight plan includes information such as the aircraft's flight number, take-off time, and expected flight path. The radar module can determine which flight plan matches the target object currently detected by the radar by comparing the real-time detection data with the flight plan, and then retrieve the matching flight plan to obtain the basic information of the target object in the flight plan.

[0180] In addition, aircraft are usually equipped with a transponder, which can automatically respond after receiving the radar signal, sending a signal containing the aircraft's identity information, real-time flight parameter coding and other data. After the radar module receives this signal, it can directly obtain the basic information of the aircraft. This information exchange using a transponder is called a secondary radar or SSR (Secondary Surveillance Radar) system.

[0181] After the radar module obtains the basic information of each target object through any of the above methods, it is also synchronously transmitted to the panoramic video monitoring module through the central control module. After the panoramic video monitoring module receives the basic information of each target object sent by the central control module, the panoramic video monitoring module can draw a point in the video image based on the plane coordinate data of each target object and use the circle function in the OpenCV algorithm. This point is used to represent the target object. Furthermore, the panoramic video monitoring module can add a text label at the point based on the basic information of each target object and use the putText function in the OpenCV algorithm. By way of example, the program code for annotation using the OpenCV algorithm can be: cv2.circle(img, ,radius,color,thickness) cv2.putText(img,text, ,font,size,color,thickness) Among them, img is the video image, is the plane coordinate data of the target object, radius is the radius of the drawn circle, color is the color, thickness is the line width, text is the text to be added, font is the font, and size is the font size. If you want to mark multiple target objects, you can do it through a loop.

[0182] It should be noted that if the target object is an aircraft, the text label may include information such as the aircraft's flight altitude, flight number, aircraft type (such as A380, B747, etc.), aircraft status (such as flight status, taxiing status, etc.), aircraft speed (such as current speed or expected speed, etc.), and even the aircraft's destination or origin. The specific information to be marked may depend on the information needs of tower staff and flight management personnel.

[0183] In this embodiment, the panoramic video monitoring module also needs to update the video image and the position of the target object in the video image in real time. Specifically, the main program of the panoramic video monitoring module can continuously read the latest intermediate coordinate data of each target object from the buffer and update the annotation information and position of the target object position on the latest video image.

[0184] In this embodiment, the panoramic video monitoring module may further include a user terminal, which has a display and can display the annotated video image through the display. It should be noted that the user terminal may include (but not limited to): mobile terminals such as mobile phones, laptops, PADs (tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc., and the user terminal is not specifically limited here. The display process mainly includes two sub-steps: image rendering and image playback.

[0185] Image rendering is the process of converting video image data containing the location annotations of target objects into images that can be displayed on a monitor. This process requires the use of rendering techniques in computer graphics, such as lighting models and texture mapping. The speed of image rendering can be accelerated by using a graphics processing unit (GPU) or a graphics processing library, such as OpenGL, Vulkan, or DirectX.

[0186] Image playback is to display the rendered images continuously on the monitor to form a continuous video screen. This process generally requires the use of video playback technology, such as video decoding, frame rate control, etc. Use existing video playback libraries, such as FFmpeg, VLC, etc., to implement the video playback function.

[0187] Furthermore, the display may also be used to display a user interface so that the user can control the video playback through the operation interface, for example, start / pause playback, fast forward / rewind, change playback speed, etc. In addition, the user interface may also provide various auxiliary functions, for example, video search, screenshot, recording, etc. The user interface may be implemented using a graphical user interface (GUI) library, such as Qt, GTK, WxWidgets, etc.

[0188] Thus, the radar module detects each target object in the airport, and the interference of rain and snow on the radar data is eliminated by denoising the radar data, so that more accurate position data of the target object can be obtained; the position data of each target object is synchronously transmitted to the panoramic video monitoring module in real time through the central control module, ensuring the synchronization of the radar data and the panoramic video image data; the panoramic video monitoring module visualizes each target object detected by the radar in the video image, and displays the marked video image through the user terminal, even in rainy, foggy and other weather conditions, the airport tower monitoring personnel can clearly and completely see the relevant information of the aircraft or other large objects in the airport through the user terminal, thereby effectively improving the effectiveness and safety of the airport operation. For example, through the airport panoramic monitoring system of this embodiment, the tower staff can see the exact position and taxiing speed of the aircraft preparing to leave the airport, so as to make more accurate arrangements and commands.

[0189] On the basis of the above embodiments, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by the processor, the computer can execute the method performed by the radar module in the airport panoramic monitoring method provided by the above methods, the method comprising: obtaining radar data of the current airport; processing the radar data to obtain the position data of each target object in the airport, and sending the position data of each target object to the central control module. Alternatively, the computer can execute the method performed by the central control module in the airport panoramic monitoring method provided by the above methods, the method comprising: performing data conversion on the position data of each target object, and synchronously outputting the converted position data of each target object to the panoramic video monitoring module. Alternatively, the computer can execute the method performed by the panoramic video monitoring module in the airport panoramic monitoring method provided by the above methods, the method comprising: obtaining a video image of the current airport; and marking each target object in the video image according to the converted position data of each target object.

[0190] On the basis of the above embodiments, on another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by the processor, the method executed by the radar module in the airport panoramic monitoring method provided by the above methods is implemented, and the method includes: obtaining radar data of the current airport; processing the radar data to obtain the position data of each target object in the airport, and sending the position data of each target object to the central control module. Alternatively, the method executed by the central control module in the airport panoramic monitoring method provided by the above methods can be executed, and the method includes: performing data conversion on the position data of each target object, and synchronously outputting the converted position data of each target object to the panoramic video monitoring module. Alternatively, the method executed by the panoramic video monitoring module in the airport panoramic monitoring method provided by the above methods can be executed, and the method includes: obtaining a video image of the current airport; marking each target object in the video image according to the converted position data of each target object. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0191] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Claims

1. An airport panoramic monitoring system, characterized in that: It includes a radar module, a panoramic video monitoring module and a central control module, wherein the central control module is respectively connected to the radar module and the panoramic video monitoring module in communication; The radar module is used to obtain radar data of the current airport, perform data processing on the radar data, obtain position data of each target object in the airport, and send the position data of each target object to the central control module; The central control module is used to convert the position data of each target object and synchronously output the converted position data of each target object to the panoramic video monitoring module; The panoramic video monitoring module is used to obtain the current video image of the airport and mark each target object in the video image according to the converted position data of each target object.

2. The airport panoramic monitoring system according to claim 1, characterized in that: The radar module is specifically used for: Converting the radar data into a radar image, and obtaining a grayscale value change rate of each pixel in the radar image; When the gray value change rate of the pixel point is less than a preset threshold, the pixel point is determined as a noise point; The noise points are denoised according to the sign of the grayscale value change rate of the noise points.

3. The airport panoramic monitoring system according to claim 1, characterized in that: The position data of the target object includes polar coordinate data of the target object, and the central control module includes middleware, and the central control module is specifically used for: Performing coordinate conversion on the polar coordinate data of each target object to obtain intermediate coordinate data of each target object in the coordinate system of the panoramic video monitoring module; Using the middleware, converting the data format of the intermediate coordinate data of each target object into a target data format, wherein the target data format is a data format corresponding to the panoramic video monitoring module; The intermediate coordinate data of each target object after data format conversion is synchronously transmitted to the panoramic video monitoring module.

4. The airport panoramic monitoring system according to claim 3, characterized in that: The panoramic video monitoring module is specifically used for: Determining geographic coordinate data of each target object according to the intermediate coordinate data of each target object; Performing coordinate conversion on the geographic coordinate data of each target object to obtain plane coordinate data of each target object, wherein the plane coordinate data of the target object is the position coordinate of the target object in the coordinate system of the video image; Each target object is marked in the video image according to the plane coordinate data of each target object.

5. The airport panoramic monitoring system according to claim 4, characterized in that: The radar module is further specifically used to obtain basic information of each target object and send the basic information of each target object to the central control module; The central control module is also specifically used to synchronously send the basic information of each target object to the panoramic video monitoring module; The panoramic video monitoring module is also specifically used to use the circle function and based on the plane coordinate data of each target object, to graphically annotate each target object in the video image, and to use the putText function and based on the basic information of each target object, to textually annotate each target object in the video image.

6. The airport panoramic monitoring system according to claim 1, characterized in that: The radar module includes at least one radar, and the panoramic video monitoring module includes at least one camera device; Each of the radars is used to obtain the radar data of the current airport; Each of the camera devices is used to obtain the video image of the current airport; The central control module is also used to perform time start point alignment and data frame synchronization processing on each of the radars and each of the camera devices.

7. The airport panoramic monitoring system according to claim 5, characterized in that: The panoramic video monitoring module also includes a display, and the display is used to display the video image after the graphic annotation and the text annotation.

8. A method for panoramic monitoring of an airport, characterized in that: Applied to the airport panoramic monitoring system according to any one of claims 1 to 7, the method comprises: The panoramic video monitoring module obtains the video image of the current airport; The radar module obtains radar data of the current airport, processes the radar data, obtains position data of each target object in the airport, and sends the position data of each target object to the central control module; The central control module performs data conversion on the position data of each target object, and synchronously outputs the converted position data of each target object to the panoramic video monitoring module; The panoramic video monitoring module marks each of the target objects in the video image according to the converted position data of each of the target objects.

9. The airport panoramic monitoring method according to claim 8, characterized in that: The step of processing the radar data to obtain the position data of each target object in the airport includes: Converting the radar data into a radar image, and obtaining a grayscale value change rate of each pixel in the radar image; When the gray value change rate of the pixel point is less than a preset threshold, the pixel point is determined as a noise point; The noise points are denoised according to the sign of the grayscale value change rate of the noise points.

10. The airport panoramic monitoring method according to claim 8, characterized in that: The position data of the target object includes polar coordinate data of the target object; The step of converting the position data of each target object comprises: Performing coordinate conversion on the polar coordinate data of each target object to obtain intermediate coordinate data of each target object in the coordinate system of the panoramic video monitoring module; Converting the data format of the intermediate coordinate data of each target object into a target data format, wherein the target data format is a data format corresponding to the panoramic video monitoring module; The intermediate coordinate data of each target object after data format conversion is synchronously transmitted to the panoramic video monitoring module.

11. The airport panoramic monitoring method according to claim 10, characterized in that: The step of marking each of the target objects in the video image according to the converted position data of each of the target objects includes: Determining geographic coordinate data of each target object according to the intermediate coordinate data of each target object; Performing coordinate conversion on the geographic coordinate data of each target object to obtain plane coordinate data of each target object, wherein the plane coordinate data of the target object is the position coordinate of the target object in the coordinate system of the video image; Each target object is marked in the video image according to the plane coordinate data of each target object.

12. The airport panoramic monitoring method according to claim 11, characterized in that: The panoramic video monitoring module converts the geographic coordinate data of the target object into the plane coordinate data of the target object by the following formula: in, is the plane coordinate data of the target object, R is the radius of the earth, λ is the longitude in the geographic coordinate data of the target object, and φ is the latitude in the geographic coordinate data of the target object.

13. The airport panoramic monitoring method according to claim 11, characterized in that: After the radar module processes the radar data to obtain the position data of each target object in the airport, the method further includes: The radar module acquires basic information of each target object and sends the basic information of each target object to the central control module; The central control module synchronously sends the basic information of each target object to the panoramic video monitoring module; The step of marking each target object in the video image according to the plane coordinate data of each target object includes: Using the circle function and according to the plane coordinate data of each target object, each target object is graphically annotated in the video image, and using the putText function and according to the basic information of each target object, each target object is textually annotated in the video image.

Images