An error handling method and system based on aviation video stream port
Through multi-level judgment and processing measures, the error handling method for aviation video streams solves the problem of low accuracy and accuracy of video stream data, improves the accuracy of target recognition and the stability of video streams, and is suitable for aviation technology. Navigation and target tracking tasks.
Patent Information
- Application Number
- CN202411298875.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-18
AI Technical Summary
In aviation technology, the accuracy and accuracy of video streaming data is low, especially in navigation and target tracking tasks, making target recognition difficult to accurately perform.
Through an error processing method based on the aerial video stream port, including multi-level judgment and corresponding processing measures, the playback status, jitter status and target type of the video stream are judged, and specific processing measures are used to correct static and dynamic targets to improve video flow accuracy and accuracy.
It effectively improves the accuracy and accuracy of video stream data related to target recognition, ensures the recognition and tracking accuracy of weak targets in aviation scenarios, and enhances the stability and display quality of video streams.
Smart Images

Figure CN118827967B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of avionics technology, and in particular to an error processing method and system based on an avionic video stream port. Background Art
[0002] In modern avionics systems, more and more information is transmitted in the form of images, which are delivered to pilots and crew members through complex onboard video systems. Video systems include: infrared sensors, optical cameras, radars, flight recorders, map systems, video overlay systems, head-up displays (HUDs), multi-function displays, and video integrated processors. Video systems are used in aircraft taxiing and takeoff assistance systems, cargo loading, navigation, target tracking, collision avoidance systems, and more stringent mission processing.
[0003] Especially for navigation, target tracking and more stringent task processing, it is more important to ensure the accuracy of video stream data. The most important technology in these aspects is target recognition technology. Due to the large number of weak targets in aerial scenes and the small amount of information they contain, and the lack of corresponding correction methods for images collected for dynamic and static targets, the collected image data (video stream data) is often of low precision and accuracy cannot be guaranteed. Summary of the invention
[0004] In order to solve the problem of low accuracy and precision of video stream data related to target recognition in existing aviation technology, the present invention provides an error handling method based on an aviation video stream port, which includes the following steps:
[0005] S1. Obtaining the original video stream, and obtaining the corresponding multi-frame images based on the original video stream;
[0006] S2. Perform a primary determination on the original video stream to determine whether the original video stream can be played normally. If so, execute step S3. If not, use the first measure to process the original video stream, and then execute step S3;
[0007] S3. Perform a secondary determination on the original video stream to determine whether the image corresponding to the original video stream is jittered. If so, use a second measure to process the original video stream, and then execute step S4. If not, execute step S4;
[0008] S4. Performing a three-level determination on the original video stream, determining a plurality of coordinate position data of a first target to be tracked based on the original video stream, and judging whether the type of the first target is a static target based on the plurality of coordinate position data, and if so, processing the original video stream using a third measure; if not, processing the original video stream using a fourth measure;
[0009] S5. Upload the processed original video stream to a preset display device for display.
[0010] The present invention is implemented through the following technical scheme: firstly, the original video stream is obtained, and the original video stream is split into multiple frames of images; the original video stream is subjected to a first-level judgment, which belongs to a basic judgment, and is used to judge whether the original video stream can be played normally. The judgment factors include but are not limited to factors such as freeze or black screen. If it is judged that the original video stream can be played normally, a second-level judgment is performed. If it is judged that the original video stream cannot be played normally, it is necessary to adopt the first measure for processing and then perform a second-level judgment. For example, for freeze, it is necessary to adjust the data transmission rate, and for black screen, it is necessary to update the driver of the display device; the second-level judgment is to judge whether the original video stream has jitter. Since the acquisition equipment and the flight equipment cannot remain stable all the time, the acquired video stream will be unstable, resulting in the phenomenon of video stream jitter, which will have a greater impact on subsequent target recognition and target tracking. Therefore, when it is judged that the original video stream has jitter, , a second measure is needed to weaken the degree of jitter of the original video stream, and then a three-level judgment is performed. When it is determined that there is no jitter in the original video stream, the three-level judgment is directly performed; due to the large number of weak targets in the aviation scene and the small amount of information contained, there is a lack of correction means for the images collected for dynamic or static targets, resulting in the accuracy of the collected images not being guaranteed. Therefore, a three-level judgment is set up, and multiple coordinate position data of the first target to be tracked are determined through the original video stream. If the multiple coordinate position data are the same or the difference is small, the type of the first target is determined to be a static target, otherwise, the type of the first target is determined to be a dynamic target, and then different measures are used for processing the static target and the dynamic target respectively, so as to further improve the accuracy and precision of the target acquisition image, that is, to improve the accuracy of the corresponding video stream data, and finally upload the processed original video stream to the preset display device for display.
[0011] Further, using the third measure to process the original video stream includes:
[0012] A1. Obtain the image at time t0 as the first image, determine at least one key feature point of the first target in the first image, and record the first position data of the key feature point;
[0013] A2. The image at time t1 is obtained and recorded as a second image, and second position data of the key feature point is obtained and recorded in the second image, where t0 and t1 are adjacent times and t0<t1;
[0014] A3. Based on the airborne device, obtain the first speed characteristic data of the flight device, and based on the first speed characteristic data and the first position data, predict the third position data of the key feature point at time t1 in the second image;
[0015] A4. Determine whether the second position data coincides with the third position data, and if so, use the second image as a new first image; if not, correct the second image based on the second position data and the third position data to obtain a second corrected image, and use the second corrected image as a new first image;
[0016] A5. Loop through steps A1 to A4 until all images of the original video stream are processed.
[0017] When the first target is determined to be a static target, the first image at time t0 is obtained, at least one key feature point of the first target is determined in the first image, and the first position data of the key feature point is recorded, and the first position data is the vector position data in the image; the actual position data (second position data) of the key feature point in the second image at the next moment is obtained; then, based on the airborne equipment, the first speed feature data of the flight equipment is obtained, and the third position data of the key feature point in the image at the next moment is predicted through the first speed feature data, and it is determined whether the second position data coincides with the third position data. If they coincide, it means that the collected second image is accurate, and the second image is used as the new first image; if they do not coincide, it means that there is a deviation in the collected second image, and the second image needs to be corrected, and the corrected second corrected image is used as the new first image; finally, the above steps are executed in a loop until all images of the original video stream are processed.
[0018] Further, performing correction processing on the second image to obtain a second corrected image includes:
[0019] Establishing a background model based on the arithmetic mean of pixels between consecutive images;
[0020] Performing pixel difference between the second image and the background model to obtain a pixel difference result, comparing the pixel difference result with a preset threshold, and if the pixel difference result is greater than the preset threshold, determining that the pixel belongs to the target area, otherwise, determining that the pixel belongs to the background area, and updating the background model based on the value of the pixel;
[0021] The key feature point belongs to the target area, and a change vector between the second position data and the third position data is calculated;
[0022] Based on the change vector, the direction and distance in which the target area needs to be translated are obtained to obtain an acquisition result;
[0023] Based on the acquisition result, the target area is processed to obtain a modified target area;
[0024] Based on the background area and the background model, generating a corrected background area of the second corrected image;
[0025] The second corrected image is obtained based on the correction target area and the correction background area.
[0026] When there is a deviation in the collected second image, the second image needs to be corrected. The core idea of the correction is to translate the target area containing key feature points in the second image. First, a background model is established based on the arithmetic mean of pixels between consecutive images. Then, the second image and the background model are differentiated by pixels to screen out the target area of the second image. The target area is the area corresponding to the first target to be tracked. The change vector between the second position data and the third position data is calculated. Through the change vector, the direction and distance of the translation of the target area can be obtained. Then, the target area is translated to obtain the corrected target area. For the corrected background area in the second corrected image, it is automatically generated through the background area and background model of the second image, and finally the second corrected image is obtained.
[0027] Further, using the fourth measure to process the original video stream includes:
[0028] Based on the multiple frames of the image, a motion trajectory map of the first target is generated, and based on the motion trajectory map, a corresponding first radar target is obtained by screening and matching in the radar map;
[0029] Acquire the image at time T and record it as a third image, determine the central feature point of the first target in the third image, and record fourth position data of the central feature point;
[0030] The image at time T+1 is obtained and recorded as a fourth image, and the fifth position data of the central feature point is obtained and recorded in the fourth image, where time T and time T+1 are adjacent times;
[0031] Based on the radar map and the first radar target, second velocity characteristic data of the first target is acquired; based on the airborne device, third velocity characteristic data of the flight device is acquired; and based on the second velocity characteristic data and the third velocity characteristic data, sixth position data of the central feature point in the fourth image at the time T+1 is predicted;
[0032] calculating a first change vector between the fifth position data and the sixth position data;
[0033] Based on the first change vector, the fourth image is corrected to obtain a fourth corrected image.
[0034] When the first target is a dynamic target, the difference from the static target image acquisition is that both the flight equipment and the first target are in motion, so different measures need to be taken to process them. Because the speed characteristic data of the first target needs to be obtained, it is necessary to use the radar map. First, based on the multi-frame collected images, the motion trajectory of the first target is processed, and the first radar target is obtained by screening and matching the motion trajectory of the first target in the radar map, that is, the first radar target is the first target in the radar map, and then the second speed characteristic data of the first target is obtained through the first radar target and the radar map. The third image at time T is acquired. Since the first target is a dynamic target, in order to better track and capture the first target, the central feature point (key feature point) of the first target is determined, and the fourth position data of the central feature point is recorded. Similarly, the fourth image at time T+1 is acquired, and the fifth position data of the central feature point is acquired and recorded in the fourth image. The sixth position data of the central feature point in the fourth image at time T+1 is predicted through the third speed feature data of the flight equipment and the second speed feature data of the first target, and the first change vector between the fifth position data and the sixth position data is calculated. The fourth image is corrected through the first change vector to obtain a fourth corrected image. The correction processing measures are similar to those for the second corrected image and will not be elaborated further.
[0035] Furthermore, using the fourth measure to process the original video stream also includes:
[0036] Acquire seventh position data of the first radar target in the radar map at time T, acquire eighth position data of the first radar target in the radar map at time T+1, and predict ninth position data of the first radar target in the radar map at time T+1 based on the first speed characteristic data, the second speed characteristic data, and the seventh position data;
[0037] calculating a second change vector between the eighth position data and the ninth position data;
[0038] Calculating a convolution sum of the first change vector and the second change vector, and updating the first change vector according to the convolution sum;
[0039] Based on the updated first change vector, the fourth image is corrected to obtain a fifth corrected image.
[0040] In order to further improve the accuracy of dynamic target image acquisition, the seventh position data of the first radar target in the radar map at time T and the eighth position data at time T+1 are obtained, and the ninth position data of the first radar target in the radar map at time T+1 is predicted based on the first speed feature data, the second speed feature data and the seventh position data, and the second change vector between the eighth position data and the ninth position data is calculated. Since the first change vector and the second change vector both describe the movement of the first target, the convolution sum of the first change vector and the second change vector is calculated, and the first change vector is updated using the convolution sum. The fourth image is corrected using the updated first change vector to obtain a fifth corrected image.
[0041] Further, performing a first-level determination on the original video stream includes:
[0042] Converting the plurality of frames of the image into a sequence of images, and extracting image information of all the images;
[0043] Based on the image information, calculating the image motion amount of all the sequence images to obtain a first calculation result;
[0044] Based on the first calculation result, calculating the average motion amount of all sequence images to obtain a second calculation result;
[0045] Based on the second calculation result, the dynamic factor is dynamically calculated, and based on the second calculation result and the dynamic factor, a judgment result is output. If the judgment result is 0, it is determined that there is no freeze in the original video stream. If the judgment result is 1, it is determined that there is freeze in the original video stream.
[0046] Among them, in video quality detection, detecting whether the video is stuck is one of the standards for video quality detection. The above method converts the video stream into a sequence of images, and calculates the difference (motion) between each frame of the image to find out whether there is a fault in the sequence of images, and then determines whether the current video has a stuck phenomenon. If the judgment result is 0, it is determined that the original video stream does not have a stuck phenomenon. If the judgment result is 1, it is determined that the original video stream has a stuck phenomenon.
[0047] Furthermore, using the first measure to process the original video stream includes: obtaining a first time period during which the original video stream is stuck, and adjusting a transmission rate of the original video stream in the first time period.
[0048] Among them, the reason for the freeze in the video stream is mostly due to the influence of the data transmission rate. Therefore, by obtaining the time period when the original video stream is freezed, the transmission rate of the video stream is adjusted (improved) within the time period.
[0049] Furthermore, performing a secondary determination on the original video stream includes: determining whether an image corresponding to the original video stream has jitter based on an image grayscale projection algorithm.
[0050] Among them, due to the jitter of image acquisition equipment and flight equipment, video acquisition is unstable, which will cause jitter in the video stream. The most significant feature of video jitter is the overall displacement between frames. After the displacement is detected, further logic is used to determine whether the video is jittering. The image grayscale projection algorithm estimates the global motion vector between adjacent frames through the correlation of the overall row and column grayscale projection curves of the image, that is, detects the translation vector between frames of the video sequence, and determines whether the video is jittering through the translation vector.
[0051] Furthermore, the second measure for processing the original video stream includes: optimizing the jitter of the image corresponding to the original video stream based on a previous and next frame association algorithm.
[0052] Among them, the use of the previous and next frame association algorithm can effectively weaken the jitter degree of the image corresponding to the original video stream. The principle of the previous and next frame association algorithm is to use the first frame and the second frame of the image corresponding to the original video stream as two adjacent frames, and match the target areas of the two adjacent frames in pairs. The matching method is the sum of the distance between the upper left corner and the lower right corner of the target area. When the sum of the two is less than the threshold, the two adjacent frames are matched successfully, and the first frame is used as the previous frame of the next frame (the third frame) for matching. When the sum of the two is greater than or equal to the threshold, the two adjacent frames fail to match, and the second frame is used as the previous frame of the next frame (the third frame), and subsequent iterations are performed.
[0053] In order to solve the problem of low accuracy and precision of video stream data related to target recognition in existing aviation technology, the present invention provides an error handling system based on an aviation video stream port, the system comprising:
[0054] An acquisition unit, used to acquire an original video stream, and based on the original video stream, acquire corresponding multiple frames of images;
[0055] A primary determination unit, configured to perform a primary determination on the original video stream to determine whether the original video stream can be played normally, and if so, execute a secondary determination unit; if not, adopt a first measure to process the original video stream, and then execute the secondary determination unit;
[0056] A secondary determination unit, configured to perform a secondary determination on the original video stream to determine whether an image corresponding to the original video stream has jitter, and if so, to process the original video stream using a second measure, and then execute the tertiary determination unit, and if not, to execute the tertiary determination unit;
[0057] A three-level determination unit is used to perform a three-level determination on the original video stream, determine a plurality of coordinate position data of a first target to be tracked based on the original video stream, and determine whether the type of the first target is a static target based on the plurality of coordinate position data, and if so, process the original video stream using a third measure; if not, process the original video stream using a fourth measure;
[0058] The display unit is used to upload the processed original video stream to a preset display device for display.
[0059] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:
[0060] The present invention ensures the normal playback of video streams by setting a first-level judgment and corresponding measures, weakens the degree of video stream jitter by setting a second-level judgment, and sets a third-level judgment to distinguish the first target of target tracking and identification as a static target and a dynamic target. When collecting images (video streams) for static or dynamic targets, different measures are used for correction and processing, so as to ensure the accuracy and precision of video stream data related to target identification.
[0061] The present invention designs a method for determining the deviation of a related image for static target image acquisition, and designs a related correction method for images with deviations, thereby improving the accuracy of static target image acquisition.
[0062] The present invention designs a method for determining relevant image deviations for dynamic target image acquisition, and designs relevant correction methods for images with deviations. Furthermore, an optimization method is designed for dynamic target acquisition images, thereby improving the accuracy of dynamic target image acquisition. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the present invention, and do not constitute a limitation on the embodiments of the present invention;
[0064] Figure 1 It is a flow chart of an error handling method based on an aviation video stream port in the present invention;
[0065] Figure 2 The present invention is a schematic diagram of an error handling system based on an aviation video stream port. DETAILED DESCRIPTION
[0066] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.
[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those within the scope of this description. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0068] Embodiment 1
[0069] Please refer to Figure 1 , Figure 1 The present invention is a flowchart of an error handling method based on an aviation video stream port, the method comprising the following steps:
[0070] S1. Obtaining the original video stream, and obtaining the corresponding multi-frame images based on the original video stream;
[0071] S2. Perform a primary determination on the original video stream to determine whether the original video stream can be played normally. If so, execute step S3. If not, use the first measure to process the original video stream, and then execute step S3;
[0072] S3. Perform a secondary determination on the original video stream to determine whether the image corresponding to the original video stream is jittered. If so, use a second measure to process the original video stream, and then execute step S4. If not, execute step S4;
[0073] S4. Performing a three-level determination on the original video stream, determining a plurality of coordinate position data of a first target to be tracked based on the original video stream, and judging whether the type of the first target is a static target based on the plurality of coordinate position data, and if so, processing the original video stream using a third measure; if not, processing the original video stream using a fourth measure;
[0074] S5. Upload the processed original video stream to a preset display device for display.
[0075] The specific embodiments of the present invention are as follows: obtaining an original video stream, splitting the original video stream into multiple frames of images, arranging the multiple frames of images in sequence, and then performing a first-level judgment on the original video stream. The first-level judgment belongs to a basic judgment, which is to ensure that the video stream can be played normally, such as judging whether there is a freeze in the video stream. If so, the efficiency of video stream data transmission is improved. If not, a second-level judgment is directly performed; the second-level judgment is to judge whether there is a jitter phenomenon in the original video stream. Since image acquisition equipment (such as optical cameras) and flight equipment have jitter problems, video acquisition will be unstable. Based on the image grayscale projection algorithm, it is judged whether there is jitter in the image corresponding to the original video stream. If so, the degree of jitter is weakened based on the previous and next frame association algorithm, and then a third-level judgment is performed. If not, a third-level judgment is directly performed; the third-level judgment is to further improve the accuracy of image acquisition. Since there are many weak and small targets in aerial scenes and they contain less information, acquisition is caused. The image data is often not guaranteed in terms of accuracy and precision. The embodiment of the present invention obtains multiple coordinate position data of the first target to be tracked based on the original video stream. If the multiple coordinate position data are the same or the difference is small (due to the error of image acquisition and the jitter of the flight equipment, the coordinate position data of the static target obtained cannot be guaranteed to remain unchanged), the first target is determined to be a static target, otherwise, the first target is determined to be a dynamic target; when the first target is a static target, the third measure is used for processing, specifically, a method for determining image deviation and a method for correcting deviation images are designed to improve the accuracy and precision of static target image acquisition. When the first target is a dynamic target, the fourth measure is used for processing, specifically, a method for determining image deviation and a method for correcting deviation images are designed, and an optimization method is further designed for dynamic target image acquisition, which improves the accuracy of dynamic target image acquisition. Finally, the processed original video stream is uploaded to a preset display device for display. The above-mentioned specific measures are all mentioned in the subsequent embodiments, and the embodiments of the present invention will not be further explained.
[0076] Wherein, adopting the third measure to process the original video stream includes:
[0077] A1. Obtain the image at time t0 as the first image, determine at least one key feature point of the first target in the first image, and record the first position data of the key feature point;
[0078] A2. The image at time t1 is obtained and recorded as a second image, and second position data of the key feature point is obtained and recorded in the second image, where t0 and t1 are adjacent times and t0<t1;
[0079] A3. Based on the airborne device, obtain the first speed characteristic data of the flight device, and based on the first speed characteristic data and the first position data, predict the third position data of the key feature point at time t1 in the second image;
[0080] A4. Determine whether the second position data coincides with the third position data, and if so, use the second image as a new first image; if not, correct the second image based on the second position data and the third position data to obtain a second corrected image, and use the second corrected image as a new first image;
[0081] A5. Loop through steps A1 to A4 until all images of the original video stream are processed.
[0082] For example, the first image at time t0 and the second image at time t1 are obtained, the first position data of the key feature point in the first image is (x1, y1), and the second position data of the key feature point in the second image is (x2, y2). Based on the airborne equipment, the first speed feature data of the aircraft equipment is obtained. The first speed feature data includes information such as the speed, direction and attitude of the aircraft. Based on the first speed feature data, the third position data (x3, y3) of the key feature point in the second image at time t1 is predicted, and it is determined whether the second position data (x2, y2) and the third position data (x3, y3) coincide with each other. If so, the second image is used as the new first image; if not, it means that there is a deviation in the collected second image, and the second image needs to be corrected, and then the corrected second image is used as the new first image. Finally, the above steps are executed repeatedly until all images of the original video stream are processed.
[0083] The step of correcting the second image to obtain a second corrected image includes:
[0084] Establishing a background model based on the arithmetic mean of pixels between consecutive images;
[0085] Performing pixel difference between the second image and the background model to obtain a pixel difference result, comparing the pixel difference result with a preset threshold, and if the pixel difference result is greater than the preset threshold, determining that the pixel belongs to the target area, otherwise, determining that the pixel belongs to the background area, and updating the background model based on the value of the pixel;
[0086] The key feature point belongs to the target area, and a change vector between the second position data and the third position data is calculated;
[0087] Based on the change vector, the direction and distance in which the target area needs to be translated are obtained to obtain an acquisition result;
[0088] Based on the acquisition result, the target area is processed to obtain a modified target area;
[0089] Based on the background area and the background model, generating a corrected background area of the second corrected image;
[0090] The second corrected image is obtained based on the correction target area and the correction background area.
[0091] For example, a background model is established based on the arithmetic mean of pixels between consecutive images. After the background model is established, the second image is processed, and pixel difference is performed between the second image and the background model to obtain a pixel difference result. If the pixel difference result is greater than a preset threshold, it means that the pixel belongs to the target area, otherwise, it belongs to the background area. After traversing the second image, the target area of the second image is screened and obtained, and the change vector of the second position data (x2, y2) and the third position data (x3, y3) is calculated. Based on the change vector, the distance and direction in which the target area needs to be translated are obtained, and the target area in the second corrected image is obtained. Then, the background area of the second corrected image is generated through the background area and the background model of the second image. The target area and the background area in the second corrected image are merged to generate a second corrected image, and the correction processing of the second image is completed.
[0092] Wherein, adopting the fourth measure to process the original video stream includes:
[0093] Based on the multiple frames of the image, a motion trajectory map of the first target is generated, and based on the motion trajectory map, a corresponding first radar target is obtained by screening and matching in the radar map;
[0094] Acquire the image at time T and record it as a third image, determine the central feature point of the first target in the third image, and record fourth position data of the central feature point;
[0095] The image at time T+1 is obtained and recorded as a fourth image, and the fifth position data of the central feature point is obtained and recorded in the fourth image, where time T and time T+1 are adjacent times;
[0096] Based on the radar map and the first radar target, second velocity characteristic data of the first target is acquired; based on the airborne device, third velocity characteristic data of the flight device is acquired; and based on the second velocity characteristic data and the third velocity characteristic data, sixth position data of the central feature point in the fourth image at the time T+1 is predicted;
[0097] calculating a first change vector between the fifth position data and the sixth position data;
[0098] Based on the first change vector, the fourth image is corrected to obtain a fourth corrected image.
[0099] Because the speed characteristic data of the first target needs to be obtained, the radar map is needed. First, based on the multi-frame collected images, the motion trajectory of the first target is processed, and the first radar target is obtained by screening and matching in the radar map through the motion trajectory of the first target, that is, the first radar target is the first target in the radar map, and then the second speed characteristic data of the first target is obtained through the first radar target and the radar map. The third image at time T is obtained. Since the first target is a dynamic target, in order to better track and capture the first target, the central feature point (key feature point) of the first target is determined, and the fourth position data of the central feature point is recorded. Similarly, the fourth image at time T+1 is obtained, and the fifth position data of the central feature point is obtained and recorded in the fourth image; through the third speed characteristic data of the flight equipment and the second speed characteristic data of the first target, the sixth position data of the central feature point in the fourth image at time T+1 is predicted, and the first change vector between the fifth position data and the sixth position data is calculated. Through the first change vector, the fourth image is corrected to obtain the fourth corrected image. The correction processing measures are similar to those of the second corrected image and will not be further elaborated. It should be noted that the first speed characteristic data and the third speed characteristic data of the flight equipment are both speed characteristic data of the flight equipment.
[0100] Wherein, adopting the fourth measure to process the original video stream further includes:
[0101] Acquire seventh position data of the first radar target in the radar map at time T, acquire eighth position data of the first radar target in the radar map at time T+1, and predict ninth position data of the first radar target in the radar map at time T+1 based on the first speed characteristic data, the second speed characteristic data, and the seventh position data;
[0102] calculating a second change vector between the eighth position data and the ninth position data;
[0103] Calculating a convolution sum of the first change vector and the second change vector, and updating the first change vector according to the convolution sum;
[0104] Based on the updated first change vector, the fourth image is corrected to obtain a fifth corrected image.
[0105] In order to further improve the accuracy of dynamic target image acquisition, the seventh position data of the first radar target in the radar map at time T and the eighth position data at time T+1 are obtained, and the ninth position data of the first radar target in the radar map at time T+1 is predicted based on the first speed feature data, the second speed feature data and the seventh position data, and the second change vector between the eighth position data and the ninth position data is calculated. Since the first change vector and the second change vector both describe the movement of the first target, the convolution sum of the first change vector and the second change vector is calculated, and the first change vector is updated using the convolution sum. The fourth image is corrected using the updated first change vector to obtain a fifth corrected image.
[0106] The first-level determination of the original video stream includes:
[0107] Converting the plurality of frames of the image into a sequence of images, and extracting image information of all the images;
[0108] Based on the image information, calculating the image motion amount of all the sequence images to obtain a first calculation result;
[0109] Based on the first calculation result, calculating the average motion amount of all sequence images to obtain a second calculation result;
[0110] Based on the second calculation result, the dynamic factor is dynamically calculated, and based on the second calculation result and the dynamic factor, a judgment result is output. If the judgment result is 0, it is determined that there is no freeze in the original video stream. If the judgment result is 1, it is determined that there is freeze in the original video stream.
[0111] In video stream quality detection, detecting whether there is a freeze in the video is one of the standards for video quality detection. First, multiple frames of video stream images are converted into sequence images, and then image information of all images is extracted, specifically including traversing the current image set, using the pixels of the next moment frame minus the pixel value of the current moment frame, calculating the difference information between two adjacent frames, setting a threshold, and when the difference value between the two frames exceeds the set threshold, it means that there are moving pixels in the sequence image, otherwise, there is no motion. This step eliminates moving pixels under low motion noise or perception ability; calculate the image motion of all sequence images, specifically including squaring the difference value between the two frames, and calculating the average value of each video frame. The average value is the motion of the current frame, and the total motion value of all frames is calculated; based on the total motion value of all frames, the average motion J of all sequence images is obtained. ; In nearly static scenes and dynamic scenes, due to the small or large changes in pixels, the stuck frames / lost frames will have a small / large amount of motion information, so it is necessary to use a dynamic threshold to determine the stuck video frames; loop through the video frames of the sequence of images to obtain the dynamic factor of each frame. The dynamic factor is calculated as Dfact=a+b*logJ. When the value of Dfact is less than c, the value of Dfact is c, and a, b, and c are all constants, which are 2.5, 1.25, and 0.1 respectively; when the average motion amount J of the sequence of images is less than or equal to the value of Dfact*Mdrop, the output result is 1, and it is considered that the video stream is stuck. When the average motion amount J of the sequence of images is greater than the value of Dfact*Mdrop, the output result is 0, and it is considered that the video stream is not stuck. Mdrop is the dynamic threshold, which is usually 0.015.
[0112] The adopting the first measure to process the original video stream includes: obtaining a first time period during which the original video stream is stuck, and adjusting the transmission rate of the original video stream in the first time period.
[0113] The reason why the video stream is stuck is mostly due to the influence of the data transmission rate. Therefore, by obtaining the time period when the original video stream is stuck, the transmission rate of the video stream is adjusted (increased) during this time period. Similarly, for the time period when the video stream is less stuck, the transmission efficiency of the video stream data can be appropriately reduced.
[0114] The second-level determination of the original video stream includes: determining whether the image corresponding to the original video stream has jitter based on an image grayscale projection algorithm.
[0115] The image grayscale projection algorithm estimates the global motion between adjacent frames through the correlation of the grayscale projection curves of the entire row and column of the image, that is, detects the translation vector between the frames of the video sequence. The image grayscale projection algorithm only performs a cross-correlation operation on the projection curves of the rows and columns of adjacent frames of the video, and can calculate the global motion offset vector between the frames of the video sequence. The algorithm has a small amount of calculation and high accuracy. In addition to the above-mentioned use of the image grayscale projection algorithm to detect whether the image corresponding to the original video stream is jittering, other algorithms such as the optical flow method and the block matching method can also be used, and the present invention does not make specific limitations.
[0116] The second measure for processing the original video stream includes: optimizing the jitter of the image corresponding to the original video stream based on a previous and next frame association algorithm.
[0117] The first and second frames of the corresponding image of the original video stream are taken as two adjacent frames, and the target areas of the two adjacent frames are matched in pairs. The matching method is the sum of the distance between the upper left corner and the lower right corner of the target area. When the sum of the two is less than the threshold, the two adjacent frames are matched successfully, and the first frame is used as the previous frame of the next frame (the third frame) for further matching. When the sum of the two is greater than or equal to the threshold, the two adjacent frames fail to match, and the second frame is used as the previous frame of the next frame (the third frame), and subsequent iterations are performed.
[0118] Embodiment 2
[0119] Please refer to Figure 2 , Figure 2 The present invention is a schematic diagram of an error handling system based on an aviation video stream port, the system comprising:
[0120] An acquisition unit, used to acquire an original video stream, and based on the original video stream, acquire corresponding multiple frames of images;
[0121] A primary determination unit, configured to perform a primary determination on the original video stream to determine whether the original video stream can be played normally, and if so, execute a secondary determination unit; if not, adopt a first measure to process the original video stream, and then execute the secondary determination unit;
[0122] A secondary determination unit, configured to perform a secondary determination on the original video stream to determine whether an image corresponding to the original video stream has jitter, and if so, to process the original video stream using a second measure, and then execute the tertiary determination unit, and if not, to execute the tertiary determination unit;
[0123] A three-level determination unit is used to perform a three-level determination on the original video stream, determine a plurality of coordinate position data of a first target to be tracked based on the original video stream, and determine whether the type of the first target is a static target based on the plurality of coordinate position data, and if so, process the original video stream using a third measure; if not, process the original video stream using a fourth measure;
[0124] The display unit is used to upload the processed original video stream to a preset display device for display.
[0125] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0126] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. An error handling method based on an aviation video stream port, characterized in that: The method comprises the following steps: S1. Obtaining the original video stream, and obtaining the corresponding multi-frame images based on the original video stream; S2. Perform a primary determination on the original video stream to determine whether the original video stream can be played normally. If so, execute step S3. If not, use the first measure to process the original video stream, and then execute step S3; S3. Perform a secondary determination on the original video stream to determine whether the image corresponding to the original video stream is jittered. If so, use a second measure to process the original video stream, and then execute step S4. If not, execute step S4; S4. Performing a three-level determination on the original video stream, determining a plurality of coordinate position data of a first target to be tracked based on the original video stream, and judging whether the type of the first target is a static target based on the plurality of coordinate position data, and if so, processing the original video stream using a third measure; if not, processing the original video stream using a fourth measure; Using the third measure to process the original video stream includes: A1. Obtain the image at time t0 as the first image, determine at least one key feature point of the first target in the first image, and record the first position data of the key feature point; A2. The image at time t1 is obtained and recorded as a second image, and second position data of the key feature point is obtained and recorded in the second image, where t0 and t1 are adjacent times and t0<t1; A3. Based on the airborne device, obtain the first speed characteristic data of the flight device, and based on the first speed characteristic data and the first position data, predict the third position data of the key feature point at time t1 in the second image; A4. Determine whether the second position data coincides with the third position data, and if so, use the second image as a new first image; if not, correct the second image based on the second position data and the third position data to obtain a second corrected image, and use the second corrected image as a new first image; A5. Loop through steps A1 to A4 until all images of the original video stream are processed; Using the fourth measure to process the original video stream includes: Based on the multiple frames of the image, a motion trajectory map of the first target is generated, and based on the motion trajectory map, a corresponding first radar target is obtained by screening and matching in the radar map; Acquire the image at time T and record it as a third image, determine the central feature point of the first target in the third image, and record fourth position data of the central feature point; The image at time T+1 is obtained and recorded as a fourth image, and the fifth position data of the central feature point is obtained and recorded in the fourth image, where time T and time T+1 are adjacent times; Based on the radar image and the first radar target, second velocity characteristic data of the first target is acquired; based on the airborne device, third velocity characteristic data of the flight device is acquired; and based on the second velocity characteristic data and the third velocity characteristic data, sixth position data of the central feature point in the fourth image at the time T+1 is predicted; calculating a first change vector between the fifth position data and the sixth position data; Based on the first change vector, correcting the fourth image to obtain a fourth corrected image; S5. Upload the processed original video stream to a preset display device for display.
2. The error handling method based on the aviation video stream port according to claim 1 is characterized in that: Correcting the second image to obtain a second corrected image includes: Establishing a background model based on the arithmetic mean of pixels between consecutive images; Performing pixel difference between the second image and the background model to obtain a pixel difference result, comparing the pixel difference result with a preset threshold, and if the pixel difference result is greater than the preset threshold, determining that the pixel belongs to the target area, otherwise, determining that the pixel belongs to the background area, and updating the background model based on the value of the pixel; The key feature point belongs to the target area, and a change vector between the second position data and the third position data is calculated; Based on the change vector, the direction and distance in which the target area needs to be translated are obtained to obtain an acquisition result; Based on the acquisition result, the target area is processed to obtain a modified target area; Based on the background area and the background model, generating a corrected background area of the second corrected image; The second corrected image is obtained based on the correction target area and the correction background area.
3. The error handling method based on the aviation video stream port according to claim 1 is characterized in that: Using the fourth measure to process the original video stream also includes: Acquire seventh position data of the first radar target in the radar map at time T, acquire eighth position data of the first radar target in the radar map at time T+1, and predict ninth position data of the first radar target in the radar map at time T+1 based on the first speed characteristic data, the second speed characteristic data, and the seventh position data; calculating a second change vector between the eighth position data and the ninth position data; Calculating a convolution sum of the first change vector and the second change vector, and updating the first change vector according to the convolution sum; Based on the updated first change vector, the fourth image is corrected to obtain a fifth corrected image.
4. The error handling method based on the aviation video stream port according to claim 1 is characterized in that: The first-level determination of the original video stream includes: Converting the plurality of frames of the image into a sequence of images, and extracting image information of all the images; Based on the image information, calculating the image motion amount of all the sequence images to obtain a first calculation result; Based on the first calculation result, calculating the average motion amount of all sequence images to obtain a second calculation result; Based on the second calculation result, the dynamic factor is dynamically calculated, and based on the second calculation result and the dynamic factor, a judgment result is output. If the judgment result is 0, it is determined that there is no freeze in the original video stream. If the judgment result is 1, it is determined that there is freeze in the original video stream.
5. The error handling method based on the aviation video stream port according to claim 1 is characterized in that: Using the first measure to process the original video stream includes: obtaining a first time period during which the original video stream is stuck, and adjusting a transmission rate of the original video stream in the first time period.
6. The error handling method based on the aviation video stream port according to claim 1 is characterized in that: The secondary determination of the original video stream includes: determining whether the image corresponding to the original video stream has jitter based on an image grayscale projection algorithm.
7. The error handling method based on the aviation video stream port according to claim 1 is characterized in that: The second measure for processing the original video stream includes: optimizing the jitter of the image corresponding to the original video stream based on a previous and next frame association algorithm.
8. An error handling system based on an aviation video stream port, implemented by an error handling method based on an aviation video stream port as claimed in claim 1, characterized in that: The system comprises: An acquisition unit, used to acquire an original video stream, and based on the original video stream, acquire corresponding multiple frames of images; A primary determination unit, configured to perform a primary determination on the original video stream to determine whether the original video stream can be played normally, and if so, execute a secondary determination unit; if not, adopt a first measure to process the original video stream, and then execute the secondary determination unit; A secondary determination unit, configured to perform a secondary determination on the original video stream to determine whether an image corresponding to the original video stream has jitter, and if so, to process the original video stream using a second measure, and then execute the tertiary determination unit, and if not, to execute the tertiary determination unit; A three-level determination unit is used to perform a three-level determination on the original video stream, determine a plurality of coordinate position data of a first target to be tracked based on the original video stream, and determine whether the type of the first target is a static target based on the plurality of coordinate position data, and if so, process the original video stream using a third measure; if not, process the original video stream using a fourth measure; The display unit is used to upload the processed original video stream to a preset display device for display.
