A method for obtaining frame-by-frame correspondence between photoelectric turret recorded data and images

Abstract

The present invention relates to a method for obtaining the frame-by-frame correspondence between the photoelectric turret recorded data and the image. By extracting the pixel position and color card information of the time in the image, the time is named as the name of the image, and the picture name and time are extracted as an independent document, corresponding to the photoelectric turret posture, position and other information recorded at the same time, so as to achieve a one-to-one correspondence between the image and the data recorded at the same time. This technology provides a basis for the secondary development and utilization of recorded data, and solves the problem that the photoelectric turret cannot effectively retrieve and use flight data. The whole process realizes automated operation, greatly saves manpower and material resources, improves business efficiency, and reduces human errors. Automatic retrieval and redefinition of images can be completed by setting parameters, and information correspondence can be performed with the help of a database.

Description

Technical Field

[0001] The present invention belongs to computer image processing, and relates to image recognition and database retrieval methods. Specifically, it relates to a method for obtaining the correspondence between the photoelectric turret recorded data and the image frame by frame, which can complete the automatic retrieval and redefinition of the image by setting software parameters, and use the database to perform the work of information frame by frame correspondence, which provides a basis for the secondary development and utilization of the recorded data, and solves the problem that the photoelectric turret cannot effectively retrieve and use the flight data. Background Art

[0002] The optoelectronic pod is an airborne optoelectronic detection system that integrates infrared, visible light, and laser, and is the main optoelectronic reconnaissance equipment for aircraft platforms. Based on aircraft, missiles and other flying platforms, it solves the basic scientific problem of accurately positioning and measuring the speed of time-sensitive targets whose position coordinates cannot be predicted before the battle. Without changing the existing combat system and its hardware platform, the optoelectronic pod needs to provide real-time information, including: time information accurate to the millisecond level, and the attitude angle, geographic location, laser ranging information of the optoelectronic turret corresponding to this moment, and the pixel deviation from the target to the center of the original image at the same time.

[0003] The photoelectric turret is equipped with a recording component to record images and data information. Images and data are transmitted through two channels, so there will be time errors. At the same time, long-term recording will generate a large amount of data and pictures, which is difficult to achieve one-to-one correspondence. If it relies on manual search, it is inefficient and prone to human errors, so it is difficult to achieve.

[0004] Therefore, an automated method is needed to achieve frame-by-frame correspondence between the photoelectric turret recorded data and images, and to name the images as display labels on the screen, which will facilitate secondary development and utilization of the data. Summary of the invention

[0005] Technical issues to be solved

[0006] In order to avoid the shortcomings of the prior art, the present invention proposes a method for obtaining the frame-by-frame correspondence between the optoelectronic turret recorded data and the image, which provides a basis for the secondary development and utilization of the recorded data, solves the problem that the optoelectronic turret cannot effectively retrieve and use flight data, and saves manpower and material resources, improves business efficiency, and reduces human errors.

[0007] Technical Solution

[0008] A method for obtaining the frame-by-frame correspondence between photoelectric turret recorded data and images, characterized by the following steps:

[0009] Step 1, extract the time information of the image: use the color card extraction pen of photoshop to extract the RGB of the time and surrounding parts of the TV and infrared original pictures provided by the recording component inside the photoelectric turret, and use the digital RGB range on the image as white, and the range outside this range as black to complete the binary processing; place the pixel position of the upper left corner of each number in the array "Left";

[0010] Step 2, extract digital templates as image numbers: randomly select one or more pictures, intercept single numbers according to the pixel position of the array "Left" and perform binarization processing until all ten numbers from 0 to 9 with a pixel size of 12*12 are obtained; use the ten numbers from 0 to 9 as 10 digital templates;

[0011] Step 3: According to the first and last pixel positions provided by the Left array, the digital part of the image is cut out in the form of a "digital bar", and only the pixels within the RGB range are retained;

[0012] Step 4: Crop the number strips one by one to get a single digital photo, and match the pixels with each template starting from "0" one by one:

[0013] If the number of different pixels is less than 3, the template is considered to be matched successfully, and the number captured on the image is the number recognized by the template at this time;

[0014] If the number of different pixels is greater than or equal to 3, it will be compared with the next template until a template with a pixel deviation less than 3 is matched;

[0015] If the matching with the template "9" fails, it is considered that the recognition fails and the number is ambiguous;

[0016] Step 5: Put the recognized numbers one by one into an array named "numbers". If the length of the numbers array recognized by the "number bar" is 9, it is considered that the image recognition of this "number bar" is successful. The image is named in the format of "hour_minute_second_millisecond.tif" and recorded in the log file "time.txt". Otherwise, the image name is not changed and the "time.txt" file is not updated.

[0017] Step 6: Import all the inertial navigation data into the database and add a new column for the picture name to form an inertial navigation database;

[0018] Step 7: By traversing, compare the time information in the image name with the existing inertial navigation time information in the database;

[0019] Use the pointer to access each row of the database, and insert the picture name that is the same as the inertial navigation time into the picture name sequence of the inertial navigation database;

[0020] When there is no picture name that is the same as the inertial navigation time, the closest time interval is used for subtraction:

[0021]

[0022] Among them, t2 is the time of the picture that does not exist in the database, t1 and t3 are respectively the previous row of data and the next row of data in the database closest to t2, x and y are respectively the corresponding record data corresponding to t1 and t3, and z is the corresponding data at time t2 obtained by difference.

[0023] After all data are imported, the new database is exported, and the corresponding data and images within the photoelectric turret cycle can be obtained.

[0024] When the inertial navigation data is imported into the database, the inertial navigation data of the exported recorder is in two formats: 1. ANSI data format, using space as a delimiter for data slicing; 2. UTF-8 data format, transcoding the character code into a string, using '\t', '\r'.'ufeff' as a delimiter for data slicing.

[0025] Beneficial Effects

[0026] The present invention proposes a method for obtaining the frame-by-frame correspondence between the photoelectric turret recorded data and the image, by extracting the pixel position and color card information of the time in the image, naming the time as the name of the image, and extracting the picture name and time as an independent document, corresponding to the photoelectric turret posture, position and other information recorded at the same time, so as to achieve a one-to-one correspondence between the image and data recorded at the same time. This technology provides a basis for the secondary development and utilization of recorded data, and solves the problem that the photoelectric turret cannot effectively retrieve and use flight data. The entire process realizes automated operation, greatly saves manpower and material resources, improves business efficiency, and reduces human errors. Automatic retrieval and redefinition of images can be completed by setting parameters, and information correspondence can be performed with the help of a database. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart for identifying time numbers in a single image and renaming the image according to the identified time numbers

[0028] Figure 2 Template images for numbers 0 to 9

[0029] Figure 3 The images are renamed according to the recognized time numbers

[0030] Figure 4 Flow chart for formatting the inertial navigation data in the recording device and importing it into the database

[0031] Figure 5In order to import the identified time information corresponding to the inertial navigation data into the database, plan the data format in the database, and export the new inertial navigation data flow chart DETAILED DESCRIPTION

[0032] The present invention will now be further described with reference to the embodiments and the accompanying drawings:

[0033] The technical solution is: extract the pixel position and RGB range of the TV and infrared original picture time information provided by the recording component inside the optoelectronic turret, obtain ten digital templates from 0 to 9, and compare the time slices word by word to obtain complete time information. If successfully recognized, it is renamed to the image name, and the time and picture name are entered into the record document. At the same time, the real-time data information of the turret provided by the recording component is imported into the database, and a new column for the picture name is added. If there is a corresponding moment for the picture, the picture name is directly inserted into the database, otherwise, a difference is made, a new row of data is inserted, and the picture name in the database at this time is marked. After all the data is imported, the new database is exported. At this time, the corresponding data and images within the optoelectronic turret cycle can be obtained.

[0034] The whole process is implemented through Python, OpenCV image recognition, and database corresponding insertion. The first step is to extract the time information (hour, minute, second, millisecond) on the image:

[0035] 1. In order to extract the time information of the image, first use the color card extraction pen of photoshop to extract the RGB of the time and the surrounding parts to complete the binary processing, that is, separate the RGB range of the numbers on the image. If it exceeds this range, it will be black, and if it is within the range, it will be white. Since the pixel size (12*12) and position of each number on the photoelectric turret image are the same, the number of numbers to be recognized is also equal, so it is only necessary to place the pixel position of the upper left corner of each number in the array "Left".

[0036] 2. Before image processing, it is necessary to extract the template to pave the way for subsequent image digital comparison. Randomly select one or more pictures, intercept single numbers according to the pixel position of the array "Left" and perform binary processing until all ten numbers from 0 to 9 with a pixel size of 12*12 are obtained, and they can be used as templates. Figure 2 That is 10 digital templates.

[0037] 3. Now we can start the image digital extraction. To prevent the program from abnormalities due to factors such as image duplication and damage, the images are divided into two categories during the recognition process: success and failure. If we use the opencv library to compare pixels one by one, there will be a large number of images with recognition errors or failures. After testing, the success rate is only 30%. Therefore, we choose to use the template matching method:

[0038] 1) According to the first and last pixel positions provided by the Left array, the digital part of the image is cut out in the form of a "digital bar", and only the pixels within the RGB range are retained.

[0039] 2) Crop the number strips one by one to get a single digital photo, and match the pixels with each template starting from "0" one by one. If the number of different pixels is less than 3, the template is considered to be matched successfully, and the number captured on the image is the number recognized by the template at this time; if the number of different pixels is greater than or equal to 3, it is compared with the next template until a template with a pixel deviation of less than 3 is matched; if the match with the template "9" is unsuccessful, it is considered that the recognition has failed and the number is blurred.

[0040] 3) Put the recognized numbers one by one into an array named "numbers". If the length of the numbers array recognized by the "number bar" is 9, the image recognition of this "number bar" is considered successful, and the image is named in the format of "hour_minute_second_millisecond.tif" and recorded in the log file "time.txt". Otherwise, the image name is not changed and the "time.txt" file is not updated.

[0041] The whole identification process flow chart is shown in the attached figure Figure 1 shown.

[0042] This method has been tested and has a success rate of 95% or more, and an accuracy rate of 98% or more. Because computers can automatically sort files by file name, this method can separate the images that have not been successfully recognized from the images that have been successfully recognized, making it easy to see at a glance.

[0043] 4. Since the communication between the subsystems in the optoelectronic turret is carried out according to their own specified cycles (but the cycle is very short, generally in milliseconds), there are timing errors and sending and receiving delays. Therefore, the recorded pictures and inertial navigation data may not correspond one to one, that is, the pictures exist, but there may be no inertial navigation data at the corresponding time. Based on this situation, consider using the difference to insert the non-existent time data information. Therefore, the database (Microsoft SQL server) was introduced. All inertial navigation data are imported first through the POS information provided by the recorder. As shown in the attached figure Figure 4 shown.

[0044] 5. Next, the time information identified by the image is compared and inserted with the existing inertial navigation time information in the database through traversal. For the traversal of the database, considering that the amount of data may be extremely large, a pointer will be used to access each row of the database, and the traversal range is set to be limited to 1s. This can greatly reduce the time required for traversal and ensure that the time recorded by the image at this time is closest to the time recorded by the inertial navigation. Therefore, if there is a corresponding time, the image name can be consistent with the recorded data; if there is no corresponding time, the closest time interval can be found for subtraction. The subtraction formula is:

[0045]

[0046] Among them, t2 is the time of the image that does not exist in the database, t1 and t3 are the previous and next rows of data in the database closest to t2, x and y are the corresponding record data corresponding to t1 and t3, and z is the corresponding data at time t2 obtained by difference. Because the time difference is very short, it can be considered that the row inserted at this time is highly reliable and can be used as analysis data.

[0047] When all the images and times can be found in the database, the data import is considered complete. Export all the data in the database and complete the correspondence between the photoelectric turret recorded data and the image frame by frame. Figure 5 That is, the inertial navigation data corresponding to the picture is inserted into the flow chart.

Claims

1. A method for obtaining frame-by-frame correspondence between photoelectric turret recorded data and images, characterized in that: Here are the steps: Step 1, extract the time information of the image: use the color card extraction pen of photoshop to extract the RGB of the time and surrounding parts of the TV and infrared original pictures provided by the recording component inside the photoelectric turret, and use the digital RGB range on the image as white, and the range outside this range as black to complete the binary processing; place the pixel position of the upper left corner of each number in the array "Left"; Step 2, extract digital templates as image numbers: randomly select one or more pictures, intercept single numbers according to the pixel position of the array "Left" and perform binarization processing until all ten numbers from 0 to 9 with a pixel size of 12*12 are obtained; use the ten numbers from 0 to 9 as 10 digital templates; Step 3: Cut out all the digital parts of the image in the form of "digital bars" according to the first and last pixel positions provided by the Left array, and only keep the pixels within the RGB range; Step 4: Crop the digital strips one by one to obtain individual digital photos, and match the pixels with each template starting from "0" one by one; If the number of different pixels is less than 3, the template is considered to be matched successfully, and the number captured on the image is the number recognized by the template at this time; If the number of different pixels is greater than or equal to 3, it will be compared with the next template until a template with a pixel deviation less than 3 is matched; If the matching with the template "9" fails, it is considered that the recognition fails and the number is ambiguous; Step 5: Put the recognized numbers one by one into an array named "numbers". If the length of the numbers array recognized by the "number bar" is 9, it is considered that the image recognition of this "number bar" is successful. The image is named in the format of "hour_minute_second_millisecond.tif" and recorded in the log file "time.txt". Otherwise, the image name is not changed and the "time.txt" file is not updated. Step 6: Import all the inertial navigation data into the database and add a new column for the picture name to form an inertial navigation database; Step 7: By traversing, compare the time information in the image name with the existing inertial navigation time information in the database; Use the pointer to access each row of the database, and insert the picture name that is the same as the inertial navigation time into the picture name sequence of the inertial navigation database; If there is no picture name that is the same as the inertial navigation time, the closest time interval is used for difference calculation; ； in, The time of the picture that does not exist in the database. , Distance The previous row of data and the next row of data in the most recent database, x and y are , The corresponding record data, z is the difference The data corresponding to each moment; After all the data are imported, the new database is exported, and the corresponding data and images within the optoelectronic turret cycle can be obtained.

2. The method for obtaining the frame-by-frame correspondence between the photoelectric turret recorded data and the image according to claim 1, characterized in that: When the inertial navigation data is imported into the database, the inertial navigation data of the exported recorder is in two formats:

1. ANSI data format, using space as a delimiter for data slicing; 2. UTF-8 data format, transcoding the character code into a string, using '\t', '\r'.'ufeff' as a delimiter for data slicing.

Images