A method for automatic data annotation of remote sensing images based on AIS data retrieval
Through the automatic labeling method based on AIS data retrieval, the problem of vessel target detection in remote sensing images is solved, the labeling efficiency and accuracy are improved, the data set production cycle is shortened, and efficient automatic labeling of vessel targets in remote sensing images is achieved.
Patent Information
- Application Number
- CN202111543132.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-12-16
AI Technical Summary
Deep learning-based object detection method relies on manual labeling of data sets when detecting and identifying ship objects in remote sensing images, resulting in long production cycle, high manpower investment and low efficiency. Especially in remote sensing images in nearshore areas, the target size of ship objects is small and easy. Confused, difficult to accurately detect.
Using the automatic labeling method based on AIS data retrieval, the coordinate range of the camera monitoring screen is calculated, the latitude and longitude coordinates and navigation information of the ship are queried, and the interpolation method and the mapping relationship between the image coordinate system and the geographical coordinate system are used to calculate the position of the ship in the image. and integrate ship type information to form automatic labeling data.
The labeling efficiency of the ship target data set in remote sensing images is improved, the production cycle of the data set is shortened, and the rapid and accurate ship target detection and recognition are achieved.
Smart Images

Figure CN114625911B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of cloud computing, and in particular relates to an automatic data annotation method for remote sensing images based on AIS data retrieval. Background Art
[0002] The object detection method based on deep learning neural network is a relatively efficient and accurate calculation method in the field of computer vision. Different object detection models can detect and identify different types and sizes of objects in the image by virtue of their deep network structure design and the training data used, and obtain the position coordinates and type prediction results of the object in the image.
[0003] However, deep learning-based target detection methods often rely on large manually annotated data sets for training to achieve better detection and recognition results. Especially when detecting and identifying ship targets in remote sensing images, in remote sensing images of nearshore areas, the ship targets in the images are small in size and easily confused with other targets such as containers, which brings great difficulties to the production of data sets. The production cycle of data sets that rely on manual annotation is long, the manpower investment is high, and the production efficiency is low. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a method for automatic annotation of ship target data based on AIS data retrieval, which can improve the annotation efficiency of large data sets of remote sensing image ship targets and shorten the production cycle of data sets.
[0005] A method for automatically labeling ship target data, characterized in that it comprises the following steps:
[0006] Step 1: Calculate the coordinate range of the camera's monitoring screen and obtain the four longitude and latitude coordinates of the vertices of the monitoring image (long1, lati1), (long2, lati2), (long3, lati3), (long4, lati4),
[0007] Step 2: Use the regional ship query API based on HTTP to query the ship navigation information within the set time period in the area with the longitude and latitude coordinates (long1, lati1), (long2, lati2), (long3, lati3), and (long4, lati4) as the four vertices;
[0008] Step 3: According to the regional ship information queried by AIS in step 2, find the time point t at which the camera image was taken p The geographic coordinates of the target ship at two adjacent timestamps t1 and t2 are obtained by interpolation method. p The latitude and longitude coordinates of;
[0009] Step 4: Based on the mapping relationship between the latitude and longitude coordinates of the target ship and the camera coordinates, calculate the image coordinate value (x c ,y c ).
[0010] Step 5: Use the ship information query API based on http to query the ship type, length, width and heading information of the ship, and integrate and calculate the vertex position coordinates of the ship in the image and the corresponding ship category information;
[0011] Step 6: Use the position coordinates and type data of the target ship obtained in step 5 to mark the target ship to form text annotation data.
[0012] Preferably, in step 1, the calculation method of the four latitude and longitude coordinates is:
[0013] Artificially select known initial reference points A, B within the shooting range, and the image pixel coordinates A(u a ,v a ),B(u b ,v b ) and record its longitude and latitude A(long a ,lati a ),B(long b ,lati b ); According to the linear relationship between the image coordinate system and the geographic coordinate system, the coordinates of the four vertices of the captured image frame are obtained as follows:
[0014]
[0015]
[0016] Among them, (long i ,lati i ) are the geographical coordinates of the four vertices in the image, i = 1, 2, 3, 4; (u i ,v i ) are the image position coordinates of the four vertices of the image, which can be obtained by camera calibration and establishing a mapping relationship between the image coordinate system and the longitude and latitude coordinates.
[0017] Preferably, in step 3, when the camera in step 1 is a satellite-borne camera, the shooting time t P Obtain timing signals from satellites for calibration.
[0018] Preferably, in step 3, when the camera in step 1 is an airborne camera, it is determined whether the clock jumps during shooting. If there is a jump, the stable value before and after the timestamp is obtained, and the shooting time t is calculated according to the shooting time interval.P make corrections;
[0019] Preferably, in step 3, when the camera in step 1 is an airborne camera, it is determined whether there is an error between the camera clock and the timestamp data of the ship's geographic coordinates updated in real time in the AIS database. If there is an error, the camera clock is periodically compared with the standard UTC time to obtain the shooting time t P Perform error correction.
[0020] Preferably, in step 4:
[0021]
[0022]
[0023] in, is the latitude and longitude coordinates of the center point of the ship target after correction in step 4; (u a ,v a )、(u b ,v b ) are the image coordinates of the known reference points A and B on the image, (long a ,lati a ),(long b ,lati b ) are the latitude and longitude coordinates of reference points A and B.
[0024] Preferably, in step 5, the length L of the ship obtained by query ship and width W ship The type of vessel and the resolution of the camera cam Convert to the length l in the image coordinate system s and width s , the calculation formula is as follows:
[0025] l s =L ship × cam
[0026] w s =W ship × cam
[0027] The center position coordinates (x c ,y c ), considering the length and width of the ship, calculate the coordinates of the four vertices of each ship target (x min ,y min )(x min ,y max )(x max ,ymax )(x max ,y min ), the calculation process is as follows:
[0028]
[0029]
[0030]
[0031]
[0032] Among them, θ is the course information of the ship.
[0033] The present invention has the following beneficial effects:
[0034] The present invention provides a method for automatically labeling ship target data based on AIS data retrieval. First, the location parameters such as the installation location, installation height and pitch angle of a monitoring camera are measured to calculate the latitude and longitude coordinate range of the area to be queried; then, the timestamp and geographic coordinates, heading, driving speed and other information of the ship target are obtained through AIS query; the location coordinates of the ship target in the image are corrected and calculated by using a method for fast matching of images and geographic coordinates; the labeling data format of the target in the image is integrated to complete the data labeling of the remote sensing image ship target, which can improve the labeling efficiency of a large data set of remote sensing image ship targets and shorten the production cycle of the data set. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0036] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0037] Step 1: Calculate the coordinate range of the camera's monitoring screen and obtain the four longitude and latitude coordinates of the vertices of the monitoring image (long1, lati1), (long2, lati2), (long3, lati3), (long4, lati4), specifically:
[0038] Artificially select known initial reference points A, B within the shooting range, and the image pixel coordinates A(u a ,v a ),B(u b ,v b ) and record its longitude and latitude A(long a ,lati a ),B(long b ,lati b). According to the linear relationship between the image coordinate system and the geographic coordinate system, the vertex coordinates of the captured image frame can be obtained as follows:
[0039]
[0040]
[0041] Among them, (long i ,lati i ) are the geographical coordinates of the four vertices in the image, i = 1, 2, 3, 4; (u i ,v i ) are the image position coordinates of the four vertices of the image, which can be obtained by camera calibration and establishing a mapping relationship between the image coordinate system and the longitude and latitude coordinates.
[0042] Step 2: Use the regional ship query API based on http to query the ship navigation information within the set time period in the area through the longitude and latitude coordinates (long1, lati1), (long2, lati2), (long3, lati3), (long4, lati4), and obtain the ship's MMSI number, longitude and latitude coordinates (lon posttime ,lat posttime ), timestamp posttime, etc. and heading course information.
[0043] Step 3: Optical remote sensing images are generally airborne remote sensing images and satellite remote sensing images. Airborne remote sensing images are sensors mounted on aircraft and other tools to collect data on the ground, while satellite remote sensing images are sensors mounted on earth-flying satellites to collect data. Satellite cameras generally do not have shooting time accuracy errors because they can obtain timing signals from satellites for calibration and thus synchronize with standard time.
[0044] If the onboard camera terminal device is not connected to the Internet, the image capture time t P This is the default system time of the camera. The device has a problem of limited clock accuracy, so there may be a gap with the standard time. It is necessary to set up and configure an NTP clock server within the network, and then fill the address of the NTP clock server into the configuration interface of each network camera or network hard disk recorder to ensure time synchronization and obtain the corrected image shooting time. The correction method may be as follows:
[0045] 1) It is necessary to determine whether the clock jumps during shooting and exclude the situation where the clock is reset. You can obtain the images taken by the camera within several hours and determine the jump situation by comparing whether their timestamps are continuous. If there is a jump, obtain the stable value before and after the timestamp and make corrections based on the shooting time interval;
[0046] 2) Determine whether there is an error between the camera clock and the timestamp data of the ship's geographic coordinates updated in real time in the AIS database. AIS data uses the positioning timestamp of the Universal Time Coordinated (UTC). The camera clock can be compared with the standard UTC time at regular intervals to correct the error. If the camera clock is highly accurate, the error should be a stable constant. If the camera clock is less accurate, the error may gradually increase over time.
[0047] Considering the above two situations, the error correction is performed on the data of the image time stamp. The formula is as follows:
[0048] t p =t q ±Δt e
[0049] Δt e is the error value, t p is the corrected shooting time, t q This is the system shooting time with errors before correction.
[0050] Step 4: Based on the regional ship information queried by AIS in step 2, find the time point t at which the camera image was taken p The geographical coordinates of the target ship at two adjacent timestamps t1 and t2
[0051] Since the AIS dynamic information update cycle is short and the navigation track of a ship in a short period of time can be approximated as a straight line, the linear difference method can be used to calculate the approximate longitude and latitude coordinates of the ship at the time of image shooting.
[0052]
[0053]
[0054] st
[0055] t1 <t p <t2
[0056] Step 5: Based on the mapping relationship between the latitude and longitude coordinates of the target ship and the camera coordinates, calculate the image coordinate value (x c ,y c ).
[0057]
[0058]
[0059] in, is the latitude and longitude coordinates of the center point of the ship target after correction in step 4; A(u a ,v a ),B(u b ,v b ) are the image coordinates of the known reference points A and B on the image.
[0060] Step 6: Use the http-based ship information query API to query the ship type, length, width, and heading information of the ship, and integrate and calculate the vertex position coordinates of the ship in the image and the corresponding ship category information.
[0061] The length L of the ship obtained according to the query ship and width W ship The type of vessel and the resolution of the camera cam Convert to the length l in the image coordinate system s and width s The calculation formula is as follows:
[0062] l s =L ship × cam
[0063] w s =W ship × cam
[0064] The center position coordinates (x c ,y c ), considering the length and width of the ship, calculate the coordinates of the four vertices of each ship target (x min ,y min )(x min ,y max )(x max ,y max )(x max ,y min ), the calculation process is as follows:
[0065]
[0066]
[0067]
[0068]
[0069] Among them, θ is the course information of the ship obtained by querying in step 2.
[0070] Step 7: Use the data obtained in step 6 to annotate and form text annotation data.
[0071] Integrate the location coordinates and type of each ship target to form a data annotation in text format (.txt or .xml format), and the annotation content is:
[0072] id,category,coordiate1,coordiate2,coordiate3,coordiate4
[0073] The following are examples of annotations:
[0074]
[0075] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for automatic labeling of ship target data, It is characterized in that The steps include: Step 1: Calculate the coordinate range of the camera's monitoring screen and obtain the four longitude and latitude coordinates of the vertices of the monitoring image (long 1 , lati 1 ), (long 2 , lati 2 ), (long 3 , lati 3 ), (long 4 , lati 4 ), Step 2: Use the regional ship query API based on http to query the longitude and latitude coordinates (long 1 , lati 1 ), (long 2 , lati 2 ), (long 3 , lati 3 ), (long 4 , lati 4 ) is the navigation information of ships within the set time period in the area of the four vertices; Step 3: According to the regional ship information queried by AIS in step 2, find the time point t at which the camera image was taken p Two adjacent timestamps t 1 , t 2 The geographic coordinates of the target ship are obtained by interpolation method at time point t p The latitude and longitude coordinates of; Step 4: Based on the mapping relationship between the latitude and longitude coordinates of the target ship and the camera coordinates, calculate the image coordinate value (x c ,y c ); Step 5: Use the ship information query API based on http to query the ship type, length, width and heading information of the ship, and integrate and calculate the vertex position coordinates of the ship in the image and the corresponding ship category information; Step 6: Use the position coordinates and type data of the target ship obtained in step 5 to mark the target ship to form text annotation data; In step 1, the calculation method of the four latitude and longitude coordinates is: Artificially select known initial reference points A, B, and image pixel coordinates A(u a , v a ), B(u b ,v b ) and record its longitude and latitude A(long a , lati a ), B(long b , lati b ); According to the linear relationship between the image coordinate system and the geographic coordinate system, the coordinates of the four vertices of the captured image frame are obtained as follows: Among them, (long i , lati i ) are the geographical coordinates of the four vertices in the image, i = 1, 2, 3, 4; (u i , v i ) are the image position coordinates of the four vertices of the image, which can be obtained by camera calibration and establishing a mapping relationship between the image coordinate system and the longitude and latitude coordinates.
2. A method for automatically marking ship target data according to claim 1, It is characterized in that In step 3, when the camera in step 1 is a satellite-borne camera, the shooting time t P Obtain timing signals from satellites for calibration.
3. A method for automatically marking ship target data according to claim 1, It is characterized in that In step 3, when the camera in step 1 is an airborne camera, it is determined whether the clock jumps during shooting. If there is a jump, the stable value before and after the timestamp is obtained, and the shooting time t is calculated according to the shooting time interval. P Make corrections.
4. A method for automatically marking ship target data as claimed in claim 3, It is characterized in that In step 3, when the camera in step 1 is an airborne camera, it is determined whether there is an error between the camera clock and the timestamp data of the ship's geographic coordinates updated in real time in the AIS database. If there is an error, the camera clock is periodically compared with the standard UTC time to obtain the shooting time t P Perform error correction.
5. A method for automatically marking ship target data according to claim 1, It is characterized in that In step 4: in, is the latitude and longitude coordinates of the center point of the ship target after correction in step 4; (u a , v a )、(u b , v b ) are the image coordinates of the known reference points A and B on the image, (long a , lati a ), (long b , lati b ) are the latitude and longitude coordinates of reference points A and B.
6. A method for automatically marking ship target data according to claim 1, It is characterized in that In step 5, the length L of the ship obtained from the query ship and width W ship The type of vessel and the resolution of the camera cam Convert to the length l in the image coordinate system s and width s , the calculation formula is as follows: l s =L ship ×r cam In s =In ship ×r cam The center position coordinates (x c ,y c ), considering the length and width of the ship, calculate the coordinates of the four vertices of each ship target (x min ,y min )(x min ,y max )(x max ,y max )(x max ,y min ), the calculation process is as follows: Among them, θ is the course information of the ship.
