System and method for quickly creating UI (user interface) sequence image data set sample

Abstract

The invention relates to the technical field of image processing, in particular to a UI sequence image data set sample rapid creation system and method, and the system comprises an interface interaction module which is used for opening a remote sensing image sequence, and determining the cutting effect, the cutting region and the cutting form of a remote sensing image in the remote sensing image sequence; the image registration module is used for registering the remote sensing image sequence; and the sequence image cutting module is used for receiving the two remote sensing image sequences and cutting the two remote sensing image sequences according to the cutting form, the cutting area and the cutting effect. According to the method, an interface interactive method is adopted, the image is segmented according to input row and column information, and processing functions based on FFT registration, image video generation and the like are added, so that a data set required by deep learning can be obtained more quickly and better.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a system and method for quickly creating a UI interface sequence image data set sample. Background Art

[0002] The creation of data sets in the field of deep learning is crucial. It can be said that without a large amount of data information, a model with good performance cannot be trained. However, in the direction of sequence image processing, the creation of data sets is very difficult and often requires a cumbersome manual processing stage. The manual processing stage specifically includes: ① Image registration is required; ② Visualize the image to determine whether there are regions or targets of interest in the image; ③ Perform unified cropping according to the pixel positions of the regions or targets of interest; ④ Generate the required video.

[0003] Current interface-based remote sensing image cropping uses professional remote sensing image software such as ENVI, ArcGIS, and PCI Geomatica. However, their cropping principles are to load the remote sensing image and then divide the image into pieces one by one according to the selected area, pixel position, threshold, etc., lacking the ability to crop sequence images and without the relevant preprocessing processes for sequence images. Therefore, the selection of sequence images by these software is extremely cumbersome and requires a large amount of manual processing costs. Summary of the Invention

[0004] In view of this, the present invention aims to provide a system and method for quickly creating a UI interface sequence image data set sample, which uses an interface interactive method to divide the image according to the input row and column information, and adds processing functions such as FFT registration and image video generation, so as to obtain the data set required for deep learning faster and better.

[0005] To achieve the above object, the technical solution of the present invention is realized as follows: A system for quickly creating a UI interface sequence image data set sample includes: An interface interaction module for opening a sequence of remote sensing images to be edited, determining the cropping effect, cropping area, and cropping form of the remote sensing images in the sequence of remote sensing images, and determining the output file path of the cropped image blocks; An image registration module for registering the sequence of remote sensing images to obtain and store the registered sequence of remote sensing images; The sequence image cropping module receives the registered remote sensing image sequence or the remote sensing image sequence, and crops the registered remote sensing image sequence or the remote sensing image sequence according to the cropping form, cropping area, and cropping effect; and stores the image blocks in the corresponding folders under the output file path according to the serial numbers.

[0006] Further, the interface interaction module includes: The row and column determination sub-module is used to input row information and column information, and transfer the row information and column information to the sequence image cropping module; The display sub-module is used to display the cropping effect according to the row information and column information; The cropping area determination sub-module is used to determine the cropping area and transfer the cropping area to the sequence image cropping module; The registration determination sub-module is used to control the image registration module to register the remote sensing image sequence; The cropping form determination sub-module is used to determine the cropping form; the cropping forms include TIFF cropping, PNG cropping, cropping by sequence, and cropping by selection; The output path determination sub-module determines the output file path of the cropped remote sensing image sequence according to the input file path of the remote sensing image sequence.

[0007] Further, in the image registration module: According to the input file path, determine the registration file path of the registered remote sensing image sequence; Use the fast Fourier transform to register the remote sensing image sequence to obtain the registered remote sensing image sequence, and store the registered remote sensing image sequence in the folder of the registration file path.

[0008] Further, the sequence image cropping module includes: The TIFF cropping sub-module, when the cropping form is TIFF cropping, the TIFF cropping sub-module calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and column information, and stores the image blocks in the corresponding TIFF folders according to the images where they are located; store the TIFF folders in the total folder under the output file path; The PNG cropping sub-module, when the cropping form is PNG cropping, the PNG cropping sub-module converts the registered remote sensing image sequence or the remote sensing image sequence to the PNG format, and then calls the PNG cropping function to crop the images of the registered remote sensing image sequence or the remote sensing image sequence in the PNG format according to the row information and column information, and stores the cropped PNG format image blocks in the corresponding PNG folders according to the images where they are located; store the PNG folders in the total folder; Sequence cropping sub-module. When the cropping form is sequence-based cropping, the sequence cropping sub-module calls the TIFF cropping function to crop the registered remote sensing image sequence or the images in the remote sensing image sequence according to the row information and column information, gathers the image blocks with the same serial number into the same sequence folder, and stores the sequence folder in the total folder. Selected area cropping sub-module. When the cropping form is selected area cropping, the selected area cropping sub-module calls the TIFF cropping function to crop the registered remote sensing image sequence or the images in the remote sensing image sequence according to the row information and column information, gathers the image blocks that meet the cropping area into the same area folder according to the serial number, and stores the area folder in the total folder.

[0009] Furthermore, sequence-based cropping includes TIFF sequence cropping and TIFF to PNG sequence cropping. Among them, when the cropping form is TIFF sequence cropping, the sequence cropping sub-module directly stores the image blocks in the sequence folder; when the cropping form is TIFF to PNG sequence cropping, the sequence cropping sub-module converts the image blocks to the PNG format and then stores the PNG format image blocks in the sequence folder.

[0010] Furthermore, selected area cropping includes TIFF selected area cropping and TIFF to PNG selected area cropping. Among them, when the cropping form is TIFF selected area cropping, the selected area cropping sub-module directly stores the area folder in the total folder; when the cropping form is TIFF to PNG selected area cropping, the selected area cropping sub-module converts the image blocks in the area folder to the PNG format and then stores the area folder containing the PNG format image blocks in the total folder.

[0011] Furthermore, the system also includes a dataset generation module, and the dataset generation module includes: Image clustering sub-module. When the cropping form is selected area cropping or sequence-based cropping, the image clustering sub-module uses a clustering algorithm to splice the image blocks according to the serial number; stores the corresponding synthesized area obtained after splicing in the area folder or sequence folder. Video generation sub-module. Converts the PNG format image blocks or the synthesized areas in PNG format into a video according to the serial number and stores the obtained video data.

[0012] Furthermore, in the image clustering sub-module, when the cropping area is not a rectangle, the cropping area is supplemented with a black background to obtain a rectangle with the smallest area after filling.

[0013] A method for quickly creating a sequence image dataset sample. According to the UI interface sequence image dataset sample quick creation system provided by the present invention, it includes: S1: Input the file path where the remote sensing image sequence is located in the interface interaction module, and determine the cropping effect, cropping area, and cropping form in the interface interaction module; and determine whether to perform image matching in the interface interaction module; if image matching is to be performed, execute step S2, if not, execute step S3; S2: The image registration module collects the remote sensing image sequence from the file path obtained in step S1, and registers the remote sensing image sequence to obtain and store the registered remote sensing image sequence; S3: The sequence image cropping module determines the cropping form determined in step S1, and combines the cropping effect and cropping area to crop the remote sensing image sequence from step S1 or the registered remote sensing image sequence from step S2, and stores the cropped remote sensing image sequence in the corresponding folder according to the output file path.

[0014] Furthermore, the creation system further includes a dataset generation module; the fast creation method further includes: S4: Determine whether image clustering is required in the interface interaction module. If image clustering is required, execute step S5; otherwise, execute step S6; S5: The dataset generation module performs clustering on the image blocks, and stores the clustered composite regions in the corresponding folders; S6: Determine whether video conversion is required in the interface interaction module. If so, the dataset generation module converts the cropped image blocks into a video according to the serial number to obtain and store the video data.

[0015] Compared with the prior art, the present invention can achieve the following beneficial effects: (1) The UI interface sequence image dataset sample fast creation system of the present invention adopts an image cropping method that divides according to rows and columns under interface interaction. The cropping is convenient and efficient, and there is no need to manually select the region of interest. At the same time, the cropped images are of the same size and the spatial structure is clear; (2) In the UI interface sequence image dataset sample fast creation system of the present invention, if the user does not perform specific region selection, the TIFF cropping function can be directly called to crop a single TIFF image, input the number of rows and columns required for cropping, and the algorithm can perform fast and efficient cropping; in addition, on the basis of row and column cropping, an image clustering sub-module is conceived, which can more specifically perform personalized cropping on the sequence images, which is convenient and efficient; (3) In the UI interface sequence image dataset sample fast creation system of the present invention, through the comprehensive processing of sequence images, integrating multiple functions such as registration, cropping, and video conversion, the fast creation of a sequence image dataset is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which form a part of the present invention, are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 It is a schematic structural diagram of a system for quickly creating a sample of a UI interface sequence image dataset according to an embodiment of the present invention; Figure 2 It is a schematic principle diagram of an image clustering sub-module according to an embodiment of the present invention; Figure 3 It is a schematic flowchart of a method for quickly creating a sample of a UI interface sequence image dataset according to an embodiment of the present invention; Figure 4 It is a flowchart block diagram of a method for quickly creating a sample of a UI interface sequence image dataset according to an embodiment of the present invention. Detailed implementation manners

[0017] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0018] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0019] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0020] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0021] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] As Figure 1 shown, the UI interface sequence image dataset sample rapid creation system described in the embodiments of the present invention includes an interface interaction module, an image registration module, and a sequence image cropping module. Among them, the interface interaction module is used to open the remote sensing image sequence to be edited, and determine the cropping effect, cropping area, and cropping form of the remote sensing images in the remote sensing image sequence, and determine the output file path of the cropped image blocks. The image registration module registers the remote sensing image sequence, obtains and stores the registered remote sensing image sequence. The sequence image cropping module receives the registered remote sensing image sequence or the remote sensing image sequence, and crops the registered remote sensing image sequence or the remote sensing image sequence according to the cropping form, cropping area, and cropping effect, and stores the image blocks into the corresponding folders under the output file path according to the serial numbers.

[0023] Since the clarity of TIFF format images is the highest and the amount of information they contain is the largest, in the field of remote sensing images, the main image format used is the TIFF format. However, TIFF images cannot directly generate videos, while PNG images can retain most of the information of TIFF images. Therefore, TIFF images are converted into PNG images for video generation.

[0024] In some embodiments, the interface interaction module includes a row-column determination sub-module, a display sub-module, a cropping area determination sub-module, a registration determination sub-module, a cropping form determination sub-module, and an output path determination sub-module. Among them, the row-column determination sub-module is used to input row information and column information, and transfer the row information and column information to the sequence image cropping module. The display sub-module is used to display the cropping effect according to the row information and column information. The cropping area determination sub-module is used to determine the cropping area and transfer the cropping area to the sequence image cropping module. The registration determination sub-module is used to control the image registration module to register the remote sensing image sequence. The cropping form determination sub-module is used to determine the cropping form. The output path determination sub-module determines the output file path of the cropped remote sensing image sequence according to the input file path of the remote sensing image sequence. The cropping forms include TIFF cropping, PNG cropping, cropping by sequence, and cropping by selection.

[0025] In one embodiment, the output path determination sub-module names the output file path as the input file path + "_OUT" based on the input file path. For example, if the path of the opened remote sensing image sequence is "C: / Users / Lenovo / Desktop / A / boy.jpg", the output path determination sub-module determines that the input file path is "C: / Users / Lenovo / Desktop / A", and then automatically creates an output folder path named "C: / Users / Lenovo / Desktop / A_OUT". Under the output folder path, corresponding folders are created according to different cropping formats and the order of cropping to store the cropped images.

[0026] In one embodiment, after the user opens the remote sensing image sequence to be edited in the UI interface, the display sub-module displays one image in the remote sensing image sequence as a thumbnail on the canvas of the UI interface (generally, when opening a new remote sensing image sequence, the display sub-module displays the first image in the remote sensing image sequence in the UI interface). The output path determination sub-module extracts the folder path where the remote sensing image sequence is located as the input file path and displays the upper-level folder path of the current image on the UI interface. The user can adjust the size of the canvas according to actual needs. The user inputs the number of rows and columns into the row-column determination sub-module through the UI interface. The display sub-module shows the cropping effect according to the number of rows and columns, specifically by determining the cropping dotted lines according to the input number of rows and columns and arranging the cropping dotted lines in rows and columns on the current image to form the corresponding number of image blocks to show the cropping effect to the user. In addition, the display sub-module displays the row and column numbers (i.e., serial numbers) where each image block is located above it, in the form of "row number_column number". The serial numbers of the selected sub-regions are automatically recorded and provided for the user's secondary editing. Secondary editing includes sorting, correcting, clearing, and selectively deleting the serial numbers, etc. Other editing methods are also available. In this embodiment, selective deletion is taken as an example. When the user clicks on the grid in the third row and third column and the grid in the fifth row and sixth column in the UI interface according to the requirements, the corresponding area grids in the left canvas and the row and column numbers (i.e., 3_3 and 5_6) stored in the right list will be displayed. If there is a misselection, the corresponding row and column numbers can be simply deleted in the right list. For example, if the grid in the third row and third column (i.e., 3_3) is misselected, then directly delete the row and column number "3_3" in the right list. Thus, the secondary editing of the serial numbers of the sub-regions is completed.

[0027] The user determines the cropping area through the UI interface to the cropping area determination sub-module. Specifically, the method can be to click on the area to be cropped with the mouse, or to input the serial number of the area to be cropped through the keyboard. After receiving the information of the cropping area, the cropping area determination sub-module transfers the information of the cropping area to the serial image cropping module. The user determines the cropping form through the UI interface to the cropping form determination sub-module. Specifically, there are relevant buttons for the cropping form set on the UI interface, corresponding to four types: TIFF cropping, PNG cropping, cropping by sequence, and cropping by selection. Cropping by sequence includes TIFF sequence cropping and TIFF to PNG sequence cropping. Cropping by selection includes TIFF selection cropping and TIFF to PNG selection cropping. Therefore, in this embodiment, a total of 6 relevant buttons for the cropping form are set on the UI interface, namely "TIFF cropping", "PNG cropping", "cropping by sequence" (corresponding to TIFF sequence cropping), "TIFF image to PNG sequence sub-image" (corresponding to TIFF to PNG sequence cropping), "cropping by selection" (corresponding to TIFF selection cropping), and "TIFF image to PNG sequence sub-image by selection" (corresponding to TIFF to PNG selection cropping). The user determines the specific cropping form by clicking on these 6 buttons, and the cropping form determination sub-module sends the determined cropping form to the serial image cropping module. For example, when cropping is required in the form of cropping by selection, in addition to selecting the corresponding button, it is also necessary to click on the area divided by the corresponding row and column numbers on the visual image of the canvas with the mouse. For example, "5_5" indicates that the user has clicked on the small area in the fifth row and fifth column. If it is cropping by selection, the system provided by the present invention will only crop this small area.

[0028] The user determines whether to register the remote sensing image sequence through the UI interface to the registration determination sub-module. Specifically, there is a "TIFF registration" option set on the UI interface. If image registration is required, this option is selected, and the registration determination sub-module controls the image registration module to register the remote sensing image sequence. Otherwise, the remote sensing image sequence is not registered.

[0029] In other embodiments, the user can also determine whether to generate video data and whether to cluster the cropped image blocks through the UI interface. Specifically, there are "DIY cropping" and "convert to video" options set on the UI interface. When the user selects the "DIY cropping" option, the dataset generation module will receive the confirmation signal for image block clustering. After the serial image cropping module completes image cropping, the cropped image blocks are clustered to form a composite area. When the user selects the "convert to video" option, the dataset generation module will receive the confirmation signal for video generation. After the serial image cropping module completes image cropping, the cropped image blocks are converted into video data.

[0030] In some embodiments, the image registration module determines the registration file path for registering the remote sensing image sequence according to the input file path. The image registration module preferably uses the fast Fourier transform to register the remote sensing image sequence, obtains the registered remote sensing image sequence, and stores the registered remote sensing image sequence in the folder of the registration file path.

[0031] In a certain embodiment, the image registration module creates a registration folder under the input file path, and the registered images are stored in the registration folder in the order of their corresponding remote sensing image sequences. For the sake of rapid registration, the fast Fourier transform registration method is preferably used for registration, which specifically includes: First, copy the first image in the remote sensing image sequence to the registration folder; Then, perform FFT registration on the second and all subsequent images with the first image as the reference image; specifically: perform frequency domain conversion on the image to be registered (i.e., the second and all subsequent images) and the reference image (i.e., the first image), and then use the Fourier transform to obtain the spectral information of the two images (including amplitude information and phase information); calculate the phase difference between the two images, perform inverse Fourier transform on the phase difference, find the phase correlation peak, and infer the translation amount; based on the reference image, translate the image to be registered according to the translation amount to obtain the registered image; Finally, save the registered images according to the serial numbers to obtain the registered remote sensing image sequence.

[0032] In some embodiments, the sequence image cropping module includes a TIFF cropping sub-module, a PNG cropping sub-module, a sequence cropping sub-module, and a selected cropping sub-module: When the cropping form is TIFF cropping, the TIFF cropping sub-module calls the TIFF cropping function to crop the registered remote sensing image sequence or the images in the remote sensing image sequence according to the row information and column information, and stores the image blocks in the corresponding TIFF folders according to the images where they are located; stores the TIFF folders in the total folder under the output file path; When the cropping form is PNG cropping, the PNG cropping sub-module converts the registered remote sensing image sequence or the remote sensing image sequence into PNG format, and then calls the PNG cropping function to crop the PNG format registered remote sensing image sequence or the images in the remote sensing image sequence according to the row information and column information, and stores the cropped PNG format image blocks in the corresponding PNG folders according to the images where they are located; stores the PNG folders in the total folder; When the cropping form is cropping by sequence, the sequence cropping sub-module calls the TIFF cropping function to crop the registered remote sensing image sequence or the images in the remote sensing image sequence according to the row information and column information, gathers the image blocks with the same serial number into the same sequence folder, and stores the sequence folder in the total folder; When the cropping form is cropping by selection, the selected cropping sub-module calls the TIFF cropping function to crop the registered remote sensing image sequence or the images in the remote sensing image sequence according to the row information and column information, and centralizes the image blocks that meet the cropping area into the same area folder according to the serial number, and stores the area folder in the total folder.

[0033] In one embodiment, the TIFF cropping function is an independent underlying function that requires 5 hyperparameters as input, namely the number of rows and columns to be cropped, the input file path, the output file path, the registration variable, and the sequence information of the cropping area. Among them, the sequence information of the cropping area is defaulted to be empty, and the registration variable is defaulted to False. At this time, it means that the cropped image sequence is not a registered remote sensing image sequence. At this time, the input file path is the file path of the remote sensing image sequence; if it is necessary to crop the registered remote sensing image sequence, then the registration variable needs to be set to True. At this time, the input file path is the path where the registration folder is located. The TIFF cropping function will create a total folder named like "tiff1", "tiff2", etc. according to the sequence number under the output file path. Inside each total folder, a TIFF folder named after each picture name will be generated separately, such as "img1_name", "img2_name", etc. Since there is no information on the cropping area, the TIFF cropping function will directly crop the image according to the input number of rows and columns, and save the cropped image blocks to the corresponding folder according to the row and column numbers, named as "number of rows_number of columns", such as "1_1", "1_2", etc. The TIFF cropping function by default uses a multi-threaded strategy, which can prevent the update of the image interface.

[0034] In one embodiment, the PNG cropping function is also an independent underlying function that requires four hyperparameters as input, namely the number of rows and columns to be cropped, the input file path, the output file path, and the registration variable. The usage of the registration variable and the way the input file path changes when registration is required are the same as those of the TIFF cropping function, which will not be elaborated here. The difference between the PNG cropping function and the TIFF cropping function is that the PNG cropping function needs to first convert the TIFF format remote sensing image sequence or registered remote sensing image sequence into the PNG format and then crop the PNG format image sequence. The PNG cropping function will create a general folder named like "png1", "png2", etc. according to the sequence number under the output file path. Inside each general folder, a PNG folder named after each image name will be generated separately, such as "img1_name", "img2_name", etc. The PNG cropping function will crop the image according to the input number of rows and columns, and save the cropped image patches to the corresponding folders named according to the row and column numbers, named as "number of rows_number of columns", such as "1_1", "1_2", etc.

[0035] In some embodiments, when the cropping form is TIFF sequence cropping, the sequence cropping sub-module directly stores the image patches into the sequence folder; when the cropping form is TIFF to PNG sequence cropping, the sequence cropping sub-module converts the image patches into the PNG format and then stores the PNG format image patches into the sequence folder.

[0036] In one embodiment, the sequence cropping sub-module has seven inputs, namely the number of rows and columns to be cropped, the input file path, the output file path, the registration variable, the clustering variable, the sequence information of the cropping area, and the video generation variable. Among them, the sequence information of the cropping area is default empty, the clustering variable and the video variable are default False. The usage of the registration variable and the way the input file path changes when registration is required are the same as those of the TIFF cropping function, which will not be elaborated here. When clustering is required for the cropped image patches, the clustering variable changes from False to True, which means that after the sequence cropping sub-module completes image cropping, the image patches need to be input into the dataset generation module for clustering.

[0037] When the cropping form is TIFF sequence cropping, the sequence cropping sub-module will create a general folder named like "tiff_series1", "tiff_series2",... etc. according to the sequence number under the output file path. Inside each general folder, the sequence cropping sub-module will separately generate a TIFF folder named after each picture name, such as "img1_name", "img2_name",... etc. At the next level of the general folder tiff_seriesX (X = 1, 2, 3,...), a sequence folder named "series_img" is created, and the number of rows × the number of columns of sub-folders are created in the sequence folder series_img, and the sub-folders are named "number of rows_number of columns", such as "1_1", "1_2",... etc. Since there is no information on the cropping area, the sequence cropping sub-module calls the TIFF cropping function to directly crop the image according to the input number of rows and columns, and saves the cropped image blocks named according to the row and column numbers into the corresponding folders. The image blocks are named "number of rows_number of columns", such as "1_1", "1_2",... etc. The sequence cropping sub-module aggregates the image blocks with the same number of rows and columns in all TIFF folders imgX_name (X = 1, 2, 3,...) into the sub-folders with the corresponding number of rows and columns in the sequence folder series_img, and names the image blocks in the sub-folders according to the image sequence of the images they are in. For example, the sequence cropping sub-module aggregates the image block 1_1 in the TIFF folder img1_name, the image block 1_1 in the TIFF folder img2_name, the image block 1_1 in the TIFF folder img3_name,..., the image block 1_1 in the TIFF folder imgX_name into the sub-folder 1_1 of the sequence folder series_img. In the sub-folder 1_1 of the sequence folder series_img, the image block 1_1 from the TIFF folder img1_name is renamed to "1_1_1", the image block 1_1 from the TIFF folder img2_name is renamed to "1_1_2", the image block 1_1 from the TIFF folder img3_name is renamed to "1_1_3", and so on.

[0038] When the cropping form is TIFF to PNG sequence cropping, the cropping and storage methods are the same as those of the sequence cropping sub-module in the case of TIFF sequence cropping, which will not be elaborated here. The difference from the sequence cropping sub-module in the case of TIFF sequence cropping is that since the video can only be generated from PNG-formatted images, the video variable can only change from False to True when the cropping form is TIFF to PNG sequence cropping. Only at this time can the PNG-formatted image blocks be sent into the dataset generation module for video conversion.

[0039] In some embodiments, the selected clipping includes TIFF selected clipping and TIFF-to-PNG selected clipping; wherein, when the clipping form is TIFF selected clipping, the selected clipping sub-module directly stores the regional folder into the total folder; when the clipping form is TIFF-to-PNG selected clipping, the selected clipping sub-module converts the image blocks in the regional folder into the PNG format, and then stores the regional folder containing the image blocks in the PNG format into the total folder.

[0040] In one embodiment, the selected clipping sub-module also has 7 inputs, which are also the number of rows and columns of the clipping, the input file path, the output file path, the registration variable, the clustering variable, the sequence information of the clipping area, and the video generation variable respectively. The difference from the sequence clipping sub-module is that the sequence information of the clipping area of the selected clipping sub-module is the sequence information of the clipping area, and the format of the sequence information of the clipping area is a List list.

[0041] When the cropping format is TIFF and cropping is selected, the selected cropping sub-module will create a master folder named "tiff_series_selected", "tiff_series_selected2",... etc. in sequence under the output file path according to the serial number. Inside each master folder, the selected cropping sub-module will separately generate TIFF folders named after each picture name, such as "img1_name", "img2_name",... etc. At the next level of the master folder tiff_series_selectedX (X = 1, 2, 3,...), create a regional folder named "series_img", and create sub-files equal to the number of cropping regions in the regional folder series_img. The sub-folders are named "selected row xX_selected column yX" (X = 1, 2, 3,...), such as "selected row x1_selected column y1", "selected row x2_selected column y2",... etc. The selected cropping sub-module calls the TIFF cropping function to crop the image according to the sequence information of the cropping region and saves it in the corresponding folder according to the position of the image block in the original image, such as "selected row x1_selected column y1", "selected row x2_selected column y2",... "selected row xX_selected column yX". The image blocks that meet the cropping region are centralized in the same regional folder, and the regional folder is stored in the master folder. The image blocks in the sub-folders are named according to the image sequence of the image they are in. For example, the selected cropping sub-module will gather the selected image blocks "selected row x1_selected column y1" in the TIFF folder img1_name, "selected row x1_selected column y1" in the TIFF folder img2_name,... "selected row x1_selected column y1" in the TIFF folder imgX_name, all in the sub-folder "selected row x1_selected column y1" of the regional folder series_img. In the sub-folder "selected row x1_selected column y1" of the regional folder series_img, the image block "selected row x1_selected column y1" from the TIFF folder img1_name is renamed to "selected row x1_selected column y1_1", the image block "selected row x1_selected column y1" from the TIFF folder img2_name is renamed to "selected row x1_selected column y1_2", the image block "selected row x1_selected column y1" from the TIFF folder img3_name is renamed to "selected row x1_selected column y1_3", and so on.

[0042] When the cropping form is TIFF to PNG and cropping is selected, the cropping and storage methods are the same as those of the sequence cropping sub-module when the cropping form is TIFF and cropping is selected, which will not be elaborated here. The difference from the selected cropping sub-module when the cropping form is TIFF and cropping is selected is that since the video can only be generated from PNG-formatted images, the video variable can only change from False to True when the cropping form is TIFF to PNG and cropping is selected. Only at this time can the PNG-formatted image blocks be sent to the dataset generation module for video conversion.

[0043] In some embodiments, the UI interface sequence image dataset sample rapid creation system provided by the present invention further includes a dataset generation module. The dataset generation module can cluster image blocks as needed to form a synthesis region, and convert the PNG-formatted image blocks into a video to obtain video data. Specifically, the dataset generation module includes an image clustering sub-module and a video generation sub-module. Among them, when the cropping form is selected cropping or sequence cropping, the image clustering sub-module uses a clustering algorithm to splice the image blocks according to the serial numbers, and stores the corresponding synthesis region obtained after splicing in the region folder or sequence folder. Further, when the cropping region is not a rectangle, the image clustering sub-module supplements the cropping region with a black background to obtain a rectangle with the smallest area after filling.

[0044] In a certain embodiment, the principle of the clustering algorithm is as follows: The serial numbers of two image blocks (i 1 , j 1 ), (i 2 , j 2 ), when or , it means that the two image blocks are adjacent in the original image. The clustering algorithm is used to splice all adjacent sub-sequence images according to the serial numbers to obtain the clustering effect as shown in Figure 2 . However, if the cropping region cannot form a rectangle, it is default to be supplemented with a black background. For example, the area shown by the yellow block in Figure 2 will be supplemented with black, and finally the required synthesis region can be obtained more precisely. The image storage method of the image clustering sub-module is: Create a synthesis folder under the region folder created by the selected cropping sub-module or the sequence folder created by the sequence cropping sub-module, and create different sub-folders named similar to "(row number x1, column number y2)-(row number x2, column number y2)" in the synthesis folder. The folder contains the sequence sub-images of different region clusters. It should be noted that since there is no determined cropping region in the sequence cropping sub-module, in the synthesis folder created under the sequence folder, the naming method of the sub-files is directly (1,1)-(row number, column number).

[0045] The video generation sub-module converts the PNG-format image blocks or the PNG-format synthesis regions into a video according to the serial number, and stores the obtained video data. In one embodiment, the video generation sub-module directly uses the existing functions for converting video to images (for example, in the Python programming language, directly call the cv2.VideoWriter function in the Opencv library) to convert the PNG-format image blocks according to the configuration of fpi = 6. The video storage method of the video generation sub-module is as follows: for example, for the sub-folder 1_1 with PNG-format image blocks, or the sub-folder "selected row x1_selected column y1" with PNG-format image blocks, directly perform video conversion on the two folders, and correspondingly obtain video files named "1_1.mp4" and "selected row x1_selected column y1.mp4", and directly save them in the corresponding sub-files.

[0046] An embodiment of the present invention also provides a method for quickly creating a sample of a sequence image dataset. As Figure 3 and Figure 4 shown, according to the UI interface sequence image dataset sample quick creation system provided by the present invention, the method includes: S1: Input the file path where the remote sensing image sequence is located in the interface interaction module, and determine the cropping effect, cropping area, and cropping form in the interface interaction module; and determine whether to perform image matching in the interface interaction module; if image matching is performed, execute step S2, if not, execute step S3; S2: The image registration module collects the remote sensing image sequence from the file path obtained in step S1, and registers the remote sensing image sequence to obtain and store the registered remote sensing image sequence; S3: The sequence image cropping module determines the cropping form determined in step S1, and combines the cropping effect and cropping area to crop the remote sensing image sequence from step S1 or the registered remote sensing image sequence from step S2, and stores the cropped remote sensing image sequence in the corresponding folder according to the output file path.

[0047] In some embodiments, the method further includes: S4: Determine whether image clustering is required in the interface interaction module. If image clustering is required, execute step S5; otherwise, execute step S6; S5: The dataset generation module clusters the image blocks and stores the clustered synthesis regions in the corresponding folders; S6: Determine whether video conversion is required in the interface interaction module. If so, the dataset generation module converts the cropped image blocks into a video according to the serial number to obtain and store the video data.

[0048] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and no limitation is imposed herein.

[0049] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A UI interface sequence image dataset sample rapid creation system, characterized by: include: An interface interaction module is used to open a remote sensing image sequence to be edited, and determine the cropping effect, cropping area and cropping form of the remote sensing images in the remote sensing image sequence, and determine the output file path of the cropped image block; An image registration module registers the remote sensing image sequence to obtain and store the registered remote sensing image sequence; A sequence image cropping module receives the registered remote sensing image sequence or the remote sensing image sequence, and crops the registered remote sensing image sequence or the remote sensing image sequence according to the cropping form, the cropping area and the cropping effect; And the image blocks are stored in corresponding folders under the output file path according to the serial numbers.

2. The UI interface sequence image data set sample rapid creation system according to claim 1, characterized in that: The interface interaction module includes: A row and column determination submodule, used for inputting row information and column information required for the cropping area, and transmitting the row information and the column information to the sequence image cropping module; A display submodule, used for displaying the cropping effect according to the row information and the column information; A cropping region determination submodule, used for determining the cropping region and transferring the cropping region to the sequence image cropping module; A registration determination submodule, used for controlling the image registration module to register the remote sensing image sequence; A cropping form determination submodule, used to determine the cropping form; the cropping forms include TIFF cropping, PNG cropping, sequence cropping and selected cropping; The output path determination submodule determines the output file path of the cropped remote sensing image sequence according to the input file path of the remote sensing image sequence.

3. The UI interface sequence image data set sample rapid creation system according to claim 2, characterized in that: In the image registration module: Determining the registration file path of the registration remote sensing image sequence according to the input file path; The remote sensing image sequence is registered by using fast Fourier transform to obtain the registered remote sensing image sequence, and the registered remote sensing image sequence is stored in the folder of the registration file path.

4. The UI interface sequence image data set sample rapid creation system according to claim 2, characterized in that: The sequence image cropping module comprises: TIFF cropping submodule, when the cropping form is the TIFF cropping, the TIFF cropping submodule calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and the column information, and stores the image blocks in the corresponding TIFF folder according to the image they are in; and stores the TIFF folder in the total folder under the output file path; PNG cropping submodule, when the cropping form is the PNG cropping, the PNG cropping submodule converts the registered remote sensing image sequence or the remote sensing image sequence into PNG format, and then calls the PNG cropping function to crop the images of the registered remote sensing image sequence or the remote sensing image sequence in PNG format according to the row information and the column information, and stores the cropped PNG format image blocks in the corresponding PNG folder according to the image they are in; and stores the PNG folder in the general folder; A sequence cropping submodule, when the cropping form is the cropping by sequence, the sequence cropping submodule calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and the column information, collects the image blocks with the same sequence number into the same sequence folder, and stores the sequence folder into the total folder; A cropping submodule is selected. When the cropping form is the selected cropping, the selected cropping submodule calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and the column information, and collects the image blocks that meet the cropping area into the same area folder according to the serial number, and stores the area folder in the total folder.

5. The UI interface sequence image data set sample rapid creation system according to claim 4, characterized in that: The cropping by sequence includes TIFF sequence cropping and TIFF to PNG sequence cropping; wherein, when the cropping form is the TIFF sequence cropping, the sequence cropping submodule directly stores the image block in the sequence folder; when the cropping form is TIFF to PNG sequence cropping, the sequence cropping submodule converts the image block into PNG format, and then stores the image block in PNG format in the sequence folder.

6. The UI interface sequence image data set sample rapid creation system according to claim 4, characterized in that: The selected cropping includes TIFF selected cropping and TIFF to PNG selected cropping; wherein, when the cropping form is the TIFF selected cropping, the selected cropping submodule directly stores the regional folder in the total folder; when the cropping form is the TIFF to PNG selected cropping, the selected cropping submodule converts the image blocks in the regional folder into PNG format, and then stores the regional folder containing the image blocks in PNG format in the total folder.

7. The UI interface sequence image data set sample rapid creation system according to claim 4, characterized in that: The system further includes a data set generation module, wherein the data set generation module includes: An image clustering submodule, when the cropping form is the cropping by selection or the cropping by sequence, the image clustering submodule uses a clustering algorithm to splice the image blocks according to the sequence numbers; and stores the synthesized regions obtained after splicing in the region folder or the sequence folder accordingly; The video generation submodule converts the image blocks in PNG format or the synthesis area in PNG format into video according to the serial number, and stores the obtained video data.

8. The system for quickly creating samples of UI interface sequence image datasets according to claim 7, characterized in that: In the image clustering submodule, when the cropping area is not a rectangle, the cropping area is supplemented with a black background so that the cropping area after filling is a rectangle with the smallest area.

9. A method for quickly creating a sample sequence image dataset, according to the system for quickly creating a sample sequence image dataset on a UI interface according to any one of claims 1 to 8, characterized in that: include: S1: inputting the file path of the remote sensing image sequence in the interface interaction module, and determining the cropping effect, cropping area and cropping form in the interface interaction module; and determining whether to perform image matching in the interface interaction module; If image matching is performed, step S2 is executed; if not, step S3 is executed; S2: the image registration module collects the remote sensing image sequence from the file path obtained in step S1, and registers the remote sensing image sequence to obtain and store the registered remote sensing image sequence; S3: The sequence image cropping module determines the cropping form determined in step S1, and crops the remote sensing image sequence from step S1 or the registered remote sensing image sequence from step S2 in combination with the cropping effect and the cropping area, and stores the cropped remote sensing image sequence in the corresponding folder according to the output file path.

10. The method for quickly creating a sequence image dataset sample according to claim 9, characterized in that: The creation system further includes a data set generation module; the rapid creation method further includes: S4: Determine in the interface interaction module whether image clustering is required, if image clustering is required, execute step S5; otherwise, execute step S6; S5: The data set generation module clusters the image blocks and stores the clustered synthesis areas in a corresponding folder; S6: Determine in the interface interaction module whether video conversion is required. If required, the data set generation module will crop the image blocks and perform video conversion according to the sequence numbers to obtain and store the video data.

Images