System and method for quickly creating samples of UI interface sequence image datasets

Abstract

The present invention relates to the field of image processing technology, and in particular to a system and method for quickly creating sample sequence image datasets using a UI interface. The system comprises: an interface interaction module for opening a remote sensing image sequence and determining the cropping effect, cropping area, and cropping form of the remote sensing images in the remote sensing image sequence; an image registration module for registering the remote sensing image sequence; and a sequence image cropping module for receiving two remote sensing image sequences and cropping them according to the cropping form, cropping area, and cropping effect. The present invention uses an interface interactive method to segment images according to input row and column information, and incorporates processing functions such as FFT-based registration and image video generation, enabling faster and better acquisition of the datasets required for deep learning.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a system and method for quickly creating samples of a UI interface sequence image data set. Background Art

[0002] The creation of datasets is crucial in the field of deep learning. Without a large amount of data, it is impossible to train a high-performance model. However, in the field of sequential image processing, creating datasets is very difficult and often requires tedious manual processing. The manual processing stage specifically includes:

[0003] ① Image registration is required;

[0004] ② Visualize the image to determine whether there is an area of ​​interest or target in the image;

[0005] ③ Perform uniform cropping according to the pixel position of the region of interest or target;

[0006] ④Generate the required video.

[0007] Current interface-based remote sensing image cropping software utilizes specialized remote sensing imaging software such as ENVI, ArcGIS, and PCI Geomatica. However, these software applications all rely on loading remote sensing images and then segmenting them one by one based on selected regions, pixel locations, and thresholds. These applications lack the ability to crop sequential images and lack the necessary preprocessing for sequential imagery. Consequently, these software applications are extremely cumbersome for sequential image selection and require significant manual effort. Summary of the Invention

[0008] In view of this, the present invention aims to provide a system and method for quickly creating samples of sequential image datasets on a UI interface. It adopts an interactive interface method to segment images according to the input row and column information, and adds processing functions such as FFT alignment and image video generation, which can obtain the dataset required for deep learning faster and better.

[0009] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0010] A system for quickly creating samples of UI interface sequence image datasets, including:

[0011] The interface interaction module is used to open the remote sensing image sequence to be edited, and determine the cropping effect, cropping area and cropping form of the remote sensing image in the remote sensing image sequence, as well as determine the output file path of the cropped image block;

[0012] Image registration module, registers the remote sensing image sequence, obtains and stores the registered remote sensing image sequence;

[0013] The sequence image cropping module receives the registered remote sensing image sequence or the remote sensing image sequence, and crops the registered remote sensing image sequence or the remote sensing image sequence according to the cropping form, cropping area and cropping effect; and the image blocks are stored in the corresponding folder under the output file path according to the serial number.

[0014] Furthermore, the interface interaction module includes:

[0015] The row and column determination submodule is used to input row information and column information and pass the row information and column information to the sequence image cropping module;

[0016] The display submodule is used to display the cropping effect according to the row information and column information;

[0017] The cropping region determination submodule is used to determine the cropping region and pass the cropping region to the sequence image cropping module;

[0018] The registration determination submodule is used to control the image registration module to register the remote sensing image sequence;

[0019] The cropping mode determination submodule is used to determine the cropping mode; the cropping modes include TIFF cropping, PNG cropping, cropping by sequence, and cropping by selection;

[0020] The output path determination submodule determines the output file path of the cropped remote sensing image sequence according to the input file path of the remote sensing image sequence.

[0021] Furthermore, in the image registration module:

[0022] According to the input file path, determine the registration file path of the remote sensing image sequence;

[0023] Fast Fourier transform is used to register the remote sensing image sequence to obtain the registered remote sensing image sequence, and the registered remote sensing image sequence is stored in the folder of the registration file path.

[0024] Furthermore, the sequence image cropping module includes:

[0025] TIFF cropping submodule: When the cropping mode is TIFF cropping, the TIFF cropping submodule calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and column information, and stores the image blocks in the corresponding TIFF folder according to the image they are in; and stores the TIFF folder in the total folder under the output file path;

[0026] PNG cropping submodule: When the cropping mode is PNG cropping, the PNG cropping submodule converts the registration remote sensing image sequence or remote sensing image sequence into PNG format, and then calls the PNG cropping function to crop the images of the registration remote sensing image sequence or remote sensing image sequence in PNG format according to the row information and column information, and stores the cropped PNG format image blocks in the corresponding PNG folder according to the image they are in; and stores the PNG folder in the main folder;

[0027] Sequence cropping submodule: When the cropping mode is sequence cropping, the sequence cropping submodule calls the TIFF cropping function to crop the images in the registration remote sensing image sequence or remote sensing image sequence according to the row information and column information, collects the image blocks with the same sequence number into the same sequence folder, and stores the sequence folder into the main folder;

[0028] Select the cropping submodule. When the cropping mode is selected, select the cropping submodule and call the TIFF cropping function to crop the images in the registered remote sensing image sequence or remote sensing image sequence according to the row information and column information. The image blocks that meet the cropping area are concentrated in the same regional folder according to the serial number, and the regional folder is stored in the total folder.

[0029] Furthermore, cropping by sequence includes TIFF sequence cropping and TIFF to PNG sequence cropping; when the cropping form is TIFF sequence cropping, the sequence cropping submodule directly stores the image blocks in the sequence folder; when the cropping form is TIFF to PNG sequence cropping, the sequence cropping submodule converts the image blocks into PNG format, and then stores the PNG format image blocks in the sequence folder.

[0030] Furthermore, selected cropping includes TIFF selected cropping and TIFF to PNG selected cropping; wherein, when the cropping form is TIFF selected cropping, the selected cropping submodule directly stores the regional folder in the total folder; when the cropping form is TIFF to PNG selected cropping, the selected cropping submodule converts the image blocks in the regional folder into PNG format, and then stores the regional folder containing the image blocks in PNG format in the total folder.

[0031] Furthermore, the system also includes a data set generation module, which includes:

[0032] Image clustering submodule: When the cropping mode is selected cropping or sequence cropping, the image clustering submodule uses a clustering algorithm to splice image blocks according to sequence numbers; the resulting composite regions are stored in the region folder or sequence folder.

[0033] The video generation submodule converts the image blocks in PNG format or the synthesis area in PNG format into video according to the serial number, and stores the obtained video data.

[0034] Furthermore, in the image clustering submodule, when the cropping area is not a rectangle, the cropping area is supplemented with a black background so that the cropping area after filling is a rectangle with the smallest area.

[0035] A method for quickly creating a sample sequence image dataset, according to a system for quickly creating a sample sequence image dataset on a UI interface provided by the present invention, comprises:

[0036] S1: Input the file path of the remote sensing image sequence in the interface interaction module, and determine the cropping effect, cropping area and cropping form in the interface interaction module; and determine whether to perform image matching in the interface interaction module; if image matching is performed, execute step S2; if not, execute step S3;

[0037] S2: The image registration module collects the remote sensing image sequence from the file path obtained in step S1, registers the remote sensing image sequence, and obtains and stores the registered remote sensing image sequence;

[0038] S3: The sequence image cropping module determines the cropping form determined in step S1, and crops the remote sensing image sequence from step S1 or the registered remote sensing image sequence from step S2 based on the cropping effect and cropping area, and stores the cropped remote sensing image sequence in the corresponding folder according to the output file path.

[0039] Furthermore, the creation system also includes a data set generation module; the rapid creation method also includes:

[0040] S4: Determine in the interface interaction module whether image clustering is required. If image clustering is required, execute step S5; otherwise, execute step S6;

[0041] S5: The dataset generation module clusters the image blocks and stores the clustered synthetic regions in the corresponding folders;

[0042] S6: Determine in the interface interaction module whether video conversion is required. If required, the data set generation module will crop the image blocks and perform video conversion according to the sequence numbers to obtain and store the video data.

[0043] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0044] (1) The present invention creates a UI interface sequence image dataset sample rapid creation system, which adopts an image cropping method based on the number of rows and columns under interface interaction. The cropping is convenient and efficient, and there is no need to manually select the area of ​​interest. At the same time, the cropped images are of uniform size and have a clear spatial structure.

[0045] (2) In the UI interface sequence image dataset sample rapid creation system created by the present invention, if the user does not select a specific area, the TIFF cropping function can be directly called to crop a single TIFF image. The number of rows and columns required for cropping can be input, and the algorithm can perform fast and efficient cropping. In addition, based on the row and column cropping, an image clustering submodule is conceived, which can perform personalized cropping of sequence images in a more targeted manner, which is convenient and efficient.

[0046] (3) The present invention creates a system for quickly creating samples of sequential image datasets on a UI interface, which integrates multiple functions such as sequential image processing, registration, cropping, and video conversion to achieve the creation of rapid sequential image datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 This is a schematic diagram of the structure of the system for quickly creating a sample of a UI interface sequence image dataset according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the principle of the image clustering submodule described in an embodiment of the present invention;

[0050] Figure 3 A flowchart of a method for quickly creating a sample of a UI interface sequence image dataset according to an embodiment of the present invention;

[0051] Figure 4 This is a flowchart of a method for quickly creating a sample of a UI interface sequence image dataset according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0053] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0054] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0055] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0056] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0057] like Figure 1 As shown, the UI interface sequence image data set sample rapid creation system described in the embodiment of the present invention includes an interface interaction module, an image registration module and a sequence image cropping module. Among them, the interface interaction module is used to open the remote sensing image sequence that needs to be edited, and determine the cropping effect, cropping area and cropping form of the remote sensing image in the remote sensing image sequence, and determine the output file path of the cropped image block. The image registration module aligns the remote sensing image sequence, obtains and stores the aligned remote sensing image sequence. The sequence image cropping module receives the aligned remote sensing image sequence or the remote sensing image sequence, and crops the aligned remote sensing image sequence or the remote sensing image sequence according to the cropping form, cropping area and cropping effect, and stores the image blocks in the corresponding folder under the output file path according to the serial number.

[0058] Since TIFF format images have the highest clarity and contain the largest amount of information, the image format used in the field of remote sensing images is mainly TIFF format. However, TIFF images cannot directly generate videos, while PNG images can retain most of the information of TIFF images, so TIFF images are converted to PNG images for video generation.

[0059] In some embodiments, the interface interaction module includes a row and column determination submodule, a display submodule, a cropping area determination submodule, a registration determination submodule, a cropping form determination submodule, and an output path determination submodule. Among them, the row and column determination submodule is used to input row information and column information, and pass the row information and column information to the sequence image cropping module. The display submodule is used to display the cropping effect based on the row information and column information. The cropping area determination submodule is used to determine the cropping area and pass the cropping area to the sequence image cropping module. The registration determination submodule is used to control the image registration module to align the remote sensing image sequence. The cropping form determination submodule is used to determine the cropping form. The output path determination submodule determines the output file path of the cropped remote sensing image sequence based on the input file path of the remote sensing image sequence. The cropping forms include TIFF cropping, PNG cropping, cropping by sequence, and cropping by selection.

[0060] In a certain embodiment, the output path determination submodule will name the output file path as the input file path + "_OUT" based on the input file path. For example, if the path of the opened remote sensing image sequence is "C: / Users / Lenovo / Desktop / A / boy.jpg", the output path determination submodule confirms that the input file path is "C: / Users / Lenovo / Desktop / A", and then automatically creates an output folder path named "C: / Users / Lenovo / Desktop / A_OUT". Under the output folder path, corresponding folders are created according to different cropping formats and the order of cropping to store the cropped images.

[0061] In one embodiment, after a user opens a remote sensing image sequence to be edited in the UI, the display submodule displays one image from the sequence as a thumbnail on the UI canvas (typically, when opening a new remote sensing image sequence, the display submodule displays the first image in the sequence). The output path determination submodule extracts the folder path containing the remote sensing image sequence as the input file path and displays the parent folder path of the current image on the UI. The user can adjust the canvas size as needed. The user enters the number of rows and columns into the row and column determination submodule through the UI. The display submodule then displays the cropping effect based on these numbers. Specifically, the display submodule determines dotted cropping lines based on the input row and column numbers and arranges them on the current image, forming a corresponding number of image blocks, to demonstrate the cropping effect to the user. Furthermore, the display submodule displays the row and column number (i.e., sequence number) above each image block in the format "row number_column number." The sequence number of the selected subarea is automatically recorded and available for secondary editing by the user. Secondary editing includes sorting, correcting, clearing, and other editing methods such as selective deletion of serial numbers. This embodiment takes selective deletion as an example. When the user uses the mouse to click on the third row and third column grid and the fifth row and sixth column grid in the UI interface as required, the corresponding area grids in the left canvas and the right list will display the row and column numbers of the clicks (i.e., 3_3 and 5_6). If an accidental selection occurs, the corresponding row and column numbers can be simply deleted from the right list. For example, if the third row and third column grid (i.e., 3_3) is accidentally selected, the row and column number "3_3" can be directly deleted from the right list. At this point, the secondary editing of the serial number of the sub-area is completed.

[0062] The user determines the cropping area in the cropping area determination submodule through the UI interface. The specific method is to click the area to be cropped with the mouse, or enter the serial number of the area to be cropped through the keyboard. After receiving the information of the cropping area, the cropping area determination submodule passes the information of the cropping area to the sequence image cropping module. The user determines the cropping form in the cropping form determination submodule through the UI interface. Specifically, the UI interface is provided with relevant buttons for the cropping form, corresponding to four categories: TIFF cropping, PNG cropping, cropping by sequence, and cropping by selection. Cropping by sequence includes TIFF sequence cropping and TIFF to PNG sequence cropping. Cropping by selection includes TIFF selected cropping and TIFF to PNG selected cropping. Therefore, in this embodiment, a total of 6 buttons for cropping forms are provided on the UI interface, namely "TIFF cropping", "PNG cropping", "cropping by sequence" (corresponding to TIFF sequence cropping), "TIFF image to PNG sequence sub-image" (corresponding to TIFF to PNG sequence cropping), "cropping by selection" (corresponding to TIFF selected cropping), and "cropping by selected TIFF image to PNG sequence sub-image" (corresponding to TIFF to PNG selected cropping). The user determines the specific cropping form by clicking these 6 buttons, and the cropping form determination submodule sends the determined cropping form to the sequence image cropping module. For example, when cropping is required in the form of selected cropping, in addition to selecting the corresponding button, you also need to use the mouse to click on the area divided by row and column numbers on the visual image of the canvas, such as "5_5", which means that the user clicked on the small area of ​​the fifth row and fifth column. If cropping is performed in the form of selected cropping, the system provided by the present invention will only crop this small area.

[0063] The user determines whether to register the remote sensing image sequence to the registration determination submodule through the UI interface. Specifically, a "TIFF registration" option is set on the UI interface. If image registration is required, select this option, and the registration determination submodule controls the image registration module to register the remote sensing image sequence. Otherwise, no image registration is performed on the remote sensing image sequence.

[0064] In other embodiments, the user may also determine through the UI interface whether to generate video data and whether to cluster the cropped image blocks. Specifically, the UI interface is provided with a "DIY cropping" option and a "convert to video" option. When the user selects the "DIY cropping" option, the data set generation module will receive a confirmation signal for image block clustering. After the sequence image cropping module completes image cropping, the cropped image blocks are clustered to form a synthetic area. When the user selects the "convert to video" option, the data set generation module will receive a confirmation signal for video generation. After the sequence image cropping module completes image cropping, the cropped image blocks are converted into video to form video data.

[0065] In some embodiments, the image registration module determines a registration file path for the remote sensing image sequence based on the input file path. The image registration module preferably uses a fast Fourier transform to register the remote sensing image sequence to obtain a registered remote sensing image sequence, and stores the registered remote sensing image sequence in a folder in the registration file path.

[0066] In one embodiment, the image registration module creates a registration folder under the input file path, and the registered images are stored in the registration folder according to the order in which they are located in the remote sensing image sequence. For the sake of rapid registration, the fast Fourier transform registration method is preferably used for registration, which specifically includes:

[0067] First, copy the first image in the remote sensing image sequence into the registration folder;

[0068] Then, the second and all subsequent images are FFT-aligned with the first image as the reference image. Specifically, the image to be aligned (i.e., the second and all subsequent images) and the reference image (i.e., the first image) are converted to the frequency domain, and then the frequency spectrum information (including amplitude and phase information) of the two images is obtained using Fourier transform. The phase difference between the two images is calculated and the phase difference is inversely Fourier transformed to find the phase correlation peak and infer the translation amount. Based on the reference image, the image to be aligned is translated according to the translation amount to obtain the aligned image.

[0069] Finally, the registered images are saved according to the serial number to obtain the registered remote sensing image sequence.

[0070] In some embodiments, the sequence image cropping module includes a TIFF cropping submodule, a PNG cropping submodule, a sequence cropping submodule, and a selected cropping submodule:

[0071] When the cropping mode is TIFF cropping, the TIFF cropping submodule calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and column information, and stores the image blocks in the corresponding TIFF folder according to the image they are in; and stores the TIFF folder in the total folder under the output file path;

[0072] When the cropping mode is PNG cropping, the PNG cropping submodule converts the registered remote sensing image sequence or remote sensing image sequence into PNG format, and then calls the PNG cropping function to crop the images of the registered remote sensing image sequence or remote sensing image sequence in PNG format according to the row information and column information, and stores the cropped PNG format image blocks in the corresponding PNG folder according to the image they are in; and stores the PNG folder in the main folder;

[0073] When the cropping mode is sequence cropping, the sequence cropping submodule calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or remote sensing image sequence according to the row information and column information, collects the image blocks with the same sequence number into the same sequence folder, and stores the sequence folder into the main folder;

[0074] When the cropping mode is selected, the cropping submodule is selected and calls the TIFF cropping function to crop the images in the registered remote sensing image sequence or the remote sensing image sequence according to the row information and column information, and the image blocks that meet the cropping area are concentrated into the same regional folder according to the serial number, and the regional folder is stored in the total folder.

[0075] In one embodiment, the TIFF cropping function is an independent, underlying function that requires five hyperparameters: the number of rows and columns to be cropped, the input file path, the output file path, the registration variable, and the sequence information of the cropped area. The sequence information of the cropped area defaults to empty, and the registration variable defaults to False, meaning that the cropped image sequence is not a registered remote sensing image sequence. In this case, the input file path is the file path of the remote sensing image sequence. If the registered remote sensing image sequence needs to be cropped, the registration variable needs to be set to True, and the input file path is the path to the registration folder. The TIFF cropping function creates sequentially numbered folders such as "tiff1," "tiff2," etc. within the output file path. Within each folder, a separate TIFF folder is generated, named after each image, such as "img1_name," "img2_name," etc. Because there's no cropping region information, the TIFF crop function crops the image directly according to the input row and column numbers. The resulting image blocks are saved in the corresponding folders, named "row_column" (e.g., "1_1," "1_2," etc.). The TIFF crop function uses a multi-threaded strategy by default, which prevents image interface updates.

[0076] In a certain embodiment, the PNG cropping function is also an independent underlying function that requires the input of four hyperparameters, namely the number of rows and columns to be cropped, the input file path, the output file path, and the alignment variable. The way of using the alignment variable and the way of changing the input file path when alignment is required are consistent with the TIFF cropping function and will not be repeated here. The difference between the PNG cropping function and the TIFF cropping function is that the PNG cropping function needs to convert the remote sensing image sequence in TIFF format or the aligned remote sensing image sequence into PNG format first, and then crop the image sequence in PNG format. The PNG cropping function will create a general folder numbered according to the sequence, such as "png1", "png2", etc. under the output file path. A PNG folder named after each picture name will be generated separately in each general folder, such as "img1_name", "img2_name", etc. The PNG cropping function crops the image according to the input number of rows and columns, and saves the cropped image blocks into the corresponding folders with names based on the row and column numbers, such as "row number_column number", "1_1", "1_2", etc.

[0077] In some embodiments, when the cropping form is TIFF sequence cropping, the sequence cropping submodule directly stores the image blocks in the sequence folder; when the cropping form is TIFF to PNG sequence cropping, the sequence cropping submodule converts the image blocks into PNG format, and then stores the image blocks in PNG format in the sequence folder.

[0078] In one embodiment, the sequence cropping submodule has 7 inputs, namely the number of rows and columns to be cropped, the input file path, the output file path, the alignment variable, the clustering variable, the sequence information of the cropped area, and the generated video variable. Among them, the sequence information of the cropped area defaults to empty, the clustering variable and the video variable default to False, the use of the alignment variable, and the change of the input file path when alignment is required are consistent with the TIFF cropping function and will not be repeated here. When the cropped image blocks need to be clustered, the clustering variable is changed from False to True, which means that after the sequence cropping submodule completes the image cropping, the image blocks need to be input into the data set generation module for clustering.

[0079] When cropping to TIFF sequence format, the Sequence Cropping submodule creates sequentially numbered folders in the output file path, such as "tiff_series1," "tiff_series2," and so on. Within each folder, the Sequence Cropping submodule generates separate TIFF folders named after each image, such as "img1_name," "img2_name," and so on. Within the tiff_seriesX folder (where X = 1, 2, 3, ...), a sequence folder named "series_img" is created. Within the sequence folder, there are also subfolders named "rows x columns," with the number of rows x number of columns, such as "1_1," "1_2," and so on. Because there's no information about the cropping region, the sequence cropping submodule calls the TIFF cropping function to directly crop the image according to the input row and column numbers. The resulting image blocks are named "row_column" according to the row and column numbers and saved in corresponding folders. The blocks are named "row_number_of_columns," such as "1_1," "1_2," and so on. The sequence cropping submodule aggregates all image blocks with the same row and column numbers in the TIFF folder imgX_name (where X = 1, 2, 3, ...) into subfolders of the sequence folder series_img with the corresponding row and column numbers. The blocks in each subfolder are named according to the image sequence of the image they contain. For example, the sequence cropping submodule aggregates image block 1_1 in the TIFF folder img1_name, image block 1_1 in the TIFF folder img2_name, image block 1_1 in the TIFF folder img3_name, ..., and image block 1_1 in the TIFF folder imgX_name into subfolder 1_1 of the sequence folder series_img. In the subfolder 1_1 of the sequence folder series_img, the image block 1_1 from the TIFF folder img1_name is renamed to "1_1_1", the image block 1_1 from the TIFF folder img2_name is renamed to "1_1_2", the image block 1_1 from the TIFF folder img3_name is renamed to "1_1_3", and so on.

[0080] When the cropping format is TIFF-to-PNG sequence cropping, the cropping and storage methods are the same as those used by the Sequence Cropping submodule when the cropping format is TIFF sequence cropping, and will not be further explained here. The difference from the Sequence Cropping submodule when the cropping format is TIFF sequence cropping is that, because video can only be generated from PNG images, only when the cropping format is TIFF-to-PNG sequence cropping can the video variable change from False to True, and only then can the PNG image blocks be fed into the Dataset Generation module for video conversion.

[0081] In some embodiments, selected cropping includes TIFF selected cropping and TIFF to PNG selected cropping; wherein, when the cropping form is TIFF selected cropping, the selected cropping submodule directly stores the regional folder in the total folder; when the cropping form is TIFF to PNG selected cropping, the selected cropping submodule converts the image blocks in the regional folder into PNG format, and then stores the regional folder containing the image blocks in PNG format into the total folder.

[0082] In one embodiment, the Selected Cropping submodule also takes seven inputs: the number of rows and columns to be cropped, the input file path, the output file path, the registration variable, the clustering variable, the sequence information of the cropped region, and the generated video variable. Unlike the Sequential Cropping submodule, the Selected Cropping submodule provides the sequence information of the cropped region in a list format.

[0083] When the cropping mode is TIFF Selected Cropping, the Selected Cropping submodule creates sequentially numbered master folders in the output file path, such as "tiff_series_selected," "tiff_series_selected2," and so on. Within each master folder, the Selected Cropping submodule generates separate TIFF folders named after each image, such as "img1_name," "img2_name," and so on. Within the master folder tiff_series_selectedX (where X=1, 2, 3...), a region folder named "series_img" is created. Within the region folder series_img, subfolders are created, each containing a number of cropped regions, named "Selected Row xX_Selected Column yX" (where X=1, 2, 3...), such as "Selected Row x1_Selected Column y1," "Selected Row x2_Selected Column y2," and so on. The selected cropping submodule calls the TIFF cropping function to crop the image according to the sequence information of the cropping region. It then names the image blocks according to their position in the original image and saves them in corresponding folders, such as "selected row x1_selected column y1," "selected row x2_selected column y2," ..., "selected row xX_selected column yX." Image blocks that match the cropping region are grouped together in the same region folder, which is then stored in a main folder. The image blocks in each subfolder are named according to the image sequence of the image they contain. For example, the selected cropping submodule groups the selected image blocks "selected row x1_selected column y1" in the TIFF folder img1_name, "selected row x1_selected column y1" in the TIFF folder img2_name, ..., and "selected row x1_selected column y1" in the TIFF folder imgX_name into the subfolder "selected row x1_selected column y1" of the region folder series_img. In the subfolder "Selected row x1_Selected column y1" of the area folder series_img, the image block "Selected row x1_Selected column y1" from the TIFF folder img1_name is renamed to "Selected row x1_Selected column y1_1", the image block "Selected row x1_Selected column y1" from the TIFF folder img2_name is renamed to "Selected row x1_Selected column y1_2", the image block "Selected row x1_Selected column y1" from the TIFF folder img3_name is renamed to "Selected row x1_Selected column y1_3", and so on.

[0084] When Cropping is selected for TIFF to PNG conversion, the cropping and storage methods are the same as when Cropping is selected for TIFF in the Sequence Cropping submodule, and are not further described here. The difference from when Cropping is selected for TIFF in the Selected Cropping submodule is that, because video can only be generated from PNG images, only when Cropping is selected for TIFF to PNG conversion does the Video variable change from False to True, allowing the PNG image blocks to be fed into the Dataset Generation module for video conversion.

[0085] In some embodiments, the UI interface sequence image dataset sample rapid creation system provided by the present invention also includes a dataset generation module. The dataset generation module can cluster the image blocks to form a synthetic area as needed, and convert the image blocks in PNG format into videos to obtain video data. Specifically, the dataset generation module includes an image clustering submodule and a video generation submodule. Among them, when the cropping form is selected cropping or sequence cropping, the image clustering submodule uses a clustering algorithm to splice the image blocks according to the serial number, and stores the synthetic area obtained after splicing in the region folder or the sequence folder. Furthermore, when the cropping area is not a rectangle, the image clustering submodule supplements the cropping area with a black background so that the filled cropping area is a rectangle with the smallest area.

[0086] In one embodiment, the principle of the clustering algorithm is: the serial numbers of two image blocks (i1, j1), (i2, j2), when or When , it means that the two image blocks are adjacent in the original image. Use the clustering algorithm to splice all adjacent subsequence images according to the sequence number, and get the following Figure 2 However, if the cropped area cannot form a rectangle, it will be supplemented with a black background by default, as shown in Figure 2 The areas shown in yellow blocks are supplemented with black, ultimately resulting in a more refined synthesis of the desired regions. The image clustering submodule stores images by creating a synthesis folder within the region folder created by the selected cropping submodule or the sequence folder created by the sequence cropping submodule. Within the synthesis folder, subfolders named similar to "(rows x1, columns y2) - (rows x2, columns y2)" are created, each containing sequence subimages from different region clusters. Note that since no cropping regions are defined in the sequence cropping submodule, the subfiles in the synthesis folder created within the sequence folder are named directly as (1, 1) - (rows, columns).

[0087] The video generation submodule converts PNG-formatted image blocks or PNG-formatted composite areas into video according to the serial number, and stores the resulting video data. In one embodiment, the video generation submodule directly uses an existing video-to-image conversion function (e.g., in the Python programming language, directly calling the cv2.VideoWriter function in the OpenCV library) to convert PNG-formatted image blocks into video according to an fpi=6 configuration. The video generation submodule stores videos in the following manner: for example, for a subfolder 1_1 containing PNG-formatted image blocks, or a subfolder "selected row x1_selected column y1" containing PNG-formatted image blocks, the two folders are directly converted into videos, resulting in corresponding video files named "1_1.mp4" and "selected row x1_selected column y1.mp4," respectively, and the videos are directly stored in the corresponding subfolders.

[0088] The embodiment of the present invention also provides a method for quickly creating a sample of a sequence image dataset. Figure 3 and Figure 4 As shown, according to the system for quickly creating a sample of a UI interface sequence image dataset provided by the present invention, the method includes:

[0089] S1: Input the file path of the remote sensing image sequence in the interface interaction module, and determine the cropping effect, cropping area and cropping form in the interface interaction module; and determine whether to perform image matching in the interface interaction module; if image matching is performed, execute step S2; if not, execute step S3;

[0090] S2: The image registration module collects the remote sensing image sequence from the file path obtained in step S1, registers the remote sensing image sequence, and obtains and stores the registered remote sensing image sequence;

[0091] S3: The sequence image cropping module determines the cropping form determined in step S1, and crops the remote sensing image sequence from step S1 or the registered remote sensing image sequence from step S2 based on the cropping effect and cropping area, and stores the cropped remote sensing image sequence in the corresponding folder according to the output file path.

[0092] In some embodiments, the method further comprises:

[0093] S4: Determine in the interface interaction module whether image clustering is required. If image clustering is required, execute step S5; otherwise, execute step S6;

[0094] S5: The dataset generation module clusters the image blocks and stores the clustered synthetic regions in the corresponding folders;

[0095] S6: Determine in the interface interaction module whether video conversion is required. If required, the data set generation module will crop the image blocks and perform video conversion according to the sequence numbers to obtain and store the video data.

[0096] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0097] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A system for quickly creating samples of a UI interface sequence image dataset, characterized in that: include: An interface interaction module is used to open a remote sensing image sequence to be edited, and determine the cropping effect, cropping area and cropping form of the remote sensing image in the remote sensing image sequence, as well as determine the output file path of the cropped image block; The cropping modes include TIFF cropping, PNG cropping, cropping by sequence and cropping by selection; An image registration module registers the remote sensing image sequence to obtain and store the registered remote sensing image sequence; a sequence image cropping module, receiving the registered remote sensing image sequence or the remote sensing image sequence, and cropping the registered remote sensing image sequence or the remote sensing image sequence according to the cropping form, the cropping area and the cropping effect; and storing the image blocks in corresponding folders under the output file path according to the serial numbers; The sequence image cropping module includes: A TIFF cropping submodule, when the cropping form is the TIFF cropping, the TIFF cropping submodule calls a TIFF cropping function to crop the registered remote sensing image sequence or the image in the remote sensing image sequence according to the row information and column information, and stores the image blocks in corresponding TIFF folders according to the images in which they are located; and stores the TIFF folders in a total folder under the output file path; A PNG cropping submodule, when the cropping form is the PNG cropping, the PNG cropping submodule converts the registered remote sensing image sequence or the remote sensing image sequence into PNG format, and then calls a PNG cropping function to crop the images of the registered remote sensing image sequence or the remote sensing image sequence in PNG format according to the row information and the column information, and stores the cropped PNG format image blocks in the corresponding PNG folder according to the image in which they are located; and stores the PNG folder in the main folder; A sequence cropping submodule, when the cropping mode is the cropping by sequence, the sequence cropping submodule calls a TIFF cropping function according to the row information and the column information to crop the images in the registered remote sensing image sequence or the remote sensing image sequence, collects image blocks with the same sequence number into the same sequence folder, and stores the sequence folder into the main folder; A cropping submodule is selected. When the cropping form is the selected cropping, the selected cropping submodule calls the TIFF cropping function according to the row information and the column information to crop the images in the registered remote sensing image sequence or the remote sensing image sequence, and collects the image blocks that meet the cropping area into the same area folder according to the serial number, and stores the area folder in the total folder.

2. The system for quickly creating UI interface sequence image dataset samples according to claim 1, characterized in that: The interface interaction module includes: A row and column determination submodule, configured to input row information and column information required for a cropping region, and transmit the row information and column information to the sequence image cropping module; A display submodule, configured to display the cropping effect according to the row information and the column information; A cropping region determination submodule, configured to determine the cropping region and transfer the cropping region to the sequence image cropping module; A registration determination submodule, configured to control the image registration module to register the remote sensing image sequence; A cropping form determination submodule, configured to determine the cropping form; The output path determination submodule determines the output file path of the cropped remote sensing image sequence according to the input file path of the remote sensing image sequence.

3. The system for quickly creating UI interface sequence image dataset samples according to claim 2, characterized in that: In the image registration module: Determining the registration file path of the registered remote sensing image sequence according to the input file path; The remote sensing image sequence is registered using fast Fourier transform to obtain the registered remote sensing image sequence, and the registered remote sensing image sequence is stored in the folder of the registration file path.

4. The system for rapidly creating UI interface sequence image dataset samples according to claim 1, characterized in that: The sequence cropping includes TIFF sequence cropping and TIFF to PNG sequence cropping; wherein, when the cropping form is the TIFF sequence cropping, the sequence cropping submodule directly stores the image block in the sequence folder; when the cropping form is TIFF to PNG sequence cropping, the sequence cropping submodule converts the image block into PNG format, and then stores the image block in PNG format in the sequence folder.

5. The UI interface sequence image dataset sample rapid creation system according to claim 1, characterized in that: The selected cropping includes TIFF selected cropping and TIFF to PNG selected cropping; wherein, when the cropping form is TIFF selected cropping, the selected cropping submodule directly stores the regional folder in the total folder; when the cropping form is TIFF to PNG selected cropping, the selected cropping submodule converts the image blocks in the regional folder into PNG format, and then stores the regional folder containing the image blocks in PNG format into the total folder.

6. The UI interface sequence image dataset sample rapid creation system according to claim 1, characterized in that: The system further includes a data set generation module, which includes: An image clustering submodule, when the cropping mode is the cropping by selection or the cropping by sequence, the image clustering submodule uses a clustering algorithm to splice the image blocks according to the sequence numbers; and stores the synthesized regions obtained after the splicing in the region folder or the sequence folder; The video generation submodule converts the image blocks in PNG format or the synthesis area in PNG format into video according to the serial number, and stores the obtained video data.

7. The system for rapidly creating UI interface sequence image dataset samples according to claim 6, characterized in that: In the image clustering submodule, when the cropping area is not a rectangle, the cropping area is supplemented with a black background so that the cropping area after filling is a rectangle with the smallest area.

8. A method for quickly creating a sequence image dataset sample, according to the system for quickly creating a sequence image dataset sample on a UI interface according to any one of claims 1 to 7, characterized in that: include: S1: inputting the file path of the remote sensing image sequence into the interface interaction module, and determining the cropping effect, cropping area and cropping form in the interface interaction module; and determining whether to perform image matching in the interface interaction module; If image matching is performed, execute step S2; if not, execute step S3; S2: The image registration module collects the remote sensing image sequence from the file path obtained in step S1, registers the remote sensing image sequence, and obtains and stores the registered remote sensing image sequence; S3: The sequence image cropping module determines the cropping form determined in step S1, and crops the remote sensing image sequence from step S1 or the registered remote sensing image sequence from step S2 in combination with the cropping effect and the cropping area, and stores the cropped remote sensing image sequence in the corresponding folder according to the output file path.

9. The method for quickly creating a sequence image dataset sample according to claim 8, characterized in that: The creation system further includes a data set generation module; and the rapid creation method further includes: S4: Determine in the interface interaction module whether image clustering is required. If image clustering is required, execute step S5; otherwise, execute step S6; S5: The data set generation module clusters the image blocks and stores the clustered synthetic regions in corresponding folders; S6: Determine in the interface interaction module whether video conversion is required. If required, the data set generation module performs video conversion on the cropped image blocks according to the sequence numbers to obtain and store video data.

Images