WebGIS-based sample set automatic generation system and method

Abstract

The invention relates to the technical field of image data processing, in particular to a WebGIS-based sample set automatic generation system and method. The system comprises an original image data acquisition module, an original file generation module, a slice data generation module, a sample set generation module, a sample library management module and a sample set task management module. Based on the slice data generation module and the sample set generation module, the generation efficiency of the sample set can be improved, and timely supply of training samples is ensured; and based on the sample set task management module, the completion state of the sample set can be monitored in real time, so that managers can know the generation process of the sample in time or find possible abnormal conditions.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to a sample set automatic generation system based on WebGIS, a sample set automatic generation method based on WebGIS, a processor and a machine-readable storage medium. Background Art

[0002] In the prior art, sample sets used for image model training are mainly imported manually, that is, R&D personnel are deeply involved in running slicing scripts to generate sample sets from original images and target annotation information, and then the sliced ​​sample sets are uploaded to the sample library management system by manually uploading compressed packages. Since the existing system platform cannot automatically convert the labeled original image data into sample sets that can be used for model training, R&D personnel are required to deeply participate in running slicing scripts. Therefore, the sample set generation efficiency is low, time-consuming and labor-intensive, and it is difficult to meet the large demand for samples in online training. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a sample set automatic generation system, method, machine-readable storage medium and processor based on WebGIS.

[0004] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a sample set automatic generation system based on WebGIS, the system comprising: an original image data acquisition module, used to acquire the annotated original image data in batches; an original file generation module, used to convert the annotated data of the original image data into an original file; a slice data generation module, used to slice the original image data and the original file according to N preset segmentation rules to obtain N slice data sets; any slice data in the slice data set consists of two parts: slice image data and corresponding label data; a sample set generation module, used to process the slice data in the slice data set, generate N sample sets that can be used for model training, and store the sample sets in a sample library management module; a sample library management module, used to store and manage sample sets; a sample set task management module, used to monitor the generation progress of N sample sets in real time, and update the progress status in real time; the progress status includes in progress and completed.

[0005] Based on the first aspect, the converting of the annotated data of the original image data into the original file includes: generating a shpfile file and a geojson file from the annotated data through a tool class.

[0006] Based on the first aspect, the slicing process of the original image data and the original file includes: slicing the original image data and the shpfile file into sliced ​​image data and label data of corresponding specifications through a slicing service.

[0007] Based on the first aspect, the restrictions on N are as follows:

[0008]

[0009] Among them, A represents the number of slice size types that meet the segmentation requirements, B represents the number of slice format types that meet the segmentation requirements, and C represents the number of slice overlap types that meet the segmentation requirements.

[0010] In the second aspect, the present application provides a sample set automatic generation method based on WebGIS, which is applicable to the above-mentioned sample set automatic generation system, and the method includes: using a sample set task management module to create N sample set generation tasks, and monitoring the generation progress of each sample set; when the corresponding sample set generation task is completed, its progress status is changed from in progress to completed.

[0011] Based on the second aspect, the method also includes: using the sample set task management module to respectively create N original file generation subtasks, slice data generation subtasks and sample set storage subtasks; the monitoring of the generation progress of each sample set includes: judging whether all the labeled data are converted into original files; if so, the progress status of the corresponding original file generation subtask is shown as completed, otherwise it is shown as in progress; judging whether the original image data and the original file have completed the slicing processing; if so, the progress status of the corresponding slice data generation subtask is shown as completed, otherwise it is shown as in progress; judging whether the sample set is successfully stored in the sample library management module; if so, the progress status of the corresponding sample set storage subtask is shown as completed, otherwise it is shown as in progress; the completion of the sample set generation task is manifested as: the progress status of the corresponding original file generation subtask, slice data generation subtask and sample set storage subtask are all shown as completed.

[0012] Based on the second aspect, the method further includes: using the sample set task management module to create an independent library table to record the sample set generation task, and updating the task status of the sample set generation task in real time according to the processing progress of the sample set generation task.

[0013] Based on the second aspect, the method also includes: using a sample set task management module to create a data storage directory, the data storage directory includes a sample annotation directory and a sample set directory; the sample annotation directory includes an original data directory and a slice format directory; wherein the original image data and the original file are stored in the original data directory; the slice image data and label data are stored in the sample set directory.

[0014] In a third aspect, the present application provides a processor configured to execute the above-mentioned WebGIS-based automatic sample set generation method.

[0015] In a fourth aspect, the present application provides a machine-readable storage medium having instructions stored thereon, which, when executed by a processor, configure the processor to execute the above-mentioned WebGIS-based automatic sample set generation method.

[0016] The system provided by the present application can divide the annotated original image data into slice data that meets the requirements according to preset rules by setting a slice data generation module, and can quickly generate sample sets for model training based on slice data sets by setting a sample set generation module, and aggregate and parse the sample sets to the sample library management module, thereby improving the efficiency of sample set generation and ensuring the timely supply of training samples. In addition, the system provided by the present application also monitors the completion status of the sample set in real time by setting a sample set task management module, so that managers can promptly understand the sample generation process or discover possible abnormal situations.

[0017] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0019] Figure 1 A system structure block diagram of a sample set automatic generation system according to an embodiment of the present application is schematically shown;

[0020] Figure 2 The following is a schematic diagram of the process of generating a sample set in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the application environment of the method in the embodiment of the present application is schematically shown;

[0022] Figure 4 A diagram schematically shows the relationship between the labeling task table and the sample set generation task library table in an embodiment of the present application;

[0023] Figure 5 A data flow diagram based on an image data directory in an embodiment of the present application is schematically shown;

[0024] Figure 6 The internal structure diagram of the computer device according to the embodiment of the present application is schematically shown.

[0025] Description of Reference Numerals

[0026] 1-Original image data acquisition module; 2-Original file generation module; 3-Slice data generation module; 4-Sample set generation module; 5-Sample library management module; 6-Sample set task management module; 102-Terminal; 104-Server; A01-Processor; A02-Network interface; A03-Internal memory; A04-Display screen; A05-Input device; A06-Non-volatile storage medium; B01-Operating system; B02-Computer program. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0028] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0029] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0030] Example 1

[0031] This embodiment provides a sample set automatic generation system based on WebGIS. Figure 1 As shown, the system comprises:

[0032] The original image data acquisition module 1 is used to acquire the annotated original image data in batches;

[0033] Generally, due to the large amount of complete image data to be processed, single-thread processing will consume too much time. For example, in this embodiment, multiple time nodes can be set to divide the collected original image into different small segments or extract a single feature image, and then the multiple segments of images or multiple feature images are divided into batches, and each batch of images or images is processed in parallel by multiple threads to improve processing efficiency.

[0034] After obtaining the annotated original image data (including image data and annotated data), the following modules are required to generate the sample set required for model training:

[0035] The original file generation module 2 is used to convert the annotation data of the original image data into an original file;

[0036] The slice data generation module 3 is used to slice the original image data and the original file according to the preset N segmentation rules to obtain N slice data sets; any slice data in the slice data set is composed of two parts: slice image data and corresponding label data;

[0037] The sample set generation module 4 is used to process the slice data in the slice data set, generate N sample sets that can be used for model training, and store the sample sets in the sample library management module 5;

[0038] Specifically, the above raw image data processing process can be summarized into three steps (such as Figure 2 As shown in the figure), the original data, slice data and sample set are generated. Because each step of program processing requires operating files and takes a long time, in this embodiment, when generating the original files, slice data and sample set data, they are processed by asynchronous process or multi-threading (each batch of original image data can be in different processing steps respectively), and the processing status is tracked and recorded in real time:

[0039] Generate raw data, including generating shpfile and geojson files, that is, generate shpfile and geojson files through tool classes from target annotation data in postgresql library.

[0040] Generate slice data, that is, slice the original image data and shpfile files into slice image data (images) and label data (labels) of corresponding specifications according to the preset segmentation rules through the slice service. Specifically, before segmentation, it is necessary to preset the slice size (for example, 1024*1024, 512*512, 256*256), format type (such as: DOTA, VOC, YOLO, COCO) and overlap properties, so the number of segmentation rules N is subject to the following restrictions:

[0041]

[0042] Among them, A represents the number of slice size types that meet the segmentation requirements, B represents the number of slice format types that meet the segmentation requirements, and C represents the number of slice overlap types that meet the segmentation requirements.

[0043] Generate sample set data, that is, aggregate and parse slice data from different sources into the sample library management module 5.

[0044] The sample library management module 5 is used to store and manage sample sets;

[0045] The sample set task management module 6 is used to monitor the generation progress of N sample sets in real time and update the progress status in real time; the progress status includes in progress and completed. Based on this module, the completion status of the sample set can be tracked in real time, which is convenient for managers to timely understand the sample generation process or discover possible abnormalities (for example, when the processing progress status of a batch of raw image data is in progress for a long time, it means that one of the above three steps may be abnormal).

[0046] Example 2

[0047] Based on the system provided in Example 1, the WebGIS-based sample set automatic generation method provided in this application can be applied to Figure 3 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0048] The above-mentioned sample set automatic generation method includes:

[0049] Use the sample set task management module 6 to create N sample set generation tasks and monitor the generation progress of each sample set; when the corresponding sample set generation task is completed, its progress status is changed from in progress to completed. The "in progress" status in the program processing logic includes three sub-states: generating original shpfile and geojson, slicing images, and parsing and storing sample sets. When the above three sub-states are completed, the sample set generation task status status is updated to completed. The details are as follows:

[0050] Using the sample set task management module 6, N original file generation subtasks, slice data generation subtasks and sample set storage subtasks are created respectively;

[0051] The monitoring of the generation progress of each sample set includes:

[0052] Determine whether all the annotated data has been converted into original files; if so, the progress status of the corresponding original file generation subtask is shown as completed, otherwise it is shown as in progress;

[0053] Determine whether the original image data and the original file have completed the slicing process; if so, the progress status of the corresponding slicing data generation subtask is shown as completed, otherwise it is shown as in progress;

[0054] Determine whether the sample set is successfully stored in the sample library management module 5; if so, the progress status of the corresponding sample set storage subtask is shown as completed, otherwise it is shown as in progress;

[0055] The completion of the sample set generation task is manifested as follows: the progress status of the corresponding original file generation subtask, slice data generation subtask and sample set storage subtask are all manifested as completed.

[0056] Based on the above method, the generation progress stage of the sample set can be further clarified. When an abnormal situation occurs, the location where the abnormal situation occurs can be determined more accurately, thereby improving the efficiency of finding abnormal points.

[0057] Furthermore, in order to facilitate the unified management of all sample set generation tasks, the sample set task management module 6 can also be used to create an independent library table (i.e., the sample set generation task library table) to record the sample set generation tasks, and update the task status of the sample set generation tasks in real time according to the processing progress of the sample set generation tasks. This method has a clear structure, is convenient for management personnel to uniformly manage and view, and is also more convenient for abnormal tracking (for example, by observing whether different sample set generation tasks will have abnormalities in the same processing step).

[0058] It should be noted that before performing the sample set generation task, it is also necessary to complete the pre-created sample labeling task to obtain the labeled original image data. After completing a sample labeling task, the user will generate different slice data sets by setting different slice sizes, overlaps, slice formats, etc. In this embodiment, one slice data set corresponds to one sample set, and one sample set corresponds to one sample set generation task. By establishing a sample labeling task table t_label_task and comparing it with the sample set generation task library table t_label_generate_task (such as Figure 4 As shown in the figure, the difference in quantity can be intuitively reflected. In t_label_generate_task, task_id represents the labeling task to which the sample set generation task belongs. When performing a sample set generation task, the main fields of its library table structure include task_id, slice_x, slice_y, overlap, and code, so as to determine the uniqueness of a sample set generation task, where code is the code value in the slice format, 1-DOTA, 2-VOC, 3-COCO, and 4-YOLO.

[0059] Example 3

[0060] To facilitate the subsequent authority division and data migration, in this embodiment, the sample set task management module 6 is used to create a data storage directory, such as Figure 5 As shown ( Figure 5 The content shown clearly shows the data flow in the process of generating the sample set, where the data storage directory mainly consists of the sample label directory / opt / label and the sample set directory / opt / sampleSet corresponding to the sample library management module 5.

[0061] The sample annotation directory includes the original data directory and the slice format directory. The original data directory contains original data, shpfile files, and geojson files; the slice format directory contains DOTA files, VOC files, COCO files, and YOLO files. In addition, the original data directory contains slice image data and annotation data.

[0062] It should also be noted that: 1) When generating a sample set, the main parameters include slice size (slice_x, slice_y), overlap (overlap), and slice format (code). Therefore, when creating a slice data directory, the directory is divided according to / opt / label / taskId / slice format / slice size / overlap to ensure the uniqueness of the slice directory. 2) Figure 5 The slice directory structure of COCO files, VOC files, and YOLO files is the same as the slice directory structure of DOTA files (not shown in the figure). 3) When the sliced ​​data is aggregated to the sample library management module 5, not only the sliced ​​image data and label data are copied to the corresponding sample set directory, but also the sliced ​​image data metadata and label data metadata need to be parsed into elasticsearch and the business statistical data need to be stored in postgresql for subsequent retrieval and statistics.

[0063] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned method for automatically generating a sample set based on WebGIS is implemented.

[0064] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, a method for automatically generating a sample set based on WebGIS is implemented. The display screen A04 of the computer device can be a liquid crystal display or an electronic ink display, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0065] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0066] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0067] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0071] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0072] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0073] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0074] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A sample set automatic generation system based on WebGIS, characterized in that: The system comprises: The original image data acquisition module is used to acquire the annotated original image data in batches; A raw file generation module, used to convert the annotation data of the raw image data into a raw file; The slice data generation module is used to slice the original image data and the original file according to the preset N segmentation rules to obtain N slice data sets; any slice data in the slice data set is composed of two parts: slice image data and corresponding label data; The sample set generation module is used to process the slice data in the slice data set, generate N sample sets that can be used for model training, and store the sample sets in the sample library management module; The sample library management module is used to store and manage sample sets; The sample set task management module is used to monitor the generation progress of the N sample sets in real time and update the progress status in real time; the progress status includes in progress and completed.

2. The WebGIS-based sample set automatic generation system according to claim 1, characterized in that: The converting the annotated data of the original image data into the original file comprises: Generate shpfile and geojson files from the labeled data through the tool class.

3. The WebGIS-based sample set automatic generation system according to claim 2, characterized in that: The slicing process of the original image data and the original file includes: The original image data and shpfile files are sliced ​​into slice image data and label data of corresponding specifications through the slicing service.

4. The WebGIS-based sample set automatic generation system according to claim 1, characterized in that: The restrictions on N are as follows: ； Among them, A represents the number of slice size types that meet the segmentation requirements, B represents the number of slice format types that meet the segmentation requirements, and C represents the number of slice overlap types that meet the segmentation requirements.

5. A method for automatically generating sample sets based on WebGIS, characterized in that: The sample set automatic generation system according to any one of claims 1 to 4, wherein the method comprises: Use the sample set task management module to create N sample set generation tasks and monitor the generation progress of each sample set; When the corresponding sample set generation task is completed, its progress status is changed from in progress to completed.

6. The WebGIS-based automatic sample set generation method according to claim 5, characterized in that: The method further comprises: using the sample set task management module to respectively create N original file generation subtasks, slice data generation subtasks and sample set storage subtasks; The monitoring of the generation progress of each sample set includes: Determine whether all the annotated data has been converted into original files; if so, the progress status of the corresponding original file generation subtask is shown as completed, otherwise it is shown as in progress; Determine whether the original image data and the original file have completed the slicing process; if so, the progress status of the corresponding slicing data generation subtask is shown as completed, otherwise it is shown as in progress; Determine whether the sample set is successfully stored in the sample library management module; if so, the progress status of the corresponding sample set storage subtask is shown as completed, otherwise it is shown as in progress; The completion of the sample set generation task is manifested as follows: the progress status of the corresponding original file generation subtask, slice data generation subtask and sample set storage subtask are all manifested as completed.

7. The method for automatically generating a sample set based on WebGIS according to claim 5 or 6, characterized in that: The method further comprises: An independent database table is created using the sample set task management module to record sample set generation tasks, and the task status of the sample set generation tasks is updated in real time according to the processing progress of the sample set generation tasks.

8. The method for automatically generating sample sets based on WebGIS according to claim 5, characterized in that: The method further comprises: Using the sample set task management module to create a data storage directory, the data storage directory includes a sample annotation directory and a sample set directory; The sample annotation directory includes an original data directory and a slice format directory; wherein the original image data and the original file are stored in the original data directory; The slice image data and label data are stored in a sample set directory.

9. A processor, characterized in that: The method is configured to execute the WebGIS-based automatic sample set generation method as described in any one of claims 5 to 8.

10. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor is configured to execute the WebGIS-based automatic sample set generation method described in any one of claims 5 to 8.

Images