A method and device for generating data set for neural network training

Abstract

The present invention relates to a data set generation method, device, equipment and computer-readable storage medium for neural network training, comprising: firstly collecting sample images from each frame image of a video including target object information, secondly performing feature region detection on each sample image after filtering and noise reduction, and determining the feature region position of each sample image; then performing random non-repetitive enhancement transformation on each sample image, cutting out a regional image set from each enhanced image of the obtained enhanced image set, and finally compressing and packaging the regional image sets of all sample images as a whole to generate a training data set. The method provided by the present invention can quickly generate a data set for training, accelerate the speed of making a data set, and automatically expand the diversity of data to achieve a data enhancement function.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and in particular to a method, device, equipment and computer-readable storage medium for generating a data set for neural network training. Background Art

[0002] With the development of deep learning technology, the application of neural networks in object classification and recognition is becoming more and more common. The training of neural networks often requires a large number of data sets of images, but in the industrial field, there is a lack of unified and open data sets. Therefore, each enterprise unit needs to establish its own product image data set to provide rich learning samples for the neural network to improve the recognition ability of the neural network model. However, establishing a neural network data set often requires a lot of manpower, material resources and other resources, and is time-consuming.

[0003] Therefore, it is necessary to propose a dataset generation method for neural network training to solve the above problems. Summary of the invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the defects of the prior art that the data set generation for neural network training is time-consuming and has few samples.

[0005] In order to solve the above technical problems, the present invention provides a method for generating a data set for neural network training, comprising:

[0006] Obtain a pre-filmed video including a target object, and collect a sample image from each frame of the video at every preset number of frames;

[0007] Performing feature region detection on each sample image to determine the feature region of each sample image;

[0008] According to the preset data enhancement parameters, a random non-repetitive enhancement transformation is performed on each sample image to obtain an enhanced image set corresponding to each sample image; wherein the enhancement transformation includes a mirror transformation and a rotation transformation;

[0009] According to the characteristic region of each sample image, a region having the same size as the characteristic region is cropped from each enhanced image in the enhanced image set to obtain a region image set of each sample image;

[0010] The regional image sets of all sample images are compressed and packaged as a whole to generate a training data set.

[0011] In one embodiment of the present invention, the performing feature region detection on each sample image to determine the feature region of each sample image includes:

[0012] Performing filtering and noise reduction processing on each sample image;

[0013] By performing a traversal operation on each sample image after the filtering and denoising process, the Harris corner points and the FAST corner points of each sample image are calculated;

[0014] The rectangular areas of the Harris corner points and the rectangular areas of the FAST corner points are counted, and k times the intersection of the two rectangular areas is taken as the feature area of ​​each sample image.

[0015] In one embodiment of the present invention, performing a random non-repetitive enhancement transformation on each sample image according to a preset data enhancement parameter to obtain an enhanced image set corresponding to each sample image includes:

[0016] According to a preset data enhancement parameter m, determining whether each sample image is subjected to a random non-repetitive enhancement transformation;

[0017] If the enhancement parameter m is equal to 1, each sample image is not subjected to random and non-repetitive enhancement transformation;

[0018] If the enhancement parameter m is greater than 1, each sample image is subjected to m-1 random non-repetitive enhancement transformations.

[0019] In one embodiment of the present invention, the mirror transformation includes: X-axis mirroring, Y-axis mirroring and XY-axis mirroring.

[0020] In one embodiment of the present invention, the rotation angle range of the rotation transformation is: 0° to 360°.

[0021] In one embodiment of the present invention, the step of cutting out a region having the same size as the feature region from each enhanced image in the enhanced image set according to the feature region of each sample image to obtain the region image set of each sample image comprises:

[0022] Determine the center point of the feature area of ​​each sample image;

[0023] Taking the center point of the characteristic region of each sample image as a reference, a region having the same size as the characteristic region is cropped from each enhanced image in the enhanced image set.

[0024] In one embodiment of the present invention, the overall compression and packaging of the regional image sets of all sample images before generating the training data set includes: normalizing the regional image sets of all sample images.

[0025] The present invention provides a data set generation device for neural network training, comprising:

[0026] An acquisition module is used to acquire a pre-shot video including a target object, and to acquire a sample image from each frame of the video at a preset frame rate;

[0027] A detection module, used to perform feature area detection on each sample image to determine the feature area of ​​each sample image;

[0028] An enhancement module, used to perform a random non-repetitive enhancement transformation on each sample image according to preset data enhancement parameters to obtain an enhanced image set corresponding to each sample image; wherein the enhancement transformation includes a mirror transformation and a rotation transformation;

[0029] A cropping module, used for cropping a region having the same size as the characteristic region from each enhanced image in the enhanced image set according to the characteristic region of each sample image, to obtain a region image set of each sample image;

[0030] The generation module is used to compress and package the regional image sets of all sample images as a whole to generate a training data set.

[0031] The present invention provides a data set generation device for neural network training, comprising:

[0032] Memory for storing computer programs;

[0033] A processor, configured to implement the steps of a method for generating a data set for neural network training as claimed in any one of claims 1 to 7 when executing the computer program.

[0034] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of a method for generating a data set for neural network training as described in any one of claims 1 to 7 are implemented.

[0035] The above technical solution of the present invention has the following advantages compared with the prior art:

[0036] The data set generation method provided by the present invention first obtains a pre-filmed video including a target object, and collects a sample image from each frame image of the video at every preset frame number; secondly, performs feature region detection on each sample image to determine the feature region position of each sample image; then performs random non-repetitive enhancement transformation on each sample image, cuts out a regional image set from each enhanced image of the obtained enhanced image set, and finally compresses and packages the regional image sets of all sample images as a whole to generate a training data set. The method provided by the present invention controls the similarity of the sample images of the data set according to the preset frame number; performs feature region detection on each sample image to accurately identify the feature region position of each sample image, thereby speeding up the production speed of the data set; and uses random non-repetitive enhancement transformation to expand the diversity of the data set and realize the function of data enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 A flowchart of a first specific embodiment of a method for generating a data set for neural network training provided by the present invention;

[0039] Figure 2 A flowchart of a second specific embodiment of the method for generating a data set for neural network training provided by the present invention;

[0040] Figure 3 A schematic diagram of the principle of the method for generating a data set for neural network training provided by the present invention;

[0041] Figure 4 A structural block diagram of a method for generating a data set for neural network training provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0043] Reference Figure 1 As shown, Figure 1 This is a flowchart of a first specific embodiment of the method for generating a data set for neural network training provided by the present invention; the specific operation steps are as follows:

[0044] Step S101: obtaining a pre-filmed video including a target object, and collecting a sample image from each frame of the video at intervals of a preset number of frames;

[0045] Step S102: performing feature region detection on each sample image to determine the feature region of each sample image;

[0046] Step S103: performing a random non-repetitive enhancement transformation on each sample image according to preset data enhancement parameters to obtain an enhanced image set corresponding to each sample image; wherein the enhancement transformation includes a mirror transformation and a rotation transformation;

[0047] Step S104: according to the characteristic region of each sample image, a region having the same size as the characteristic region is cropped from each enhanced image in the enhanced image set to obtain a region image set of each sample image;

[0048] Step S105: compress and package the regional image sets of all sample images as a whole to generate a training data set.

[0049] The method provided in this embodiment quickly generates a data set for training by shooting multi-angle and multi-background videos of the target object, and automatically expands the diversity of the data to achieve data enhancement function; it also provides rich learning samples for the training of neural networks.

[0050] Based on the above embodiments, this embodiment is a further explanation of the above embodiments; please refer to Figure 2 and Figure 3 , Figure 2 A flowchart of a second specific embodiment of the method for generating a data set for neural network training provided by the present invention; Figure 3 The following is a schematic diagram of the principle of the method for generating a data set for neural network training provided by the present invention; the specific operation steps are as follows:

[0051] Step S201: obtaining a video including a target object that is shot in advance at multiple angles, and collecting a sample image from each frame of the video every f frames according to a preset sampling parameter f;

[0052] When f=0, the sample images are all the video frames; each video frame only includes information of one or one type of target object, and the video shooting background color is single and there is no complex contour or edge information of other objects.

[0053] Step S202: performing filtering and noise reduction processing on each sample image;

[0054] In the present invention, bilateral filtering, Gaussian filtering and box filtering can be selected to perform filtering and noise reduction processing on each sample image. During the filtering and noise reduction processing, a preset filtering function is used to control the difference between each adjacent pixel point in the image within a certain range, correct the unreasonable pixel points in the image, thereby reducing and eliminating the noise interference in the image.

[0055] Step S203: traversing each sample image after the filtering and denoising process to calculate the Harris corner points and FAST corner points of each sample image;

[0056] Step S204: Count the rectangular areas of the Harris corner points and the rectangular areas of the FAST corner points, and take k times the intersection of the two rectangular areas as the feature area of ​​each sample image;

[0057] The rectangular areas of the Harris corner points and the FAST corner points are counted, all the Harris corner points are surrounded by rectangle A, all the FAST corner points are surrounded by rectangle B, and k times the intersection of rectangles A and B are taken as the feature area of ​​each sample image. The k times is an artificially set magnification factor that can modify the feature area window. The magnification factor k is used to enlarge and reduce the intersection area, thereby ensuring that the target object is contained in the feature area.

[0058] Step S205: determining the coordinates O of the center point of the feature area of ​​each sample image according to the pixel coordinate positions of the feature area of ​​each sample image;

[0059] Step S206: performing a random non-repetitive enhancement transformation on each sample image according to preset data enhancement parameters to obtain an enhanced image set corresponding to each sample image;

[0060] The enhanced transformation includes mirror transformation and rotation transformation, the mirror transformation includes X-axis mirror, Y-axis mirror and XY-axis mirror, and the rotation transformation is that the system randomly selects the rotation angle θ of the image, and performs a rotation transformation on the sample image with the coordinate O of the center point of the feature area of ​​each sample image as the center. The rotation angle θ of the rotation transformation is 15°, 30°, ..., 330°, 345°.

[0061] According to the preset data enhancement parameter m, determine whether each sample image is subjected to a random and non-repetitive enhancement transformation; if the enhancement parameter m is equal to 1, each sample image is not subjected to a random and non-repetitive enhancement transformation; if the enhancement parameter m is greater than 1, each sample image is subjected to m-1 random and non-repetitive enhancement transformations to generate m enhanced images.

[0062] Step S207: taking the coordinate O of the center point of the characteristic region of each sample image as a reference, cropping a region having the same size as the characteristic region from each enhanced image in the enhanced image set to obtain a region image set of each sample image;

[0063] Step S208: performing normalization processing on the regional image sets of all sample images;

[0064] Step S209: compress and package the regional image set of all sample images that have completed the normalization process as a whole to generate a training data set.

[0065] Normalize the regional image sets of all sample images, and re-convert the size of all sample image regional image sets to a set size (the default is 320×320). The size can be determined according to the actual situation and is not limited in this embodiment. Write the file name and save path of all sample image regional image sets, as well as the category label information, into a txt file to provide the address information and label information of the image when the neural network reads the data. Finally, compress and package the regional image sets of all sample images as a whole to generate a resource compression package with the suffix ".gz".

[0066] The method provided in this embodiment calculates the rectangular area of ​​Harris corner points and FAST corner points, and takes the intersection of the two rectangular areas to determine the feature area detection of each sample image, which can accurately identify the feature area position of each sample image and speed up the production of data sets. Random non-repetitive rotation and mirror enhancement transformation are used to crop the regional image set of each sample image to expand the diversity of the data set and realize the data enhancement function.

[0067] Please refer to Figure 4 , Figure 4 A structural block diagram of a method for generating a data set for neural network training provided in an embodiment of the present invention; a specific device may include:

[0068] The acquisition module 100 is used to acquire a pre-shot video including a target object, and acquire a sample image from each frame image of the video every preset number of frames;

[0069] The detection module 200 is used to perform feature region detection on each sample image to determine the feature region of each sample image;

[0070] The enhancement module 300 is used to perform a random non-repetitive enhancement transformation on each sample image according to a preset data enhancement parameter to obtain an enhanced image set corresponding to each sample image; wherein the enhancement transformation includes a mirror transformation and a rotation transformation;

[0071] A cropping module 400 is used to crop a region having the same size as the characteristic region from each enhanced image in the enhanced image set according to the characteristic region of each sample image, so as to obtain a region image set of each sample image;

[0072] The generation module 500 is used to compress and package the regional image sets of all sample images as a whole to generate a training data set.

[0073] The data set generating device for neural network training of this embodiment is used to implement the aforementioned data set generating method for neural network training. Therefore, the specific implementation method of the data set generating device for neural network training can be seen in the embodiment part of the data set generating method for neural network training in the previous text. For example, the acquisition module 100, the detection module 200, the enhancement module 300, the cropping module 400, and the generation module 500 are respectively used to implement steps S101, S102, S103, S104 and S105 in the aforementioned data set generating method for neural network training. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0074] A specific embodiment of the present invention also provides a data set generation device for neural network training, including: a memory for storing a computer program; a processor for implementing the steps of the above-mentioned data set generation method for neural network training when executing the computer program.

[0075] A specific embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for generating a data set for neural network training are implemented.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0080] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A method for generating a data set for neural network training, characterized in that: include: Obtain a pre-filmed video including a target object, and collect a sample image from each frame of the video at every preset number of frames; Performing feature region detection on each sample image to determine the feature region of each sample image; According to the preset data enhancement parameters, a random non-repetitive enhancement transformation is performed on each sample image to obtain an enhanced image set corresponding to each sample image; wherein the enhancement transformation includes a mirror transformation and a rotation transformation; According to the characteristic region of each sample image, a region having the same size as the characteristic region is cropped from each enhanced image in the enhanced image set to obtain a region image set of each sample image; The regional image sets of all sample images are compressed and packaged as a whole to generate a training data set; The performing feature region detection on each sample image to determine the feature region of each sample image comprises: Performing filtering and noise reduction processing on each sample image; By performing a traversal operation on each sample image after the filtering and denoising process, the Harris corner points and the FAST corner points of each sample image are calculated; Counting the rectangular areas of the Harris corner points and the rectangular areas of the FAST corner points, and taking k times the intersection of the two rectangular areas as the feature area of ​​each sample image; The step of performing random and non-repetitive enhancement transformation on each sample image according to the preset data enhancement parameters to obtain an enhanced image set corresponding to each sample image includes: According to a preset data enhancement parameter m, determining whether each sample image is subjected to a random non-repetitive enhancement transformation; If the enhancement parameter m is equal to 1, each sample image is not subjected to random and non-repetitive enhancement transformation; If the enhancement parameter m is greater than 1, each sample image is subjected to m-1 random non-repetitive enhancement transformations; The step of cutting out a region having the same size as the feature region from each enhanced image in the enhanced image set according to the feature region of each sample image to obtain the region image set of each sample image comprises: Determine the center point of the feature area of ​​each sample image; Taking the center point of the characteristic region of each sample image as a reference, a region having the same size as the characteristic region is cropped from each enhanced image in the enhanced image set.

2. The method according to claim 1, characterized in that The mirror transformation includes: X-axis mirroring, Y-axis mirroring and XY-axis mirroring.

3. The method according to claim 1, characterized in that: The rotation angle range of the rotation transformation is: 0°~360°.

4. The method according to claim 1, characterized in that: The overall compression and packaging of the regional image sets of all sample images before generating the training data set includes: normalizing the regional image sets of all sample images.

5. A data set generation device for neural network training, using a data set generation method for neural network training as described in any one of claims 1 to 4, characterized in that ,include: An acquisition module is used to acquire a pre-shot video including a target object, and to acquire a sample image from each frame of the video at a preset frame rate; A detection module, used to perform feature area detection on each sample image to determine the feature area of ​​each sample image; An enhancement module, used to perform a random non-repetitive enhancement transformation on each sample image according to preset data enhancement parameters to obtain an enhanced image set corresponding to each sample image; wherein the enhancement transformation includes a mirror transformation and a rotation transformation; A cropping module, used for cropping a region having the same size as the characteristic region from each enhanced image in the enhanced image set according to the characteristic region of each sample image, to obtain a region image set of each sample image; The generation module is used to compress and package the regional image sets of all sample images as a whole to generate a training data set.

6. A data set generation device for neural network training, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of a method for generating a data set for neural network training as claimed in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a method for generating a data set for neural network training as claimed in any one of claims 1 to 4 are implemented.

Images