Method and device for generating training samples of a mass production model, storage medium, apparatus

Abstract

The application provides a method and device for generating a large number of model training samples, a storage medium and equipment, wherein the method comprises: acquiring a plurality of original pictures containing a target object; the target object is an image that needs to be recognized by the computer vision model; determining the image type of the target object; wherein the image type comprises a fixed style image and a non-fixed style image; and performing image expansion processing on the original pictures corresponding to the target object by using a preset processing rule matched with the image type of the target object, to obtain a plurality of picture samples. Through the application, a large number of qualified picture samples to be trained can be quickly generated from a small number of original pictures.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, apparatus, storage medium, and device for generating large batches of model training samples. Background Technology

[0002] Computer vision is a technology that uses cameras and computers to identify, track, and measure targets, replacing the human eye, and further processes the images to create images more suitable for human observation or transmission to instruments for detection. Training computer vision models often requires a large amount of image data. For example, training a computer vision model that can recognize cars from scratch requires a large number of image samples containing cars; relying on manual photography for collection is extremely time-consuming and labor-intensive. To train a relatively accurate model, the number of image samples needs to be at least tens of thousands, or even hundreds of thousands or millions. Therefore, the availability of image samples is undoubtedly the biggest obstacle to training computer vision models. However, in practical project applications, there are often only a very small number of image samples available, requiring continuous collection over a long period. Summary of the Invention

[0003] Therefore, the present invention provides a method, apparatus, storage medium, and device for generating large batches of model training samples, aiming to solve the technical problem that existing technologies cannot quickly and in batches generate computer vision model training samples.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] According to a first aspect of the present invention, the present invention provides a method for generating a large number of model training samples, the method comprising:

[0006] Acquire multiple original images containing a target object; the target object is the image that the computer vision model needs to recognize.

[0007] Determine the image type of the target object; wherein, the image type includes fixed-style images and non-fixed-style images;

[0008] Using preset processing rules that match the image type of the target object, the original image corresponding to the target object is subjected to image expansion processing to obtain multiple image samples.

[0009] Optionally, determining the image type of the target object includes:

[0010] For multiple original images containing the same target object, generate fingerprint codes corresponding to each original image;

[0011] Compare the similarity of each fingerprint code corresponding to each original image. If the similarity exceeds a preset similarity threshold, the image type of the target object is a fixed-style image.

[0012] If the similarity does not exceed the preset similarity threshold, the image type of the target object is a non-fixed style image.

[0013] Optionally, the image type of the target object is a fixed-style image;

[0014] The step of performing image expansion processing on the original image corresponding to the target object using preset processing rules that match the image type of the target object includes:

[0015] Step S1: Use image segmentation technology to perform image matting on the original image containing the target object to obtain the target image;

[0016] Step S2: Obtain multiple base images, and use image fusion technology to merge the target image with each of the base images to obtain multiple fused images.

[0017] Optionally, the method further includes:

[0018] Determine whether the total number of images in the fused images reaches a preset number of images; if the number of images does not reach the preset number of samples, repeat step S2 until the number of images reaches the preset number of images; and / or,

[0019] Each of the fused images is determined to have a preset image quality. If the image quality meets the preset image quality, the fused image is used as an image sample. If the image quality does not meet the preset image quality, the fused image is deleted.

[0020] The preset image quality includes at least the degree of distinction between the target image and the negative in the fused image.

[0021] Optionally, the image type of the target object is a non-fixed style image;

[0022] The preset processing rules include simulated occlusion processing and / or image editing processing;

[0023] The simulated occlusion process involves using mosaic technology to occlude the target object in the original image at random locations.

[0024] The image scaling process involves using image editing tools to scale, stretch, or crop the original image to obtain image samples of different sizes.

[0025] Optionally, before generating the fingerprint codes corresponding to each of the original images, the method further includes:

[0026] Determine whether the image positions of each target object are the same in the corresponding original image;

[0027] If the image positions of each target object are the same in the corresponding original image, then the portion of the original image corresponding to the image position is cropped, and the portion of the original image is updated to the original image of the target object.

[0028] Optionally, the fixed-style image is an image whose image style does not change with scene factors;

[0029] The non-fixed style image is an image whose style changes with scene factors;

[0030] The scene factors include at least one of lighting, shooting angle, and image position.

[0031] According to a second aspect of the present invention, the present invention provides an apparatus for generating large batches of model training samples, the apparatus comprising:

[0032] The original image acquisition module is used to acquire multiple original images containing a target object; the target object is the image that the computer vision model needs to recognize.

[0033] An image type determination module is used to determine the image type of the target object; wherein, the image type includes fixed-style images and non-fixed-style images;

[0034] The image sample expansion module is used to perform image expansion processing on the original image corresponding to the target object using preset processing rules that match the image type of the target object, so as to obtain multiple image samples.

[0035] According to a third aspect of the present invention, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform a method for generating large batches of model training samples as described in any one of the first aspects of the present invention.

[0036] According to a fourth aspect of the present invention, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements a method for generating large batches of model training samples as described in any one of the first aspects of the present invention.

[0037] The present invention, by adopting the above technical solution, has at least the following beneficial effects:

[0038] The present invention obtains multiple original images containing a target object; the target object is the image that the computer vision model needs to recognize; the image type of the target object is determined; wherein, the image type includes fixed-style images and non-fixed-style images; using preset processing rules that match the image type of the target object, image expansion processing is performed on the original images corresponding to the target object to obtain multiple image samples. Thus, a large number of qualified, training-ready image samples can be quickly generated using a small number of original images.

[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a method for generating large batches of model training samples according to an embodiment of the present invention is shown.

[0042] Figure 2 A schematic diagram of the structure of an apparatus for generating large batches of model training samples according to an embodiment of the present invention is shown.

[0043] Figure 3 A schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0044] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0045] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0046] This invention provides a method for generating large batches of model training samples, such as... Figure 1 As shown, it may include at least the following steps S101 to S103:

[0047] Step S101: Obtain multiple original images containing the target object.

[0048] The target object is the object that the computer vision model needs to recognize. For example, if you want to train a computer vision model that can recognize cars, then the car is the target object. The original images are pre-obtained pictures containing the target object, collected from the internet through various means.

[0049] Step S102: Determine the image type of the target object.

[0050] This invention categorizes target object images into fixed-style images and non-fixed-style images. Fixed-style images are those whose image style does not change with scene factors; non-fixed-style images are those whose image style changes with scene factors. Scene factors can include lighting, shooting angle, and image position. For example, the CCTV1 channel logo always maintains the same image style whether viewed in a television screenshot or a mobile phone screenshot, and it is always displayed in the upper left corner of the screenshot. There will never be only half a logo, or the logo will be tilted or rotated. Therefore, the CCTV1 channel logo is a fixed-style image. Similarly, when a dog is the target object, even if the dog is standing still, different shooting angles (up, down, left, right) will result in different dog images; when the dog moves, even if the scene factors do not change, various dog images will still be produced. Therefore, a dog is a non-fixed-style image.

[0051] In determining the image type of a target object, this embodiment of the invention can specifically generate fingerprint codes corresponding to each of multiple original images containing the same target object; compare the similarity of each fingerprint code corresponding to each original image; if the similarity exceeds a preset similarity threshold, the image type of the target object is a fixed-style image; if the similarity does not exceed the preset similarity threshold, the image type of the target object is a non-fixed-style image. Before generating fingerprint codes corresponding to each original image, it can also be determined whether the image positions of each target object in the corresponding original image are the same; if the image positions of each target object in the corresponding original image are the same, then a portion of the original image corresponding to that image position is cropped, and this portion of the original image is updated to the original image of the target object.

[0052] In other words, for multiple original images containing the same target object, to enhance the accuracy of determining the image type of the target object, for original images with known fixed image positions, a portion of the original image corresponding to the known fixed image position can be cropped and used as the original image of the target object for similarity comparison. For example, the CCTV1 channel logo is always located in the upper left corner of the original image, so a portion of the original image containing the CCTV1 channel logo in the upper left corner can be cropped and used as the original image of the CCTV1 channel logo for subsequent similarity comparison. Then, an OpenCV image recognition and localization system can be used to generate fingerprint codes corresponding to each original image, and the image type of the target object can be determined based on the fingerprint codes. Specifically, since there are many original images, the fingerprint codes of the original images can be compared pairwise to obtain the similarity between each pair of original images. If the similarity between each pair of original images exceeds a preset similarity threshold, the target object corresponding to these original images can be considered a fixed-style image. If the similarity between two original images does not exceed the preset similarity threshold, the target object corresponding to these original images is considered a non-fixed-style image. It should be noted that the preset similarity threshold proposed in this embodiment can be set according to actual conditions and needs, and this invention does not limit it.

[0053] Step S103: Using preset processing rules that match the image type of the target object, perform image expansion processing on the original image corresponding to the target object to obtain multiple image samples.

[0054] Understandably, different image types of target objects require different image expansion processing methods. For target objects with fixed-style images, image segmentation techniques can be used to cut out the original image containing the target object to obtain the target image; multiple base images are then obtained, and image fusion techniques are used to merge the target image with each base image to obtain multiple fused images.

[0055] In this embodiment of the invention, the target image is the image of the target object. For example, if the original image is horses galloping on a grassland and the target object is horses, the original image can be cut out to obtain an image containing only horses, which is the target image. Specifically, image segmentation techniques such as Photoshop (Adobe Photoshop image processing software) can be used to extract the target image from the original image. Multiple background images can be obtained by downloading them from the Internet or existing image resource libraries using web scraping technology. After obtaining the target image and a preset number of background images, the target image and background images can be batch merged using the PIL library, which is programmed in Java or Python, to obtain multiple merged images containing the target object with the background images as the background.

[0056] Furthermore, embodiments of the present invention can also perform image quantity checks and image quality checks on the fused images. Specifically, regarding the image quantity, it can be determined whether the total number of images in the fused images reaches a preset number of images. If the number of images does not reach the preset sample number, a new background image is acquired, and the target image and the background image are merged to obtain a fused image, until the number of images in the fused images reaches the preset number of images. Regarding image quality, it can be determined whether the image quality of each fused image reaches a preset image quality; if the image quality reaches the preset image quality, the fused image is used as an image sample; if the image quality does not reach the preset image quality, the fused image is deleted. The preset image quality can be the degree of distinguishability between the target image and the background in the fused image. It is understood that, in order to improve the training efficiency of computer vision models, it is required that the target image and the background are easily distinguishable; fused images where the target image and the background are difficult to distinguish are considered not to have reached the preset image quality and are deleted. Specifically, to determine whether the target image and the background are easily distinguishable, the fused image obtained in the above steps can first be segmented using segment-anything technology. Segment-anything is an image segmentation technology that can accurately identify objects in an image. Then, the contours of the fused image after image segmentation are extracted using the Canny edge detection algorithm, resulting in an extracted contour image. The extracted contour image is compared with the target image. If the images are identical, the target image and the negative are considered easily distinguishable, and the fused image meets the image quality requirements; otherwise, the target image and the negative are considered difficult to distinguish, and the fused image does not meet the image quality requirements and is deleted.

[0057] For target objects in non-fixed-style images, pre-defined processing rules can include simulated occlusion and / or image editing. Simulated occlusion involves using mosaic technology to randomly occlude the target object in the original image. This allows for the simulation of occlusion of different objects or in different scenes. For example, when the target object is a dog, mosaic technology can be used to occlude the dog's body, simulating an image where only the dog's head is visible behind a sofa. Similarly, mosaic technology can be used to simulate occlusion of the head, tail, one leg, two legs, one foot, etc.

[0058] Building upon the simulated occlusion processing, embodiments of the present invention can also utilize image editing tools to scale, stretch, or crop the original image to obtain image samples of different sizes. Specifically, image scaling techniques based on Java or Python programming can be used to scale the image after simulated occlusion processing to produce more images of different sizes. For example, using mosaic technology, 1000 images with different occlusion positions are generated from 10 existing original images. Then, image scaling techniques are used again to compress or enlarge these 1000 images into 10 different sizes, thereby generating 10,000 images. Similarly, image stretching techniques can be used to generate even more images.

[0059] Furthermore, based on simulated occlusion processing and image scaling and stretching, this embodiment of the invention can also utilize image cropping technology to crop different regions of the image, thereby effectively solving the problem of repetitive backgrounds in an image. Thus, for a small number of original images, based on different image types, a large number of different types of image samples can be generated in a short time.

[0060] This invention provides a method for generating a large number of training samples for a computer vision model. The method involves acquiring multiple original images containing a target object; the target object being the image that the computer vision model needs to recognize; determining the image type of the target object; wherein the image type includes fixed-style images and non-fixed-style images; and using preset processing rules that match the image type of the target object, performing image expansion processing on the original images corresponding to the target object to obtain multiple image samples. This allows for the rapid generation of a large number of qualified, training image samples using only a small number of original images.

[0061] Furthermore, as Figure 1 In specific implementation, embodiments of the present invention provide a device for generating large batches of model training samples, such as... Figure 2 As shown, the device may include: an original image acquisition module 210, an image type determination module 220, and an image sample expansion module 230.

[0062] The original image acquisition module 210 can be used to acquire multiple original images containing a target object; the target object is the image that the computer vision model needs to recognize.

[0063] The image type determination module 220 can be used to determine the image type of the target object; wherein, the image type includes fixed-style images and non-fixed-style images; a fixed-style image is an image whose image style does not change with scene factors; a non-fixed-style image is an image whose image style changes with scene factors; wherein, scene factors include at least one of lighting, shooting angle, and image position;

[0064] Image sample expansion module 230 can be used to perform image expansion processing on the original image corresponding to the target object using preset processing rules that match the image type of the target object, so as to obtain multiple image samples.

[0065] Optionally, the image type determination module 220 can also be used to generate fingerprint codes corresponding to each of the multiple original images containing the same target object; compare the similarity of each fingerprint code corresponding to each original image; if the similarity exceeds a preset similarity threshold, the image type of the target object is a fixed-style image; if the similarity does not exceed the preset similarity threshold, the image type of the target object is a non-fixed-style image.

[0066] Optionally, the image sample extension module 230 can also be used to: when the image type of the target object is a fixed-style image, use image segmentation technology to perform image matting on the original image containing the target object to obtain the target image; acquire multiple background images, and use image fusion technology to merge the target image with each background image to obtain multiple fused images; determine whether the total number of fused images reaches a preset number of images; if the number of images does not reach the preset number of samples, repeatedly acquire multiple background images, and use image fusion technology to merge the target image with each background image to obtain multiple fused images until the number of images reaches the preset number of images; and / or determine whether the image quality of each fused image reaches a preset image quality; if the image quality reaches the preset image quality, use the fused image as an image sample; if the image quality does not reach the preset image quality, delete the fused image; wherein, the preset image quality includes at least the degree of distinction between the target image and the negative in the fused image.

[0067] Optionally, the image sample expansion module 230 can also be used to perform image expansion processing on the original image corresponding to the target object when the image type of the target object is a non-fixed style image, using simulated occlusion processing and / or image editing processing; wherein, simulated occlusion processing is to use mosaic technology to occlude the target object in the original image at random positions; and image scaling processing is to use image editing tools to scale, stretch, or crop the original image to obtain image samples of different sizes.

[0068] Optionally, the image type determination module 220 can also be used to determine whether the image positions of each target object are the same in the corresponding original image; if the image positions of each target object are the same in the corresponding original image, then the part of the original image corresponding to the image position is cropped and the part of the original image is updated to the original image of the target object.

[0069] It should be noted that other corresponding descriptions of the functional modules involved in the apparatus for generating large batches of model training samples provided in this embodiment of the invention can be found in [reference]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0070] Based on the above, Figure 1 Accordingly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for generating large batches of model training samples as described in any of the above embodiments.

[0071] Based on the above, Figure 1 The method shown and as Figure 2 The embodiment of the device shown in the invention also provides a physical structure diagram of a computer device, such as... Figure 3 As shown, the computer device may include a communication bus, a processor, a memory, and a communication interface. It may also include input / output interfaces and a display device. The various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory, performing the steps of the method for generating large batches of model training samples described in the above embodiments.

[0072] Those skilled in the art will clearly understand that the specific working process of the systems, devices, modules and units described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0073] Furthermore, the functional units in the various embodiments of the present invention can be physically independent of each other, or two or more functional units can be integrated together, or all functional units can be integrated into one processing unit. The integrated functional units described above can be implemented in hardware, or in software or firmware.

[0074] Those skilled in the art will understand that if the integrated functional unit is implemented in software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computing device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of the present invention when running the instructions. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as a computing device, personal computer, server, or network device) related to program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the computing device, the computing device executes all or part of the steps of the methods described in the various embodiments of the present invention.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to depart from the protection scope of the present invention.

Claims

1. A method for generating large batches of model training samples, characterized in that, The method includes: Acquire multiple original images containing the target object; the target object is the object that the computer vision model needs to identify. Determine the image type of the target object; wherein, the image type includes fixed-style images and non-fixed-style images; Using preset processing rules that match the image type of the target object, the original image corresponding to the target object is subjected to image expansion processing to obtain multiple image samples; Determining the image type of the target object includes: For multiple original images containing the same target object, generate fingerprint codes corresponding to each original image; Compare the similarity of each fingerprint code corresponding to each original image. If the similarity exceeds a preset similarity threshold, the image type of the target object is a fixed-style image. If the similarity does not exceed the preset similarity threshold, the image type of the target object is a non-fixed style image; The image type of the target object is a fixed-style image; the image expansion processing of the original image corresponding to the target object using preset processing rules that match the image type of the target object includes: Step S1: Use image segmentation technology to perform image cutout processing on the original image containing the target object to obtain the target image; Step S2: Obtain multiple base images, and use image fusion technology to merge the target image with each of the base images to obtain multiple fused images; The target object's image type is a non-fixed style image; the preset processing rules include simulated occlusion processing and / or image editing processing.

2. The method according to claim 1, characterized in that, The method further includes: Determine whether the total number of images in the fused images reaches a preset number of images; if the number of images does not reach the preset number of samples, repeat step S2 until the number of images reaches the preset number of images; and / or, Each of the fused images is determined to have a preset image quality. If the image quality meets the preset image quality, the fused image is used as an image sample. If the image quality does not meet the preset image quality, the fused image is deleted. The preset image quality includes at least the degree of distinction between the target image and the negative in the fused image.

3. The method according to claim 1, characterized in that, The simulated occlusion process involves using mosaic technology to occlude the target object in the original image at random locations. The image scaling process involves using image editing tools to scale, stretch, or crop the original image to obtain image samples of different sizes.

4. The method according to claim 1, characterized in that, Before generating the fingerprint codes corresponding to each of the original images, the method further includes: Determine whether the image positions of each target object are the same in the corresponding original image; If the image positions of each target object are the same in the corresponding original image, then the portion of the original image corresponding to the image position is cropped, and the portion of the original image is updated to the original image of the target object.

5. The method according to any one of claims 1 to 4, characterized in that, The fixed-style image is an image whose style does not change with scene factors; The non-fixed style image is an image whose style changes with scene factors; The scene factors include at least one of lighting, shooting angle, and image position.

6. An apparatus for generating large batches of model training samples, characterized in that, The device includes: The original image acquisition module is used to acquire multiple original images containing a target object; the target object is an image that the computer vision model needs to recognize. An image type determination module is used to determine the image type of the target object; wherein, the image type includes fixed-style images and non-fixed-style images; The image sample expansion module is used to perform image expansion processing on the original image corresponding to the target object using preset processing rules that match the image type of the target object, so as to obtain multiple image samples; The image type determination module is further configured to generate fingerprint codes corresponding to each of the multiple original images containing the same target object; compare the similarity of each fingerprint code corresponding to each original image; when the similarity exceeds a preset similarity threshold, define the image type of the target object as a fixed-style image; when the similarity does not exceed the preset similarity threshold, define the image type of the target object as a non-fixed-style image. The image sample expansion module is further configured to: when the image type of the target object is a fixed-style image; use image segmentation technology to perform image cutout processing on the original image containing the target object to obtain the target image; acquire multiple base images, and use image fusion technology to merge the target image with each of the base images to obtain multiple fused images; when the image type of the target object is a non-fixed-style image; the preset processing rules include simulated occlusion processing and / or image editing processing.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method for generating large batches of model training samples as described in any one of claims 1 to 5.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for generating large batches of model training samples as described in any one of claims 1 to 5.

Images