Image data enhancement method, device, terminal device and storage medium

Abstract

The present application is applicable to the field of image processing technology and provides an image data enhancement method, apparatus, terminal device, and storage medium, comprising: obtaining a preset number of source images; scaling each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images; wherein the reference size is larger than the input size corresponding to the source image; stitching and truncating each of the images to be stitched according to preset stitching points of a stitching image region to obtain a target stitching image; wherein the size of the stitching image region is the reference size; and generating a data-enhanced image of the input size based on the target stitching image. Embodiments of the present application can efficiently and accurately implement image data enhancement.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to an image data enhancement method, apparatus, terminal device, and storage medium. Background Art

[0002] With the development of artificial intelligence (AI), deep learning methods, which effectively enable machines to mimic human activities such as seeing, hearing, and thinking, have become widely used. Deep learning methods typically require big data, but collecting and labeling data is time-consuming and costly, necessitating data augmentation methods to obtain more data. Data augmentation is a technique that expands existing data to maximize its use and achieve optimal model performance.

[0003] Image-based deep learning methods also require image data augmentation to expand image data. However, current image data augmentation methods produce poor images. Summary of the Invention

[0004] In view of this, embodiments of the present application provide an image data enhancement method, apparatus, terminal device, and storage medium to solve the problem of how to efficiently and accurately achieve image data enhancement in the prior art.

[0005] A first aspect of an embodiment of the present application provides a method for image data enhancement, comprising:

[0006] Get a preset number of source images;

[0007] Scaling each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images; wherein the reference size is larger than the input size corresponding to the source image;

[0008] According to the preset stitching points of the stitching image area, stitching and truncating the images to be stitched together to obtain a target stitching image; wherein the size of the stitching image area is the reference size;

[0009] Generate a data augmented image with a size of the input size according to the target stitched image.

[0010] A second aspect of an embodiment of the present application provides an image data enhancement device, including:

[0011] A source image acquisition unit, configured to acquire a preset number of source images;

[0012] a scaling unit, configured to scale each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images; wherein the reference size is larger than an input size corresponding to the source image;

[0013] a stitching unit, configured to stitch and truncate each of the images to be stitched according to a preset stitching point of the stitching image area to obtain a target stitching image; wherein the size of the stitching image area is the reference size;

[0014] A generating unit is configured to generate a data augmented image having a size equal to the input size according to the target stitched image.

[0015] A third aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the terminal device implements the steps of the image data enhancement method.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the terminal device implements the steps of the image data enhancement method.

[0017] A fifth aspect of the embodiments of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the steps of the image data enhancement method as described in the first aspect.

[0018] Compared with the prior art, the embodiments of the present application have the following advantages: In the embodiments of the present application, a preset number of source images are first obtained, and then each source image is scaled according to a preset reference size to obtain images to be stitched, each of which is larger than the input size; thereafter, each image to be stitched is stitched and truncated according to the stitching points of the preset stitching image area to obtain a stitched image of the reference size; finally, the stitched image is scaled to the input size to obtain a data-enhanced image. Since the data-enhanced image can be obtained based on the stitching of a preset number of source images, the diversity of the image background can be enhanced and a more diverse image can be generated; and since the source images are first scaled to images to be stitched larger than the input size based on the reference size before stitching, that is, the source images are first enlarged in size before stitching, the effect of small target detection is avoided due to the reduction in the scale of the detection target caused by image stitching, thereby improving the image effect of the generated data-enhanced image and improving the accuracy of subsequent image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is a schematic diagram of an implementation flow of an image data enhancement method provided in an embodiment of the present application;

[0021] Figure 2 This is an example diagram of a spliced ​​image area provided in an embodiment of the present application;

[0022] Figure 3 This is another example diagram of a stitched image area provided in an embodiment of the present application;

[0023] Figure 4 This is an example diagram of a spliced ​​image provided by an embodiment of the present application;

[0024] Figure 5 is an example diagram of a target stitching image provided in an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of an image data enhancement device provided in an embodiment of the present application;

[0026] Figure 7 It is a schematic diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0028] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0029] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0030] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0031] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0032] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0033] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0034] Currently, image-based deep learning methods require image data augmentation to expand image data. Traditional color or geometric shape augmentation methods typically transform a single image in both color and geometric scale space, leaving the image background unchanged. Advanced data augmentation methods, such as cutout (a data augmentation method that randomly adds grayscale to a small region of a single image), mixup (a data augmentation method that superimposes channels of two images at a certain ratio), and mosaic (a data augmentation method that combines four images into a single image using a 2x2 stitching scheme with variable stitching centers), enhance image content diversity to a certain extent. However, cutout methods rarely enhance background diversity, while mixup methods produce aliased images that deviate significantly from the real image. While mosaic methods enhance background diversity, the stitching reduces the object scale by a factor of 2 compared to the single image, making small objects even smaller and significantly impacting detection. Consequently, current image data augmentation methods produce poor quality images.

[0035] In order to solve the above-mentioned technical problems, an embodiment of the present application provides an image data enhancement method, apparatus, terminal device and storage medium, including: obtaining a preset number of source images; scaling each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images; wherein the reference size is larger than the input size corresponding to the source image; according to preset stitching points of a stitching image area, stitching and truncating each of the images to be stitched to obtain a target stitching image; wherein the size of the stitching image area is the reference size; scaling the target stitching image to the input size to obtain a data enhanced image.

[0036] Since a data-enhanced image can be obtained based on the splicing of a preset number of source images, the diversity of the image background can be enhanced and a more diverse image can be generated. Moreover, since the source image is first scaled to an image to be spliced ​​that is larger than the input size based on a preset size before splicing, that is, the source image is first enlarged in size before splicing, the problem of poor detection performance of small targets in the mosaic data enhancement method can be avoided. While enriching the diversity of the training input image background, the target scale within the image is kept within a reasonable range as much as possible, thereby balancing the performance of the network on large and small targets. At the same time, it can also avoid the presence of a large number of blank areas in the spliced ​​image, avoid wasting computing resources, and thus improve the convergence speed of the network trained based on the data-enhanced image. That is, through the data enhancement method of the image of the embodiment of the present application, the image effect of the generated data-enhanced image can be improved, and the accuracy of subsequent image recognition can be improved.

[0037] In one application scenario, the image data augmentation method of the embodiments of the present application can expand the sample data of the target detection network. That is, the image data augmentation method can accurately generate data-enhanced images suitable for training the target detection network, thereby improving the accuracy of target detection network training and the accuracy of subsequent target detection by the target detection network. The target detection network can be a neural network used to detect human faces, vehicles, or product targets on a production line.

[0038] Example 1:

[0039] Figure 1 A flow chart of an image data enhancement method provided in an embodiment of the present application is shown, and is described in detail as follows:

[0040] In S101 , a preset number of source images are acquired.

[0041] In an embodiment of the present application, the source image is an original sample image containing a detection target. The size of the source image is referred to as the input size, which refers to the image input size required by the target detection network using the source image as a sample image. In one embodiment, the size of the longest side of the source image is equal to the image input size required by the target detection network using the source image as a sample image. For example, if the image input size required by the target detection network is: the longest side of the input image is d, then the input size of the source image is the longest side d.

[0042] In the embodiment of the present application, a preset number of source images of the input size may be obtained from the sample database, where the preset number is greater than 2. For example, the preset number may be 4.

[0043] In S102 , each of the source images is scaled according to a preset reference size to obtain an image to be stitched corresponding to each of the source images; wherein the reference size is larger than an input size corresponding to the source image.

[0044] In this embodiment of the present application, after obtaining a preset number of source images, a reference size larger than the input size corresponding to the source images is first determined. Each source image is then scaled according to the reference size to obtain the corresponding image to be stitched. Because the reference size is larger than the input size, the scaling process here is specifically image magnification, resulting in the size of the detection target contained in the image to be stitched being larger than the original size of the detection target in the corresponding source image.

[0045] In S103 , the images to be stitched are stitched and truncated according to the preset stitching points of the stitching image area to obtain a target stitching image; wherein the size of the stitching image area is the reference size.

[0046] In the embodiment of the present application, the stitching image area is a blank image area with a pre-set size as a reference size. Before step S103, one or more pixel points can be randomly determined in the stitching image area as the stitching points of the stitching image area. The stitching points can divide the stitching image area into a preset number of areas to be filled with the images to be stitched corresponding to the respective source images. In one embodiment, the preset number is 4, and the number of stitching points set in the stitching image area is 1, then the stitching points can divide the stitching image area into 4 image areas, such as Figure 2 In another embodiment, the preset number is 9, and the number of stitching points set in the stitching image area is 2, then the stitching points can divide the stitching image area into 9 image areas.

[0047] After determining the images to be stitched, the images to be stitched are stitched together with the stitching point of the preset stitching image area as the center. Since the size of the preset stitching image area is the reference size, and each image to be stitched is scaled based on the reference size, i.e., the size of each image to be stitched is close to or equal to the reference size, it can be seen that the sum of the sizes of the images to be stitched is greater than the size of the stitching image area. Therefore, after stitching the images to be stitched together within the stitching image area with the stitching point as the center, there will be image areas within the images to be stitched that are outside the stitching image area. In this case, for each image to be stitched, the image areas outside the stitching image area are truncated and deleted to obtain the target stitched image with the reference size.

[0048] In S104 , a data augmented image having a size equal to the input size is generated according to the target stitched image.

[0049] After stitching together a target stitched image of the reference size, the target stitched image can be scaled or affine transformed to obtain a data-augmented image of the input size. This data-augmented image is a newly generated sample image that can serve as a sample for the target detection network based on a preset number of source images, thereby expanding the sample data of the target detection network. For example, assuming the side length of the reference size is 2d and the side length of the input size is d, the original 2d×2d target stitched image can be transformed to a d×d image size through an affine transformation to obtain a data-augmented image.

[0050] In the embodiment of the present application, since a data-enhanced image can be obtained based on the splicing of a preset number of source images, the diversity of the image background can be enhanced and a more diverse image can be generated; and since the source image is first scaled to an image to be spliced ​​that is larger than the input size based on a preset size before splicing, that is, the source image is first enlarged in size before splicing, the problem of poor detection performance of small targets in the mosaic data enhancement method can be avoided. While enriching the diversity of the background of the training input image, the target scale within the image is kept within a reasonable range as much as possible, thereby balancing the performance of the network on large and small targets; at the same time, it can also avoid the presence of a large number of blank areas in the spliced ​​image, avoid wasting computing resources, and thus improve the convergence speed of the network trained based on the data-enhanced image. That is, through the data enhancement method of the image in the embodiment of the present application, the image effect of the generated data-enhanced image can be improved, and the accuracy of subsequent image recognition can be improved.

[0051] Optionally, before scaling the source images according to a preset reference size to obtain the images to be stitched corresponding to the source images, the method further includes:

[0052] The input size is multiplied by a preset multiple to obtain a reference size.

[0053] In the embodiment of the present application, after determining the input size corresponding to the source image, the input size is multiplied by a preset multiple to obtain the reference size. The preset multiple can be determined based on a preset number. For example, when the preset number is 4, the preset multiple can be 2 (i.e., assuming the long side of the input size is d, the long side of the reference size is 2d); when the preset number is 9, the preset multiple can be 3.

[0054] Since the input size can be multiplied by a preset multiple to obtain a reference size, each source image can be magnified exponentially based on the reference size. As a result, after being spliced ​​and scaled to the input size, the size of the detection target in the source image is basically not reduced, thereby improving the image effect of the generated data-augmented image and improving the accuracy of subsequent small target detection.

[0055] Optionally, the scaling process is performed on each of the source images according to a preset reference size to obtain the images to be stitched corresponding to each of the source images, including:

[0056] For each source image, a target size corresponding to the source image is determined according to the reference size and the size of the detection target contained in the source image; the source image is scaled to the target size to obtain an image to be stitched corresponding to the source image.

[0057] In an embodiment of the present application, for each source image, a target size that is floating at a certain scale based on the reference size is determined as the target size to be scaled to for the source image, respectively, based on the size distribution of the detection targets contained in the source image. In one embodiment, the smaller the size of the detection target contained in the source image, the larger the determined target size; conversely, the larger the size of the detection target contained in the source image, the smaller the determined target size. That is, for detection targets with smaller sizes, the source image is enlarged to a larger target size to improve the subsequent detection effect of the small-sized detection target; and for detection targets with larger sizes, the relatively small degree of enlargement will not affect the detection effect, and the smaller degree of enlargement can reserve more image enlargement space for other source images containing detection targets with smaller sizes.

[0058] For each source image, after determining the target size according to the size distribution of the detection targets contained therein, the source image is scaled to the target size to obtain the image to be stitched corresponding to the source image.

[0059] In an embodiment of the present application, since the source image can be adaptively scaled to a corresponding scale according to the size of the detection target contained in the source image, the scale space occupied by the detection target of each source image in the stitched image area is more balanced and reasonable, thereby further improving the effect of the generated data-enhanced image and improving the accuracy of subsequent target recognition.

[0060] Optionally, determining the target size corresponding to the source image according to the reference size and the size of the detection target contained in the source image includes:

[0061] If the number of small targets contained in the source image is greater than the number of large targets, a size larger than the reference size is determined as the target size corresponding to the source image; wherein the small target is a detection target whose size is less than or equal to a preset size threshold, and the large target is a detection target whose size is greater than the preset size threshold.

[0062] In an embodiment of the present application, each detection target contained in a source image is classified into small targets and large targets based on a preset size threshold. Small targets are targets whose size is less than or equal to the preset size threshold, and large targets are targets whose size is greater than the preset size threshold. In one embodiment, the preset size threshold can be determined based on a reference size, for example, the preset size threshold can be one-tenth of the reference size.

[0063] For each source image, if the number of small objects detected in the source image is greater than the number of large objects, it indicates that the current source image contains a large number of small objects. In this case, the source image size can be appropriately increased. Therefore, a size slightly larger than the reference size is determined as the target size. For example, if the reference size is 2d, then Δd can be added to the reference size to obtain the target size, where Δd can be a real number greater than 0 and less than d.

[0064] In an embodiment of the present application, since a size larger than the reference size is determined as the target size when the source image contains a large number of small targets, the scale of the source image can be magnified to a greater extent, thereby improving the data enhancement effect of the source image and improving the subsequent small target detection effect.

[0065] Optionally, determining the target size corresponding to the source image according to the reference size and the size of the detection target contained in the source image includes:

[0066] If the number of the large objects contained in the source image is greater than the number of the small objects, a size between the reference size and the input size is determined as the object size corresponding to the source image.

[0067] Conversely, for each source image, if the number of large objects detected in the source image is greater than the number of small objects, it indicates that the current source image contains a large number of large objects. In this case, the source image size can be appropriately reduced. Therefore, a size between the reference size and the input size is determined as the target size corresponding to the source image. This ensures that the source image can be enlarged based on the input size while ensuring that the enlargement degree is smaller than the reference size. For example, let the input size be d and the reference size be 2d, then: d < target size < 2d.

[0068] In an embodiment of the present application, since the source image contains a large number of large targets, a size between the reference size and the input size is determined as the target size, the magnification degree of the source image can be reduced to better balance the scale space of the subsequent target stitching image and the data enhancement image, thereby improving the data enhancement effect of the image.

[0069] Optionally, at the stitching points of the preset stitching image area, stitching and truncating the images to be stitched to obtain a target stitching image includes:

[0070] According to the preset stitching points of the stitching image area, stitching and truncating the images to be stitched together to obtain an initial stitching image;

[0071] For each image area in the initial stitched image corresponding to the source image, the detection targets in the image area that meet the preset filtering conditions are filtered out to obtain a target stitched image; wherein the preset filtering conditions include any one or more of the following: the aspect ratio of the detection target is less than a first preset threshold, the width is less than a second preset threshold, the height is less than a third preset threshold, and the ratio of the area occupied by the detection target after the source image is truncation to the area occupied by the detection target before the source image is truncation is less than a fourth preset threshold.

[0072] In the embodiment of the present application, after obtaining the images to be stitched, stitching and truncation are performed according to stitching points randomly determined in the stitching image area, and the resulting preliminary stitched image is called an initial stitched image.

[0073] Since the initial stitched image is obtained after truncation, some detection targets may be located just at the boundary of the stitched image area and are truncated, leaving only a small part of them. Therefore, after obtaining the initial stitched image, the embodiment of the present application further filters out the detection targets with low quality caused by truncation, so as to avoid these low-quality detection targets from interfering with the subsequent training of the target detection network. Specifically, after obtaining the initial stitched image, for each image area corresponding to the source image in the initial stitched image, that is, the stitched area of ​​the source image, it is detected whether there is an inspection target that meets the preset filtering conditions; if so, the detection target is filtered out as a low-quality detection target. After the image areas corresponding to each source image in the initial stitched image are all subjected to the low-quality detection target filtering operation, a target stitched image with better quality can be obtained.

[0074] The above-mentioned preset filtering conditions may include any one or more of the following:

[0075] (1) The aspect ratio of the detection target is less than a first preset threshold. When the aspect ratio of the detection target is less than the first preset threshold, it means that the detection target may be truncated, resulting in only a small and elongated area. In this case, it is judged as a low-quality detection target.

[0076] (2) The width is less than the second preset threshold and / or the height is less than the third preset threshold. When the width and / or height of the detection target is too small, it means that a large part of the detection target may be cut off in the width direction or the height direction, resulting in the remaining area being insufficient to accurately represent the original detection target features. Therefore, it is judged as a low-quality detection target.

[0077] (3) The ratio of the area occupied by the detection target after the source image is truncated to the area occupied by the detection target before the source image is truncated is less than a fourth preset threshold. Assume that the area occupied by the detection target in the source image before truncation is S1, and the area occupied by the detection target after truncation is S2. If the area ratio S2 / S1 is less than the fourth preset threshold, it means that most of the area of ​​the current detection target has been truncated, resulting in the remaining area being insufficient to accurately represent the original detection target features. Therefore, it is determined to be a low-quality detection target.

[0078] In an embodiment of the present application, since low-quality detection targets caused by scaling and truncation in each source image can be filtered out through preset filtering conditions, the quality of the detection targets contained in the data-enhanced image can be guaranteed, and the training effect and detection effect of the target inspection network subsequently trained based on the data-enhanced image can be improved, thereby reducing the probability of target false detection.

[0079] Optionally, generating a data augmented image having a size of the input size according to the target stitched image includes:

[0080] Performing color transformation and / or geometric transformation on the target stitched image to obtain a transformed image;

[0081] A data augmented image having a size of the input size is generated according to the transformed image.

[0082] In an embodiment of the present application, after obtaining the target stitched image, the target stitched image can be further transformed to obtain a transformed image. The transformation process includes color transformation and / or geometric transformation. Color transformation can include grayscale processing, brightness dithering, contrast dithering, etc., and color transformation can further enhance the color of the image data. Geometric transformation can include translation, rotation, and other transformation processes, and geometric shape enhancement can further be achieved through geometric transformation.

[0083] After obtaining the transformed image, the transformed image is scaled or affine transformed to the input size to obtain a data augmented image.

[0084] In an embodiment of the present application, after obtaining the target stitched image, the target stitched image can be further subjected to color transformation processing and / or geometric transformation processing, and after obtaining the transformed image, a data enhanced image can be obtained based on the transformed image. Therefore, it is possible to combine color enhancement and / or geometric shape enhancement to obtain a data enhanced image with richer image information, further improve the data enhancement effect of the image, and provide richer sample images for subsequent target detection network training, thereby improving the recognition effect of the target detection network.

[0085] In order to further illustrate the effect of the embodiment of the present application, a specific example is given below:

[0086] Assume that the input size of the source image is: the longest side is d, the input source images include four source images A, B, C, and D, and the base size is: 2d×2d (i.e. the side length is 2d). Under this condition, if the input source images A, B, C, and D are directly spliced ​​according to the original input size with the longest side being d, the schematic diagram of the spliced ​​image is as follows Figure 4 As shown. Figure 4 When the stitched image with a side length of 2d is transformed to the input size (i.e., the size of d×d), each source image in the stitched image is also reduced to half of its original size, so that the scale of the detection target contained in the source image is also reduced by half, resulting in a poor display effect of the detection target (especially small targets) in the final image. By using the image data enhancement method of the embodiment of the present application, each source image is first enlarged to a size with the longest side of 2d based on the reference size, and then the image is stitched, and the following can be obtained: Figure 5The stitched image shown in the figure has a square frame with a side length of 2d, which is the stitched image area. By truncating the image area outside the stitched image area, a target stitched image with a side length of 2d can be obtained. Afterwards, when the target stitched image is transformed into a data augmented image with the input size, the size of the source image is transformed from 2d to the original input size d, so that the scale of the detection target contained in the source image remains unchanged, thereby improving the image display effect and facilitating the subsequent small target detection. At the same time, Figure 5 The blank area of ​​the target stitched image is also smaller than Figure 4 As shown in the blank areas of the stitched image, it can be seen that the data enhanced image finally obtained by the method of the embodiment of the present application has fewer blank areas and can provide richer image information.

[0087] In summary, it can be seen that through the image data enhancement method of the embodiment of the present application, data-enhanced images with richer background information and more balanced target scales can be obtained, providing richer and higher-quality training samples for the target detection network, and significantly improving the detection effect of the target detection network on small targets.

[0088] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0089] Example 2:

[0090] Figure 6 A schematic diagram of the structure of an image data enhancement device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown:

[0091] The image data enhancement device includes: a source image acquisition unit 61, a scaling unit 62, a splicing unit 63 and a generating unit 64.

[0092] The source image acquisition unit 61 is configured to acquire a preset number of source images.

[0093] The scaling unit 62 is configured to scale each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images; wherein the reference size is larger than the input size corresponding to the source image.

[0094] The stitching unit 63 is configured to stitch and truncate the images to be stitched according to the preset stitching points of the stitching image area to obtain a target stitching image; wherein the size of the stitching image area is the reference size.

[0095] The generating unit 64 is configured to generate a data-enhanced image having a size equal to the input size according to the target stitched image.

[0096] Optionally, the image data enhancement device further includes:

[0097] The reference size determining unit is configured to multiply the input size by a preset multiple to obtain a reference size.

[0098] Optionally, the scaling unit 62 is specifically configured to determine, for each source image, a target size corresponding to the source image based on the reference size and the size of the detection target contained in the source image; and scale the source image to the target size to obtain an image to be stitched corresponding to the source image.

[0099] Optionally, in the scaling unit 62, determining the target size corresponding to the source image according to the reference size and the size of the detection target contained in the source image includes:

[0100] If the number of small targets contained in the source image is greater than the number of large targets, a size larger than the reference size is determined as the target size corresponding to the source image; wherein the small target is a detection target whose size is less than or equal to a preset size threshold, and the large target is a detection target whose size is greater than the preset size threshold.

[0101] Optionally, in the scaling unit 62, determining the target size corresponding to the source image according to the reference size and the size of the detection target contained in the source image includes:

[0102] If the number of the large objects contained in the source image is greater than the number of the small objects, a size between the reference size and the input size is determined as the object size corresponding to the source image.

[0103] Optionally, the stitching unit 63 is specifically configured to stitch and truncate each of the images to be stitched according to a preset stitching point of the stitching image area to obtain an initial stitched image; for each image area in the initial stitched image corresponding to the source image, filter out the detection targets in the image area that meet a preset filtering condition to obtain a target stitched image; wherein the preset filtering condition includes any one or more of the following: the aspect ratio of the detection target is less than a first preset threshold, the width is less than a second preset threshold, the height is less than a third preset threshold, and the ratio of the area occupied by the detection target after the source image is truncation to the area occupied by the detection target before the source image is truncation is less than a fourth preset threshold.

[0104] Optionally, the generating unit 64 is specifically configured to perform color transformation processing and / or geometric transformation processing on the target stitched image to obtain a transformed image; and generate a data enhanced image having a size equal to the input size based on the transformed image.

[0105] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0106] Example 3:

[0107] Figure 7 This is a schematic diagram of a terminal device provided by an embodiment of the present application. Figure 7 As shown, the terminal device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as an image data enhancement program. When the processor 70 executes the computer program 72, the steps in the above-mentioned various image data enhancement method embodiments are implemented, such as Figure 1 Alternatively, when the processor 70 executes the computer program 72, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 6 The functions of the source image acquisition unit 61 to the generation unit 64 are shown.

[0108] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 72 in the terminal device 7. For example, the computer program 72 may be divided into a source image acquisition unit, a scaling unit, a splicing unit, and a generation unit, with the specific functions of each unit being as follows:

[0109] The source image acquisition unit is used to acquire a preset number of source images.

[0110] The scaling unit is configured to scale each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images; wherein the reference size is larger than the input size corresponding to the source image.

[0111] The stitching unit is configured to stitch and truncate the images to be stitched according to a preset stitching point of a stitching image area to obtain a target stitching image; wherein the size of the stitching image area is the reference size.

[0112] A generating unit is configured to generate a data augmented image having a size equal to the input size according to the target stitched image.

[0113] The terminal device 7 can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device can include, but is not limited to, a processor 70 and a memory 71. It can be understood by those skilled in the art that Figure 7 It is only an example of the terminal device 7 and does not constitute a limitation on the terminal device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0114] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0115] The memory 71 may be an internal storage unit of the terminal device 7, such as a hard disk or memory of the terminal device 7. The memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Furthermore, the memory 71 may include both an internal storage unit of the terminal device 7 and an external storage device. The memory 71 is used to store the computer program and other programs and data required by the terminal device. The memory 71 may also be used to temporarily store data that has been output or is about to be output.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0117] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0118] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0121] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0122] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0123] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A data enhancement method for an image, characterized in that: include: Get a preset number of source images; Scaling each of the source images according to a preset reference size to obtain an image to be stitched that is larger than the input size and corresponds to each of the source images; wherein the reference size is larger than the input size corresponding to the source image; According to the preset stitching points of the stitching image area, stitching and truncating the images to be stitched that are larger than the input size to obtain a target stitching image; wherein the size of the stitching image area is the reference size; A data augmented image having a size of the input size is generated according to the target stitched image; the size of the target in the data augmented image is within a reasonable range.

2. The image data enhancement method according to claim 1, wherein: Before scaling the source images according to the preset reference size to obtain the images to be stitched corresponding to the source images, the method further includes: The input size is multiplied by a preset multiple to obtain a reference size.

3. The image data enhancement method according to claim 1, wherein: The step of scaling each of the source images according to a preset reference size to obtain images to be stitched corresponding to each of the source images includes: For each source image, a target size corresponding to the source image is determined according to the reference size and the size of the detection target contained in the source image; the source image is scaled to the target size to obtain an image to be stitched corresponding to the source image.

4. The image data enhancement method according to claim 3, wherein: The determining, based on the reference size and the size of the detection target contained in the source image, the target size corresponding to the source image includes: If the number of small targets contained in the source image is greater than the number of large targets, a size larger than the reference size is determined as the target size corresponding to the source image; wherein the small target is a detection target whose size is less than or equal to a preset size threshold, and the large target is a detection target whose size is greater than the preset size threshold.

5. The image data enhancement method according to claim 4, wherein: The determining, based on the reference size and the size of the detection target contained in the source image, the target size corresponding to the source image includes: If the number of the large objects contained in the source image is greater than the number of the small objects, a size between the reference size and the input size is determined as the object size corresponding to the source image.

6. The image data enhancement method according to claim 1, wherein: The method of performing stitching and truncation processing on each of the images to be stitched according to the preset stitching points of the stitching image area to obtain a target stitching image includes: According to the preset stitching points of the stitching image area, stitching and truncating the images to be stitched together to obtain an initial stitching image; For each image area in the initial stitched image corresponding to the source image, the detection targets in the image area that meet the preset filtering conditions are filtered out to obtain a target stitched image; wherein the preset filtering conditions include any one or more of the following: the aspect ratio of the detection target is less than a first preset threshold, the width is less than a second preset threshold, the height is less than a third preset threshold, and the ratio of the area occupied by the detection target after the source image is truncation to the area occupied by the detection target before the source image is truncation is less than a fourth preset threshold.

7. The image data enhancement method according to any one of claims 1 to 6, wherein: Generating a data augmented image having a size of the input size according to the target stitched image includes: Performing color transformation and / or geometric transformation on the target stitched image to obtain a transformed image; A data augmented image having a size of the input size is generated according to the transformed image.

8. An image data enhancement device, characterized in that: include: A source image acquisition unit, configured to acquire a preset number of source images; A scaling unit, configured to scale each of the source images according to a preset reference size to obtain an image to be stitched that is larger than an input size and corresponds to each of the source images; wherein the reference size is larger than the input size corresponding to the source image; a stitching unit, configured to stitch and truncate the images to be stitched that are larger than the input size according to a preset stitching point of the stitching image area, to obtain a target stitching image; wherein the size of the stitching image area is the reference size; A generating unit is configured to generate a data augmented image having a size of the input size based on the target stitched image; the size of the target in the data augmented image is within a reasonable range.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the terminal device implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the terminal device implements the steps of the method according to any one of claims 1 to 7.

Images