A data enhancement method and device, electronic equipment and storage medium
By combining global and local data augmentation, the problems of low training data diversity and model overfitting were solved, thereby improving the model's generalization ability and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU HIKROBOT TECH CO LTD
- Filing Date
- 2023-10-31
- Publication Date
- 2026-08-04
AI Technical Summary
Existing data augmentation techniques suffer from low training data diversity, cannot effectively prevent model overfitting, and cannot improve the model's generalization ability and robustness.
After processing the image using a global data augmentation method, the object region is extracted as the second object region, and then local data augmentation processing is performed, combining global and local data augmentation methods.
It increases the diversity of training data, effectively avoids model overfitting, and improves the model's generalization ability and robustness.
Smart Images
Figure CN117409181B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image data processing technology, and in particular to a data enhancement method, apparatus, electronic device, and storage medium. Background Technology
[0002] Data augmentation is a common machine learning technique that can be applied to various machine learning tasks, especially when the amount of data is small or the data distribution is imbalanced. Data augmentation can expand the dataset, improving the model's generalization ability and robustness. Examples of data augmentation applications include image classification, object detection, image segmentation, text classification, and speech recognition.
[0003] In commonly used data augmentation techniques, the image of the region containing the object is usually copied from the global image to obtain a local image, which is then augmented. However, augmenting only the local image results in low diversity of training data, and with a limited amount of training data, it is also impossible to effectively avoid model overfitting, let alone effectively improve the model's generalization ability and robustness. Summary of the Invention
[0004] The purpose of this application is to provide a data augmentation method, apparatus, electronic device, and storage medium to solve the problems of low training data diversity, inability to effectively avoid model overfitting, and inability to improve the generalization ability and robustness of existing methods. The specific technical solution is as follows:
[0005] In a first aspect, embodiments of this application provide a data augmentation method, the method comprising:
[0006] Obtain a first image to be processed that is labeled with a first object region, wherein the first object region is the region in the first image to be processed where the object is located;
[0007] Global data augmentation processing is performed on the first image to be processed to obtain a first data augmented image, and the region of the first object region in the first data augmented image is obtained as the second object region;
[0008] Extract the image within the second object region from the first data-enhanced image, and use it as the second image to be processed;
[0009] The second image to be processed is subjected to local data augmentation to obtain a second data-enhanced image.
[0010] Optionally, the global data augmentation process includes at least one or more of the following operations:
[0011] The first image to be processed is flipped, the first image to be processed is randomly perturbed, the first object region is rotated, and the first object region is expanded.
[0012] Optionally, the random perturbation of the first image to be processed includes at least one or more of the following operations:
[0013] Add noise to the first image to be processed, adjust the brightness of the first image to be processed, adjust the contrast of the first image to be processed, and adjust the resolution of the first image to be processed.
[0014] Optionally, obtaining the region of the first object region in the first data-enhanced image as the second object region includes:
[0015] An object is identified in the first data-enhanced image using a preset algorithm, and the region of the object in the first data-enhanced image is obtained as the second object region.
[0016] Optionally, obtaining the region of the first object region in the first data-enhanced image as the second object region includes:
[0017] Based on the first object region and the transformation rule, the region of the object in the first data-enhanced image is determined as the second object region. The transformation rule is related to the operation type of the global data enhancement process.
[0018] Optionally, the local data augmentation process includes at least one or more of the following operations:
[0019] The process involves flipping the second image to be processed, adding noise to the second image to be processed, adjusting the brightness of the second image to be processed, adjusting the contrast of the second image to be processed, adjusting the resolution of the second image to be processed, performing a cut-Mix operation on the second image to be processed, and performing a full-image mixup operation on the second image to be processed.
[0020] Secondly, embodiments of this application provide a data enhancement apparatus, the apparatus comprising:
[0021] The acquisition module is used to acquire a first image to be processed that is marked with a first object region, wherein the first object region is the region in the first image to be processed where the object is located;
[0022] The first data augmentation module is used to perform global data augmentation processing on the first image to be processed to obtain a first data augmented image, and to obtain the region of the first object region in the first data augmented image as the second object region;
[0023] The extraction module is used to extract the image within the second object region from the first data-enhanced image as the second image to be processed;
[0024] The second data augmentation module is used to perform local data augmentation processing on the second image to be processed to obtain a second data-enhanced image.
[0025] Optionally, the global data augmentation process includes at least one or more of the following operations:
[0026] The first image to be processed is flipped, the first image to be processed is randomly perturbed, the first object region is rotated, and the first object region is expanded.
[0027] Optionally, the random perturbation of the first image to be processed includes at least one or more of the following operations:
[0028] Add noise to the first image to be processed, adjust the brightness of the first image to be processed, adjust the contrast of the first image to be processed, and adjust the resolution of the first image to be processed.
[0029] Optionally, the first data augmentation module is used for:
[0030] An object is identified in the first data-enhanced image using a preset algorithm, and the region of the object in the first data-enhanced image is obtained as the second object region.
[0031] Optionally, the first data augmentation module is used for:
[0032] Based on the first object region and the transformation rule, the region of the object in the first data-enhanced image is determined as the second object region. The transformation rule is related to the operation type of the global data enhancement process.
[0033] Optionally, the local data augmentation process includes at least one or more of the following operations:
[0034] The process involves flipping the second image to be processed, adding noise to the second image to be processed, adjusting the brightness of the second image to be processed, adjusting the contrast of the second image to be processed, adjusting the resolution of the second image to be processed, performing a cut-Mix operation on the second image to be processed, and performing a full-image mixup operation on the second image to be processed.
[0035] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory;
[0036] The memory is used to store computer programs;
[0037] When the processor executes the program stored in the memory, it implements the data augmentation method described in this application.
[0038] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the data augmentation method described in this application.
[0039] Beneficial effects of the embodiments in this application:
[0040] The data augmentation method, apparatus, electronic device, and storage medium provided in this application embodiment perform global data augmentation processing on a first image to be processed, which is labeled with a first object region, to obtain a first data-augmented image. An image including the region of the first object region within the first data-augmented image is then acquired and extracted as a second image to be processed. Local data augmentation processing is then performed on the second image to obtain a second data-augmented image. Compared to performing data augmentation processing only on a local image, by performing global data augmentation processing on the global image and then extracting the region of the first object region within the first data-augmented image as the second object region, subsequent local data augmentation processing can be performed on the image of the second object region. This provides a data augmentation method that combines global and local data augmentation, improving the diversity of training data during training and effectively avoiding model overfitting, thus enhancing the model's generalization ability and robustness.
[0041] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.
[0043] Figure 1 This is a schematic diagram of a data augmentation method according to an embodiment of this application;
[0044] Figure 2a This is a schematic diagram illustrating the horizontal flipping of a first image to be processed according to an embodiment of this application.
[0045] Figure 2b This is a schematic diagram illustrating the vertical flipping of a first image to be processed according to an embodiment of this application.
[0046] Figure 3This is a schematic diagram illustrating rotating a first object region in a first image to be processed by a preset angle according to an embodiment of this application.
[0047] Figure 4 This is a schematic diagram illustrating the expansion of a first object region in a first image to be processed according to an embodiment of this application.
[0048] Figure 5 This is a schematic diagram illustrating global data augmentation processing of a first image to be processed according to an embodiment of this application.
[0049] Figure 6 This is a schematic diagram of local data enhancement processing of a second image to be processed according to an embodiment of this application;
[0050] Figure 7 This is a schematic diagram of a data enhancement device according to an embodiment of this application;
[0051] Figure 8 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.
[0053] First, the technical terms used in this application will be explained.
[0054] Data augmentation: In the process of training machine learning and deep learning models, the input data is transformed to increase the diversity of the training data.
[0055] Global data augmentation: Data augmentation operations performed on the image before copying the target region. For example, in a fabric defect detection task, if the defective area only occupies a small portion of the fabric area, then the data augmentation operation performed on the entire fabric image is a global data augmentation operation.
[0056] Local data augmentation: This refers to data augmentation operations performed on the image of the target region after copying it from the image. For example, in a fabric defect detection task, the defect area is located using a bounding rectangle, and the region within the bounding rectangle is copied from the image. The data augmentation operation performed on the copied target region is called local data augmentation.
[0057] In related techniques, a local image including the target region is copied from the global image, and data augmentation is performed only on this local image. However, data augmentation on only a local image leads to low diversity in the training data, and with a small amount of training data, it is also impossible to effectively avoid model overfitting, let alone effectively improve the model's generalization ability and robustness.
[0058] In view of this, embodiments of this application provide a data augmentation method, see below. Figure 1 The method includes:
[0059] S101, Obtain the first image to be processed marked with the first object region, wherein the first object region is the region where the object is located in the first image to be processed.
[0060] The first image to be processed is a sample image that can be used for machine learning tasks. For example, the first image to be processed is an image containing a defective region obtained during defect detection in an industrial scene, and the first object region is the region where the defective region is located in the first image to be processed.
[0061] Labeling the first object region in the first image to be processed is essentially for the purpose of locating the objects in the first image to be processed. In one possible embodiment, a bounding rectangle of a preset size can be used to label the objects in the first image to be processed, thereby obtaining the first object region, which is used to label the objects, and a label is added to the bounding rectangle to identify the object category of the objects within the bounding rectangle.
[0062] S102, perform global data augmentation processing on the first image to be processed to obtain a first data augmented image, and obtain the region of the first object region in the first data augmented image as the second object region.
[0063] After performing global data augmentation on the first image to be processed, the region where the object is located in the first image to be processed will change. For example, taking an image containing defective regions as an example, after performing global data augmentation, the defective regions in the first image to be processed will change. For example, as... Figure 2aAs shown, in the first image to be processed without global data augmentation, the object's location is in the upper left region of the image, while in the first image to be processed with global data augmentation, the object's location is in the upper right region of the image. Therefore, while performing global data augmentation on the first image to be processed to obtain the first data-augmented image, it is also necessary to obtain the region of the first object region within the first data-augmented image and use it as the second object region so that subsequent local data augmentation processing can be performed on the image extracted based on the second object region. For example, after performing global data augmentation on an image containing a defective region, it is also necessary to obtain the region of the defective region within the image after global data augmentation and use it as the second object region.
[0064] S103, extract the image within the second object region from the first data-enhanced image as the second image to be processed.
[0065] In this embodiment, after performing global data augmentation on a first image to be processed to obtain a first data-enhanced image, local data augmentation is performed on a second object region within the first data-enhanced image. Before performing local data augmentation on the second object region in the first data-enhanced image, an image of the second object region can be extracted from the first data-enhanced image using image data copying technology. This extracted image of the second object region is then used as the second image to be processed, so that local data augmentation can be performed on the second image to be processed subsequently. For example, an image of the defective region can be extracted from an image containing a defective region that has undergone global data augmentation using image data copying technology. This extracted image of the defective region is then used as the image to be processed, and local data augmentation is subsequently performed on the image to be processed.
[0066] S104, perform local data augmentation processing on the second image to be processed to obtain the second data-enhanced image.
[0067] In one possible embodiment, local data augmentation is performed on an image containing defective regions extracted from an image that has undergone global data augmentation to obtain an image with local data augmentation.
[0068] In this embodiment, a first data-augmented image is obtained by performing global data augmentation on a first image to be processed, which is labeled with a first object region. An image containing the region of the first object region within the first data-augmented image is then obtained and extracted as a second image to be processed. Local data augmentation is then performed on the second image to be processed to obtain a second data-augmented image. Compared to performing data augmentation only on a local image, by performing global data augmentation on the global image and then extracting the region of the first object region within the first data-augmented image as the second object region, subsequent local data augmentation can be performed on the image of this second object region. This provides a data augmentation method that combines global and local data augmentation, improving the diversity of training data during training and enhancing the model's generalization ability and robustness.
[0069] In one possible embodiment, the global data augmentation process includes at least one of flipping the first image to be processed, randomly perturbing the first image to be processed, rotating the first object region, and expanding the first object region, and the above operations may be performed once or multiple times.
[0070] In one possible embodiment, randomly perturbing the first image to be processed includes at least adding noise to the first image to be processed, adjusting the brightness of the first image to be processed, adjusting the contrast of the first image to be processed, and adjusting the resolution of the first image to be processed. Details regarding randomly perturbing the image are provided below and will not be described in detail here.
[0071] In one possible embodiment, when flipping the first image to be processed, it is determined whether to flip the first image to be processed. When the determination result is a preset probability p1, the first image to be processed is flipped in the horizontal or vertical direction, 0≦p1≦1; when the determination result is 1-p1, the first image to be processed is not flipped.
[0072] like Figure 2a As shown, Figure 2a This is a schematic diagram of horizontally flipping the first image to be processed. The large rectangle on the left represents the first image to be processed, the small rectangle on the left represents the first object region, the large rectangle on the right represents the first image to be processed after horizontal flipping (the first data-enhanced image), and the small rectangle on the right represents the first object region after horizontal flipping (the second object region).
[0073] like Figure 2b As shown, Figure 2bThis is a schematic diagram of vertically flipping the first image to be processed. The large rectangle on the left represents the first image to be processed, the small rectangle on the left represents the first object region, the large rectangle on the right represents the first image to be processed (first data augmentation image) after vertical flipping, and the small rectangle on the right represents the first object region (second object region) after vertical flipping.
[0074] In one possible embodiment, the first object region in the first image to be processed is rotated by a preset angle α.
[0075] like Figure 3 As shown, Figure 3 This is a schematic diagram of rotating the first object region in the first image to be processed by a preset angle. The large rectangle on the left represents the first image to be processed, the small rectangle on the left represents the first object region, the large rectangle on the right represents the first image to be processed (the first data-enhanced image) after rotating the first object region by a preset angle, the small rectangle on the right filled with lines represents the rotated first object region (the second object region), and the small rectangle on the right without lines filled with lines represents the first object region.
[0076] In one possible embodiment, the first object region is a bounding rectangle, and the first object region is extended outward by 1-k pixels along a direction perpendicular to any boundary line of the bounding rectangle.
[0077] like Figure 4 As shown, Figure 4 This is a schematic diagram of expanding the first object region in the first image to be processed. The large rectangle on the left represents the first image to be processed, the small rectangle on the left represents the first object region, the large solid rectangle on the right represents the first image to be processed (first data augmentation image) with the first object region expanded, the dashed rectangle on the right represents the expanded first object region (second object region), and the small solid rectangle on the right represents the first object region.
[0078] For example, see Figure 5 , Figure 5 This is a schematic diagram of global data augmentation processing on a first image to be processed. For example, the first image to be processed is flipped to obtain a first data augmentation image 1. The object region in the first data augmentation image 1 is rotated to obtain a first data augmentation image 2. The object region in the first data augmentation image 2 is expanded to obtain a first data augmentation image 3. The first data augmentation image 3 is used as the target first data augmentation image for subsequent local data augmentation processing.
[0079] In one possible embodiment, obtaining a region of a first object region in a first data-enhanced image as a second object region includes: determining an object in the first data-enhanced image using a preset algorithm, and obtaining a region of the object in the first data-enhanced image as the second object region.
[0080] In one possible embodiment, a coordinate system is established with any pixel in the first image to be processed as the origin, and the position coordinates of the first object region are recorded. After global data augmentation processing is performed on the first image to be processed, a preset algorithm is used to determine the object in the first data-augmented image, and the position coordinates of the region of the object in the first data-augmented image are obtained. The region corresponding to the position coordinates is taken as the second object region. For example, a coordinate system is established with any pixel in an image containing a defect region as the origin, and the position coordinates of the defect region in the image are recorded. After global data augmentation processing is performed on the image, a preset algorithm is used to determine the defect region in the globally data-augmented image, and the position coordinates of the defect region in the region of the globally data-augmented image are obtained. The region corresponding to the position coordinates is taken as the region where the defect region is located in the globally data-augmented image.
[0081] In one possible embodiment, the preset algorithm includes, but is not limited to, object detection algorithm, semantic segmentation algorithm, and instance segmentation algorithm. This application does not limit the preset algorithm.
[0082] In one possible embodiment, obtaining a region of the first object region in the first data-enhanced image as the second object region includes:
[0083] The region of the object in the first data-enhanced image is determined based on the first object region and the transformation law, and is used as the second object region. The transformation law is related to the operation type of the global data enhancement processing.
[0084] In one possible embodiment, a coordinate system is established with any pixel in the first image to be processed as the origin, and the position coordinates of the first object region are recorded. After performing global data augmentation processing on the first image to be processed, the operation type of the global data augmentation processing is determined, the transformation law is determined based on the operation type, and the region of the object in the image after global data augmentation processing is determined based on the position coordinates of the first object region and the transformation law, and is used as the second object region. For example, as... Figure 4 As shown, the global data augmentation process expands the first object region. The transformation rule is to increase the size of the first object region by a preset multiple, which can also increase the position coordinates of the first object region by a preset multiple. The region formed by the position coordinates after the above processing is determined as the second object region.
[0085] In one possible embodiment, a coordinate system is established with any pixel in the first image to be processed as the origin, and the position coordinates of the first object region are recorded. After performing global data augmentation processing on the first image to be processed, the operation type and operation order of the global data augmentation processing are determined. Based on the operation type and operation order of the global data augmentation processing, a transformation rule is determined. Based on the position coordinates of the first object region and the transformation rule, the region of the object in the globally data augmented image is determined as the second object region. For example, if the global data augmentation processing first expands the first object region and then horizontally flips the expanded first image to be processed, the transformation rule is to first enlarge the size of the first object region by a preset multiple, and then move the first object region, whose size has been enlarged by the preset multiple, to the right by a preset distance. In this way, the position coordinates of the first object region can be first enlarged to a preset multiple, and then the horizontal coordinates of the position coordinates enlarged to the preset multiple can be increased to a preset coordinate. The region formed by the position coordinates after the above processing is determined as the second object region.
[0086] In one possible embodiment, the local data enhancement processing includes at least one of the following: flipping the second image to be processed, adding noise to the second image to be processed, adjusting the brightness of the second image to be processed, adjusting the contrast of the second image to be processed, adjusting the resolution of the second image to be processed, performing a cropping and mixing (CutMix) on the second image to be processed, and performing a full-image mixing (Mixup) on the second image to be processed.
[0087] In one possible implementation, Mixup data augmentation involves blending two images at a certain ratio to generate a new image; CutMix data augmentation algorithm involves cutting out a region from one image and then cropping out a region of the same size from another image to fill in the cut-out region to achieve blending and generate a new image.
[0088] In one possible embodiment, when flipping the second image to be processed, it is determined whether to flip the second image to be processed. If the determination result is a preset probability p2, the first image to be processed is flipped horizontally or vertically, 0 ≤ p2 ≤ 1; if the determination result is 1 ≤ p2, the second image to be processed is not flipped. When adding noise to the second image to be processed, it is determined whether to add noise to the second image to be processed. If the determination result is a preset probability p3, noise is added to the second image to be processed, 0 ≤ p3 ≤ 1; if the determination result is 1 ≤ p3, noise is not added to the second image to be processed. The noise can be Gaussian noise and / or Poisson noise, and this application does not limit the type of noise. When adjusting the brightness of the second image to be processed, it is determined whether to adjust the brightness of the second image to be processed. If the determination result is a preset probability p4, the brightness of the second image to be processed is adjusted, 0 ≤ p4 ≤ 1; if the determination result is 1 ≤ p4, the brightness of the second image to be processed is not adjusted. When adjusting the contrast of the second image to be processed, a decision is made as to whether to adjust the contrast. If the decision result is a preset probability p5, the contrast of the second image to be processed is adjusted (0 ≤ p5 ≤ 1); if the decision result is 1 ≤ p5, the contrast of the second image to be processed is not adjusted. When adjusting the resolution of the second image to be processed, the resolution of the second image to be processed is adjusted to a preset scale.
[0089] In one possible embodiment, the second image to be processed is subjected to one or more local data augmentation processes to obtain a second data-enhanced image, wherein the multiple local data augmentation processes performed may be the same or different.
[0090] For example, see Figure 6 , Figure 6 This is a schematic diagram illustrating local data augmentation processing of a second image to be processed. The second image to be processed is flipped to obtain a second data-augmented image 1; noise is added to the second data-augmented image 1 to obtain a second data-augmented image 2; the brightness of the second data-augmented image 2 is adjusted to obtain a second data-augmented image 3; the contrast of the second data-augmented image 3 is adjusted to obtain a second data-augmented image 4; the resolution of the second data-augmented image 4 is adjusted to obtain a second data-augmented image 5. The second data-augmented image 5 is used as the target second data-augmented image for training the model.
[0091] This application also provides a data enhancement device, see [link to relevant documentation]. Figure 7 The device includes:
[0092] The acquisition module 701 is used to acquire a first image to be processed marked with a first object region, wherein the first object region is the region in the first image to be processed where the object is located;
[0093] The first data augmentation module 702 is used to perform global data augmentation processing on the first image to be processed to obtain a first data augmented image, and to obtain the region of the first object region in the first data augmented image as the second object region;
[0094] Extraction module 703 is used to extract an image within a second object region from a first data-enhanced image, as a second image to be processed;
[0095] The second data enhancement module 704 is used to perform local data enhancement processing on the second image to be processed to obtain a second data-enhanced image.
[0096] Optionally, global data augmentation processing includes at least one or more of the following operations:
[0097] The first image to be processed is flipped, the first image to be processed is randomly perturbed, the first object region is rotated, and the first object region is expanded.
[0098] Optionally, randomly perturbing the first image to be processed includes at least one or more of the following operations:
[0099] Add noise to the first image to be processed, adjust the brightness of the first image to be processed, adjust the contrast of the first image to be processed, and adjust the resolution of the first image to be processed.
[0100] Optional, the first data augmentation module is used for:
[0101] The object is determined in the first data augmentation image using a preset algorithm, and the region of the object in the first data augmentation image is obtained as the second object region.
[0102] Optional, the first data augmentation module is used for:
[0103] The region of the object in the first data-enhanced image is determined based on the first object region and the transformation law, and is used as the second object region. The transformation law is related to the operation type of the global data enhancement processing.
[0104] Optionally, local data augmentation processing includes at least one or more of the following operations:
[0105] The process includes: flipping the second image to be processed, adding noise to the second image to be processed, adjusting the brightness of the second image to be processed, adjusting the contrast of the second image to be processed, adjusting the resolution of the second image to be processed, cropping and mixing the second image to be processed (CutMix), and mixing the second image to be processed (Mixup).
[0106] This application also provides an electronic device, such as... Figure 8 As shown, it includes:
[0107] Memory 801 is used to store computer programs;
[0108] When processor 802 executes a program stored in memory 801, it performs the following steps:
[0109] Obtain a first image to be processed that is labeled with a first object region, wherein the first object region is the region in the first image to be processed where the object is located;
[0110] Global data augmentation is performed on the first image to be processed to obtain a first data augmented image, and the region of the first object region in the first data augmented image is obtained as the second object region;
[0111] Extract the image within the second object region from the first data-enhanced image, and use it as the second image to be processed;
[0112] Local data augmentation processing is performed on the second image to be processed to obtain the second data-augmented image.
[0113] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 802, the communication interface, and the memory 801 communicating with each other via the communication bus.
[0114] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0115] The communication interface is used for communication between the aforementioned electronic devices and other devices.
[0116] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0117] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0118] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above data augmentation methods.
[0119] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the data augmentation methods described above.
[0120] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.
[0121] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0122] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the method, apparatus, electronic device, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0123] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.
Claims
1. A data augmentation method, characterized in that, The method includes: Obtain a first image to be processed that is labeled with a first object region, wherein the first object region is the region in the first image to be processed where the object is located; Global data augmentation processing is performed on the first image to be processed to obtain a first data augmented image, and the region of the first object region in the first data augmented image is obtained as the second object region; Extract the image within the second object region from the first data-enhanced image, and use it as the second image to be processed; The second image to be processed is subjected to local data augmentation processing to obtain a second data-enhanced image; The global data augmentation process includes at least one or more of the following operations: The first image to be processed is flipped, the first image to be processed is randomly perturbed, the first object region is rotated, and the first object region is expanded. The step of obtaining the region of the first object region in the first data-enhanced image as the second object region includes: Based on the first object region and the transformation rule, the region of the object in the first data-enhanced image is determined as the second object region. The transformation rule is related to the operation type of the global data enhancement process.
2. The method as described in claim 1, characterized in that, The random perturbation of the first image to be processed includes at least one or more of the following operations: Add noise to the first image to be processed, adjust the brightness of the first image to be processed, adjust the contrast of the first image to be processed, and adjust the resolution of the first image to be processed.
3. The method as described in claim 1, characterized in that, The step of obtaining the region of the first object region in the first data-enhanced image as the second object region includes: An object is identified in the first data-enhanced image using a preset algorithm, and the region of the object in the first data-enhanced image is obtained as the second object region.
4. The method as described in claim 1, characterized in that, The local data augmentation process includes at least one or more of the following operations: The process involves flipping the second image to be processed, adding noise to the second image to be processed, adjusting the brightness of the second image to be processed, adjusting the contrast of the second image to be processed, adjusting the resolution of the second image to be processed, performing a cut-Mix operation on the second image to be processed, and performing a full-image mixup operation on the second image to be processed.
5. A data augmentation device, characterized in that, The device includes: The acquisition module is used to acquire a first image to be processed that is marked with a first object region, wherein the first object region is the region in the first image to be processed where the object is located; The first data augmentation module is used to perform global data augmentation processing on the first image to be processed to obtain a first data augmented image, and to obtain the region of the first object region in the first data augmented image as the second object region; The extraction module is used to extract the image within the second object region from the first data-enhanced image as the second image to be processed; The second data augmentation module is used to perform local data augmentation processing on the second image to be processed to obtain a second data-enhanced image; The global data augmentation process includes at least one or more of the following operations: The first image to be processed is flipped, the first image to be processed is randomly perturbed, the first object region is rotated, and the first object region is expanded. The first data enhancement module is used for: Based on the first object region and the transformation rule, the region of the object in the first data-enhanced image is determined as the second object region. The transformation rule is related to the operation type of the global data enhancement process.
6. The apparatus as claimed in claim 5, characterized in that, The random perturbation of the first image to be processed includes at least one or more of the following operations: Add noise to the first image to be processed, adjust the brightness of the first image to be processed, adjust the contrast of the first image to be processed, and adjust the resolution of the first image to be processed.
7. The apparatus as claimed in claim 5, characterized in that, The first data enhancement module is used for: An object is identified in the first data-enhanced image using a preset algorithm, and the region of the object in the first data-enhanced image is obtained as the second object region.
8. The apparatus as claimed in claim 5, characterized in that, The local data augmentation process includes at least one or more of the following operations: The process involves flipping the second image to be processed, adding noise to the second image to be processed, adjusting the brightness of the second image to be processed, adjusting the contrast of the second image to be processed, adjusting the resolution of the second image to be processed, performing a cut-Mix operation on the second image to be processed, and performing a full-image mixup operation on the second image to be processed.
9. An electronic device, characterized in that, Including processor and memory; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the method described in any one of claims 1-4.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-4.