A method for generating large-scale target detection data samples

Abstract

The present invention discloses a method for generating large-scale target detection data samples, which specifically relates to the field of target detection and is used to solve the problem in the prior art that the long-tail distribution of data sets leads to insufficient detection performance of rare category samples; by obtaining an initial image data set containing annotations of multiple categories of target objects, calculating the occurrence frequency of target objects of each category, and determining that the category with an occurrence frequency lower than a preset threshold is a rare category; performing dynamic behavior and geometric complexity analysis on the image of the rare category target object to determine whether the degree of distinguishability between the target object and the background is abnormal; for the target object with an abnormal degree of distinguishability, adjusting the pixel intensity through a contrast enhancement algorithm, and screening a background image that meets the characteristics of the target object based on a scene matching algorithm; embedding the adjusted target object into the screened background image to generate a synthetic image, and adding the synthetic image to the initial data set to form an expanded image data set.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and more specifically, to a method for generating large-scale target detection data samples. Background Art

[0002] Object detection is an important research direction in the field of computer vision and is widely used in scenarios such as autonomous driving, security monitoring, and intelligent manufacturing. In object detection tasks, training models usually rely on large-scale image datasets. However, the category distribution of image datasets in the prior art often presents a long-tail distribution, that is, the number of minority category samples is extremely small, while the number of majority category samples is abundant. This uneven data distribution will cause the rare category samples to be underweighted during the training process, thereby affecting the model's ability to detect rare categories.

[0003] In order to improve the balance of the data set, existing technologies often generate synthetic images through data augmentation methods, which usually do not fully consider the overall matching relationship between the target object and the background and its impact on data quality, which easily leads to poor fusion of the target object and the background in the synthetic image. This will result in low quality of synthetic data, which in turn affects the model's detection performance for rare category targets.

[0004] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a method for generating large-scale target detection data samples to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for generating large-scale target detection data samples comprises the following steps:

[0008] S1: Obtain an initial image dataset containing annotations of target objects of multiple categories, calculate the occurrence frequency of target objects of each category, and determine that the category with an occurrence frequency lower than a preset threshold is a rare category;

[0009] S2: For images containing rare target objects: by analyzing the motion characteristics of the target object and the background, evaluate whether there are abnormalities in the dynamic behavior of the target object and the background; by analyzing the geometric complexity of the target object and the background, evaluate whether the structural characteristics of the target object in the background match;

[0010] S3: judging whether the degree of distinguishability between the target object and the background is abnormal based on whether the dynamic behaviors of the target object and the background are abnormal and whether the structural characteristics of the target object in the background match;

[0011] S4: For target objects with abnormal distinguishability from the background, a contrast enhancement algorithm is used to adjust the pixel intensity of the target object, and a background image that meets the characteristics of the target object is screened based on a scene matching algorithm;

[0012] S5: embed the adjusted target object into the filtered background image to generate a synthetic image, and add the synthetic image to the initial image dataset to form an expanded image dataset.

[0013] In a preferred embodiment, an initial image data set containing annotations of target objects of multiple categories is obtained, the occurrence frequency of target objects of each category is calculated, and the category with an occurrence frequency lower than a preset threshold is determined as a rare category, specifically:

[0014] Collecting original images containing multi-category target objects, and preprocessing the collected original images, the preprocessing includes image quality correction, size standardization and noise filtering;

[0015] The preprocessed original image is annotated with a target, and the annotated information includes the category of the target object and the position information in the image, so as to form annotated image data;

[0016] Integrate the annotated image data into an initial image dataset, perform statistical analysis on the target object categories in the initial image dataset, and calculate the occurrence frequency of each target object category in the entire initial image dataset;

[0017] Target object categories whose occurrence frequency is lower than a preset threshold are determined as rare categories.

[0018] In a preferred embodiment, the motion characteristics of the target object and the background are analyzed to evaluate whether the dynamic behaviors of the target object and the background are abnormal, specifically:

[0019] Extract image sequences containing rare category target objects and calculate the motion vector field of the target object and the background through the optical flow algorithm;

[0020] Model the trajectory of the motion vector of the target object and generate the motion trajectory of the target object based on the time series characteristics;

[0021] Cluster analysis is performed through the motion vector field of the background area to construct the background motion pattern and extract the background dynamic behavior characteristics;

[0022] Compare and analyze the target object's motion trajectory with the background dynamic behavior characteristics, calculate the motion matching degree and record the difference characteristics;

[0023] The motion matching degree is used to determine whether there is any abnormality in the dynamic behavior of the target object and the background.

[0024] In a preferred embodiment, the geometric complexity of the target object and the background is analyzed to evaluate whether the structural characteristics of the target object in the background match, specifically:

[0025] Extract the regional features of rare category target objects and their images, and separate the target object area and background area;

[0026] Calculate the geometric complexity of the target object area and the background area respectively, and the geometric complexity includes edge density and fractal dimension;

[0027] Compare and analyze the geometric complexity of the target object area and the background area to quantify the complexity difference between the target object area and the background area;

[0028] Calculate the geometric matching degree based on the complexity difference between the target object area and the background area;

[0029] According to the degree of deviation of geometric matching, it is judged whether the structural characteristics of the target object in the background match.

[0030] In a preferred embodiment, whether the degree of distinguishability between the target object and the background is abnormal is determined based on whether the dynamic behaviors of the target object and the background are abnormal and whether the structural characteristics of the target object in the background match, specifically:

[0031] When there is no abnormality in the dynamic behavior of the target object and the background, and the structural characteristics of the target object in the background match, the degree of distinguishability between the target object and the background is determined to be normal; otherwise, the degree of distinguishability between the target object and the background is determined to be abnormal.

[0032] In a preferred embodiment, for a target object whose distinguishability from the background is abnormal, a contrast enhancement algorithm is used to adjust the pixel intensity of the target object, specifically:

[0033] Extract the pixel values ​​of the target object area, and redistribute the pixel intensity values ​​based on the histogram equalization method to stretch the pixel contrast range of the target object;

[0034] The pixel values ​​are further subjected to adaptive contrast limiting to avoid image artifacts caused by over-enhancement;

[0035] The pixel values ​​of the corresponding area in the original image are replaced by the adjusted pixel values ​​of the target object to form a target object area with enhanced contrast.

[0036] In a preferred embodiment, the background image that meets the characteristics of the target object is screened based on the scene matching algorithm, specifically:

[0037] Generate a multi-dimensional feature vector of the target object by extracting feature information of the target object, wherein the feature information of the target object includes color distribution, texture feature and shape feature;

[0038] Extracting feature information of each background image from a background image database, and representing each background image as a feature vector;

[0039] Based on the similarity calculation results of the multidimensional feature vector of the target object and the feature vector of the background image, matching is performed through a preset similarity metric, the background image with the highest similarity is screened, and the background image that meets the characteristics of the target object is determined.

[0040] In a preferred embodiment, the adjusted target object is embedded in the filtered background image to generate a synthetic image, and the synthetic image is added to the initial image data set to form an expanded image data set, specifically:

[0041] Perform transparency processing on the adjusted target object to extract pixel information and contour characteristics of the target object;

[0042] Determine the embedding position of the target object in the background image according to the geometric matching relationship between the target object and the filtered background image;

[0043] Scale the target object to the target size according to the preset ratio to keep the proportion of the target object and the background image coordinated;

[0044] The target object is embedded into the background image using the image fusion algorithm, and the edge pixels of the target object are adjusted according to the color characteristics of the background area during fusion;

[0045] The generated synthetic images are added to the initial image dataset, the structure of the expanded image dataset is updated, and the category information and position relationship of the newly added images are marked.

[0046] Technical effects and advantages of a method for generating large-scale target detection data samples of the present invention:

[0047] 1. The present invention optimizes the presentation effect of rare category target objects in synthetic images by comprehensively analyzing the dynamic behavior and geometric complexity characteristics between the target object and the background, combined with the evaluation results of the degree of distinguishability between the target object and the background. Compared with the prior art methods that simply rely on random data enhancement or simple geometric transformation, the present invention can effectively solve the problem of poor matching between the target object and the background. Through dynamic behavior analysis, the consistency of the motion pattern of the target object and the background is ensured; through geometric complexity analysis, the structural characteristics of the target object in the background are optimized. This comprehensive analysis significantly improves the naturalness of the fusion of the target object and the background, thereby improving the quality of the synthetic image and providing more reliable training data for subsequent target detection models.

[0048] 2. By adopting the target object embedding strategy based on the scene matching algorithm, the background image that best matches the characteristics of the target object is screened, and the pixel intensity of the target object is adjusted in combination with the contrast enhancement algorithm, so that the target object is more clearly visible in the synthetic image, and the detectability of rare category targets is improved. By adding the generated synthetic image to the initial data set and forming an expanded image data set, the present invention significantly improves the balance of the data set, especially in the expansion quality and quantity of rare category samples, and provides efficient and accurate data support for the training of the target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a method for generating large-scale target detection data samples according to the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0051] Figure 1 A method for generating a large-scale target detection data sample is provided in the present invention, which comprises the following steps:

[0052] S1: Obtain an initial image dataset containing annotations of target objects of multiple categories, calculate the occurrence frequency of target objects of each category, and determine that the category with an occurrence frequency lower than a preset threshold is a rare category.

[0053] S2: For images containing rare category target objects: by analyzing the motion characteristics of the target object and the background, evaluate whether there are any abnormalities in the dynamic behavior of the target object and the background; by analyzing the geometric complexity of the target object and the background, evaluate whether the structural characteristics of the target object in the background match.

[0054] S3: Based on whether there is an abnormality in the dynamic behaviors of the target object and the background and whether the structural characteristics of the target object in the background match, it is determined whether the degree of distinguishability between the target object and the background is abnormal.

[0055] S4: For target objects whose distinguishability from the background is abnormal, a contrast enhancement algorithm is used to adjust the pixel intensity of the target object, and a background image that meets the characteristics of the target object is screened based on a scene matching algorithm.

[0056] S5: embed the adjusted target object into the filtered background image to generate a synthetic image, and add the synthetic image to the initial image dataset to form an expanded image dataset.

[0057] Obtain an initial image dataset containing annotations of multiple categories of target objects, calculate the frequency of occurrence of target objects of each category, and determine that the category with an occurrence frequency lower than a preset threshold is a rare category, specifically:

[0058] Collect original images containing multi-category target objects and preprocess the collected original images. The preprocessing includes image quality correction, size standardization and noise filtering:

[0059] Raw image data containing multiple categories of target objects are collected in different scenes and conditions through a variety of image acquisition devices (such as high-definition cameras, drone cameras, and mobile phone cameras). These scenes include indoors, outdoors, during the day, at night, and in various complex environments (such as rainy and snowy weather, backlight, etc.). The setting parameters for image acquisition include resolution, frame rate, and exposure time to ensure that the collected raw images have sufficient clarity and diversity. The acquisition devices are arranged in multiple geographical locations to increase the coverage of target object categories.

[0060] Image quality correction: Use the image enhancement algorithm based on convolutional neural network to repair the blurred areas in the image and correct the brightness of overexposed or underexposed areas.

[0061] Size normalization: All original images are uniformly resized to a fixed resolution (e.g., 1280 × 720 pixels) and the key features of the image are retained using a bilinear interpolation method.

[0062] Noise filtering: Remove random noise from the image based on the non-local means denoising algorithm while retaining the texture details of the target object to ensure that the image quality is suitable for subsequent target annotation and analysis.

[0063] The preprocessed original image is annotated with targets. The annotation information includes the category of the target object and the location information in the image to form annotated image data:

[0064] Through the trained target detection model (such as the YOLO series or the Transformer-based target detection network), the target object is automatically detected and a preliminary bounding box and category label are generated.

[0065] The location information of the target object is stored in the form of the four corner coordinates of the bounding box to facilitate subsequent calculation and analysis.

[0066] The labeled image data is integrated into the initial image dataset, and the target object categories in the initial image dataset are statistically analyzed to calculate the frequency of occurrence of each target object category in the entire initial image dataset:

[0067] The image data is classified according to the target object category based on the annotation information for subsequent statistics and analysis; the image data is analyzed for similarity using the perceptual hashing algorithm to eliminate redundant or overly repetitive images; the integrated initial image data set is saved in a distributed storage structure to ensure data reading and processing efficiency.

[0068] Read the annotation information of each image and calculate the total number of times each target object category appears in the entire data set; calculate the frequency of occurrence of the target object category; and generate a frequency distribution diagram of the target object category to intuitively understand the occurrence of different categories.

[0069] Identify target object categories whose occurrence frequency is lower than a preset threshold as rare categories:

[0070] Select an appropriate preset threshold based on the training data requirements and the distribution characteristics of rare categories.

[0071] The target object categories with an occurrence frequency lower than a preset threshold (such as 1% or lower) are defined as rare categories, and the rare categories and related information are recorded; the name of the rare category, the location information of the corresponding target object and the path of the image are stored in an index file for subsequent processing.

[0072] By analyzing the motion characteristics of the target object and the background, it is evaluated whether there are any abnormalities in the dynamic behaviors of the target object and the background, specifically:

[0073] Extract an image sequence containing rare category target objects and calculate the motion vector field of the target object and the background through the optical flow algorithm:

[0074] Image sequences containing rare categories of target objects are screened out from the initial image dataset, and the inter-frame difference method is used to extract the motion area of ​​the target object in consecutive frames to ensure that only the effective motion range of the target object is included.

[0075] Based on the image sequence, the dense optical flow algorithm is used to calculate the motion vector field of the target object and the background area in each frame, and the following formula is defined: ;in, Indicates at time Moment, Pixel The motion vector field of and Along and The grayscale gradient of the image in the direction.

[0076] The motion vector field is recorded, including the vector information of the target object and the motion information of the background area, which are stored in independent motion characteristic matrices respectively.

[0077] Output of vector field extraction: independent motion vector field matrices for target objects and background regions.

[0078] The motion vector of the target object is modeled and the motion trajectory of the target object based on the time series characteristics is generated:

[0079] The spatial coordinate points of the target object at different times are integrated in chronological order to generate the trajectory of the target object.

[0080] In order to capture the dynamic change characteristics of the trajectory, cubic spline interpolation is used to smooth the trajectory and generate the motion trajectory model of the target object. The formula is as follows: ;in, represents the smooth trajectory function of the target object, are the parameters of the interpolation fit.

[0081] in, is the fitting parameter of the cubic spline interpolation function, which is used to construct a smooth model of the target object's motion trajectory. By performing a time series analysis on the trajectory points of the target object, a cubic polynomial is fitted to the spatial coordinates of the trajectory points in the time dimension. The fitting process uses the least squares method to solve the problem of minimizing the residual sum of squares of the trajectory points. The specific value of is used to ensure that the trajectory model is consistent with the actual motion trajectory and to achieve smoothing of the trajectory.

[0082] Cluster analysis is performed through the motion vector field of the background area to construct the background motion pattern and extract the background dynamic behavior characteristics:

[0083] Based on the background area motion vector field obtained in the first step, the density clustering algorithm (DBSCAN) is used to group the motion characteristics of the background and divide the background into multiple regions with similar motion characteristics. The clustering process is defined as follows:

[0084] Let the motion vector be ,in and are the horizontal and vertical components of the vector respectively. For each background area, calculate its centroid motion vector: ;in, is the centroid motion vector of the background area, is the number of vectors in the cluster, is the number of the vector within the cluster.

[0085] Through the clustering results, the background motion pattern is constructed, including dynamic features such as the centroid vector, boundary shape and motion direction of the background area.

[0086] Compare and analyze the target object's motion trajectory with the background dynamic behavior characteristics, calculate the motion matching degree and record the difference characteristics:

[0087] Calculate the matching degree between the target object trajectory and the background features. The matching degree is defined as the cosine similarity between the motion vector of the target object and the motion vector of the centroid of the background area: ;in, Indicates the matching degree between the target object trajectory and the background features, is the motion vector of the target object, Represents the dot product of two vectors.

[0088] After calculating the matching degree between the target object trajectory and the background features, the difference features are recorded. The difference features represent the difference between the target object and the background motion features, including the motion direction deviation and speed difference.

[0089] Determine whether there is any abnormality in the dynamic behavior of the target object and the background based on the motion matching degree:

[0090] Set the preset threshold corresponding to the matching degree between the target object trajectory and the background features .

[0091] when When , it is determined that the dynamic behavior of the target object and the background is abnormal; otherwise, it is determined that the dynamic behavior of the target object and the background is not abnormal.

[0092] The preset threshold The setting is based on the distribution statistics of motion matching between the target object and the background. A large amount of normal and abnormal sample data is used to select the best dividing point through ROC curve analysis to balance the accuracy and recall rate of judgment. At the same time, the threshold is dynamically adjusted in combination with the application needs of actual scenarios.

[0093] By analyzing the geometric complexity of the target object and the background, it is evaluated whether the structural characteristics of the target object in the background match, specifically:

[0094] Extract the regional features of rare category target objects and their images, and separate the target object area and background area:

[0095] In the image of a rare class target object, the target object region and the background region are separated using the bounding box segmentation method based on the target object's annotation information. The target object region is defined as the set of pixels within the bounding box, and the background region is defined as the set of pixels outside the bounding box but within the same image.

[0096] In order to ensure the accuracy of region segmentation, a method based on region growing algorithm is further adopted to remove background pixels near the boundary of the target object to reduce the impact of boundary blur on complexity analysis.

[0097] The region segmentation result is stored as an independent matrix, including a target object region matrix and a background region matrix, wherein each element in the target object region matrix represents the gray value of a target object pixel, wherein each element in the background region matrix represents the gray value of a background pixel.

[0098] Calculate the geometric complexity of the target object area and the background area respectively. The geometric complexity includes edge density and fractal dimension:

[0099] The Canny edge detection algorithm is used to extract the edge pixel sets of the target object area and the background area, and the edge density is calculated. The edge density is the ratio of the number of edge pixels in the area to the total number of pixels in the area.

[0100] The fractal dimension of the target object area and the background area is calculated using the fractal box counting method, which is defined as: ;in, is the fractal dimension, Indicates that the side length is The minimum number of boxes required to cover the area, Represents the side length of the box.

[0101] The geometric complexity includes but is not limited to edge density and fractal dimension, which will not be elaborated here.

[0102] Compare and analyze the geometric complexity of the target object area and the background area to quantify the complexity difference between the target object area and the background area:

[0103] Compute the difference between edge density and fractal dimension, respectively:

[0104] Complexity differences include differences in edge density and fractal dimension.

[0105] The absolute value of the difference between the edge density of the target object region and the background region is marked as the edge density difference.

[0106] The absolute value of the difference between the fractal dimensions of the target object region and the background region is marked as the fractal dimension difference.

[0107] Store all difference values ​​in a geometric complexity difference vector.

[0108] Calculate the geometric matching degree based on the complexity difference between the target object area and the background area:

[0109] Calculate the geometric matching degree between the target object area and the background area: ;in, is the geometric matching degree, Indicates The complexity difference, For the The weight of the complexity difference, is the number of complexity differences.

[0110] For example, the edge density difference included in the complexity difference is marked as 1, and the fractal dimension difference included in the complexity difference is marked as 2.

[0111] in, The setting is based on the influence of the complexity characteristics of the target object and the background area, and is determined through statistical analysis of the complexity differences of a large number of samples. Regression analysis or machine learning methods, such as gradient boosting trees, are used to optimize the weight distribution so that the matching function has the best weight ratio for different complexity characteristics.

[0112] According to the degree of deviation of geometric matching, it is judged whether the structural characteristics of the target object in the background match:

[0113] Set the preset threshold corresponding to the geometric matching degree .

[0114] when , it is determined that the structural characteristics of the target object in the background do not match; otherwise, it is determined that the structural characteristics of the target object in the background match.

[0115] The preset threshold The setting is based on the geometric complexity matching feature statistics of the target object and the background. By analyzing a large amount of training data, the distribution range of the geometric matching degree under normal and mismatching conditions is calculated, and the best segmentation point in the ROC curve analysis is selected as the threshold. This threshold is used to distinguish the matching state of the target object and the background structural characteristics, ensuring high discrimination accuracy and robustness in practical applications.

[0116] Based on whether there are abnormalities in the dynamic behaviors of the target object and the background and whether the structural characteristics of the target object in the background match, it is judged whether the degree of distinguishability between the target object and the background is abnormal, specifically:

[0117] When there is no abnormality in the dynamic behavior of the target object and the background, and the structural characteristics of the target object in the background match, the degree of distinguishability between the target object and the background is determined to be normal; otherwise, the degree of distinguishability between the target object and the background is determined to be abnormal.

[0118] This judgment is based on the analysis of the comprehensive relationship between the target object and the background in terms of dynamic behavior and structural characteristics. Abnormal dynamic behavior and mismatched structural characteristics are the two main factors for the poor fusion of the target object and the background. Dynamic behavior reflects the motion coordination between the target object and the background in the time dimension, such as whether the trajectory of the target object conforms to the overall flow pattern of the background; while structural characteristics indicate the consistency of the spatial geometric characteristics of the target object and the background, such as whether the edge complexity of the target object is adapted to the texture characteristics of the background.

[0119] When there are no abnormalities in the dynamic behavior and the structural characteristics match, the motion and geometric features of the target object are coordinated with the background, indicating that the target object appears naturally in the background and has good distinguishability. Therefore, the degree of distinguishability can be judged as normal. On the contrary, any abnormality will lead to poor fusion of the target object in the background, thereby reducing distinguishability and need to be judged as abnormal.

[0120] For target objects whose distinguishability from the background is abnormal, a contrast enhancement algorithm is used to adjust the pixel intensity of the target object, specifically:

[0121] Extract the pixel values ​​of the target object area and redistribute the pixel intensity values ​​based on the histogram equalization method to stretch the pixel contrast range of the target object:

[0122] Extract the pixel area where the target object is located from the image. Specifically, according to the boundary information of the target object, lock the rectangular range of the target object area, and read the pixel values ​​within the rectangular range into a separate storage space. In order to ensure that the extracted pixel values ​​can fully describe the contrast characteristics of the target object, the area near the boundary that may contain background pixels is eliminated, and the main pixel features of the target object are retained.

[0123] After the extraction is completed, the pixel area of ​​the target object is converted into a grayscale image for subsequent contrast adjustment processing.

[0124] Calculate the histogram distribution of pixel values ​​in the target object area, count the frequency of occurrence of each grayscale value, and generate a grayscale histogram.

[0125] According to the grayscale histogram, the pixel values ​​are redistributed so that the distribution of pixel intensity covers the entire grayscale range. By stretching the less used grayscale range, the overall contrast of pixel intensity is improved.

[0126] The adjusted pixel values ​​are mapped back to the target object region to form preliminarily enhanced target object region pixels.

[0127] After histogram equalization processing, the pixel intensity of the target object is more evenly distributed in different grayscale value ranges, which improves the brightness and contrast characteristics of the target object.

[0128] The pixel values ​​are further subjected to adaptive contrast limiting to avoid image artifacts caused by over-enhancement:

[0129] In order to avoid over-enhancement that may be caused by histogram equalization, the adaptive contrast limitation method is used to further optimize the pixel values ​​of the target object area. The steps are as follows:

[0130] The target object area is divided into several sub-blocks, and each sub-block is subjected to contrast limitation processing separately to avoid the loss of local pixel details due to overall enhancement.

[0131] The local histogram is calculated for each sub-block, a pixel intensity limit threshold is set, highlight or shadow pixels exceeding the threshold are cropped, and the cropped pixels are redistributed within a suitable grayscale range.

[0132] The adjustment results of all sub-blocks are merged to regenerate the complete target object pixel area.

[0133] Adaptive contrast limiting can effectively enhance the local contrast characteristics of the target object while avoiding image artifacts caused by over-enhancement, such as overflow in highlight areas or loss of details in shadow areas.

[0134] The adjusted pixel values ​​of the target object are used to replace the pixel values ​​of the corresponding area in the original image to form a target object area with enhanced contrast:

[0135] The target object area that has been contrast enhanced is replaced with the original pixel value of the target object by the enhanced pixel value according to the position in the original image. When replacing, ensure that the enhanced area maintains edge continuity with the original image to avoid abrupt connection between the target object area and the background.

[0136] After the replacement is completed, the pixel intensity of the target object area is adjusted, and the enhanced image can be used as input for subsequent scene matching and synthesis processing.

[0137] The background image that meets the characteristics of the target object is screened based on the scene matching algorithm, specifically:

[0138] A multi-dimensional feature vector of the target object is generated by extracting the feature information of the target object. The feature information of the target object includes color distribution, texture features and shape features:

[0139] The target object area is converted into a standardized color space (such as CIELAB), the mean and variance of each color component are calculated, and the color feature vector of the target object is generated.

[0140] The texture features of the target object, including energy, contrast, entropy and correlation, are calculated based on the gray-level co-occurrence matrix to form the texture feature vector of the target object.

[0141] The boundaries of the target objects are contoured and shape features are calculated, including area, perimeter, boundary complexity, and moment invariants.

[0142] The above features are normalized and combined into a multi-dimensional feature vector of the target object, represented as a vectorized data structure, and stored for subsequent matching.

[0143] Extract the feature information of each background image from the background image database and represent each background image as a feature vector:

[0144] Each background image is read from the background image database in turn, and the feature information of the background image is extracted in the same way as the target object feature extraction, including color distribution, texture features and shape features.

[0145] The color components of the background image are calculated in blocks, local color distribution features are generated in each area, and the local features are summarized into global color features.

[0146] The texture complexity of the background image is analyzed using a multi-scale gray-level co-occurrence matrix, and the texture feature vector with the same dimension as the target object is extracted.

[0147] By analyzing the main contour features of the background image in blocks, the global shape features are calculated, including the boundary complexity and contour distribution characteristics of the region.

[0148] The feature information of the background image is represented as a feature vector and stored in the form of a database index for matching calculation.

[0149] Based on the similarity calculation results of the multi-dimensional feature vector of the target object and the feature vector of the background image, matching is performed through a preset similarity metric, the background image with the highest similarity is screened, and the background image that meets the characteristics of the target object is determined:

[0150] For the multidimensional feature vector of the target object and the multidimensional feature vector of the background image, cosine similarity is used as a similarity measurement method. The similarity is expressed by calculating the cosine value of the angle between the two feature vectors, and its value ranges from 0 to 1. The calculation method includes the following steps:

[0151] Calculate the inner product of the feature vector: multiply each component of the target object feature vector by the corresponding component of the background image feature vector, and sum the results of all the products to obtain the inner product value of the two vectors.

[0152] Calculate the modulus of the feature vector: Calculate the modulus of the target object feature vector and the background image feature vector respectively. The modulus is the square root of the sum of the squares of the vector components.

[0153] Calculate the cosine value: Divide the inner product value by the product of the two vector norms to obtain the similarity score between the target object and the background image feature vector.

[0154] The final similarity score is used for sorting, and the background image with the highest score is selected as the background image that meets the characteristics of the target object.

[0155] In this way, the overall matching degree between the target object and the background image feature vector can be measured.

[0156] The adjusted target object is embedded into the filtered background image to generate a synthetic image, and the synthetic image is added to the initial image dataset to form an expanded image dataset, specifically:

[0157] Make the adjusted target object transparent and extract the pixel information and contour characteristics of the target object:

[0158] First, the adjusted target object image is made transparent, and the background pixels of the target object are set to be completely transparent, and only the main area of ​​the target object is retained. The contour characteristics of the target object, including edge position and curve shape, are extracted through the boundary detection algorithm. At the same time, the pixel values ​​of the target object are stored as a matrix, recording the color, brightness, transparency and other information of each pixel.

[0159] After the transparency processing, the target object is independently represented as a transparent layer, which is convenient for subsequent fusion processing with the background image.

[0160] According to the geometric matching relationship between the target object and the filtered background image, the embedding position of the target object in the background image is determined:

[0161] Extract the geometric characteristics of the background image: Analyze the main area distribution of the background image, including boundary shape and local feature dense areas.

[0162] Calculate the embedding position: Align the geometric features of the target object (such as center point, boundary size) with the geometric features of the background image, and select a blank or low-complexity area of ​​the background image as the embedding position.

[0163] Record embedding parameters: Save the coordinate information of the embedding position and the size range of the embedding area for subsequent processing.

[0164] Scale the target object to the target size according to the preset ratio to keep the proportion of the target object and the background image coordinated:

[0165] Calculate scaling ratio: Set the scaling ratio range based on the size ratio of the target object and the background image to ensure that the target object maintains the same proportion as the objects in the background.

[0166] Perform a scaling operation: Use a bilinear interpolation algorithm to scale the pixel matrix of the target object, and store the scaling result as a new pixel matrix after adjustment.

[0167] Verify the scaling result: Check whether the target object meets the preset ratio requirements after scaling to ensure that there is no visual deviation that is too large or too small.

[0168] The target object is embedded into the background image using the image fusion algorithm. During fusion, the edge pixels of the target object are adjusted according to the color characteristics of the background area:

[0169] Analyze the color features of the background image embedding position, including brightness, contrast and hue, calculate the color gradient value of the edge area, perform color gradient processing on the edge pixels of the target object so that its color gradually transitions to be consistent with the background area, and use the weighted average method to fuse the target object pixels with the background image pixels, and embed the target object into the preset position of the background image.

[0170] Add the generated synthetic images to the initial image dataset, update the structure of the expanded image dataset, and mark the category information and position relationship of the newly added images:

[0171] Save the resulting synthetic image as a standard format file (such as PNG or JPEG) and add it to the initial image dataset.

[0172] Add a new record to the initial image dataset to record the file path, category labeling information, and embedding position parameters of the synthetic image; label the target object in the synthetic image and record the specific position of the target object in the image (such as boundary coordinates); save the updated image dataset to the distributed storage system to ensure that the expanded dataset can be used for subsequent model training.

[0173] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0174] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or may be transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0175] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0178] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0180] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0181] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0182] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating large-scale target detection data samples, characterized in that: The steps include: S1: Obtain an initial image dataset containing annotations of target objects of multiple categories, calculate the occurrence frequency of target objects of each category, and determine that the category with an occurrence frequency lower than a preset threshold is a rare category; S2: For images containing rare target objects: by analyzing the motion characteristics of the target object and the background, evaluate whether there are abnormalities in the dynamic behavior of the target object and the background; by analyzing the geometric complexity of the target object and the background, evaluate whether the structural characteristics of the target object in the background match; S3: judging whether the degree of distinguishability between the target object and the background is abnormal based on whether the dynamic behaviors of the target object and the background are abnormal and whether the structural characteristics of the target object in the background match; S4: For target objects with abnormal distinguishability from the background, a contrast enhancement algorithm is used to adjust the pixel intensity of the target object, and a background image that meets the characteristics of the target object is screened based on a scene matching algorithm; S5: embedding the adjusted target object into the filtered background image to generate a synthetic image, and adding the synthetic image to the initial image dataset to form an expanded image dataset; By analyzing the motion characteristics of the target object and the background, it is evaluated whether there are any abnormalities in the dynamic behaviors of the target object and the background, specifically: Extract image sequences containing rare category target objects and calculate the motion vector field of the target object and the background through the optical flow algorithm; Model the trajectory of the motion vector of the target object and generate the motion trajectory of the target object based on the time series characteristics; Cluster analysis is performed through the motion vector field of the background area to construct the background motion pattern and extract the background dynamic behavior characteristics; Compare and analyze the target object's motion trajectory with the background dynamic behavior characteristics, calculate the motion matching degree and record the difference characteristics; Determine whether there is any abnormality in the dynamic behavior of the target object and the background based on the motion matching degree; By analyzing the geometric complexity of the target object and the background, it is evaluated whether the structural characteristics of the target object in the background match, specifically: Extract the regional features of rare category target objects and their images, and separate the target object area and background area; Calculate the geometric complexity of the target object area and the background area respectively, and the geometric complexity includes edge density and fractal dimension; Compare and analyze the geometric complexity of the target object area and the background area to quantify the complexity difference between the target object area and the background area; Calculate the geometric matching degree based on the complexity difference between the target object area and the background area; According to the degree of deviation of geometric matching, it is judged whether the structural characteristics of the target object in the background match; Based on whether there are abnormalities in the dynamic behaviors of the target object and the background and whether the structural characteristics of the target object in the background match, it is judged whether the degree of distinguishability between the target object and the background is abnormal, specifically: When there is no abnormality in the dynamic behavior of the target object and the background, and the structural characteristics of the target object in the background match, the degree of distinguishability between the target object and the background is judged to be normal; Otherwise, it is determined that the degree of distinguishability between the target object and the background is abnormal.

2. The method for generating a large-scale target detection data sample according to claim 1, characterized in that: Obtain an initial image dataset containing annotations of multiple categories of target objects, calculate the frequency of occurrence of target objects of each category, and determine that the category with an occurrence frequency lower than a preset threshold is a rare category, specifically: Collecting original images containing multi-category target objects, and preprocessing the collected original images, the preprocessing includes image quality correction, size standardization and noise filtering; The preprocessed original image is annotated with a target, and the annotated information includes the category of the target object and the position information in the image, so as to form annotated image data; Integrate the annotated image data into an initial image dataset, perform statistical analysis on the target object categories in the initial image dataset, and calculate the occurrence frequency of each target object category in the entire initial image dataset; Target object categories whose occurrence frequency is lower than a preset threshold are determined as rare categories.

3. The method for generating a large-scale target detection data sample according to claim 1, characterized in that: For target objects whose distinguishability from the background is abnormal, a contrast enhancement algorithm is used to adjust the pixel intensity of the target object, specifically: Extract the pixel values ​​of the target object area, and redistribute the pixel intensity values ​​based on the histogram equalization method to stretch the pixel contrast range of the target object; The pixel values ​​are further subjected to adaptive contrast limiting to avoid image artifacts caused by over-enhancement; The pixel values ​​of the corresponding area in the original image are replaced by the adjusted pixel values ​​of the target object to form a target object area with enhanced contrast.

4. The method for generating a large-scale target detection data sample according to claim 1, characterized in that: The background image that meets the characteristics of the target object is screened based on the scene matching algorithm, specifically: Generate a multi-dimensional feature vector of the target object by extracting feature information of the target object, wherein the feature information of the target object includes color distribution, texture feature and shape feature; Extracting feature information of each background image from a background image database, and representing each background image as a feature vector; Based on the similarity calculation results of the multidimensional feature vector of the target object and the feature vector of the background image, matching is performed through a preset similarity metric, the background image with the highest similarity is screened, and the background image that meets the characteristics of the target object is determined.

5. The method for generating a large-scale target detection data sample according to claim 1, characterized in that: The adjusted target object is embedded into the filtered background image to generate a synthetic image, and the synthetic image is added to the initial image dataset to form an expanded image dataset, specifically: Perform transparency processing on the adjusted target object to extract pixel information and contour characteristics of the target object; Determine the embedding position of the target object in the background image according to the geometric matching relationship between the target object and the filtered background image; Scale the target object to the target size according to the preset ratio to keep the proportion of the target object and the background image coordinated; The target object is embedded into the background image using the image fusion algorithm, and the edge pixels of the target object are adjusted according to the color characteristics of the background area during fusion; The generated synthetic images are added to the initial image dataset, the structure of the expanded image dataset is updated, and the category information and position relationship of the newly added images are marked.

Images