Sample image processing method and device, electronic equipment and storage medium

By using Python packets and OpenCV library for image enhancement and labeling adjustment in deep model training, the problem of limited data enhancement methods and high time cost is solved, and the data generalization requirements are achieved under a small number of samples while reducing manpower and time investment.

CN119091443APending Publication Date: 2024-12-06SICHUAN HONGHE COMM CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411136558.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When the prior art improves the generalization of deep models, the data enhancement method is limited and cannot meet the needs of certain specific scenarios. It is also cost high, and requires a large amount of sample and labeling work.

Method used

By obtaining the initial sample image containing labeling information, calling pre-configured Python packets for image enhancement processing and labeling adjustment, generating sample images for model training, and automatically selecting appropriate data enhancement methods and adjustments using OpenCV library and labeling modification files. Label information.

Benefits of technology

Complete the generalization requirements of algorithm data in the case of only a small number of samples, reducing labor and time costs in the inference process and saving model training time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091443A_ABST
    Figure CN119091443A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a sample image processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring an initial sample image containing annotation information; calling a pre-configured Python data packet to perform enhancement processing on the initial sample image according to a target image enhancement mode corresponding to the initial sample image, and performing annotation adjustment processing on the annotation information to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information; and generating a sample image for model training based on the enhanced image and the target annotation information. According to the embodiment of the invention, a proper data enhancement mode is selected from the perspective of data feature distribution, the requirement of most model algorithms on data generalization can be met only by a small number of samples, and only a small amount of manpower and time labeling work is needed in the reasoning process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a sample image processing method, device, electronic device and storage medium. Background Art

[0002] The various existing solutions for improving models by adding data require a lot of work such as data collection, labeling, and cleaning to train different deep models and deploy them online after testing.

[0003] Currently, there are two main methods for improving the generalization of models through target detection image enhancement: 1. Use the data enhancement methods (mosaic, copy_pasted, etc.) that come with common target detection models (such as YOLOv5, YOLOv8, etc.) to perform data enhancement. 2. Independently perform data enhancement to obtain new data, improve the generalization of the data, and then improve the generalization of the model, train the deep model, and deploy it after training.

[0004] There are two problems with these two recognition methods: 1. The data enhancement method is limited. The data enhancement method of common open source models cannot meet the actual working scenarios of some algorithms. For example, when detecting the abnormal opening and closing of the distribution box door, the shooting angle may not be straight, and it may be skewed or mirrored. 2. The time cost is high. It takes a long time to provide data generalization only through customized data enhancement methods, which also involves data preparation, labeling, cleaning and deep model training. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present application is to provide a sample image processing method, device, electronic device and storage medium, so as to achieve the purpose of meeting the data generalization requirements of most algorithms with only a small number of samples and requiring only a small amount of manpower and time for labeling work during the reasoning process.

[0006] In a first aspect, an embodiment of the present application provides a sample image processing method, the method comprising:

[0007] Obtain an initial sample image containing annotation information;

[0008] Calling a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information;

[0009] Based on the enhanced image and the target annotation information, a sample image for model training is generated.

[0010] Optionally, the Python data packet encapsulates an OpenCV library and a marked modification file.

[0011] The calling of the pre-configured Python data packet performs enhancement processing on the initial sample image according to the target image enhancement method corresponding to the initial sample image, and performs annotation adjustment processing on the annotation information to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information, including:

[0012] Importing the initial sample image into the Python data package;

[0013] Calling the OpenCV library to perform enhancement processing on the initial sample image according to the target image enhancement method corresponding to the initial sample image, so as to obtain an enhanced image corresponding to the initial sample image;

[0014] The annotation modification file is called to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library to obtain target annotation information corresponding to the annotation information.

[0015] Optionally, the calling of the OpenCV library to perform enhancement processing on the initial sample image according to a target image enhancement mode corresponding to the initial sample image to obtain an enhanced image corresponding to the initial sample image includes:

[0016] Calling the OpenCV library to determine the target image enhancement method corresponding to the initial sample image according to the image data distribution information of the initial sample image;

[0017] The specified enhancement function in the OpenCV library is called to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image.

[0018] Optionally, the calling of a specified enhancement function in the OpenCV library to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image includes:

[0019] Calling the specified enhancement function to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an intermediate enhanced image;

[0020] Performing color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image.

[0021] Optionally, performing color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image includes:

[0022] Convert each pixel of the intermediate enhanced image from the BGR color space to the CIELAB color space to obtain a CIELAB color space image;

[0023] Calculate the mean and variance of the CIELAB color space image;

[0024] Normalizing the CIELAB color space image based on the mean and the variance to obtain a normalized image;

[0025] Each pixel of the normalized image is converted from the CIELAB color space to the BGR color space to obtain the enhanced image.

[0026] Optionally, calling the annotation modification file to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library to obtain target annotation information corresponding to the annotation information includes:

[0027] Calling the annotation modification file to determine the image change parameter corresponding to the initial sample image according to the enhanced image and the initial sample image;

[0028] The annotation information is adjusted based on the image change parameter to obtain the target annotation information.

[0029] Optionally, generating a sample image for model training based on the enhanced image and the target annotation information includes:

[0030] replacing the original annotation information in the enhanced image with the target annotation information to obtain an annotated image, and using the annotated image as the sample image; or

[0031] The enhanced image is stored in a preset image format, and the target annotation information is stored in a preset file format, to obtain a sample image for model training.

[0032] In a second aspect, an embodiment of the present application provides a sample image processing device, the device comprising:

[0033] An initial image acquisition module, used to acquire an initial sample image containing annotation information;

[0034] An enhanced image acquisition module is used to call a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information;

[0035] A sample image generation module is used to generate a sample image for model training based on the enhanced image and the target annotation information.

[0036] Optionally, the Python data packet encapsulates an OpenCV library and a marked modification file.

[0037] The enhanced image acquisition module comprises:

[0038] An initial image importing unit, used for importing the initial sample image into the Python data package;

[0039] An enhanced image acquisition unit, used for calling the OpenCV library to perform enhancement processing on the initial sample image according to the target image enhancement mode corresponding to the initial sample image, so as to obtain an enhanced image corresponding to the initial sample image;

[0040] The target annotation acquisition unit is used to call the annotation modification file to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library, so as to obtain target annotation information corresponding to the annotation information.

[0041] Optionally, the enhanced image acquisition unit includes:

[0042] A target enhancement mode determination subunit, used for calling the OpenCV library to determine the target image enhancement mode corresponding to the initial sample image according to the image data distribution information of the initial sample image;

[0043] The enhanced image acquisition subunit is used to call the specified enhancement function in the OpenCV library to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image.

[0044] Optionally, the enhanced image acquisition subunit includes:

[0045] An intermediate image acquisition subunit is used to call the specified enhancement function to perform enhancement processing on the initial sample image based on the target image enhancement mode to obtain an intermediate enhanced image;

[0046] The first image acquisition subunit is used to perform color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image.

[0047] Optionally, the enhanced image acquisition subunit includes:

[0048] A spatial image acquisition subunit, used for converting each pixel of the intermediate enhanced image from the BGR color space to the CIELAB color space to obtain a CIELAB color space image;

[0049] A mean-variance calculation subunit, used for calculating the mean and variance of the CIELAB color space image;

[0050] A normalized image acquisition subunit, configured to perform normalization processing on the CIELAB color space image based on the mean and the variance to obtain a normalized image;

[0051] The second image acquisition subunit is used to convert each pixel of the normalized image from the CIELAB color space to the BGR color space to obtain the enhanced image.

[0052] Optionally, the target annotation acquisition unit includes:

[0053] An image change parameter determination subunit, configured to call the annotation modification file to determine the image change parameter corresponding to the initial sample image according to the enhanced image and the initial sample image;

[0054] The target annotation information acquisition subunit is used to adjust the annotation information based on the image change parameter to obtain the target annotation information.

[0055] Optionally, the sample image generation module includes:

[0056] a first sample image acquisition unit, configured to replace original annotation information in the enhanced image with the target annotation information to obtain an annotated image, and use the annotated image as the sample image;

[0057] The second sample image acquisition unit is used to store the enhanced image in a preset image format and the target annotation information in a preset file format to obtain a sample image for model training.

[0058] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0059] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the sample image processing methods described above is implemented.

[0060] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the sample image processing methods described above.

[0061] Compared with the prior art, the embodiments of the present application have the following advantages:

[0062] In an embodiment of the present application, an initial sample image containing annotation information is obtained. A pre-configured Python data package is called to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information. Based on the enhanced image and the target annotation information, a sample image for model training is generated. In an embodiment of the present application, a suitable data enhancement method is selected from the perspective of data feature distribution by a pre-configured Python data package to perform data enhancement on the image, and the annotation information of the enhanced image can be automatically modified, so that the requirements of most algorithms for data generalization can be completed with only a small number of samples, and only a small amount of manpower and time are required for annotation work during the reasoning process, saving manpower costs and model training time.

[0063] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A flow chart of the steps of a sample image processing method provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of an image processing process provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of the structure of a sample image processing device provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0069] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0070] Embodiment 1

[0071] Reference Figure 1 , shows a flow chart of the steps of a sample image processing method provided by an embodiment of the present application, such as Figure 1 As shown, the sample image processing method may include: step 101, step 102 and step 103.

[0072] Step 101: Obtain an initial sample image containing annotation information.

[0073] The embodiments of the present application can be applied to a scenario in which a pre-configured Python data package is used to select a specified enhancement method to enhance an image according to image data distribution information, and the image annotation is adjusted in combination with the enhanced image.

[0074] In this example, the initial sample image may be an image used for target detection, such as an image containing targets such as people and animals. It may also be an image used for smoking behavior detection, such as an image containing smoking behavior. Different types of initial sample images may be used to train corresponding neural network models, and this embodiment does not limit the type of initial sample images.

[0075] When performing generalization processing of sample images, an initial sample image containing annotation information may be obtained first. Specifically, taking animal detection as an example, images containing different animals may be obtained first, and the obtained images may be annotated by manual annotation or tool annotation to obtain an initial sample image containing annotation information.

[0076] It is understandable that the annotation information of the initial sample image can be directly annotated on the image, such as annotating the detection box position, type and other information on the image. The annotation information of the initial sample image can also be an image annotation file, etc. Specifically, the form of the annotation information can be determined according to actual conditions, and this embodiment does not limit this.

[0077] After the initial sample image containing the annotation information is acquired, step 102 is performed.

[0078] Step 102: Call a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and perform annotation adjustment processing on the annotation information to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information.

[0079] The Python data package refers to a data package pre-installed in the system for image data enhancement and annotation adjustment. In this example, the Python data package may be a tool package for image enhancement and annotation modification.

[0080] The enhanced image refers to the image obtained after the initial sample image is enhanced.

[0081] The target annotation information refers to the annotation information obtained after adjusting the original annotation information of the initial sample image.

[0082] After obtaining the initial sample image containing the annotation information, the previously configured Python data packet can be called to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain the enhanced image corresponding to the initial sample image and the target annotation information corresponding to the annotation information. In this embodiment, the Python data packet encapsulates the OpenCV library and the annotation modification file, wherein the OpenCV library contains different data enhancement algorithms, which can support data enhancement methods such as adding noise, adjusting brightness, performing cutout, rotating, cropping, translating, and mirroring.

[0083] Among them, OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. It can support multiple operating systems, such as Windows, Linux, macOS, iOS, and Android. In specific implementation, in OpenCV, image enhancement can be achieved in many ways, including brightness adjustment, contrast adjustment, histogram equalization, sharpening, blurring, etc.

[0084] The annotation modification file is a file used to modify the original annotation of the image. In this example, a Python script with an annotation modification function can be added to the annotation modification file.

[0085] The image enhancement processing process and the annotation adjustment process may be described in detail in conjunction with the following specific implementation methods.

[0086] In a specific implementation of the present application, the above step 102 may include:

[0087] Sub-step A1: Import the initial sample image into the Python data package.

[0088] In this embodiment, after the initial sample image containing the annotation information is obtained, the initial sample image can be imported into a Python data package.

[0089] After the initial sample image is imported into the Python data package, sub-step A2 is performed.

[0090] Sub-step A2: calling the OpenCV library to perform enhancement processing on the initial sample image according to the target image enhancement method corresponding to the initial sample image, so as to obtain an enhanced image corresponding to the initial sample image.

[0091] After the initial sample image is imported into the Python data package, the OpenCV library can be called to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and obtain the enhanced image corresponding to the initial sample image. Specifically, the OpenCV library can be called to determine the target image enhancement method corresponding to the initial sample image according to the image data distribution information of the initial sample image. Then, the specified enhancement function in the OpenCV library is called to enhance the initial sample image based on the target image enhancement method, and the enhanced image corresponding to the initial sample image is obtained.

[0092] In the specific implementation, the OpenCV library can be called first to determine the target image enhancement method corresponding to the initial sample image according to the image data distribution information of the initial sample image. For example, when detecting the abnormal opening and closing of the distribution box door, the shooting angle may not be straight, and there may be skewness and mirror flipping. At this time, the image enhancement method of radial transformation can be used, that is, the radial transformation enhancement method is used as the target image enhancement method. Alternatively, when the image data only has a frontal perspective and the daytime light is very strong, it is definitely unrealistic to improve the generalization of the model under different weather conditions and different perspectives. Therefore, you can choose to adjust the brightness and add noise to perform data enhancement under different lighting conditions, and you can perform data enhancement under different perspectives by rotation, mirroring, etc.

[0093] Of course, it is not limited to this. In practical applications, the target enhancement image can also include:

[0094] 1. For images with insufficient light or overexposure, the overall brightness can be adjusted by increasing or decreasing the brightness value of the image. At this time, the target image enhancement method can be brightness enhancement to improve the visual effect of the image.

[0095] 2. For images with low or high contrast, the contrast of the image can be enhanced by changing the distribution of brightness values ​​in the image to make the details in the image clearer. At this time, the target image enhancement method can be contrast enhancement to improve the detail expression of the image.

[0096] 3. For scenes that require adjustment of image color saturation, the color performance of the image can be affected by changing the color saturation. Increasing the saturation can make the color more vivid, while reducing the saturation will make the color tend to gray. At this time, the target image enhancement method can be the saturation adjustment method.

[0097] 4. For images with color distortion or requiring specific color effects. If the color distribution of the initial sample image is uneven or there is color cast, color correction can be used to adjust the overall tone and color balance of the image. At this time, the target image enhancement method can be color correction enhancement.

[0098] 5. For scenes with low light conditions and image transmission interference, if there is obvious noise (such as Gaussian noise, salt and pepper noise, etc.) in the initial sample image, the noise can be suppressed by smoothing filtering (such as mean filtering, median filtering, etc.) or deep learning-based methods (such as convolutional autoencoders). At this time, the target image enhancement method can be noise suppression enhancement.

[0099] 6. For scenes that need to improve image clarity or enhance image edge information, sharpening filters (such as Laplace filters, high-pass filters, etc.) or high-frequency enhancement filters can be used to enhance image edge information and details. At this time, the target image enhancement method can be detail enhancement.

[0100] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0101] After calling the OpenCV library to determine the target image enhancement method corresponding to the initial sample image according to the image data distribution information of the initial sample image, the specified enhancement function in the OpenCV library can be called to enhance the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image. In this example, the specified enhancement function can be a function provided by OpenCV, such as cv2.imread(), cv2.IMREAD_GRAYSCALE, etc.

[0102] In this embodiment, after the target image enhancement method is used to perform image data enhancement processing on the initial sample image, the enhanced image may also be color enhanced to improve the visual effect of the image. The implementation process may be described in detail in conjunction with the following specific implementation methods.

[0103] In another specific implementation of the present application, the above sub-step A2 may include:

[0104] Sub-step B1: calling the specified enhancement function to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an intermediate enhanced image.

[0105] In this embodiment, after the target image enhancement mode corresponding to the initial sample image is determined, a designated enhancement function may be called to perform enhancement processing on the initial sample image based on the target image enhancement mode to obtain an intermediate enhanced image.

[0106] After calling the designated enhancement function to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an intermediate enhanced image, sub-step B2 is executed.

[0107] Sub-step B2: performing color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image.

[0108] After calling the specified enhancement function to enhance the initial sample image based on the target image enhancement method to obtain an intermediate enhanced image, the intermediate enhanced image can be color enhanced to obtain an enhanced image corresponding to the initial sample image. The specific process of color enhancement is as follows:

[0109] 1. Convert each pixel of the intermediate enhanced image from the BGR color space to the CIELAB color space to obtain a CIELAB color space image.

[0110] In this example, the BGR color space is a variant of the RGB color space, which is mainly used in computer graphics and visual processing. In the BGR color space, colors are composed of three basic color channels: blue (B), green (G), and red (R). The values ​​of these three channels are usually quantized to a range of 0 to 255, representing different intensities of blue, green, and red. When the values ​​of these three channels are mixed in different proportions, a variety of colors can be produced.

[0111] The CIELAB color space (also known as the L*a*b* color space) is a uniform color space based on human visual perception. In the CIELAB color space, color is described by three parameters: L* (brightness), a* (chromaticity coordinates of the red-green axis), and b* (chromaticity coordinates of the yellow-blue axis).

[0112] After obtaining the intermediate enhanced image, each pixel of the intermediate enhanced image can be converted from the BGR color space to the CIELAB color space to obtain a CIELAB color space image. Specifically, since the CIELAB color space is based on the CIEXYZ color space, the BGR image needs to be converted to the XYZ color space first. Using the conversion formula from CIEXYZ to CIELAB, the values ​​in the XYZ color space are converted to the L* (brightness), a* (red-green axis chromaticity coordinates), and b* (yellow-blue axis chromaticity coordinates) values ​​in the CIELAB color space. These conversion formulas take into account the nonlinear perceptual characteristics of color to ensure that the converted color values ​​remain visually consistent.

[0113] 2. Calculate the mean and variance of the CIELAB color space image.

[0114] After obtaining the CIELAB color space image, the mean and variance of the CIELAB color space image can be calculated. Specifically, for the three channels of L*, a*, and b*, their means are calculated respectively. The variance is used to measure the degree of deviation of the data from its mean. The variance of the CIELAB color space image can be calculated by setting the variance calculation formula.

[0115] 3. Normalize the CIELAB color space image based on the mean and the variance to obtain a normalized image.

[0116] After the mean and variance of the CIELAB color space image are calculated, the CIELAB color space image can be normalized based on the mean and variance to obtain a normalized image.

[0117] 4. Convert each pixel of the normalized image from the CIELAB color space to the BGR color space to obtain the enhanced image.

[0118] After the CIELAB color space image is normalized based on the mean and variance to obtain a normalized image, each pixel of the normalized image can be converted from the CIELAB color space to the BGR color space to obtain an enhanced image.

[0119] After calling the specified enhancement function in the OpenCV library to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain the enhanced image corresponding to the initial sample image, sub-step A3 is executed.

[0120] Sub-step A3: calling the annotation modification file to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library, so as to obtain target annotation information corresponding to the annotation information.

[0121] After the specified enhancement function in the OpenCV library is called to enhance the initial sample image based on the target image enhancement method, and the enhanced image corresponding to the initial sample image is obtained, the annotation modification file can be called to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library to obtain the target annotation information corresponding to the annotation information. In this embodiment, the annotation modification file can be called to determine the image change parameters corresponding to the initial sample image according to the enhanced image and the initial sample image, and then the annotation information can be adjusted based on the image change parameters to obtain the target annotation information. Specifically, for data enhancement methods that do not modify the overall position of the image (such as adding noise, adjusting brightness, cutout, cropping, etc.), the annotation content of the original image can be directly saved. For data enhancement methods that modify the overall position of the image (rotation, translation, mirroring), at this time, it is necessary to adjust the annotation content according to the specific enhancement method. For example, for an enhancement method that performs translation, the direction and distance of the translation must be retained to modify the annotation content (x_min, y_min, x_max, y_max, where x_min, y_min represent the coordinate point of the upper left corner of the annotation box; the rest are similar, for the lower right corner) and the like. However, it is worth noting that the modified annotation content cannot exceed the boundary of the image.

[0122] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0123] The embodiment of the present application encapsulates the OpenCV library and the annotation modification file in a Python data packet, and can enhance the image data by adopting a specified enhancement method according to the image data distribution information. At the same time, the annotation can be automatically modified according to the enhanced image, without the need for manual selection of the enhancement method or manual annotation modification. While only a small number of samples are needed to meet the requirements of data generalization for most algorithms, only a small amount of manpower and time are required for annotation work during the reasoning process, saving manpower costs and model training time.

[0124] After obtaining the enhanced image corresponding to the initial sample image and the target annotation information corresponding to the annotation information, step 103 is performed.

[0125] Step 103: Based on the enhanced image and the target annotation information, a sample image for model training is generated.

[0126] After obtaining the enhanced image corresponding to the initial sample image and the target annotation information corresponding to the annotation information, a sample image for model training can be generated based on the enhanced image and the target annotation information. Specifically, the original annotation information on the enhanced image can be replaced based on the target annotation information, that is, the original annotation information on the enhanced image is replaced with the target annotation information to obtain a sample image. Alternatively, the enhanced image is stored in a preset image format, and the target annotation information is stored in a preset file format to obtain a sample image for model training. When saving the enhanced image and the target annotation information, the file names of the two must be the same, but the suffix names are different. The image can be saved in common formats such as jpg and png, and the target annotation information can be saved in formats such as xml.

[0127] Next, combine Figure 2 The image enhancement process is described in detail. Figure 2 , which shows a schematic diagram of an image processing flow provided by an embodiment of the present application. Figure 2 As shown, the process may include:

[0128] 1. Import the original data and annotation files. That is, import the original sample images and annotation information into the pre-configured Python data package.

[0129] 2. Select the data enhancement method. Analyze the functions that the algorithm needs to complete and the distribution of the imported image files, and complete the corresponding data enhancement method according to the functions that the algorithm needs to complete. For example, taking the abnormal recognition of the distribution box as an example, the ideal normal shooting angle is the front view, but in actual conditions, the camera angle may be offset or flipped. At this time, the radial transformation enhancement method can be used.

[0130] 3. Annotation file. Update the corresponding annotation file according to the enhancement method. For example, if the enhancement method is rotation, flipping, etc., which can offset the target of interest in the original image (for example, in the abnormal identification of the distribution box, we focus on the door of the distribution box), it is necessary to use affine transformation to obtain the final enhancement data and the corresponding annotation file.

[0131] The sample image processing method provided in the embodiment of the present application obtains an initial sample image containing annotation information. Call a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and perform annotation adjustment processing on the annotation information to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information. Based on the enhanced image and the target annotation information, a sample image for model training is generated. The embodiment of the present application selects a suitable data enhancement method to enhance the image from the perspective of data feature distribution through a pre-configured Python data package, and can automatically modify the annotation information of the enhanced image, so that only a small number of samples are needed to complete the requirements of most algorithms for data generalization, and only a small amount of manpower and time are required for annotation work during the reasoning process, saving manpower costs and model training time.

[0132] Embodiment 2

[0133] Reference Figure 3 , which shows a schematic diagram of the structure of a sample image processing device provided in an embodiment of the present application, such as Figure 3 As shown, the sample image processing device 300 may include the following modules:

[0134] An initial image acquisition module 310 is used to acquire an initial sample image containing annotation information;

[0135] The enhanced image acquisition module 320 is used to call a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information;

[0136] The sample image generation module 330 is used to generate a sample image for model training based on the enhanced image and the target annotation information.

[0137] Optionally, the Python data packet encapsulates an OpenCV library and a marked modification file.

[0138] The enhanced image acquisition module comprises:

[0139] An initial image importing unit, used for importing the initial sample image into the Python data package;

[0140] An enhanced image acquisition unit, used for calling the OpenCV library to perform enhancement processing on the initial sample image according to the target image enhancement mode corresponding to the initial sample image, so as to obtain an enhanced image corresponding to the initial sample image;

[0141] The target annotation acquisition unit is used to call the annotation modification file to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library, so as to obtain target annotation information corresponding to the annotation information.

[0142] Optionally, the enhanced image acquisition unit includes:

[0143] A target enhancement mode determination subunit, used for calling the OpenCV library to determine the target image enhancement mode corresponding to the initial sample image according to the image data distribution information of the initial sample image;

[0144] The enhanced image acquisition subunit is used to call the specified enhancement function in the OpenCV library to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image.

[0145] Optionally, the enhanced image acquisition subunit includes:

[0146] An intermediate image acquisition subunit is used to call the specified enhancement function to perform enhancement processing on the initial sample image based on the target image enhancement mode to obtain an intermediate enhanced image;

[0147] The first image acquisition subunit is used to perform color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image.

[0148] Optionally, the enhanced image acquisition subunit includes:

[0149] A spatial image acquisition subunit, used for converting each pixel of the intermediate enhanced image from the BGR color space to the CIELAB color space to obtain a CIELAB color space image;

[0150] A mean-variance calculation subunit, used for calculating the mean and variance of the CIELAB color space image;

[0151] A normalized image acquisition subunit, configured to perform normalization processing on the CIELAB color space image based on the mean and the variance to obtain a normalized image;

[0152] The second image acquisition subunit is used to convert each pixel of the normalized image from the CIELAB color space to the BGR color space to obtain the enhanced image.

[0153] Optionally, the target annotation acquisition unit includes:

[0154] An image change parameter determination subunit, configured to call the annotation modification file to determine the image change parameter corresponding to the initial sample image according to the enhanced image and the initial sample image;

[0155] The target annotation information acquisition subunit is used to adjust the annotation information based on the image change parameter to obtain the target annotation information.

[0156] Optionally, the sample image generation module includes:

[0157] a first sample image acquisition unit, configured to replace original annotation information in the enhanced image with the target annotation information to obtain an annotated image, and use the annotated image as the sample image;

[0158] The second sample image acquisition unit is used to store the enhanced image in a preset image format and the target annotation information in a preset file format to obtain a sample image for model training.

[0159] The sample image processing device provided in the embodiment of the present application obtains an initial sample image containing annotation information. Call a pre-configured Python data packet to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information. Based on the enhanced image and the target annotation information, a sample image for model training is generated. The embodiment of the present application selects a suitable data enhancement method to enhance the image from the perspective of data feature distribution through a pre-configured Python data packet, and can automatically modify the annotation information of the enhanced image, so that only a small number of samples are needed to complete the requirements of most algorithms for data generalization, and only a small amount of manpower and time are required for annotation work during the reasoning process, saving manpower costs and model training time.

[0160] Embodiment 3

[0161] An embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned sample image processing method when executed by the processor.

[0162] Figure 4 FIG. 4 is a schematic diagram showing the structure of an electronic device 400 according to an embodiment of the present invention. Figure 4As shown, the electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 402 or computer program instructions loaded from a storage unit 408 into a random access memory (RAM) 403. In RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. CPU 401, ROM 402 and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0163] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, a microphone, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0164] The various processes and processing described above may be performed by the processing unit 401. For example, the method of any of the above embodiments may be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the CPU 401, one or more actions in the method described above may be performed.

[0165] Embodiment 4

[0166] In addition, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned sample image processing method is implemented.

[0167] The above is a detailed introduction to a sample image processing method, a sample image processing device, an electronic device and a computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A sample image processing method, characterized in that: The method comprises: Obtain an initial sample image containing annotation information; Calling a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information; Based on the enhanced image and the target annotation information, a sample image for model training is generated.

2. The method according to claim 1, characterized in that The Python data package contains the OpenCV library and the annotation modification file. The calling of the pre-configured Python data packet performs enhancement processing on the initial sample image according to the target image enhancement method corresponding to the initial sample image, and performs annotation adjustment processing on the annotation information to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information, including: Importing the initial sample image into the Python data package; Calling the OpenCV library to perform enhancement processing on the initial sample image according to the target image enhancement method corresponding to the initial sample image, so as to obtain an enhanced image corresponding to the initial sample image; The annotation modification file is called to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library to obtain target annotation information corresponding to the annotation information.

3. The method according to claim 2, characterized in that The calling of the OpenCV library to perform enhancement processing on the initial sample image according to the target image enhancement mode corresponding to the initial sample image to obtain an enhanced image corresponding to the initial sample image includes: Calling the OpenCV library to determine the target image enhancement method corresponding to the initial sample image according to the image data distribution information of the initial sample image; The specified enhancement function in the OpenCV library is called to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image.

4. The method according to claim 3, characterized in that The calling of the specified enhancement function in the OpenCV library to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an enhanced image corresponding to the initial sample image includes: Calling the specified enhancement function to perform enhancement processing on the initial sample image based on the target image enhancement method to obtain an intermediate enhanced image; Performing color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image.

5. The method according to claim 4, characterized in that The performing color enhancement processing on the intermediate enhanced image to obtain an enhanced image corresponding to the initial sample image includes: Convert each pixel of the intermediate enhanced image from the BGR color space to the CIELAB color space to obtain a CIELAB color space image; Calculate the mean and variance of the CIELAB color space image; Normalizing the CIELAB color space image based on the mean and the variance to obtain a normalized image; Each pixel of the normalized image is converted from the CIELAB color space to the BGR color space to obtain the enhanced image.

6. The method according to claim 2, characterized in that The calling of the annotation modification file to perform annotation adjustment processing on the annotation information according to the enhanced image output by the OpenCV library to obtain target annotation information corresponding to the annotation information includes: Calling the annotation modification file to determine the image change parameter corresponding to the initial sample image according to the enhanced image and the initial sample image; The annotation information is adjusted based on the image change parameter to obtain the target annotation information.

7. The method according to claim 1, characterized in that The step of generating a sample image for model training based on the enhanced image and the target annotation information includes: replacing the original annotation information in the enhanced image with the target annotation information to obtain an annotated image, and using the annotated image as the sample image; or The enhanced image is stored in a preset image format, and the target annotation information is stored in a preset file format, to obtain a sample image for model training.

8. A sample image processing device, characterized in that: The device comprises: An initial image acquisition module, used to acquire an initial sample image containing annotation information; An enhanced image acquisition module is used to call a pre-configured Python data package to enhance the initial sample image according to the target image enhancement method corresponding to the initial sample image, and to perform annotation adjustment processing on the annotation information, so as to obtain an enhanced image corresponding to the initial sample image and target annotation information corresponding to the annotation information; A sample image generation module is used to generate a sample image for model training based on the enhanced image and the target annotation information.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the sample image processing method according to any one of claims 1 to 7 is implemented when the processor executes the program.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the sample image processing method according to any one of claims 1 to 7.