Image processing methods, apparatuses, electronic devices, and computer-readable storage media
By performing clarity detection and filtering on labeled images, blurry targets are eliminated and clear targets are retained, solving the problem of low detection accuracy of roadside equipment, achieving high-precision object detection and type recognition, and improving user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIDAO NETWORK TECH (BEIJING) CO LTD
- Filing Date
- 2023-05-15
- Publication Date
- 2026-07-17
AI Technical Summary
Due to limitations in computing power of roadside processing equipment and deep learning technology, existing detection models suffer from blurred semantic features of distant objects when processing high-definition images, resulting in low detection accuracy and an inability to provide high-precision object detection information.
By performing sharpness detection and filtering on the labeled targets in the labeled images, blurry and distant labeled targets are removed, and clear and key labeled target images are retained as training images. This adapts to the input of smaller-sized detection models and improves the accuracy of training images.
It improves the accuracy of the detection model in detecting and identifying objects in images, provides high-precision object detection information, and optimizes the user experience.
Smart Images
Figure CN116664919B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology for roadside sensing, and more specifically, to an image processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] In some road sections, the road scenes are relatively open. When roadside processing equipment captures images of the road conditions, it magnifies the original high-definition images, revealing objects 200 or even 300 meters away. However, current deep learning-based detection methods often require manual annotation of objects in the images. This annotated data is then used to train the detection model, enabling it to detect and identify objects in unannotated images.
[0003] Due to the limitations of computing power in roadside processing equipment and current deep learning technology, low-computing-power models are typically used for detection and recognition. Furthermore, because the original image and the input size of the detection model differ, the original image is scaled when inputting the image, resulting in blurred semantic features of distant objects in the shrunk image. These objects with unclear semantic features negatively impact the training of the detection model, causing errors in object detection and type recognition. Consequently, the detection model has low accuracy and cannot provide users with high-precision object detection information. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide an image processing method, apparatus, electronic device and computer-readable storage medium to improve the problem of low detection accuracy of detection models caused by unclear labeled objects in training data in the prior art.
[0005] To address the aforementioned problems, in a first aspect, embodiments of this application provide an image processing method, the method comprising:
[0006] Extract multiple labeled target images based on the bounding boxes in the labeled images;
[0007] For each of the labeled target images, a sharpness test is performed to obtain the corresponding sharpness result;
[0008] Training images are obtained by filtering multiple labeled target images based on multiple clarity results.
[0009] In the above implementation process, by extracting individual labeled target images corresponding to multiple labeled targets in the labeled image, the clarity of each labeled target image can be detected. Based on the detected clarity, labeled targets are filtered to remove blurry and distant labeled targets, retaining clear and crucial labeled target images as training images. This allows data from high-resolution images to be transferred and reused in smaller images to filter out unclear labeled targets, effectively improving the accuracy of the obtained training images. This, in turn, improves the accuracy of the detection model in detecting and identifying roadside objects when trained on these images. It provides users with high-precision object detection information to achieve various functions such as warnings and prompts, optimizing the user experience.
[0010] Optionally, the step of extracting multiple labeled target images based on the labeled bounding boxes in the labeled images includes:
[0011] The corresponding cropping region is determined by expanding each of the labeled boxes in the labeled image.
[0012] Based on each of the cropping regions, cropping is performed in the labeled image to obtain the corresponding labeled target image.
[0013] In the above implementation process, when extracting the labeled target images corresponding to each labeled target in the labeled image, in order to better distinguish the labeled target from the background region, each labeled box can be expanded outward to obtain a larger cropping area. This cropping area is then used to obtain the labeled target image containing both the labeled target and the background region. By using the expansion processing to crop the image containing both the labeled target and the background region, the integrity of the labeled target image is effectively improved, facilitating subsequent sharpness detection of the labeled target image.
[0014] Optionally, the step of performing sharpness detection on each of the labeled target images to obtain the corresponding sharpness result includes:
[0015] Based on the variance algorithm, the variance data of each labeled target image is calculated;
[0016] The sharpness result of the corresponding labeled target image is obtained based on the variance data.
[0017] In the above implementation process, the variance data corresponding to each labeled target image is calculated through multiple variance algorithms, and the sharpness result corresponding to the labeled target is determined based on the variance data. This enables fast and accurate detection of image sharpness, effectively improving the accuracy and validity of the sharpness results.
[0018] Optionally, obtaining the sharpness result of the corresponding labeled target image based on the variance data includes:
[0019] If the variance data is determined to be higher than or equal to a preset variance threshold, then the sharpness result of the labeled target image is sharp.
[0020] If the variance data is determined to be lower than the preset variance threshold, then the sharpness result of the labeled target image is blurred.
[0021] In the above implementation process, a corresponding variance threshold can be preset according to the actual situation and needs. If the variance data is higher than or equal to the preset variance threshold, it indicates that the pixel variation of the labeled target image is large, the image contrast is high, and the corresponding sharpness result is sharp. If the variance data is lower than the preset variance threshold, it indicates that the pixel variation of the labeled target image is small, the image contrast is low, and the corresponding sharpness result is blurry. The ability to determine the image sharpness result based on the magnitude of the variance data further improves the accuracy and effectiveness of the sharpness result.
[0022] Optionally, the step of filtering multiple labeled target images based on multiple sharpness results to obtain training images includes:
[0023] If the sharpness result of the labeled target image is sharp, then the first target and the corresponding first label box in the sharp labeled target image are retained;
[0024] If the clarity result of the labeled target image is blurry, then the size feature of the blurry labeled target image is obtained; and based on the size feature, the second target and the corresponding second label box in the blurred labeled target image are filtered in the labeled image to obtain the key target and the corresponding key label box;
[0025] In the labeled image, the retained first target and first bounding box, as well as the key target and key bounding box, are integrated as training targets and training bounding boxes to obtain the training image.
[0026] In the above implementation process, the processing methods differ depending on the sharpness of the labeled target images. Labeled targets and their corresponding bounding boxes in sharp images can be retained, while those in blurry images can be filtered to retain only the key labeled targets and their corresponding bounding boxes. These retained labeled targets and bounding boxes are then integrated as training targets and training bounding boxes to obtain the corresponding training images. This filtering and retention process effectively ensures the sharpness and effectiveness of training targets and bounding boxes in the training images, thereby improving the effectiveness of the training images and ultimately enhancing the detection performance of the detection model trained on them.
[0027] Optionally, the step of filtering the second target and its corresponding second bounding box in the blurred labeled target image based on the size feature to obtain the key target and its corresponding key bounding box includes:
[0028] The size feature is compared with the size threshold corresponding to the labeled image;
[0029] If the size feature is determined to be greater than or equal to the size threshold, then the second target in the blurred labeled target image is taken as the key target, and the second label box corresponding to the second target is taken as the key label box.
[0030] In the above implementation process, since the detection model needs to identify not only the type of target object but also detect the corresponding target object, for blurred labeled target images, it can also compare their corresponding size features with size thresholds to determine whether the labeled targets are key targets from a size perspective, thus preserving blurred key targets and their corresponding key bounding boxes. This method can filter key targets from multiple blurred labeled targets through size comparison. Although key targets may be blurred, the detection model still needs to detect these types of targets to improve the completeness of labeled targets in the training image, thereby improving the detection model's performance in detecting key targets.
[0031] Optionally, the method further includes:
[0032] The acquired images are labeled to obtain the initial image;
[0033] The initial image is resized based on the input dimensions of the detection model to obtain the labeled image;
[0034] The labeled image includes multiple labeled targets and corresponding labeled boxes.
[0035] In the above implementation process, the images acquired by the roadside processing equipment can be annotated first to label multiple target objects in the acquired images, resulting in an initial image with multiple labeled targets and corresponding bounding boxes. Furthermore, since there is a certain difference between the initial image and the input size of the detection model, in order to transfer the various labeled data from the high-resolution image to the smaller image of the corresponding size, the initial image can be resized based on the input size of the detection model to obtain an annotated image that meets the input conditions of the detection model. This allows for image size adjustment to accommodate detection models with various input sizes.
[0036] Secondly, embodiments of this application also provide an image processing apparatus, the apparatus comprising: an extraction module, a detection module, and a filtering module;
[0037] The extraction module is used to extract multiple labeled target images based on the labeled bounding boxes in the labeled images;
[0038] The detection module is used to perform sharpness detection on each of the labeled target images and obtain the corresponding sharpness result;
[0039] The filtering module is used to filter multiple labeled target images based on multiple clarity results to obtain training images.
[0040] In the above implementation process, the extraction module extracts individual labeled target images corresponding to multiple labeled targets in the labeled image, the detection module detects whether each labeled target image is clear, and the filtering module filters the labeled targets according to the detection clarity results to remove some blurry and distant labeled targets and retain clear and key labeled target images as training images.
[0041] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above-described implementations of the image processing method.
[0042] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which are read and executed by a processor to perform steps in any of the above-described implementations of the image processing method.
[0043] In summary, the embodiments of this application provide an image processing method, apparatus, electronic device, and computer-readable storage medium that can transfer and reuse the marked targets and bounding boxes in high-definition images to smaller-sized images to filter out unclear marked targets in the images. This effectively improves the accuracy of the obtained training images, thereby improving the accuracy of the detection model in detecting and identifying roadside objects in the images when trained based on the training images. It can provide users with high-precision object detection information to achieve multiple functions such as warnings and prompts, and optimize the user experience. Attached Figure Description
[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application;
[0046] Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application;
[0047] Figure 3 A detailed flowchart of step S200 provided for an embodiment of this application;
[0048] Figure 4 A detailed flowchart of step S300 provided for an embodiment of this application;
[0049] Figure 5 A detailed flowchart of step S400 provided for an embodiment of this application;
[0050] Figure 6 A detailed flowchart of step S420 provided for an embodiment of this application;
[0051] Figure 7 A flowchart illustrating another image processing method provided in an embodiment of this application;
[0052] Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application.
[0053] Icons: 100 - Electronic device; 111 - Memory; 112 - Memory controller; 113 - Processor; 114 - Peripheral interface; 115 - Input / output unit; 116 - Display unit; 600 - Image processing device; 610 - Extraction module; 620 - Detection module; 630 - Screening module. Detailed Implementation
[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.
[0055] Due to the computational limitations of roadside processing equipment, low-computing-power detection models are typically used to detect and identify various objects on the road. Furthermore, when processing images, the original image acquired by the processing equipment differs in size from the input image of the detection model. The input image is scaled down, resulting in blurred semantic features of distant objects in the reduced image. These unclear semantic features negatively impact the training of the detection model, leading to errors in object detection and type recognition. Consequently, the detection model's accuracy is low, failing to provide users with high-precision object detection information.
[0056] To address the aforementioned issues, this application provides an image processing method applied to electronic devices, such as servers, personal computers (PCs), tablets, smartphones, and personal digital assistants (PDAs) with logical computing capabilities. This method enables the transfer and reuse of data annotated on high-definition images onto smaller images, thereby filtering out unclear annotated targets and improving the accuracy of the acquired training images.
[0057] Optionally, please refer to Figure 1 , Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.
[0058] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.
[0059] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.
[0060] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.
[0061] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.
[0062] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 can be, but is not limited to, a mouse and a keyboard.
[0063] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In this embodiment, the display unit 116 can display various images and data such as labeled images, labeled boxes, labeled targets, labeled target images, and training images.
[0064] The electronic device in this embodiment can be used to execute various steps in the various image processing methods provided in the embodiments of this application. The implementation process of the image processing methods is described in detail below through several embodiments.
[0065] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application. The method may include steps S200-S400.
[0066] Step S200: Extract multiple labeled target images based on the labeled bounding boxes in the labeled images.
[0067] The labeled image includes multiple labeled targets and corresponding bounding boxes. The labeled targets are various objects present in the image, such as humans, vehicles, road signs, streetlights, plants, and obstacles. The bounding boxes can be frames set according to the contours of various labeled targets. The labeled targets and their corresponding bounding boxes can be extracted from the complete labeled image to obtain a small image of each labeled target, allowing for individual processing of each labeled target image.
[0068] Step S300: Perform sharpness detection on each labeled target image to obtain the corresponding sharpness result.
[0069] Due to factors such as the distance between each labeled target and the shooting device during shooting, as well as the position and state of the labeled targets, the sharpness of the labeled targets in each image varies. Therefore, a sharpness test can be performed on each labeled target image individually to determine the corresponding sharpness result.
[0070] Step S400: Select multiple labeled target images based on multiple resolution results to obtain training images.
[0071] When the detection model detects and identifies types of blurry and uncritical labeled targets, it is easy to fail to detect or misidentify types. Therefore, in order to reduce the adverse effects of labeled targets in blurry and uncritical labeled target images on the training of the detection model, the images can be filtered according to the clarity of each labeled target image to include clear or critical labeled targets and bounding boxes, so as to obtain the corresponding training images.
[0072] Optionally, multiple different labeled images can be processed to obtain multiple different training images, which are then combined to form a training dataset.
[0073] exist Figure 2 In the illustrated embodiment, the data labeled in the high-definition image can be transplanted and reused in a smaller image to filter out unclear labeled targets in the image, effectively improving the accuracy of the obtained training image. This improves the accuracy of the detection model in detecting and identifying roadside objects in the image when it is trained based on the training image. It can provide users with high-precision object detection information to achieve multiple functions such as warnings and prompts, and optimize the user experience.
[0074] Optionally, please refer to Figure 3 , Figure 3 The following is a detailed flowchart of step S200 provided in an embodiment of this application. Step S200 may also include steps S210-S220.
[0075] Step S210: Expand each bounding box in the labeled image to determine the corresponding cropping area.
[0076] In the process of extracting the labeled target images corresponding to each labeled target in the labeled image, in order to better distinguish the labeled target from the background area, each labeled box can be expanded outward to obtain a larger cropping area.
[0077] For example, the expansion process can be to expand the pixel range of the annotation box outward in each of the top, bottom, left, and right directions by a certain threshold, such as expanding by 20 pixels each, or to expand the pixel range of the four vertices of the annotation box along the diagonal by a certain threshold, such as expanding by 10 pixels each.
[0078] Step S220: Based on each cropping region, cropping is performed in the labeled image to obtain the corresponding labeled target image.
[0079] Specifically, it can crop within the complete labeled image based on the cropping region, thus obtaining a labeled target image that includes both the labeled target and the background region.
[0080] exist Figure 3 In the illustrated embodiment, the image containing the labeled target and background regions is cropped by expanding the image, which effectively improves the integrity of the labeled target image and facilitates subsequent sharpness detection of the labeled target image.
[0081] Optionally, please refer to Figure 4 , Figure 4 This is a detailed flowchart of step S300 provided in an embodiment of the present application. Step S300 may also include steps S310-S320.
[0082] Step S310: Based on the variance algorithm, calculate the variance data of each labeled target image.
[0083] Among them, the variance algorithm can include various algorithms such as the Laplacian gradient method or Sobel, which can calculate the variance data of the gradient based on each labeled target image.
[0084] Step S320: Obtain the sharpness result of the corresponding labeled target image based on the variance data.
[0085] Among them, the overall situation of the labeled target image can be determined based on the magnitude of the variance data, thereby determining the sharpness result of the labeled target image.
[0086] Optionally, a corresponding variance threshold can be preset according to actual conditions and needs. The variance threshold can be set to a relatively high threshold to represent a clearer image. The sharpness result can be determined by comparing the variance data with the preset variance threshold. It should be noted that if the variance data is determined to be higher than or equal to the preset variance threshold, it indicates that the pixel variation of the labeled target image is large, the image contrast is high, and the corresponding sharpness result of the labeled target image is clear; if the variance data is determined to be lower than the preset variance threshold, it indicates that the pixel variation of the labeled target image is small, the image contrast is low, and the corresponding sharpness result of the labeled target image is blurry.
[0087] exist Figure 4 In the illustrated embodiment, the image sharpness result can be determined based on the magnitude of the variance data, further improving the accuracy and effectiveness of the sharpness result.
[0088] Optionally, please refer to Figure 5 , Figure 5 This is a detailed flowchart of step S400 provided in an embodiment of the present application. Step S400 may also include steps S410-S440.
[0089] Step S410: If the clarity result of the labeled target image is clear, then retain the first target and the corresponding first label box in the clear labeled target image.
[0090] The processing methods differ depending on the sharpness of the labeled target image. When the sharpness of the labeled target image is clear, the first target within the image is clear, resulting in a high success rate for detection and accuracy in type identification by the detection model. Therefore, the first target and the first bounding box can be preserved.
[0091] Step S420: If the clarity result of the labeled target image is blurry, then obtain the size features of the blurry labeled target image.
[0092] When the clarity result of the labeled target image is blurry, the second target in the labeled target image is also blurry. However, considering that some key targets, such as close objects and large objects, may be blurry during detection, they still need to be detected. Therefore, the size features of the blurry labeled target image can be obtained first for subsequent filtering.
[0093] Step S430: Based on size features, filter the second target and the corresponding second annotation box in the blurred annotation target image in the annotation image to obtain the key target and the corresponding key annotation box.
[0094] Specifically, the fuzzy second target and its corresponding second bounding box can be filtered based on size features to determine the key targets and key bounding boxes that need to be retained.
[0095] Step S440: In the labeled image, the retained first target and first bounding box, as well as key targets and key bounding boxes, are integrated as training targets and training bounding boxes to obtain a training image.
[0096] Among them, the retained first target and first bounding box, as well as key targets and key bounding boxes, can be integrated in the original image to obtain the corresponding training image.
[0097] exist Figure 5 In the illustrated embodiment, clear and key labeled targets and bounding boxes can be filtered and retained, effectively ensuring the clarity and effectiveness of training targets and bounding boxes in the training images, thereby improving the effectiveness of the training images and thus improving the detection performance of the detection model after training based on the training images.
[0098] Optionally, please refer to Figure 6 , Figure 6 The following is a detailed flowchart of step S420 provided in an embodiment of this application. Step S420 may also include steps S421-S422.
[0099] Step S421: Compare the size features with the size threshold corresponding to the labeled image.
[0100] Among them, the size features can be the length, width, height and other feature data of the annotation box in the annotation image. The corresponding size threshold can be set according to the actual shooting position, wide angle and other parameters of the annotation image. The size threshold can also include the length, width and height data, so that the size features and the various types of data in the size threshold can be compared separately.
[0101] Step S422: If the size feature is determined to be greater than or equal to the size threshold, then the second target in the fuzzy labeled target image is taken as the key target, and the second label box corresponding to the second target is taken as the key label box.
[0102] Among them, when any one or more of the length, width, and height data in the size features are greater than or equal to the size threshold, it indicates that the second target in the fuzzy labeled target image is larger or closer, and such a second target can be used as the key target, and the corresponding second annotation box can be used as the key annotation box for preservation processing.
[0103] It should be noted that if the size feature is determined to be smaller than the size threshold, for example, if the length, width and height data in the size feature are all smaller than the length, width and height data in the size threshold, then it indicates that the second target in the fuzzy labeled target image is small or far away, and the semantic features of the second target are not obvious. In this case, such a second target is a non-critical target, and the non-critical target and the corresponding non-critical label box are deleted in the labeled image.
[0104] exist Figure 6 In the illustrated embodiment, key targets can be filtered from multiple blurred labeled targets by size comparison. Although key targets are blurred, the detection model still needs to detect these targets to improve the completeness of labeled targets in the training image, thereby improving the detection model's performance in detecting key targets.
[0105] Optionally, please refer to Figure 7, Figure 7 This is a flowchart illustrating another image processing method provided in an embodiment of this application. The method may further include steps S510-S520.
[0106] Step S510: The acquired images are annotated to obtain the initial image.
[0107] In this process, the images collected by the roadside processing equipment can be labeled using methods such as model labeling and manual labeling to label multiple target objects in the collected images, thus obtaining an initial image with multiple labeled targets and corresponding label boxes.
[0108] Step S520: Adjust the size of the initial image based on the input size of the detection model to obtain the labeled image.
[0109] Since there is a certain difference between the initial image and the input size of the detection model, in order to transfer the various data annotated in the high-definition image to the corresponding small-sized image, the initial image can be resized based on the input size of the detection model to obtain an annotated image that meets the input conditions of the detection model.
[0110] For example, when the initial image size is 4k, such as a resolution of 3840*3840, the input size of the detection model is 960*960. Therefore, the initial image can be scaled down proportionally to the input storage, and the bounding boxes in the initial image will also be scaled down accordingly.
[0111] exist Figure 7 In the illustrated embodiment, the image size can be adjusted to suit detection models with various input sizes.
[0112] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device 600 may include: an extraction module 610, a detection module 620, and a filtering module 630.
[0113] Extraction module 610 is used to extract multiple labeled target images based on the labeled bounding boxes in the labeled images;
[0114] The detection module 620 is used to perform sharpness detection on each labeled target image and obtain the corresponding sharpness result;
[0115] The filtering module 630 is used to filter multiple labeled target images based on multiple sharpness results to obtain training images.
[0116] In an optional implementation, the extraction module 610 may include an expansion submodule and a trimming submodule;
[0117] The expansion submodule is used to expand each bounding box in the labeled image to determine the corresponding cropping region;
[0118] The cropping submodule is used to crop the labeled image based on each cropping region to obtain the corresponding labeled target image.
[0119] In an optional implementation, the detection module 620 is specifically used to: calculate the variance data of each labeled target image based on a variance algorithm; and obtain the sharpness result of the corresponding labeled target image based on the variance data.
[0120] In an optional implementation, the detection module 620 is specifically used to: if the variance data is determined to be higher than or equal to a preset variance threshold, the sharpness result of the labeled target image is sharp; if the variance data is determined to be lower than the preset variance threshold, the sharpness result of the labeled target image is blurry.
[0121] In an optional implementation, the filtering module 630 may include a clear submodule, a fuzzy submodule, and an integration submodule;
[0122] If the sharpness result of the labeled target image is sharp, the sharpness submodule is used to retain the first target and the corresponding first label box in the sharp labeled target image;
[0123] If the clarity result of the labeled target image is blurry, the blurry submodule is used to obtain the size features of the blurry labeled target image; and based on the size features, the second target and the corresponding second annotation box in the blurred labeled target image are filtered in the labeled image to obtain the key target and the corresponding key annotation box;
[0124] The integration submodule is used to integrate the retained first target and first bounding box, as well as key targets and key bounding boxes, as training targets and training bounding boxes in the labeled image to obtain a training image.
[0125] In an optional implementation, the fuzzing submodule is specifically used to: compare the size feature with the size threshold corresponding to the labeled image; if it is determined that the size feature is greater than or equal to the size threshold, then the second target in the fuzzed labeled target image is taken as the key target, and the second labeling box corresponding to the second target is taken as the key labeling box.
[0126] In an optional embodiment, the image processing apparatus 600 may further include: an adjustment module for performing annotation processing on the acquired image to obtain an initial image; adjusting the size of the initial image based on the input size of the detection model to obtain an annotated image; wherein the annotated image includes multiple annotated targets and corresponding annotation boxes.
[0127] Since the principle of the image processing device 600 in this embodiment is similar to that of the aforementioned image processing method embodiment, the implementation of the image processing device 600 in this embodiment can refer to the description in the above-mentioned image processing method embodiment, and repeated details will not be repeated.
[0128] This application also provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, they perform the steps of any of the image processing methods provided in this embodiment.
[0129] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0130] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0131] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0132] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0133] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
[0134] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. An image processing method, characterized in that, The method includes: Extract multiple labeled target images based on the bounding boxes in the labeled images; For each of the labeled target images, a sharpness test is performed to obtain the corresponding sharpness result; Training images are obtained by filtering multiple labeled target images based on multiple clarity results; The step of filtering multiple labeled target images based on multiple sharpness results to obtain a training image includes: if the sharpness result of the labeled target image is sharp, then retaining the first target and the corresponding first annotation box in the sharp labeled target image; if the sharpness result of the labeled target image is blurry, then obtaining the size features of the blurry labeled target image; and based on the size features, filtering the second target and the corresponding second annotation box in the blurry labeled target image in the labeled image to obtain the key target and the corresponding key annotation box; and integrating the retained first target and first annotation box, as well as the key target and the key annotation box, as training targets and training annotation boxes in the labeled image to obtain the training image.
2. The method according to claim 1, characterized in that, The extraction of multiple labeled target images based on the bounding boxes in the labeled images includes: The corresponding cropping region is determined by expanding each of the labeled boxes in the labeled image. Based on each of the cropping regions, cropping is performed in the labeled image to obtain the corresponding labeled target image.
3. The method according to claim 1, characterized in that, The step of performing sharpness detection on each of the labeled target images to obtain the corresponding sharpness result includes: Based on the variance algorithm, the variance data of each labeled target image is calculated; The sharpness result of the corresponding labeled target image is obtained based on the variance data.
4. The method according to claim 3, characterized in that, The step of obtaining the sharpness result of the corresponding labeled target image based on the variance data includes: If the variance data is determined to be higher than or equal to a preset variance threshold, then the sharpness result of the labeled target image is sharp. If the variance data is determined to be lower than the preset variance threshold, then the sharpness result of the labeled target image is blurred.
5. The method according to claim 1, characterized in that, Based on the size feature, the process of filtering the second target and its corresponding second bounding box in the blurred labeled target image to obtain key targets and their corresponding key bounding boxes includes: The size feature is compared with the size threshold corresponding to the labeled image; If the size feature is determined to be greater than or equal to the size threshold, then the second target in the blurred labeled target image is taken as the key target, and the second label box corresponding to the second target is taken as the key label box.
6. The method according to claim 1, characterized in that, The method further includes: The acquired images are labeled to obtain the initial image; The initial image is resized based on the input dimensions of the detection model to obtain the labeled image; The labeled image includes multiple labeled targets and corresponding labeled boxes.
7. An image processing apparatus, characterized in that, The device includes: an extraction module, a detection module, and a screening module; The extraction module is used to extract multiple labeled target images based on the labeled bounding boxes in the labeled images; The detection module is used to perform sharpness detection on each of the labeled target images to obtain the corresponding sharpness result; The filtering module is used to filter multiple labeled target images based on multiple sharpness results to obtain training images; The filtering module is specifically used for: if the sharpness result of the labeled target image is sharp, then retaining the first target and the corresponding first annotation box in the sharp labeled target image; if the sharpness result of the labeled target image is blurry, then obtaining the size features of the blurry labeled target image; and based on the size features, filtering the second target and the corresponding second annotation box in the blurry labeled target image in the labeled image to obtain the key target and the corresponding key annotation box; and integrating the retained first target and first annotation box, as well as the key target and the key annotation box, as training targets and training annotation boxes in the labeled image to obtain the training image.
8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores program instructions, and when the processor executes the program instructions, it performs the steps of the method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1-6.