Image processing method and device, computer readable storage medium and electronic device
By using edge detection and brightness distribution recognition technologies, the problem of motion blur in vehicle-mounted equipment images under nighttime conditions is solved, ensuring the effective execution of machine vision tasks and avoiding image quality degradation caused by reduced exposure time.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING HORIZON INFORMATION TECH CO LTD
- Filing Date
- 2023-05-12
- Publication Date
- 2026-07-31
AI Technical Summary
In nighttime environments, images captured by vehicle-mounted equipment are prone to motion blur, which affects the performance of machine vision tasks. Existing technologies reduce motion blur by reducing exposure time, but this leads to a decrease in image signal-to-noise ratio and brightness.
By performing edge detection on the image to be processed, the bounding box of the moving object is determined, and the target ghosting is identified based on a preset brightness distribution ratio range. Pre-defined processing is then performed to remove the ghosting and avoid reducing the exposure time.
It effectively removes motion blur, ensuring the performance of machine vision tasks while avoiding the negative impact of reduced exposure time on image quality.
Smart Images
Figure CN116597415B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of driving technology, and in particular to an image processing method, apparatus, computer-readable storage medium, and electronic device. Background Technology
[0002] Vehicle-mounted devices (such as vehicle-mounted cameras) often capture images of vehicles traveling at high speeds. As a result, in nighttime environments, the increased exposure time can easily cause motion blur in the images captured by these devices. Summary of the Invention
[0003] To address the technical problem of motion blur appearing in images captured by vehicle-mounted devices in nighttime environments, which can negatively impact the performance of machine vision tasks when these images are used, this disclosure is proposed. Embodiments of this disclosure provide an image processing method, apparatus, computer-readable storage medium, and electronic device.
[0004] According to one aspect of the present disclosure, an image processing method is provided, comprising:
[0005] Edge detection is performed on the image to be processed to determine the bounding boxes of moving objects in the image to be processed;
[0006] Based on the preset brightness distribution ratio range and the brightness values of the pixels in the region enclosed by the bounding box, the target trailing recognition result of the moving object is obtained.
[0007] Based on the target trailing image recognition result, a predetermined process is performed on the image to be processed.
[0008] According to another aspect of the present disclosure, an image processing apparatus is provided, comprising:
[0009] The first determining module is used to perform edge detection on the image to be processed and determine the bounding box of the moving object in the image to be processed;
[0010] The acquisition module is used to obtain the target trailing image recognition result of the moving object based on the brightness values of the pixels in the region enclosed by the bounding box determined by the first determination module and a preset brightness distribution ratio range.
[0011] The execution module is used to perform predetermined processing on the image to be processed based on the target trailing recognition result obtained by the acquisition module.
[0012] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the above-described image processing method.
[0013] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:
[0014] processor;
[0015] Memory used to store the processor's executable instructions;
[0016] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image processing method described above.
[0017] Based on the image processing method, apparatus, computer-readable storage medium, and electronic device provided in the above embodiments of this disclosure, edge detection can be performed on the image to be processed to determine the bounding box of the moving object in the image. Combined with a preset brightness distribution ratio range, motion blur recognition can be performed to obtain the target motion blur recognition result. The target motion blur recognition result can be used for predetermined processing of the image to be processed, so as to efficiently and reliably achieve the predetermined machine vision task and avoid the adverse effects of motion blur on the normal execution of the machine vision task, thereby helping to ensure the execution effect of the machine vision task. Furthermore, in the embodiments of this disclosure, it is not necessary to reduce the exposure time in night scene environments to reduce motion blur; instead, the image can be captured with normal exposure time. This helps to avoid the adverse effects of reduced exposure time on the image signal-to-noise ratio and brightness, thereby helping to ensure the execution effect of the machine vision task.
[0018] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0019] Figure 1 This is a schematic flowchart of an image processing method provided in an exemplary embodiment of this disclosure.
[0020] Figure 2 This is a schematic flowchart illustrating the process of obtaining target motion blur recognition results in an image processing method provided by an exemplary embodiment of this disclosure.
[0021] Figure 3 This is a schematic diagram of the process of dividing the region enclosed by the bounding box into a first shadow region and a first main body region in an image processing method provided by an exemplary embodiment of the present disclosure.
[0022] Figure 4 This is a flowchart illustrating the process of determining whether a first motion blur region meets a preset integrity condition in an image processing method provided by an exemplary embodiment of this disclosure.
[0023] Figure 5 This is another flowchart illustrating the process of determining whether a first motion blur region meets a preset integrity condition in an image processing method provided by an exemplary embodiment of this disclosure.
[0024] Figure 6 This is a schematic diagram of a moving object and a motion blur region in an image processing method provided by an exemplary embodiment of this disclosure.
[0025] Figure 7 This is a schematic flowchart illustrating the process of determining the bounding box of a moving object in an image to be processed in an image processing method provided by an exemplary embodiment of this disclosure.
[0026] Figure 8 This is a schematic flowchart of an image processing method provided in another exemplary embodiment of this disclosure.
[0027] Figure 9 This is a schematic diagram of the structure of an image processing apparatus provided in an exemplary embodiment of the present disclosure.
[0028] Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in another exemplary embodiment of the present disclosure.
[0029] Figure 11 This is a schematic diagram of the structure of an image processing apparatus provided in another exemplary embodiment of the present disclosure.
[0030] Figure 12 This is a schematic diagram of the structure of an image processing apparatus provided in yet another exemplary embodiment of this disclosure.
[0031] Figure 13 This is a schematic diagram of the structure of an image processing apparatus provided in yet another exemplary embodiment of this disclosure.
[0032] Figure 14 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation
[0033] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.
[0034] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of this disclosure.
[0035] Application Overview
[0036] In night scene environments, due to the increased exposure time, images captured by vehicle-mounted equipment are prone to ghosting. Ghosting generally refers to the traces that appear when moving objects are imaged on a Complementary Metal Oxide Semiconductor (CMOS) image due to excessively long exposure times, with the edges repeating.
[0037] In the process of realizing this disclosure, the inventors discovered that due to the presence of motion blur, when using images captured by vehicle-mounted equipment for machine vision tasks, it can easily affect the performance of machine vision tasks, such as causing task failure.
[0038] To mitigate the adverse effects of motion blur, a common approach is to reduce exposure time in low-light conditions. However, reducing exposure time affects the image signal-to-noise ratio and brightness, leading to a decrease in image quality and consequently impacting the performance of machine vision tasks. Therefore, a more effective measure is needed to reduce the negative impact of motion blur.
[0039] Exemplary methods
[0040] Figure 1 This is a schematic flowchart of an image processing method provided in an exemplary embodiment of this disclosure. Figure 1 The method shown may include steps 110, 120 and 130, which are described below.
[0041] Step 110: Perform edge detection on the image to be processed to determine the bounding boxes of moving objects in the image.
[0042] In step 110, an edge detection algorithm can be used to perform edge detection on the image to be processed, so as to identify pixels with obvious brightness changes in the image to be processed, thereby determining the bounding box of the moving object in the image to be processed.
[0043] Optionally, the edge detection algorithm may include the following steps: 1. Smoothing the image using a filter (e.g., a Gaussian filter); 2. Calculating the gradient magnitude and direction of the image; 3. Performing non-maximum suppression on the gradient magnitude; 4. Detecting and connecting edges using a double thresholding algorithm.
[0044] Optionally, the image to be processed can be an RGB image captured in real time by the in-vehicle device, or an RGB image captured historically by the in-vehicle device; where R represents red, G represents green, and B represents blue.
[0045] Optionally, the moving object can be an object with mobility, or a part of an object with mobility; wherein, the object with mobility includes, but is not limited to, pedestrians, vehicles, trains, intelligent robots, etc.
[0046] Step 120: Based on the brightness values of pixels in the region enclosed by the preset brightness distribution ratio range and the bounding box, obtain the target trailing image recognition result of the moving object.
[0047] It should be noted that the preset brightness distribution ratio range can refer to a pre-set brightness distribution ratio range that can be used to distinguish between the moving object itself and its trailing shadow. Optionally, the preset brightness distribution ratio range can be represented as [P1, P2], where P1 can be 5%, 10%, 15% or other values, and P2 can be 80%, 85%, 90% or other values, which will not be listed here.
[0048] In step 120, the brightness values of all pixels (assuming there are N pixels) within the bounding box region can be determined to obtain N brightness values. Referring to a preset brightness distribution range and the N brightness values, it can be determined which pixels among the N pixels represent the moving object itself and which pixels represent the motion blur of the moving object, thereby determining the target motion blur recognition result. Optionally, the target motion blur recognition result may include: those pixels used to represent the motion blur of the moving object; or, the target motion blur recognition result may include: the region in the image to be processed where those pixels used to represent the motion blur of the moving object are located.
[0049] Step 130: Based on the target trailing image recognition result, perform predetermined processing on the image to be processed.
[0050] In step 130, the target ghosting recognition result can be referenced to remove the ghosting from the image to be processed, and a predetermined process can be performed on the image to be processed after ghosting removal to achieve the predetermined machine vision task.
[0051] Optionally, the image to be processed may include a foreground portion and a background portion. The bounding box of the moving object may be located in the foreground portion. Referring to the target shadow recognition result, the brightness values of those pixels in the image to be processed that represent the shadow of the moving object can be updated to be consistent with the average brightness value of the background portion, so as to obtain the image to be processed after shadow removal.
[0052] Optionally, the predetermined machine vision tasks include, but are not limited to, image segmentation tasks, recognition tasks (e.g., obstacle recognition tasks), state prediction tasks (e.g., pedestrian state prediction tasks), etc.
[0053] Of course, in step 130, it is also possible not to remove the trailing shadow from the image to be processed, but to refer to the target trailing shadow recognition result and directly perform image segmentation, recognition and other processing on the area of the image to be processed, excluding the area where the pixels used to represent the trailing shadow of the moving object are located.
[0054] Based on the image processing method provided in the above embodiments of this disclosure, edge detection can be performed on the image to be processed to determine the bounding box of the moving object in the image. Then, combined with a preset brightness distribution ratio range, motion blur recognition can be performed to obtain the target motion blur recognition result. The target motion blur recognition result can be used for predetermined processing of the image to be processed, so as to efficiently and reliably achieve the predetermined machine vision task and avoid the adverse effects of motion blur on the normal execution of the machine vision task, thereby helping to ensure the execution effect of the machine vision task. Furthermore, in the embodiments of this disclosure, it is not necessary to reduce the exposure time in night scene environments to reduce motion blur; instead, the image can be captured with normal exposure time. This helps to avoid the adverse effects of reduced exposure time on the image signal-to-noise ratio and brightness, thereby helping to ensure the execution effect of the machine vision task.
[0055] In an optional example, such as Figure 2 As shown, step 120 includes steps 1201 and 1203.
[0056] Step 1201: Based on the preset brightness distribution ratio range and the brightness values of the pixels in the region surrounded by the bounding box, perform motion blur recognition on the region surrounded by the bounding box to divide the region surrounded by the bounding box into a first motion blur region and a first main body region.
[0057] In one alternative implementation, such as Figure 3 As shown, step 1201 includes steps 12011, 12013 and 12035.
[0058] Step 12011: Determine the first average brightness value based on the brightness values of the pixels in the region enclosed by the bounding box.
[0059] In step 12011, the brightness values of N pixels within the region encompassed by the bounding box can be obtained to acquire N brightness values. An average value is then calculated based on these N brightness values to obtain a first average brightness value. Optionally, the average of the N brightness values can be directly calculated, and this average value can be used as the first average brightness value; alternatively, the two brightness values with the highest and lowest brightness values can be removed from the N brightness values, and the average of the remaining N-2 brightness values can be calculated, with this average value used as the first average brightness value.
[0060] Step 12013: Determine the first brightness range based on the first average brightness and the preset brightness distribution ratio range.
[0061] In step 12013, the minimum proportion in the preset brightness distribution ratio range can be multiplied by the first brightness average value to obtain the first brightness value, and the maximum proportion in the preset brightness distribution ratio range can be multiplied by the first brightness average value to obtain the second brightness value. The first brightness range can be a brightness range from the first brightness value to the second brightness value. If the preset brightness distribution ratio range is represented as [P1, P2], and the first brightness average value is represented as L, then the first brightness value can be represented as P1*L, the second brightness value can be represented as P2*L, and the first brightness range can be represented as [P1*L, P2*L].
[0062] Of course, the method for determining the first brightness range is not limited to this. For example, the first brightness value can be multiplied by a preset coefficient (which can be 1.1, 1.2, or other coefficients) to obtain the third brightness value, and the second brightness value can be multiplied by the preset coefficient to obtain the fourth brightness value. The first brightness range can be the brightness range from the third brightness value to the fourth brightness value. As another example, the second smallest proportion in the preset brightness distribution ratio range can be multiplied by the first brightness average value to obtain the fifth brightness value, and the second largest proportion in the preset brightness ratio range can be multiplied by the first brightness average value to obtain the sixth brightness value. The first brightness range can be the brightness range from the fifth brightness value to the sixth brightness value.
[0063] Step 12015: The region in which the brightness values of all pixels distributed in the area included by the bounding box are within the first brightness range is defined as the first motion blur region, and the region in which the brightness values of all pixels distributed in the area included by the bounding box are outside the first brightness range is defined as the first main body region.
[0064] In step 12015, the N pixels within the bounding box can be traversed. If the brightness value of a pixel among the N pixels is within a first brightness range, the location of that pixel can be considered to belong to the first motion blur region. If the brightness value of a pixel among the N pixels is outside the first brightness range, the location of that pixel can be considered to belong to the first main body region. After traversing the N pixels, the region within the bounding box can be divided into two parts: the first main body region and the first motion blur region. The first main body region can be considered as the region where the moving object itself is located, as determined by a single motion blur recognition, and the first motion blur region can be considered as the region where the motion blur of the moving object is located, as determined by a single motion blur recognition.
[0065] In this implementation, by referring to the brightness values of the N pixels in the region included by the bounding box, a first average brightness value can be determined efficiently and reliably through simple averaging. Combined with a preset brightness distribution ratio range, a first brightness range can be determined efficiently and reliably. By referring to the distribution of the brightness values of the N pixels relative to the first brightness range, the region included by the bounding box can be efficiently segmented, so as to determine the first shadow region and the first main body region through segmentation.
[0066] Of course, the implementation of step 1201 is not limited to this. For example, after determining the first average brightness value, the third brightness value, and the fourth brightness value in sequence, the first brightness range can be determined without considering the first brightness range. Instead, the region in which the brightness values of the pixels distributed in the area included by the bounding box are all greater than or equal to the third brightness value and less than or equal to the fourth brightness value can be directly determined as the first shadow region, and the remaining region in the area included by the bounding box can be determined as the first main body region.
[0067] Step 1203: Based on the first trailing shadow region and the first main body region, obtain the target trailing shadow recognition result of the moving object.
[0068] In one alternative implementation, step 1203 includes:
[0069] In response to the first trailing shadow region satisfying the preset integrity condition, the first trailing shadow region is used as the target trailing shadow recognition result of the moving object;
[0070] In response to the first motion blur region not meeting the preset integrity condition, motion blur recognition is performed on the first main body region based on motion blur recognition reference information that is associated with the preset brightness distribution ratio range and the brightness values of the pixels in the first main body region, so as to divide the first main body region into a second motion blur region and a second main body region, and based on the second motion blur region and the second main body region, the target motion blur recognition result of the moving object is obtained.
[0071] In this implementation, after dividing the area enclosed by the bounding box into a first shadow area and a first main body area, it can be determined whether the first shadow area meets the preset integrity condition.
[0072] If the first trailing shadow area meets the preset integrity condition, it can be considered that the first trailing shadow area completely contains the trailing shadow of the moving object. That is, the trailing shadow of the moving object has been accurately identified. Then, the first trailing shadow area can be directly used as the target trailing shadow identification result of the moving object.
[0073] If the first motion blur region does not meet the preset integrity condition, it can be considered that the first motion blur region does not completely contain the motion blur of the moving object. That is, the motion blur of the moving object is not accurately identified. Then, motion blur identification reference information that is associated with the preset brightness distribution ratio range and the brightness values of the pixels in the first main area can be determined.
[0074] Optionally, the number of pixels in the first main body region can be M, and the motion blur recognition reference information can include: a preset brightness distribution ratio range and the brightness values of the M pixels in the first main body region. Thus, referring to the specific implementation of step 1201 above, the first main body region can be divided into a second motion blur region and a second main body region in a similar manner, based on the preset brightness distribution ratio range and the brightness values of the M pixels in the first main body region.
[0075] Alternatively, referring to the specific implementation of step 12011 above, a second average brightness value is determined based on the brightness values of the M pixels in the first main area. Referring to the specific implementation of step 12013 above, a second brightness range is determined based on the second average brightness value and a preset brightness distribution ratio range. Referring to the specific implementation of step 12015 above, the first main area is divided into a second shadow area and a second main area based on the distribution of the brightness values of the M pixels in the first main area relative to the second brightness range.
[0076] After segmenting the first main body region, the target trailing shadow recognition result of the moving object can be obtained based on the second trailing shadow region and the second main body region. For example, it can be determined whether the second trailing shadow region meets the preset integrity condition. If the second trailing shadow region meets the preset integrity condition, it can be directly used as the target trailing shadow recognition result. If the second trailing shadow region does not meet the preset integrity condition, trailing shadow recognition can be performed again on the second main body region to segment the second main body region into a third trailing shadow region and a third main body region. The subsequent process is similar to the process of segmenting the first main body region into the second trailing shadow region and the second main body region, and will not be described in detail here.
[0077] In this implementation, by referring to whether the first trailing region meets the preset integrity condition, it can be decided whether to directly use the first trailing region as the target trailing recognition result, or to perform further trailing recognition based on the first main region. This helps to ensure the accuracy and reliability of the final target trailing recognition result.
[0078] Of course, the implementation of step 1203 is not limited to this. For example, if the first ghosting area meets the preset integrity condition, the first ghosting area may not be directly used as the target ghosting recognition result. Instead, the first ghosting area may be marked and presented to the user, who can then correct the first ghosting area and use the corrected first ghosting area as the target ghosting recognition result.
[0079] In the embodiments of this disclosure, by using the brightness values of pixels in the region enclosed by the preset brightness distribution ratio range and the bounding box, the region enclosed by the bounding box can be efficiently and reliably divided into a first shadow region and a first main body region through shadow recognition. The first shadow region and the first main body region can then be used to obtain the target shadow recognition result. Using the target shadow recognition result for machine vision tasks helps to ensure the performance of machine vision tasks.
[0080] like Figure 4 As shown, in an exemplary embodiment of this disclosure, the process of determining whether the first ghosting region meets the preset integrity conditions may include steps 410, 420, 430 and 440.
[0081] Step 410: Determine the second average brightness value based on the brightness values of the pixels in the first main area.
[0082] It should be noted that the specific implementation process of step 410 can be referred to the description of step 12011, and will not be repeated here.
[0083] Step 420: Determine the second brightness range based on the second average brightness and the preset brightness distribution ratio range.
[0084] It should be noted that the specific implementation process of step 420 can be referred to the description of step 12013, and will not be repeated here.
[0085] Step 430: In response to the fact that the brightness values of the pixels in the first main area are all outside the second brightness range, it is determined that the first motion blur area meets the preset integrity condition.
[0086] Step 440: In response to at least a portion of the brightness values of the pixels in the first main area being within the second brightness range, it is determined that the first motion blur region does not meet the preset integrity condition.
[0087] After determining the second brightness range, the M pixels in the first main area can be traversed. If the brightness value of at least one of the M pixels is within the second brightness range, it can be determined that the first ghosting region does not meet the preset integrity condition. If the brightness values of all M pixels are outside the second brightness range, it can be determined that the first ghosting region meets the preset integrity condition.
[0088] In the embodiments of this disclosure, by referring to the brightness values of the M pixels in the first main area, a second brightness average can be determined efficiently and reliably through simple mean calculation. Combined with a preset brightness distribution ratio range, a second brightness range can be determined efficiently and reliably. By referring to the distribution of the brightness values of the M pixels relative to the second brightness range, it can be determined efficiently and reliably whether the first ghosting area meets the preset integrity condition, so as to determine whether further ghosting recognition is needed for the first ghosting area.
[0089] In one optional example, the number of moving objects can be one.
[0090] In another alternative example, the number of moving objects can be at least two, and the at least two moving objects include a first moving object and a second moving object, such as... Figure 5 As shown, in another exemplary embodiment of the present disclosure, the process of determining whether the first ghosting region meets the preset integrity conditions may include steps 510, 520, 530 and 540.
[0091] Step 510: Determine the first size relationship between the dimensions of the first moving object and the second moving object.
[0092] Optionally, the first moving object and the second moving object can have the same shape, such as both being rectangular or sector-shaped (see details). Figure 6 )wait.
[0093] Taking the case where both the first and second moving objects are sector-shaped as an example, the dimensions of the first moving object can include a first central angle and a first sector radius, while the dimensions of the second moving object can include a second central angle and a second sector radius. Assuming the first and second central angles are very close, the primary size relationship between the dimensions of the first and second moving objects can be characterized by the size relationship between the first and second sector radii. If the first sector radius is greater than the second sector radius, this primary size relationship can be used to indicate that the dimension of the first moving object is greater than the dimension of the second moving object.
[0094] Taking the case where both the first moving object and the second moving object are rectangular as an example, the object size of the first moving object may include: a first length and a first width, and the object size of the second moving object may include: a second length and a second width. The first size relationship between the object sizes of the first moving object and the second moving object can be characterized by the size relationship between the first length and the second length, and the size relationship between the first width and the second width.
[0095] Step 520: Determine the second size relationship between the area dimensions of the first trailing shadow areas corresponding to the first moving object and the second moving object, respectively.
[0096] Assume the first motion shadow region corresponding to the first moving object is Figure 6 In region S1, the first trailing shadow region corresponding to the second moving object is Figure 6 In region S2, the second size relationship between the area sizes of the first trailing shadow regions corresponding to the first moving object and the second moving object can be characterized by the size relationship between the areas of region S1 and region S2. If the area of region S1 is greater than the area of region S2, the second size relationship can be used to indicate that the area size of the first trailing shadow region corresponding to the first moving object is greater than the area size of the first trailing shadow region corresponding to the second moving object.
[0097] Step 530: In response to the matching of the first size relationship and the second size relationship, it is determined that the first trailing regions corresponding to the first moving object and the second moving object respectively meet the preset integrity conditions.
[0098] Step 540: In response to the mismatch between the first size relationship and the second size relationship, it is determined that the first trailing area corresponding to the moving object with the larger object size in the first moving object and the second moving object does not meet the preset integrity condition.
[0099] If the first size relationship is used to characterize that the size of the first moving object is greater than the size of the second moving object, and the second size relationship is used to characterize that the size of the first trailing shadow area corresponding to the first moving object is greater than the size of the first trailing shadow area corresponding to the second moving object, it can be considered that the relative size between different moving objects and the relative size between the trailing shadows of different moving objects are matched. Then, it can be determined that the first trailing shadow areas corresponding to the first moving object and the second moving object respectively satisfy the preset integrity condition.
[0100] If the first size relationship is used to characterize that the size of the first moving object is larger than the size of the second moving object, and the second size relationship is used to characterize that the size of the first trailing shadow area corresponding to the first moving object is smaller than the size of the first trailing shadow area corresponding to the second moving object, it can be considered that the relative size between different moving objects and the relative size between the trailing shadows of different moving objects do not match. This is likely because the trailing shadow of the first moving object has not been accurately identified, and there are still unidentified trailing shadows in the first main body area. Therefore, it can be determined that the first trailing shadow area corresponding to the first moving object does not meet the preset integrity condition.
[0101] In the embodiments of this disclosure, by referring to whether the relative size between different moving objects matches the relative size between the trails of different moving objects, it is possible to efficiently and reliably determine whether the first trailing region meets the preset integrity condition, so as to determine whether further trailing recognition is needed for the first main body region.
[0102] In an optional example, such as Figure 7 As shown, step 110 includes steps 1101, 1103 and 1105.
[0103] Step 1101: Perform edge detection on the image to be processed to identify moving objects in the image.
[0104] In step 1101, an edge detection algorithm can be used to perform edge detection on the image to be processed in order to identify moving objects in the image to be processed. The moving objects can be objects with mobility as mentioned above.
[0105] Step 1103: In response to the motion object including a color, the motion object is identified as a moving object, and the bounding box of the moving object is determined.
[0106] Step 1105: In response to the motion object including at least two colors, the motion object is segmented into motion objects corresponding to at least two colors respectively, and the bounding box of each segmented motion object is determined.
[0107] After identifying the moving object, the pixel values of multiple pixels (let's say K pixels) representing the moving object in the R, G, and B channels can be extracted from the image to be processed. By analyzing these extracted pixel values, the color of each of the K pixels in the user's field of vision can be determined, thereby determining the number of colors included in the moving object (specifically, the number of colors presented in the user's field of vision).
[0108] If the moving object contains only one color, it can be directly used as the moving object, and the bounding box of the moving object can be directly used as the bounding box of the moving object.
[0109] If the moving object includes at least two colors, it can be segmented into moving objects corresponding to at least two colors based on the color of each of the K pixels in the user's field of vision, and the bounding box of each segmented moving object can be further determined.
[0110] For each segmented moving object, a trailing shadow recognition can be performed to obtain the target trailing shadow recognition result corresponding to the moving object. The specific recognition method can be referred to the above description, and will not be repeated here.
[0111] In the embodiments of this disclosure, by performing edge detection on the image to be processed, the moving objects in the image to be processed can be determined first. By referring to the number of colors included in the moving objects, it can be decided whether to directly treat the moving objects as moving objects or to segment the moving objects to obtain at least two moving objects with different colors. Thus, in the embodiments of this disclosure, each moving object that needs to be recognized by trailing shadows includes only one color. The single color can make the recognition efficiency higher and the recognition result more reliable.
[0112] In an optional example, such as Figure 8 As shown, an image with a normal exposure time can be input (which can be used as the image to be processed in the above text). Then, edge detection is performed on the image to determine the bounding box of the moving object. By analyzing the changes in brightness at the edges of the moving object, the true edge of the moving object can be statistically determined. This allows for the segmentation of the region enclosed by the bounding box into a first motion blur region and a first main body region through motion blur recognition. Assuming that the average brightness value of all pixels in the region enclosed by the bounding box is 100, and the preset brightness distribution ratio is [5%, 90%], the region where the brightness values of all pixels in the region enclosed by the bounding box are between [5, 90] can be determined as the first motion blur region, and the remaining region in the region enclosed by the bounding box can be determined as the first main body region. Afterward, it can be determined whether the first motion blur region meets the preset integrity condition. If it does, the first motion blur region can be directly used as the target motion blur recognition result. If it does not meet the condition, motion blur recognition can be repeated, for example, performing a second motion blur recognition on the first main body region. Further decisions can be made based on the actual situation to continue motion blur recognition until the motion blur is completely recognized, thus obtaining the target motion blur recognition result. Finally, the target trailing image recognition results can be used for machine vision tasks to ensure the performance of these tasks.
[0113] Any of the image processing methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the image processing methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the image processing methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.
[0114] Exemplary device
[0115] Figure 9 This is a schematic diagram of the structure of an image processing apparatus provided in an exemplary embodiment of the present disclosure. Figure 9 The apparatus shown includes a first determining module 910, an acquiring module 920, and an execution module 930.
[0116] The first determining module 910 is used to perform edge detection on the image to be processed and determine the bounding box of the moving object in the image to be processed.
[0117] The acquisition module 920 is used to obtain the target trailing image recognition result of a moving object based on the brightness values of the pixels in the region enclosed by the bounding box determined by the first determination module 910 and the preset brightness distribution ratio range.
[0118] The execution module 930 is used to perform predetermined processing on the image to be processed based on the target trailing recognition result obtained by the acquisition module 920.
[0119] In an optional example, such as Figure 10 As shown, the acquisition module 920 includes:
[0120] The segmentation submodule 9201 is used to perform motion blur recognition on the region surrounded by the bounding box determined by the first determining module 910 based on the brightness values of the pixels in the region surrounded by the bounding box determined by the first determining module 910, based on the preset brightness distribution ratio range and the bounding box values of the region surrounded by the bounding box determined by the first determining module 910, so as to segment the region surrounded by the bounding box determined by the first determining module 910 into a first motion blur region and a first main body region.
[0121] The acquisition submodule 9203 is used to obtain the target trailing recognition result of the moving object based on the first trailing region and the first main body region segmented by the segmentation submodule 9201.
[0122] In an optional example, such as Figure 10 As shown, the segmented submodule 9201 includes:
[0123] The first determining unit 92011 is used to determine a first average brightness value based on the brightness values of the pixels in the region enclosed by the bounding box determined by the first determining module 910.
[0124] The second determining unit 92013 is used to determine the first brightness range based on the first brightness average value determined by the first determining unit 92011 and the preset brightness distribution ratio range;
[0125] The segmentation unit 92015 is used to define the region in which the brightness values of the pixels distributed in the region included by the bounding box determined by the first determining module 910 are all within the first brightness range determined by the second determining unit 92013 as the first motion blur region, and to define the region in which the brightness values of the pixels distributed in the region included by the bounding box determined by the first determining module 910 are all outside the first brightness range determined by the second determining unit 92013 as the first main body region.
[0126] In an optional example, such as Figure 10 As shown, submodule 9203 is obtained, including:
[0127] The third determining unit 92031 is used to, in response to the first trailing region obtained by the segmentation submodule 9201 satisfying the preset integrity condition, take the first trailing region obtained by the segmentation submodule 9201 as the target trailing recognition result of the moving object.
[0128] The fourth determining unit 92033 is used to respond to the fact that the first shadow region obtained by the segmentation submodule 9201 does not meet the preset integrity condition, and to perform shadow recognition on the first main body region obtained by the segmentation submodule 9201 based on the shadow recognition reference information that is associated with the preset brightness distribution ratio range and the brightness values of the pixels in the first main body region obtained by the segmentation submodule 9201, so as to divide the first main body region obtained by the segmentation submodule 9201 into a second shadow region and a second main body region, and to obtain the target shadow recognition result of the moving object based on the second shadow region and the second main body region.
[0129] In an optional example, such as Figure 11 As shown, the apparatus provided in the embodiments of this disclosure further includes:
[0130] The second determining module 940 is used to determine a second average brightness value based on the brightness values of the pixels in the first main body region segmented by the segmentation submodule 9201.
[0131] The third determining module 950 is used to determine the second brightness range based on the second brightness average value determined by the second determining module 940 and the preset brightness distribution ratio range;
[0132] The fourth determining module 960 is used to determine that the first shadow region segmented by the segmenting submodule 9201 satisfies the preset integrity condition in response to the fact that the brightness values of the pixels in the first main area segmented by the segmentation submodule 9201 are all outside the second brightness range determined by the third determining module 950.
[0133] The fifth determining module 970 is used to determine that the first trailing region segmented by the segmenting submodule 9201 does not meet the preset integrity condition in response to at least a portion of the brightness values of the pixels in the first main area segmented by the segmentation submodule 9201 being within the second brightness range determined by the third determining module 950.
[0134] In one optional example, the number of moving objects is at least two, and the at least two moving objects include a first moving object and a second moving object;
[0135] like Figure 12 As shown, the apparatus provided in the embodiments of this disclosure further includes:
[0136] The sixth determining module 980 is used to determine a first size relationship between the respective object dimensions of the first moving object and the second moving object;
[0137] The seventh determining module 985 is used to determine a second size relationship between the area dimensions of the first trailing shadow areas corresponding to the first moving object and the second moving object, respectively.
[0138] The eighth determining module 990 is used to determine, in response to the matching of the first size relationship determined by the sixth determining module 980 and the second size relationship determined by the seventh determining module 985, that the first trailing regions corresponding to the first moving object and the second moving object respectively meet the preset integrity conditions.
[0139] The ninth determining module 995 is used to determine, in response to the mismatch between the first size relationship determined by the sixth determining module 980 and the second size relationship determined by the seventh determining module 985, that the first trailing area corresponding to the moving object with the larger object size in the first moving object and the second moving object does not meet the preset integrity condition.
[0140] In an optional example, such as Figure 13 As shown, the first determining module 910 includes:
[0141] The first determining submodule 9101 is used to perform edge detection on the image to be processed and determine moving objects in the image to be processed.
[0142] The second determining submodule 9103 is used to, in response to the first determining submodule 9101 determining that the moving object includes a color, take the moving object determined by the first determining submodule 9101 as a moving object, and determine the bounding box of the moving object;
[0143] The third determining submodule 9105 is used to, in response to the first determining submodule 9101 determining that the moving object includes at least two colors, divide the moving object determined by the first determining submodule 9101 into moving objects corresponding to at least two colors respectively, and determine the bounding box of each of the divided moving objects.
[0144] In the apparatus disclosed herein, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve the corresponding functions and effects, and this disclosure does not list them all.
[0145] Exemplary electronic devices
[0146] Figure 14 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure, including at least one processor 11 and a memory 12.
[0147] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
[0148] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the methods and / or other desired functions of the various embodiments of this disclosure described above.
[0149] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).
[0150] The input device 13 may also include, for example, a keyboard, a mouse, etc.
[0151] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.
[0152] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.
[0153] Exemplary computer program products and computer-readable storage media
[0154] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.
[0155] Computer program products can be written in any combination of one or more programming languages to perform the operations of embodiments of this disclosure. These programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0156] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.
[0157] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0158] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0159] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.
Claims
1. An image processing method, comprising: Edge detection is performed on the image to be processed to determine the bounding boxes of moving objects in the image to be processed; Based on the preset brightness distribution ratio range and the brightness values of the pixels in the region enclosed by the bounding box, the target trailing image recognition result of the moving object is obtained; wherein, the preset brightness distribution ratio range refers to the brightness distribution ratio range set in advance to distinguish between the moving object itself and the trailing image of the moving object; Based on the target trailing image recognition result, a predetermined process is performed on the image to be processed in order to achieve the predetermined machine vision task; The method of obtaining the target trailing image recognition result of the moving object based on the brightness values of pixels in the region enclosed by the preset brightness distribution ratio range and the bounding box includes: A first average brightness value is determined based on the brightness values of the pixels in the region enclosed by the bounding box. The first brightness range is determined based on the first average brightness value and the preset brightness distribution ratio range; The region in which the brightness values of all pixels distributed in the area included by the bounding box are within the first brightness range is defined as the first shadow region, and the region in which the brightness values of all pixels distributed in the area included by the bounding box are outside the first brightness range is defined as the first main body region. Based on the first trailing shadow region and the first main body region, the target trailing shadow recognition result of the moving object is obtained.
2. The method according to claim 1, wherein, The step of obtaining the target trailing image recognition result of the moving object based on the first trailing image region and the first main body region includes: In response to the first trailing region satisfying a preset integrity condition, the first trailing region is taken as the target trailing recognition result of the moving object; In response to the first motion blur region not meeting the preset integrity condition, motion blur recognition is performed on the first main body region based on motion blur recognition reference information that is associated with the preset brightness distribution ratio range and the brightness values of the pixels in the first main body region, so as to divide the first main body region into a second motion blur region and a second main body region, and based on the second motion blur region and the second main body region, the target motion blur recognition result of the moving object is obtained.
3. The method according to claim 2, further comprising: The second average brightness value is determined based on the brightness values of the pixels in the first main area; The second brightness range is determined based on the second average brightness value and the preset brightness distribution ratio range; In response to the fact that the brightness values of all pixels in the first main area are outside the second brightness range, it is determined that the first ghosting area meets the preset integrity condition; In response to at least a portion of the brightness values of the pixels in the first main area being within the second brightness range, it is determined that the first ghosting area does not meet the preset integrity condition.
4. The method according to claim 2, wherein, The number of moving objects is at least two, and the at least two moving objects include a first moving object and a second moving object; The method further includes: Determine a first size relationship between the respective object sizes of the first moving object and the second moving object; Determine a second size relationship between the area sizes of the first trailing shadow regions corresponding to the first moving object and the second moving object, respectively; In response to the matching of the first size relationship and the second size relationship, it is determined that the first trailing regions corresponding to the first moving object and the second moving object respectively meet the preset integrity condition; In response to the mismatch between the first size relationship and the second size relationship, it is determined that the first trailing shadow region corresponding to the moving object with the larger object size in the first moving object and the second moving object does not meet the preset integrity condition.
5. The method according to any one of claims 1-4, wherein, The process of edge detection in the image to be processed, determining the bounding boxes of moving objects in the image to be processed, includes: Edge detection is performed on the image to be processed to determine moving objects in the image; In response to the moving object including a color, the moving object is identified as a moving object, and the bounding box of the moving object is determined; In response to the moving object including at least two colors, the moving object is segmented into moving objects corresponding to the at least two colors respectively, and the bounding box of each segmented moving object is determined.
6. An image processing apparatus, comprising: The first determining module is used to perform edge detection on the image to be processed and determine the bounding box of the moving object in the image to be processed; The acquisition module is used to obtain the target trailing shadow recognition result of the moving object based on the brightness values of the pixels in the region enclosed by the bounding box determined by the first determination module and a preset brightness distribution ratio range; wherein, the preset brightness distribution ratio range refers to a pre-set brightness distribution ratio range used to distinguish between the moving object itself and the trailing shadow of the moving object. An execution module is used to perform predetermined processing on the image to be processed based on the target trailing recognition result obtained by the acquisition module, so as to achieve a predetermined machine vision task. The acquisition module includes: The first determining unit is used to determine a first average brightness value based on the brightness values of the pixels in the region enclosed by the bounding box; The second determining unit is used to determine the first brightness range based on the first average brightness value and the preset brightness distribution ratio range; The segmentation unit is used to define a region in which the brightness values of all pixels distributed in the region included by the bounding box are within the first brightness range as a first shadow region, and to define a region in which the brightness values of all pixels distributed in the region included by the bounding box are outside the first brightness range as a first main body region. The acquisition submodule is used to obtain the target trailing recognition result of the moving object based on the first trailing region and the first main body region.
7. A computer-readable storage medium storing a computer program for performing the image processing method according to any one of claims 1-4.
8. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image processing method according to any one of claims 1-4.