Image processing method and apparatus, computer-readable storage medium, and electronic device

By identifying motion blur through edge detection and brightness distribution ratio range recognition, the problem of motion blur in images captured by vehicle-mounted devices in night scene environments is solved, ensuring the efficient and reliable execution of machine vision tasks.

WO2024235021A1PCT designated stage expired Publication Date: 2024-11-21BEIJING HORIZON INFORMATION TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/090992
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-12
Filing Date
2024-04-30
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

In nighttime environments, images captured by vehicle-mounted equipment are prone to motion blur, which can affect the normal execution of machine vision tasks.

Method used

Edge detection is used to determine the bounding box of the moving object, and target trailing shadows are identified based on a preset brightness distribution ratio range. Trailing shadow recognition and removal are then performed to ensure that the image processing method is effectively executed under normal exposure time.

Benefits of technology

It effectively eliminates motion blur, ensuring the performance of machine vision tasks and avoiding the adverse effects of reduced exposure time on image signal-to-noise ratio and brightness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024090992_21112024_PF_FP_ABST
    Figure CN2024090992_21112024_PF_FP_ABST
Patent Text Reader

Abstract

The embodiments of the present disclosure disclose an image processing method and apparatus, a computer-readable storage medium, and an electronic device, wherein the method comprises: performing edge detection on an image to be processed, and determining a bounding box for a moving object in the image to be processed; based on a preset brightness distribution proportion range and the brightness values of pixels in the area surrounded by the bounding box, obtaining a target blur recognition result for the moving object; and based on the target blur recognition result, performing predetermined processing on the image to be processed. The embodiments of the present disclosure avoid adverse effects caused by blur, and help ensure the execution effect of machine vision tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Image processing method, device, computer-readable storage medium, and electronic device

[0001] This disclosure claims priority to Chinese patent application number CN202310539281.7 filed with the Patent Office of China on May 12, 2023, entitled “Image processing method, device, computer-readable storage medium and electronic device,” the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of driving technology, and in particular to an image processing method, device, computer-readable storage medium, and electronic device. Background Art

[0003] The objects captured by vehicle-mounted equipment (such as vehicle-mounted cameras) are often vehicles with high speeds. Therefore, in a night scene environment, due to the increased exposure time of the night scene, ghosting is likely to occur in the images captured by the vehicle-mounted equipment.

[0004] Summary of the Invention

[0005] Embodiments of the present disclosure provide an image processing method, apparatus, computer-readable storage medium, and electronic device.

[0006] According to one aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0007] Performing edge detection on the image to be processed to determine a bounding box of a moving object in the image to be processed;

[0008] Obtaining a target smear recognition result of the moving object based on a preset brightness distribution ratio range and brightness values ​​of pixels in an area surrounded by the bounding box;

[0009] Based on the target smear recognition result, predetermined processing is performed on the image to be processed.

[0010] According to another aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0011] A first determination module is configured to perform edge detection on the image to be processed and determine a bounding box of a moving object in the image to be processed;

[0012] an acquisition module, configured to obtain a target smear recognition result of the moving object based on a preset brightness distribution ratio range and the brightness values ​​of the pixels in the area surrounded by the bounding box determined by the first determination module;

[0013] An execution module is used to perform predetermined processing on the image to be processed based on the target smear recognition result obtained by the acquisition module.

[0014] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned image processing method.

[0015] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device including:

[0016] processor;

[0017] a memory for storing instructions executable by the processor;

[0018] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-mentioned image processing method.

[0019] According to another aspect of the embodiments of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor, the above-mentioned image processing method is implemented.

[0020] Based on the image processing methods, devices, computer-readable storage media, electronic devices and products provided by the above-mentioned embodiments of the present disclosure, it is possible to perform edge detection on the image to be processed to determine the bounding box of the moving object in the image to be processed, and then, in combination with the preset brightness distribution ratio range, obtain the target smear recognition result of the moving object by smear recognition. The target smear recognition result can be used for the predetermined processing of the image to be processed to efficiently and reliably achieve the predetermined machine vision task, avoid the adverse effects of smear on the normal execution of the machine vision task, and thus help to ensure the execution effect of the machine vision task. In addition, in the embodiments of the present disclosure, there is no need to reduce the exposure time in the night scene environment to reduce the smear, but the image can be captured at a normal exposure time, which helps to avoid the adverse effects of the reduction in exposure time on the image signal-to-noise ratio and brightness, and thus helps to ensure the execution effect of the machine vision task.

[0021] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a flowchart of an image processing method provided by an exemplary embodiment of the present disclosure.

[0023] FIG2 is a schematic diagram of a flow chart of obtaining a target smear recognition result in an image processing method provided by an exemplary embodiment of the present disclosure.

[0024] FIG3 is a schematic diagram of a flow chart of dividing an area surrounded by a bounding box into a first smear area and a first main body area in an image processing method provided by an exemplary embodiment of the present disclosure.

[0025] FIG4 is a schematic diagram of a flow chart of determining whether a first smear area satisfies a preset integrity condition in an image processing method provided by an exemplary embodiment of the present disclosure.

[0026] FIG5 is another flowchart of determining whether a first smear area satisfies a preset integrity condition in an image processing method provided by an exemplary embodiment of the present disclosure.

[0027] FIG6 is a schematic diagram of a moving object and a smear area in an image processing method provided by an exemplary embodiment of the present disclosure.

[0028] FIG7 is a schematic diagram of a flow chart of determining a bounding box of a moving object in an image to be processed in an image processing method provided by an exemplary embodiment of the present disclosure.

[0029] FIG8 is a flowchart of an image processing method provided by another exemplary embodiment of the present disclosure.

[0030] FIG9 is a schematic structural diagram of an image processing apparatus provided by an exemplary embodiment of the present disclosure.

[0031] FIG10 is a schematic structural diagram of an image processing apparatus provided by another exemplary embodiment of the present disclosure.

[0032] FIG11 is a schematic structural diagram of an image processing apparatus provided by yet another exemplary embodiment of the present disclosure.

[0033] FIG12 is a schematic structural diagram of an image processing apparatus provided by yet another exemplary embodiment of the present disclosure.

[0034] FIG13 is a schematic structural diagram of an image processing apparatus provided by yet another exemplary embodiment of the present disclosure.

[0035] FIG14 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0037] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0038] Application Overview

[0039] In night scenes, due to the increased exposure time, smear is prone to appear in the images captured by vehicle-mounted equipment. Among them, smear generally refers to the traces produced by the repeated edges of moving objects when imaging on the complementary metal oxide semiconductor (CMOS) due to the long exposure time.

[0040] Exemplary Methods

[0041] FIG1 is a flow chart of an image processing method provided by an exemplary embodiment of the present disclosure. The method shown in FIG1 may include steps 110 , 120 , and 130 .

[0042] Step 110 : performing edge detection on the image to be processed to determine a bounding box of a moving object in the image to be processed.

[0043] In an optional example, step 110 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by the first determination module executed by the processor.

[0044] In step 110 , an edge detection algorithm may be used to perform edge detection on the image to be processed, so as to identify pixels with obvious brightness changes in the image to be processed, thereby determining a bounding box of a moving object in the image to be processed.

[0045] Optionally, the edge detection algorithm may include the following steps: 1. smoothing the image using a filter (such as a Gaussian filter); 2. calculating the gradient magnitude and direction of the image; 3. performing non-maximum suppression on the gradient magnitude; and 4. detecting and connecting edges using a dual threshold algorithm.

[0046] Optionally, the image to be processed may be an RGB image taken in real time by the vehicle-mounted device, or an RGB image taken historically by the vehicle-mounted device; wherein R represents red, G represents green, and B represents blue.

[0047] Optionally, the moving object may be an object with the ability to move, or a part of an object with the ability to move; wherein the object with the ability to move may include but is not limited to pedestrians, vehicles, trains, intelligent robots, etc.

[0048] Step 120 : Obtain a target smear recognition result of the moving object based on the preset brightness distribution ratio range and the brightness values ​​of the pixels in the area surrounded by the bounding box.

[0049] In an optional example, step 120 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by an acquisition module executed by the processor.

[0050] It should be noted that the preset brightness distribution ratio range may refer to a preset brightness distribution ratio range that can be used to distinguish between a moving object and a trailing shadow of the moving object. Optionally, the preset brightness distribution ratio range may be expressed as [P1, P2], where P1 may be 5%, 10%, 15%, or other values, and P2 may be 80%, 85%, 90%, or other values, which are not listed here.

[0051] In step 120, the brightness values ​​of all pixels (assuming N pixels) within the area encompassed by the bounding box can be determined to obtain N brightness values. Referring to a preset brightness distribution ratio range and the N brightness values, it can be determined which of the N pixels represent the moving object itself and which represent the moving object's afterimage, thereby determining a target afterimage recognition result for the moving object. Optionally, the target afterimage recognition result can include: the pixels representing the moving object's afterimage; or, the target afterimage recognition result can include: the area in the image to be processed where the pixels representing the moving object's afterimage are located.

[0052] Step 130 : Based on the target smear recognition result, perform predetermined processing on the image to be processed.

[0053] In an optional example, step 130 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by an execution module run by the processor.

[0054] In step 130 , the target smear recognition result may be referred to to remove smear from the image to be processed, and predetermined processing may be performed on the image to be processed after the smear is removed, so as to achieve a predetermined machine vision task.

[0055] Optionally, the image to be processed may include a foreground part and a background part, and the bounding box of the moving object may be located in the foreground part. Referring to the target smear recognition result, the brightness values ​​of those pixel points in the image to be processed used to represent the smear of the moving object may be updated to be consistent with the average brightness value of the background part to obtain the image to be processed after the smear is removed.

[0056] Optionally, the predetermined machine vision tasks may include, but are not limited to, image segmentation tasks, recognition tasks (such as obstacle recognition tasks), state prediction tasks (such as pedestrian state prediction tasks), etc.

[0057] Of course, in step 130, it is also possible not to remove the ghost from the image to be processed, but to refer to the target ghost recognition result and directly perform image segmentation, recognition and other processing on the area of ​​the image to be processed except for the area where the pixel points used to represent the ghost of the moving object are located.

[0058] Based on the image processing method provided by the above-mentioned embodiment of the present disclosure, the bounding box of the moving object in the image to be processed can be determined by edge detection on the image to be processed, and then combined with the preset brightness distribution ratio range, the target smear recognition result of the moving object can be obtained by smear recognition. The target smear recognition result can be used for the predetermined processing of the image to be processed to efficiently and reliably realize the predetermined machine vision task, avoid the adverse effect of smear on the normal execution of the machine vision task, and thus help to ensure the execution effect of the machine vision task. In addition, in the embodiment of the present disclosure, there is no need to reduce the exposure time in the night scene environment to reduce the smear, but the image can be captured at a normal exposure time, which helps to avoid the adverse effect of the reduction of exposure time on the image signal-to-noise ratio and brightness, and thus helps to ensure the execution effect of the machine vision task.

[0059] In an optional example, as shown in FIG2 , step 120 may include step 1201 and step 1203 .

[0060] Step 1201 : Based on a preset brightness distribution ratio range and brightness values ​​of pixels in the area surrounded by the bounding box, perform smear recognition on the area surrounded by the bounding box to divide the area surrounded by the bounding box into a first smear area and a first main area.

[0061] In an optional example, step 1201 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a segmentation submodule executed by the processor.

[0062] In an optional implementation, as shown in FIG3 , step 1201 may include step 12011 , step 12013 , and step 12015 .

[0063] Step 12011: determine a first brightness mean based on the brightness values ​​of the pixels in the area surrounded by the bounding box.

[0064] In an optional example, step 12011 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a first determination unit executed by the processor.

[0065] In step 12011, the brightness values ​​of N pixels in the area encompassed by the bounding box may be obtained to obtain N brightness values, and an average operation may be performed on the N brightness values ​​to obtain a first average brightness value. Alternatively, the average of the N brightness values ​​may be directly calculated and used as the first average brightness value; or, the two brightness values ​​with the largest and smallest brightness values ​​may be eliminated from the N brightness values, and the average of the remaining N-2 brightness values ​​may be calculated and used as the first average brightness value.

[0066] Step 12013: Determine a first brightness range based on the first brightness mean and a preset brightness distribution ratio range.

[0067] In an optional example, step 12013 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a second determination unit executed by the processor.

[0068] In step 12013, the minimum ratio in the preset brightness distribution ratio range may be multiplied by the first brightness mean to obtain a first brightness value, and the maximum ratio in the preset brightness distribution ratio range may be multiplied by the first brightness mean to obtain a second brightness value. The first brightness range may be a brightness range from the first brightness value to the second brightness value. If the preset brightness distribution ratio range is represented as [P1, P2] and the first brightness mean is represented as L, the first brightness value may be represented as P1*L, the second brightness value may be represented as P2*L, and the first brightness range may be represented as [P1*L, P2*L].

[0069] Of course, the method for determining the first brightness range is not limited to this. For example, the first brightness value can be multiplied by a preset coefficient (which can be 1.1, 1.2, or other coefficients) to obtain a third brightness value, and the second brightness value can be multiplied by the preset coefficient to obtain a fourth brightness value. The first brightness range can be a brightness range from the third brightness value to the fourth brightness value. For another example, the second smallest ratio in the preset brightness distribution ratio range can be multiplied by the first brightness mean to obtain a fifth brightness value, and the second largest ratio in the preset brightness ratio range can be multiplied by the first brightness mean to obtain a sixth brightness value. The first brightness range can be a brightness range from the fifth brightness value to the sixth brightness value.

[0070] In step 12015, an area in which the brightness values ​​of the pixels distributed in the area included in the bounding box are all within the first brightness range is determined as a first trailing shadow area, and an area in which the brightness values ​​of the pixels distributed in the area included in the bounding box are all outside the first brightness range is determined as a first main area.

[0071] In an optional example, step 12015 can be executed by the processor calling the corresponding instructions stored in the memory, or can be executed by the segmentation unit executed by the processor.

[0072] In step 12015, the N pixels in the area included in the bounding box can be traversed. If the brightness value of a pixel in the N pixels is within the first brightness range, the location of the pixel can be considered to belong to the first trailing shadow area. If the brightness value of a pixel in the N pixels is outside the first brightness range, the location of the pixel can be considered to belong to the first main area. After traversing the N pixels, the area included in the bounding box can be divided into the first main area and the first trailing shadow area. The first main area can be considered to be the area where the moving object itself is located, as determined by the first trailing shadow recognition, and the first trailing shadow area can be considered to be the area where the trailing shadow of the moving object is located, as determined by the first trailing shadow recognition.

[0073] In this embodiment, the first brightness mean can be determined efficiently and reliably by means of a mean operation based on the brightness values ​​of the N pixels in the area included in the reference bounding box. The first brightness range can be determined efficiently and reliably by means of a preset brightness distribution ratio range. The area included in the bounding box can be efficiently segmented based on the distribution of the brightness values ​​of the N pixels relative to the first brightness range, so as to determine the first trailing shadow area and the first main body area through segmentation.

[0074] Of course, the implementation of step 1201 is not limited thereto. For example, after sequentially determining the first brightness mean value, the third brightness value, and the fourth brightness value, the first brightness range may be determined without determining the first brightness range. Instead, the area within the bounding box where the brightness values ​​of the pixels distributed therein are all greater than or equal to the third brightness value and less than or equal to the fourth brightness value may be directly determined as the first smear area, and the remaining area within the bounding box may be determined as the first main body area.

[0075] Step 1203 : Obtain a target smear recognition result of the moving object based on the first smear area and the first main body area.

[0076] In an optional example, step 1203 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by an obtaining submodule executed by the processor.

[0077] In an optional implementation, step 1203 may include:

[0078] In response to the first smear area meeting a preset integrity condition, taking the first smear area as a target smear recognition result of the moving object;

[0079] In response to the first trailing shadow area not satisfying the preset integrity condition, trailing shadow recognition is performed on the first main body area based on trailing shadow recognition reference information associated with the preset brightness distribution ratio range and the brightness values ​​of the pixels in the first main body area, so as to divide the first main body area into a second trailing shadow area and a second main body area, and based on the second trailing shadow area and the second main body area, a target trailing shadow recognition result of the moving object is obtained.

[0080] In an optional example, in response to the first smear area meeting a preset integrity condition, the step of using the first smear area as the target smear identification result of the moving object can be executed by the processor calling the corresponding instructions stored in the memory, or by a third determination unit executed by the processor.

[0081] In an optional example, in response to the first trailing shadow area not meeting a preset integrity condition, trailing shadow identification is performed on the first main body area based on trailing shadow identification reference information associated with a preset brightness distribution ratio range and the brightness values ​​of pixels in the first main body area to divide the first main body area into a second trailing shadow area and a second main body area, and based on the second trailing shadow area and the second main body area, the step of obtaining a target trailing shadow identification result of a moving object can be executed by the processor calling the corresponding instructions stored in the memory, or can be executed by a fourth determination unit executed by the processor.

[0082] In this implementation, after the area enclosed by the bounding box is divided into the first smear area and the first main area, it may be determined whether the first smear area meets a preset integrity condition.

[0083] If the first aftersmear area meets the preset integrity condition, it can be considered that the first aftersmear area completely contains the aftersmear of the moving object, that is, the aftersmear of the moving object is accurately identified. Then, the first aftersmear area can be directly used as the target aftersmear identification result of the moving object.

[0084] If the first afterimage area does not meet the preset integrity condition, it can be considered that the first afterimage area does not completely contain the afterimage of the moving object, that is, the afterimage of the moving object is not accurately identified. Then, the afterimage identification reference information associated with the preset brightness distribution ratio range and the brightness value of the pixel points in the first main area can be determined.

[0085] Optionally, the number of pixels in the first body region may be M, and the smear identification reference information may include: a preset brightness distribution ratio range and the brightness values ​​of each of the M pixels in the first body region. Thus, referring to the specific implementation of step 1201 above, the first body region may be segmented into a second smear region and a second body region based on the preset brightness distribution ratio range and the brightness values ​​of each of the M pixels in the first body region in a similar manner.

[0086] In some embodiments, the specific implementation of step 12011 above can be referred to, and in a similar manner, the second brightness mean can be determined based on the brightness values ​​of the M pixels in the first main area. The specific implementation of step 12013 above can be referred to, and in a similar manner, the second brightness range can be determined based on the second brightness mean and the preset brightness distribution ratio range. The specific implementation of step 12015 above can be referred to, and in a similar manner, the first main area can be divided into a second trailing shadow area and a second main area based on the distribution of the brightness values ​​of the M pixels in the first main area relative to the second brightness range.

[0087] After completing the segmentation of the first main body area, a target smear recognition result of the moving object can be obtained based on the second smear area and the second main body area. For example, it can be determined whether the second smear area meets a preset integrity condition. If the second smear area meets the preset integrity condition, the second smear area can be directly used as the target smear recognition result. If the second smear area does not meet the preset integrity condition, smear recognition can be continued on the second main body area to segment the second main body area into a third smear area and a third main body area. The subsequent process is similar to the processing process of segmenting the first main body area into the second smear area and the second main body area, and will not be repeated here.

[0088] In this embodiment, by referring to whether the first afterimage area meets the preset integrity condition, it can be decided whether to directly use the first afterimage area as the target afterimage recognition result, or to perform further afterimage recognition based on the first main area. This is conducive to ensuring the accuracy and reliability of the final target afterimage recognition result.

[0089] Of course, the implementation method of step 1203 is not limited to this. For example, when the first afterimage area meets the preset integrity condition, the first afterimage area may not be directly used as the target afterimage recognition result. Instead, the first afterimage area may be marked and presented to the user, and the user may correct the first afterimage area and use the corrected first afterimage area as the target afterimage recognition result.

[0090] In the embodiments of the present disclosure, by using a preset brightness distribution ratio range and the brightness values ​​of the pixel points in the area surrounded by the bounding box, the area surrounded by the bounding box can be efficiently and reliably divided into a first trailing shadow area and a first main body area through trailing shadow recognition, so that the first trailing shadow area and the first main body area can be used to obtain the target trailing shadow recognition result, and the target trailing shadow recognition result can be used for machine vision tasks, which is conducive to ensuring the execution effect of the machine vision task.

[0091] As shown in FIG. 4 , the process of determining whether the first smear area meets the preset integrity condition in an exemplary embodiment of the present disclosure may include step 410 , step 420 , step 430 and step 440 .

[0092] Step 410: Determine a second average brightness value based on the brightness values ​​of the pixels in the first main body area.

[0093] In an optional example, step 410 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a second determination module executed by the processor.

[0094] It should be noted that the specific implementation process of step 410 can refer to the description of step 12011 and will not be repeated here.

[0095] Step 420: Determine a second brightness range based on the second brightness mean and the preset brightness distribution ratio range.

[0096] In an optional example, step 420 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a third determination module executed by the processor.

[0097] It should be noted that the specific implementation process of step 420 can refer to the description of step 12013 and will not be repeated here.

[0098] Step 430 : In response to the brightness values ​​of the pixels in the first main body area all being outside the second brightness range, determining whether the first smear area meets a preset integrity condition.

[0099] In an optional example, step 430 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a fourth determination module executed by the processor.

[0100] Step 440 : In response to at least a portion of the brightness values ​​of the pixels in the first main body area being within a second brightness range, determining that the first smear area does not meet a preset integrity condition.

[0101] In an optional example, step 440 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a fifth determination module executed by the processor.

[0102] After determining the second brightness range, the M pixels in the first main area can be traversed. If the brightness value of at least one of the M pixels is within the second brightness range, it can be determined that the first afterimage area does not meet the preset integrity condition. If the brightness values ​​of the M pixels are all outside the second brightness range, it can be determined that the first afterimage area meets the preset integrity condition.

[0103] In an embodiment of the present disclosure, by referring to the brightness values ​​of each of the M pixels in the first main area, the second brightness mean can be determined efficiently and reliably through mean operation, and then combined with the preset brightness distribution ratio range, the second brightness range can be determined efficiently and reliably. By referring to the distribution of the brightness values ​​of each of the M pixels relative to the second brightness range, it can be determined efficiently and reliably whether the first trailing shadow area meets the preset integrity condition, so as to determine whether further trailing shadow identification is required for the first trailing shadow area.

[0104] In an optional example, the number of the moving object may be one.

[0105] In another optional example, the number of moving objects may be at least two, and the at least two moving objects may include a first moving object and a second moving object. As shown in FIG5 , the process of determining whether the first smear area satisfies a preset integrity condition in an exemplary embodiment of the present disclosure may include steps 510 , 520 , 530 , and 540 .

[0106] Step 510: Determine a first size relationship between the sizes of the first moving object and the second moving object.

[0107] In an optional example, step 510 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a sixth determination module executed by the processor.

[0108] Optionally, the first moving object and the second moving object may be in the same shape, for example, both are rectangular, sector-shaped (see FIG. 6 for details), etc.

[0109] Taking the case where both the first moving object and the second moving object are fan-shaped as an example, the object size of the first moving object may include: a first central angle and a first fan radius, and the object size of the second moving object may include: a second central angle and a second fan radius. Assuming that the first central angle and the second central angle are very close, the first size relationship between the object sizes of the first moving object and the second moving object can be characterized by the size relationship between the first fan radius and the second fan radius. If the first fan radius is greater than the second fan radius, the first size relationship can be used to characterize that the object size of the first moving object is greater than the object size of the second moving object.

[0110] Taking the case where the first moving object and the second moving object are both rectangular as an example, the object size of the first moving object may include: a first length and a first width, and the object size of the second moving object may include: a second length and a second width. The first size relationship between the respective object sizes of the first moving object and the second moving object can be characterized by the size relationship between the first length and the second length, and the size relationship between the first width and the second width.

[0111] Step 520 : Determine a second size relationship between the area sizes of the first smear areas corresponding to the first moving object and the second moving object.

[0112] In an optional example, step 520 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a seventh determination module executed by the processor.

[0113] Assuming that the first afterimage region corresponding to the first moving object is region S1 in FIG. 6 , and the first afterimage region corresponding to the second moving object is region S2 in FIG. 6 , the second size relationship between the area sizes of the first afterimage regions corresponding to the first moving object and the second moving object can be represented by the size relationship between the area of ​​region S1 and the area of ​​region S2. If the area of ​​region S1 is larger than the area of ​​region S2, the second size relationship can be used to represent that the area size of the first afterimage region corresponding to the first moving object is larger than the area size of the first afterimage region corresponding to the second moving object.

[0114] Step 530 : In response to the first size relationship matching the second size relationship, determining whether the first smear regions corresponding to the first moving object and the second moving object respectively meet a preset integrity condition.

[0115] In an optional example, step 530 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by an eighth determination module executed by the processor.

[0116] Step 540 : In response to the mismatch between the first size relationship and the second size relationship, determining that a first smear region corresponding to the larger moving object between the first moving object and the second moving object does not meet a preset integrity condition.

[0117] In an optional example, step 540 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a ninth determination module executed by the processor.

[0118] If the first size relationship is used to characterize that the object size of the first moving object is larger than the object size of the second moving object, and the second size relationship is used to characterize that the area size of the first afterimage area corresponding to the first moving object is larger than the area size of the first afterimage area corresponding to the second moving object, it can be considered that the relative sizes between different moving objects and the relative sizes between the afterimages of different moving objects match each other, then it can be determined that the first afterimage areas corresponding to the first moving object and the second moving object respectively meet the preset integrity conditions.

[0119] If the first size relationship is used to characterize that the object size of the first moving object is larger than the object size of the second moving object, and the second size relationship is used to characterize that the area size of the first afterimage area corresponding to the first moving object is smaller than the area size of the first afterimage area corresponding to the second moving object, it can be considered that the relative sizes between different moving objects and the relative sizes between the afterimages of different moving objects do not match. This is most likely because the afterimage of the first moving object is not accurately identified, and there is still an unidentified afterimage in the first main area. Then, it can be determined that the first afterimage area corresponding to the first moving object does not meet the preset integrity condition.

[0120] In an embodiment of the present disclosure, by referring to whether the relative sizes between different moving objects match the relative sizes between the afterimages of different moving objects, it is possible to efficiently and reliably determine whether the first afterimage area meets the preset integrity condition, so as to determine whether further afterimage identification is required for the first main body area.

[0121] In an optional example, as shown in FIG7 , step 110 may include step 1101 , step 1103 and step 1105 .

[0122] Step 1101 : performing edge detection on the image to be processed to determine the moving objects in the image to be processed.

[0123] In an optional example, step 1101 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by the first determination submodule executed by the processor.

[0124] In step 1101, an edge detection algorithm may be used to perform edge detection on the image to be processed to determine a moving object in the image to be processed. The moving object may be the object with the ability to move mentioned above.

[0125] Step 1103 : In response to the moving object including a color, the moving object is regarded as a moving object, and a bounding box of the moving object is determined.

[0126] In an optional example, step 1103 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a second determining submodule executed by the processor.

[0127] Step 1105 : In response to the moving object including at least two colors, the moving object is segmented into moving objects corresponding to the at least two colors, and a bounding box of each moving object obtained by segmentation is determined.

[0128] In an optional example, step 1105 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a third determining submodule executed by the processor.

[0129] After determining the moving object, the pixel values ​​of multiple pixel points (assuming K pixels) used to represent the moving object in the R, G, and B channels can be extracted from the image to be processed. By analyzing these extracted pixel values, the color of each of the K pixels presented in the user's field of view can be determined, thereby determining the number of colors included in the moving object (specifically, the number of colors presented in the user's field of view).

[0130] If the moving object only includes one color, the moving object can be directly used as the moving object, and the bounding box of the moving object can be directly used as the bounding box of the moving object.

[0131] If the moving object includes at least two colors, the moving object can be segmented into moving objects corresponding to at least two colors based on the colors of the K pixels presented in the user's field of view, and the bounding box of each moving object obtained by segmentation can be further determined.

[0132] For each moving object obtained by segmentation, the moving object can be subjected to trailing shadow recognition to obtain a target trailing shadow recognition result corresponding to the moving object. The specific recognition method can be referred to the above introduction and will not be repeated here.

[0133] In the embodiment of the present disclosure, by performing edge detection on the image to be processed, the moving object in the image to be processed can be first determined. Referring to the number of colors included in the moving object, it can be decided whether to directly treat the moving object as a moving object, or to segment the moving object to obtain at least two moving objects with corresponding different colors. In this way, in the embodiment of the present disclosure, each moving object that needs to be identified for smear can include only one color. The single color can make the recognition efficiency higher and the recognition results more reliable.

[0134] In an optional example, as shown in FIG8 , an image with a normal exposure time (which can be used as the image to be processed above) can be input, and then edge detection is performed on the image to determine the bounding box of the moving object. By the degree of change in the brightness of the edge of the moving object, the true edge of the moving object can be counted, so that the area surrounded by the bounding box can be divided into a first drag area and a first main area through drag recognition. Assuming that the mean brightness value of all pixels in the area surrounded by the bounding box is 100, and the preset brightness distribution ratio range is [5%, 90%], the area where the brightness values ​​of the pixels distributed in the area surrounded by the bounding box are between [5, 90] can be determined as the first drag area, and the remaining area in the area surrounded by the bounding box can be determined as the first main area. Afterwards, it can be determined whether the first drag area meets the preset integrity condition. If the first drag area meets the preset integrity condition, the first drag area can be directly used as the target drag recognition result. If the first smear area does not meet the preset completeness condition, smear recognition can be repeated. For example, smear recognition can be performed twice on the first main area. Subsequently, the system can decide whether to continue smear recognition based on the actual situation until the smear is completely identified, thereby obtaining the target smear recognition result. Finally, the target smear recognition result can be used in the machine vision task to ensure the execution effect of the machine vision task.

[0135] Any image processing method provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to a terminal device and a server. Alternatively, any image processing method provided in the embodiments of the present disclosure can be executed by a processor, such as a processor that executes any image processing method mentioned in the embodiments of the present disclosure by invoking corresponding instructions stored in a memory. This will not be further described below.

[0136] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0137] Exemplary devices

[0138] FIG9 is a schematic diagram of the structure of an image processing apparatus provided by an exemplary embodiment of the present disclosure. The apparatus shown in FIG9 can be used to implement any of the above-mentioned method embodiments of the present disclosure. The apparatus shown in FIG9 can include a first determination module 910, an acquisition module 920, and an execution module 930.

[0139] A first determination module 910 is configured to perform edge detection on the image to be processed and determine a bounding box of a moving object in the image to be processed;

[0140] An acquisition module 920 is configured to obtain a target smear recognition result of a moving object based on a preset brightness distribution ratio range and the brightness values ​​of pixels in the area surrounded by the bounding box determined by the first determination module 910;

[0141] The execution module 930 is configured to execute predetermined processing on the image to be processed based on the target smear recognition result obtained by the acquisition module 920 .

[0142] In an optional example, as shown in FIG10 , the acquisition module 920 may include:

[0143] a segmentation submodule 9201 for performing smear recognition on the area enclosed by the bounding box determined by the first determination module 910 based on a preset brightness distribution ratio range and brightness values ​​of pixels in the area enclosed by the bounding box determined by the first determination module 910, so as to segment the area enclosed by the bounding box determined by the first determination module 910 into a first smear area and a first main body area;

[0144] The obtaining submodule 9203 is configured to obtain a target smear recognition result of the moving object based on the first smear region and the first main region segmented by the segmentation submodule 9201 .

[0145] In an optional example, as shown in FIG10 , the segmentation submodule 9201 may include:

[0146] A first determining unit 92011 is configured to determine a first brightness mean based on brightness values ​​of pixels in the area surrounded by the bounding box determined by the first determining module 910;

[0147] A second determining unit 92013 is configured to determine a first brightness range based on the first brightness mean value determined by the first determining unit 92011 and a preset brightness distribution ratio range;

[0148] The segmentation unit 92015 is used to determine an area in which the brightness values ​​of the pixel points distributed in the area included in the bounding box determined by the first determination module 910 are all within the first brightness range determined by the second determination unit 92013 as a first shadow area, and to determine an area in which the brightness values ​​of the pixel points distributed in the area included in the bounding box determined by the first determination module 910 are all outside the first brightness range determined by the second determination unit 92013 as a first main area.

[0149] In an optional example, as shown in FIG10 , obtaining submodule 9203 may include:

[0150] The third determining unit 92031 is configured to use the first smear region segmented by the segmentation submodule 9201 as a target smear recognition result of the moving object in response to the first smear region segmented by the segmentation submodule 9201 meeting a preset integrity condition;

[0151] The fourth determination unit 92033 is used to perform shadow recognition on the first main body area segmented by the segmentation submodule 9201 in response to the first shadow area segmented by the segmentation submodule 9201 not meeting the preset integrity condition, based on the shadow recognition reference information associated with the preset brightness distribution ratio range and the brightness values ​​of the pixels in the first main body area segmented by the segmentation submodule 9201, so as to segment the first main body area segmented by the segmentation submodule 9201 into a second shadow area and a second main body area, and obtain a target shadow recognition result of the moving object based on the second shadow area and the second main body area.

[0152] In an optional example, as shown in FIG11 , the apparatus provided by an embodiment of the present disclosure may further include:

[0153] A second determination module 940 is configured to determine a second brightness mean based on the brightness values ​​of the pixels in the first main body region segmented by the segmentation submodule 9201;

[0154] A third determining module 950 is configured to determine a second brightness range based on the second brightness mean value determined by the second determining module 940 and a preset brightness distribution ratio range;

[0155] a fourth determining module 960 configured to determine, in response to the brightness values ​​of the pixels in the first main body region segmented by the segmentation submodule 9201 all being outside the second brightness range determined by the third determining module 950, whether the first smear region segmented by the segmentation submodule 9201 satisfies a preset integrity condition;

[0156] The fifth determination module 970 is used to determine that the first trailing shadow area segmented by the segmentation submodule 9201 does not meet the preset integrity condition in response to at least part of the brightness values ​​of the pixel points in the first main area segmented by the segmentation submodule 9201 being within the second brightness range determined by the third determination module 950.

[0157] In an optional example, the number of the moving objects is at least two, and the at least two moving objects include a first moving object and a second moving object;

[0158] As shown in FIG12 , the apparatus provided by the embodiment of the present disclosure may further include:

[0159] A sixth determining module 980 is configured to determine a first size relationship between respective object sizes of the first moving object and the second moving object;

[0160] A seventh determining module 985 is configured to determine a second size relationship between area sizes of first smear areas corresponding to the first moving object and the second moving object;

[0161] An eighth determining module 990 is configured to determine that the first smear regions corresponding to the first moving object and the second moving object respectively meet a preset integrity condition in response to the first size relationship determined by the sixth determining module 980 matching the second size relationship determined by the seventh determining module 985;

[0162] The ninth determination module 995 is used to determine that the first trailing shadow area corresponding to the moving object with a larger object size between the first moving object and the second moving object does not meet the preset integrity condition in response to the mismatch between the first size relationship determined by the sixth determination module 980 and the second size relationship determined by the seventh determination module 985.

[0163] In an optional example, as shown in FIG13 , the first determining module 910 may include:

[0164] The first determination submodule 9101 is configured to perform edge detection on the image to be processed and determine the moving object in the image to be processed;

[0165] a second determining submodule 9103 configured to, in response to the first determining submodule 9101 determining that the moving object includes a color, take the moving object determined by the first determining submodule 9101 as a moving object and determine a bounding box of the moving object;

[0166] The third determination submodule 9105 is used to segment the moving object determined by the first determination submodule 9101 into moving objects corresponding to the at least two colors in response to the moving object determined by the first determination submodule 9101 including at least two colors, and determine the boundary box of each moving object obtained by segmentation.

[0167] In the device disclosed in the present invention, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve corresponding functions and effects, and are not listed one by one in the present disclosure.

[0168] Exemplary electronic devices

[0169] FIG14 is a structural diagram of an electronic device (eg, electronic device 10 ) provided in an embodiment of the present disclosure, including at least one processor 11 and a memory 12 .

[0170] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0171] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the one or more computer program instructions to implement the method of any method embodiment of the present disclosure described above and / or other desired functions.

[0172] In an optional example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0173] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0174] The output device 14 can output various information to the outside. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0175] Of course, for simplicity, FIG14 only shows some of the components of the electronic device 10 related to the present disclosure, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may further include any other appropriate components depending on the specific application.

[0176] Exemplary computer program products and computer-readable storage media

[0177] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method of any method embodiment of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0178] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0179] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method of any method embodiment of the present disclosure described in the above "Exemplary Method" section.

[0180] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0181] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0182] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. An image processing method, comprising: Performing edge detection on the image to be processed to determine the boundary box of the moving object in the image to be processed; Obtaining a target smear recognition result of the moving object based on a preset brightness distribution ratio range and brightness values ​​of pixels in an area surrounded by the bounding box; Based on the target smear recognition result, a predetermined process is performed on the image to be processed.

2. The method according to claim 1, wherein: The obtaining of the target smear recognition result of the moving object based on the preset brightness distribution ratio range and the brightness values ​​of the pixels in the area surrounded by the bounding box includes: Based on a preset brightness distribution ratio range and brightness values ​​of pixels in the area surrounded by the bounding box, performing smear recognition on the area surrounded by the bounding box to divide the area surrounded by the bounding box into a first smear area and a first main area; A target smear recognition result of the moving object is obtained based on the first smear area and the first main body area.

3. The method according to claim 2, wherein: The method of performing smear recognition on the area surrounded by the bounding box based on the preset brightness distribution ratio range and the brightness values ​​of the pixels in the area surrounded by the bounding box to divide the area surrounded by the bounding box into a first smear area and a first main area includes: Determining a first brightness mean based on brightness values ​​of pixels in the area surrounded by the bounding box; Determining a first brightness range based on the first brightness mean value and the preset brightness distribution ratio range; An area in which the brightness values ​​of the pixels distributed in the area included in the bounding box are all within the first brightness range is determined as the first smear area, and an area in which the brightness values ​​of the pixels distributed in the area included in the bounding box are all outside the first brightness range is determined as the first main area.

4. The method according to claim 2, wherein: The obtaining, based on the first smear area and the first main body area, a target smear recognition result of the moving object includes: In response to the first smear area satisfying a preset integrity condition, using the first smear area as a target smear recognition result of the moving object; In response to the first trailing shadow area not satisfying a preset integrity condition, trailing shadow identification is performed on the first main body area based on trailing shadow identification reference information associated with the preset brightness distribution ratio range and the brightness values ​​of the pixels in the first main body area, so as to divide the first main body area into a second trailing shadow area and a second main body area, and based on the second trailing shadow area and the second main body area, a target trailing shadow identification result of the moving object is obtained.

5. The method according to claim 4, further comprising: Determining a second brightness mean value based on brightness values ​​of pixels in the first main body area; Determining a second brightness range based on the second brightness mean value and the preset brightness distribution ratio range; In response to the brightness values ​​of the pixels in the first main body area all being outside the second brightness range, determining that the first smear area satisfies a preset integrity condition; In response to at least a portion of the brightness values ​​of the pixels in the first main body area being within the second brightness range, it is determined that the first smear area does not satisfy a preset integrity condition.

6. The method according to claim 4, wherein: The number of the moving objects is at least two, and the at least two moving objects include a first moving object and a second moving object; The method further comprises: determining a first size relationship between respective object sizes of the first moving object and the second moving object; determining a second size relationship between area sizes of the first smear areas corresponding to the first moving object and the second moving object respectively; In response to the first size relationship matching the second size relationship, determining that the first smear areas corresponding to the first moving object and the second moving object respectively satisfy a preset integrity condition; In response to the first size relationship not matching the second size relationship, it is determined that the first smear region corresponding to the moving object with a larger object size between the first moving object and the second moving object does not satisfy a preset integrity condition.

7. The method according to any one of claims 1 to 6, wherein: The performing edge detection on the image to be processed to determine the boundary box of the moving object in the image to be processed includes: Performing edge detection on the image to be processed to determine the moving object in the image to be processed; In response to the moving object including a color, treating the moving object as a moving object and determining a bounding box of the moving object; In response to the moving object including at least two colors, the moving object is segmented into moving objects corresponding to the at least two colors respectively, and a boundary box of each moving object obtained by segmentation is determined.

8. An image processing device, comprising: A first determination module is used to perform edge detection on the image to be processed and determine a bounding box of a moving object in the image to be processed; an acquisition module, configured to obtain a target smear recognition result of the moving object based on a preset brightness distribution ratio range and brightness values ​​of pixels in the area surrounded by the bounding box determined by the first determination module; An execution module is used to perform predetermined processing on the image to be processed based on the target smear recognition result obtained by the acquisition module.

9. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image processing method according to any one of claims 1 to 7.

10. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the image processing method described in any one of claims 1-7.

11. A computer program product, when instructions in the computer program product are executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Image processing method and device, computer readable storage medium and electronic equipment

    CN116597415A

  • Moving target background segmentation method based on improved Vibe+ algorithm

    CN110992381A

  • Method for eliminating smear phenomenon, terminal and storage medium

    CN111340730A

  • Imaging device and smear correction processing device

    JP2006211368A

  • Imaging device

    JP2007174032A