An image processing method, device, equipment and readable storage medium

Abstract

The application provides an image processing method, device and equipment and a readable storage medium. The method comprises the following steps: detecting all target motion regions in an event image collected by an event camera; matching each target motion region to a corresponding pre-agreed region according to a preset region matching algorithm; and outputting a sub-image corresponding to each matched target motion region in the event image. Through the implementation of the application, the detected target motion region is preprocessed with reference to the pre-agreed region, so as to improve the detectability of each target motion region in the event image and ensure the accuracy of subsequent target detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and readable storage medium. Background Technology

[0002] With the continuous development of science and technology, computer vision technology has become increasingly mature. The emergence of event cameras has attracted more and more attention in the field of machine vision. Event cameras simulate the human retina, responding to pixel pulses caused by changes in brightness due to motion. Therefore, they can capture changes in scene brightness at extremely high frame rates, recording events at specific times and locations in the image, forming an event stream rather than a frame stream. This solves the problems of information redundancy, large amounts of data storage, and real-time processing associated with traditional cameras.

[0003] When using event images output by an event camera for motion detection, related technologies typically output the detected target motion area directly to the terminal application platform for processing. This approach can reduce overall system power consumption and improve response speed to a certain extent. However, in practical applications, when the moving targets are too large, too small, or too numerous, it can easily lead to data loss and low resolution, resulting in low accuracy of subsequent target recognition. Summary of the Invention

[0004] This application provides an image processing method, apparatus, device, and readable storage medium, which can at least solve the problem of low target detection accuracy in motion detection schemes provided in related technologies.

[0005] The first aspect of this application provides an image processing method, including:

[0006] Detect all moving areas of the target in the event images captured by the event camera;

[0007] According to the preset region matching algorithm, each target motion region is matched to the corresponding pre-defined region;

[0008] The sub-images corresponding to the target motion regions after each matching process in the event image are output.

[0009] A second aspect of this application provides an image processing apparatus, comprising:

[0010] The detection module is used to detect all moving areas of the target in the event images captured by the event camera;

[0011] The matching module is used to match each of the target motion regions to a corresponding pre-defined region according to a preset region matching algorithm;

[0012] The output module is used to output the sub-images of the target motion regions corresponding to each matched processing step in the event image.

[0013] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory, and when the processor executes the computer program, it implements the steps of the image processing method provided in the first aspect of this application.

[0014] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps of the image processing method provided in the first aspect of this application.

[0015] As can be seen from the above, the image processing method, apparatus, device, and readable storage medium provided in this application detect all target motion regions in the event image acquired by the event camera; match each target motion region to a corresponding pre-defined region according to a preset region matching algorithm; and output the sub-images in the event image corresponding to each matched target motion region. Through the implementation of this application, the detected target motion regions are pre-processed with reference to pre-defined regions to improve the detectability of each target motion region in the event image, ensuring the accuracy of subsequent target detection. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the basic process of an image processing method provided in the first embodiment of this application;

[0017] Figure 2 A schematic diagram of region matching provided for the first embodiment of this application;

[0018] Figure 3 This is a schematic diagram of a plurality of target motion regions provided in the first embodiment of this application;

[0019] Figure 4 A schematic diagram of another plurality of target motion regions provided in the first embodiment of this application;

[0020] Figure 5 A detailed flowchart illustrating an image processing method provided in the second embodiment of this application;

[0021] Figure 6 This is a schematic diagram of the program modules of the image processing apparatus provided in the third embodiment of this application;

[0022] Figure 7 This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of this application. Detailed Implementation

[0023] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the description of the embodiments of this application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0025] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0026] In the embodiments of this application, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0027] The above description is merely a preferred embodiment of this application and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

[0028] To address the issue of low target detection accuracy in motion detection schemes provided in related technologies, the first embodiment of this application provides an image processing method, such as... Figure 1 This is a basic flowchart illustrating the image processing method provided in this embodiment. The image processing method includes the following steps:

[0029] Step 101: Detect all target motion regions in the event images captured by the event camera.

[0030] Specifically, in this embodiment, the event camera, also known as the event-based vision sensor (EVS), is a novel image sensor. This image sensor includes a pixel array composed of multiple pixels, each of which works independently. An event is only output when the brightness change of a certain pixel reaches a certain threshold. In this embodiment, the event data is obtained by aggregating multiple frames of event data in the event stream information cached by the event camera, and then an event image is generated based on the target event data.

[0031] It should be noted that, on the one hand, because event cameras have very low light requirements and can work well in both very low and very bright light conditions, they generate event images by changing light intensity and output only the image outline of the part where the light intensity changes. This effectively reduces redundant information compared to traditional cameras, greatly reducing the computational load of image processing and lowering power consumption. On the other hand, the frame rate of event cameras is much higher than that of traditional cameras. With its microsecond-level response and large dynamic range, it is more suitable for motion detection.

[0032] In one embodiment of this example, the step of detecting all target motion regions in the event image acquired by the event camera includes: projecting event pixels in the event image acquired by the event camera in the row direction and column direction respectively to obtain event pixel projection information; and identifying all target motion regions in the event image based on the event pixel projection information.

[0033] Specifically, in this embodiment, for pixels in the event image where the event occurred, projections are performed in both the X and Y directions. That is, the sum of event pixels in each row of the pixel array is calculated to obtain the event pixel projection information in the X direction, and the sum of event pixels in each column of the pixel array is calculated to obtain the event pixel projection information in the Y direction. Further, first motion range data of the event pixel projection information in the X direction and second motion range data of the event pixel projection information in the Y direction are obtained. Finally, the first motion range data and the second motion range data are combined to determine all target motion regions in the event image.

[0034] Step 102: Match each target motion region to the corresponding pre-defined region according to the preset region matching algorithm.

[0035] Specifically, in practical applications, to avoid data loss and low resolution caused by moving targets being too large or too small in event images, this embodiment defines a pre-defined region for each target's motion region. This pre-defined region meets the target recognition requirements. In practical applications, target motion regions may be larger or smaller than the pre-defined region, or their positions may be offset from the pre-defined region. This embodiment matches the target motion region to the corresponding pre-defined region, that is, adjusts the size and position of the target motion region to match the pre-defined region.

[0036] It should be understood that the matching of the target motion area with the corresponding pre-agreed area in this embodiment is not limited to the target motion area having the same size and position as the pre-agreed area. In practical applications, the size of the target motion area can be adjusted to be proportional to the pre-agreed area, and the center position of the target motion area can be adjusted to coincide with the pre-agreed area. This embodiment does not limit this to a single aspect.

[0037] In one embodiment of this example, the step of matching each target motion region to a corresponding pre-defined region according to a preset region matching algorithm includes: obtaining position comparison information and / or size comparison information between each target motion region and the pre-defined region; and calling the corresponding region matching algorithm according to the position comparison information and / or size comparison information to match each target motion region to the corresponding pre-defined region.

[0038] Specifically, in practical applications, there are various mismatch states between the target motion area and the pre-agreed area. This embodiment measures the specific mismatch state based on the position comparison information and / or size comparison information between the two, and then adaptively selects the region matching algorithm to achieve the matching between the target motion area and the pre-agreed area, so as to improve the efficiency and accuracy of region matching.

[0039] Further, in one embodiment of this example, the step of calling the corresponding region matching algorithm based on position comparison information and / or size comparison information to match each target motion region to the corresponding pre-agreed region includes: when the size comparison information indicates that the size of the target motion region is larger than the pre-agreed region, calling the region shrinkage transformation algorithm; wherein, the region shrinkage transformation algorithm is expressed as D(x,y)=S([x*b],[y*b]), (x,y) represents the pixel position, D(x,y) represents the pixel value at pixel position (x,y) in the pre-agreed region, S(x,y) represents the pixel value at pixel position (x,y) in the target motion region, b represents the multiple of the size of the target motion region relative to the pre-agreed region, and [] represents the rounding operation; through the region shrinkage transformation algorithm, calculating the mapping position of the target pixel position in the pre-agreed region relative to the target motion region; and assigning the pixel value of the mapping position in the target motion region to the target pixel position in the pre-agreed region.

[0040] like Figure 2 The diagram shown illustrates a region matching method provided in this embodiment. Images 201 to 204 represent four event images, where 201, 202, and 203 represent the event images before matching, and 204 represents the event image obtained after the region matching algorithm is executed. Solid lines within each image represent pre-defined regions, and dashed lines represent the detected target motion region. In the diagram, 203 indicates a target motion region larger than the pre-defined region. In this case, a reduction transformation of the target motion region is required. Therefore, this embodiment calls a region reduction transformation algorithm to reduce the target motion region horizontally and vertically. Considering the specific algorithm model of the above-mentioned region reduction transformation algorithm, for example, the coordinates of point D are (2,2), and the value of b is 5, that is, the size of the target motion region is 5 times that of the pre-agreed region. Then D(2,2)=S(2*5,2*5)=S(10,10). Therefore, in this embodiment, the value of the pixel at position (10,10) in the target motion region can be assigned to the pixel at position (2,2) in the pre-agreed region. In this way, the above operation is performed on all pixels in the target motion region to obtain the reduced target motion region. After the reduction transformation, the target motion region matches the pre-agreed region.

[0041] It should be noted that in other implementations, the scaling transformation can also be achieved by adjusting the resolution mode. For example, if the initial resolution mode of the target motion region is binning mode, it can be adjusted to binning+subsample mode, such as merging four pixels in the target motion region into one pixel to achieve a scaling down of the target motion region by a factor. Alternatively, a sampling method can be used to achieve the scaling transformation, that is, selecting a portion of pixels from each unit pixel region of the target motion region for merging, while discarding other pixels. The above implementation methods in this embodiment are illustrative and can be adapted to specific application scenarios in actual applications.

[0042] Furthermore, in another embodiment of this example, the step of calling the corresponding region matching algorithm based on position comparison information and / or size comparison information to match each target motion region to the corresponding pre-agreed region includes: when the size comparison information indicates that the size of the target motion region is smaller than the pre-agreed region, calling the region magnification transformation algorithm; wherein, the region magnification transformation algorithm is expressed as D(x,y)=S([x / a],[y / a]), (x,y) represents the pixel position, D(x,y) represents the pixel value at pixel position (x,y) in the pre-agreed region, S(x,y) represents the pixel value at pixel position (x,y) in the target motion region, a represents the multiple of the size of the pre-agreed region relative to the target motion region, and [] represents the rounding operation; through the region magnification transformation algorithm, calculating the mapping position of the target pixel position in the pre-agreed region relative to the target motion region; and assigning the pixel value of the mapping position in the target motion region to the target pixel position in the pre-agreed region.

[0043] Please refer to it again. Figure 2 In the event image shown in Figure 201, the size of the target motion region in the event image is smaller than the pre-agreed region. In this case, it is necessary to enlarge the target motion region. Therefore, this embodiment calls the region shrinkage transformation algorithm to enlarge the target motion region horizontally and vertically. Referring to the specific algorithm model of the above-mentioned region shrinkage transformation algorithm, for example, the coordinates of point D are (10,10), and the value of a is 5, that is, the size of the pre-agreed region is 5 times that of the target motion region. Then D(10,10)=S([10 / 5],[10 / 5])=S(2,2). Thus, this embodiment can assign the value of the pixel at position (2,2) in the target motion region to the pixel at position (10,10) in the pre-agreed region. In this way, the above operation is performed on all pixels in the target motion region to obtain the enlarged target motion region. After the enlargement transformation, the target motion region matches the pre-agreed region.

[0044] Of course, in practical applications, magnification can also be achieved by adjusting the resolution mode. For example, if the initial resolution mode of the target moving area is binning mode, it can be adjusted to standard mode to achieve magnification. It is worth noting that magnification can also be achieved through interpolation, that is, calculating blank pixel values ​​based on the pixel values ​​of the surrounding area. This will not be elaborated on in this embodiment.

[0045] Furthermore, in another embodiment of this example, the aforementioned position comparison information is the position offset vector between the target motion area and the same reference point (e.g., geometric center point, corner point, etc.) on the pre-agreed area. Accordingly, the steps described above, which call the corresponding region matching algorithm based on position comparison information and / or size comparison information to match each target motion region to the corresponding pre-agreed region, include: calling the corresponding region alignment algorithm based on position comparison information; wherein, the region alignment algorithm includes a first region alignment algorithm and a second region alignment algorithm, the first region alignment algorithm is expressed as D(x,y)=S([x+m],[y+n]), the second region alignment algorithm is expressed as D(x,y)=S([xm],[yn]), (x,y) represents the pixel position, D(x,y) represents the pixel value at pixel position (x,y) in the pre-agreed region, S(x,y) represents the pixel value at pixel position (x,y) in the target motion region, m represents the position offset vector value on the x-axis, m represents the position offset vector value on the y-axis, and [] represents the rounding operation; calculating the mapping position of the target pixel position in the pre-agreed region relative to the target motion region through the region alignment algorithm; and assigning the pixel value of the mapping position in the target motion region to the target pixel position in the pre-agreed region.

[0046] Specifically, in practical applications, if the target moving area has a certain positional offset from the pre-agreed area, it is necessary to align the target moving area with reference to the pre-agreed area. Since there are various types of area offsets, such as left offset, right offset, up offset, and down offset, this embodiment sets two different area alignment algorithms for different area offset types, so that the area alignment algorithm can be called accordingly based on the position comparison information. For example, if the target moving area is offset two pixels to the left relative to the pre-agreed area, for the point with target moving coordinates (10,10)D, the first area alignment algorithm is used for transformation, then D(10,10)=S([10+2],[10+2])=S(12,12). Thus, this embodiment can assign the value of the pixel with position (12,12) in the target moving area to the pixel with position (10,10) in the pre-agreed area. In this way, the above operation is performed on all pixels in the target moving area, effectively realizing the alignment of the target moving area with the pre-agreed area.

[0047] It is worth noting that in the actual region matching process, multiple algorithm models may need to be combined. This embodiment does not impose a unique limitation on the execution order of multiple algorithm models. For example, the region scaling / reduction transformation algorithm can be executed first, followed by the region alignment algorithm; or the region alignment algorithm can be executed first, followed by the region scaling / reduction transformation algorithm. The specific settings can be flexibly configured according to the specific application scenario.

[0048] Step 103: Output the sub-images of the target motion regions corresponding to each matched target motion region in the event image.

[0049] Specifically, in this embodiment, after obtaining the processed event image, the sub-images of each pre-defined region are output to the terminal application platform for subsequent target recognition. The processed event image data is complete and has high resolution, which can effectively improve the accuracy of subsequent target recognition.

[0050] In one embodiment of this example, the step of outputting the sub-images corresponding to each matched target motion region in the event image includes: reordering the sub-images corresponding to each matched target motion region in the event image according to the image size to obtain the image output order; and outputting each sub-image in sequence according to the image output order.

[0051] Specifically, in practical applications, there are usually multiple target motion regions. Therefore, this embodiment needs to output multiple sub-images of target motion regions. However, the sizes of these multiple target motion regions are different, corresponding to different resolutions, such as... Figure 3 The diagram shown illustrates multiple target motion regions provided in this embodiment. Sub-images 301, 303, and 305 have the same size, i.e., the same resolution, while sub-images 302, 304, and 306 have the same resolution. Figure 3 If the six sub-images are output sequentially in the order of sub-image 301, sub-image 302, sub-image 303, sub-image 304, sub-image 305, and sub-image 306, the output unit needs to switch resolution modes five times, resulting in significant system latency and low processing efficiency. Therefore, this embodiment reorders the required output sub-images according to their image size. Figure 4 The diagram shown is another example of multiple target motion regions provided in this embodiment. Since images of the same resolution are arranged adjacently, according to... Figure 4 When outputting images in a numbered order, the resolution mode only needs to be switched once between the output sub-image 303 and sub-image 304, which can effectively improve the image output efficiency.

[0052] Of course, in some other embodiments of this example, the step of outputting the sub-images corresponding to each target motion region in the event image after matching processing includes: obtaining the image attributes of the sub-images corresponding to each target motion region in the event image after matching processing, and obtaining the task attributes of the target recognition task; determining the image output order of the multiple sub-images according to the task attributes and image attributes; and outputting each sub-image in sequence according to the image output order.

[0053] In some further embodiments of this example, the step of outputting the sub-images corresponding to each matched target motion region in the event image includes: obtaining the image attributes of the sub-images corresponding to each matched target motion region in the event image, and obtaining the task attributes of the target recognition task; determining the sub-images to be output from the multiple sub-images whose image attributes match the task attributes; and outputting the sub-images to be output.

[0054] Specifically, this embodiment can output different sub-images according to different output priorities based on subsequent target recognition needs, or it can output sub-images of the effective region of interest based on subsequent target recognition needs, so as to meet the diverse image output needs under different target recognition scenarios.

[0055] Based on the technical solution of the embodiments of this application described above, all target motion regions in the event image acquired by the event camera are detected; each target motion region is matched to a corresponding pre-defined region according to a preset region matching algorithm; and sub-images corresponding to each matched target motion region in the event image are output. Through the implementation of this application's solution, the detected target motion regions are pre-processed with reference to pre-defined regions to improve the detectability of each target motion region in the event image, ensuring the accuracy of subsequent target detection.

[0056] Figure 5 The method described in the second embodiment of this application is a refined image processing method, which includes:

[0057] Step 501: Project the event pixels in the event image captured by the event camera in both the row and column directions to obtain the event pixel projection information.

[0058] Specifically, in this embodiment, the sum of event pixels in each row of the pixel array is calculated to obtain event pixel projection information in the X direction, and the sum of event pixels in each column of the pixel array is calculated to obtain event pixel projection information in the Y direction.

[0059] Step 502: Obtain first motion range data based on event pixel projection information in the row direction, and obtain second motion range data based on event pixel projection information in the column direction.

[0060] Step 503: Identify all target motion regions in the event image based on the first motion range data and the second motion range data.

[0061] Specifically, in this embodiment, the target motion region is the region where the moving target is located in the image. In practical applications, the number of target motion regions is greater than or equal to 1.

[0062] Step 504: Obtain position comparison information and / or size comparison information between each target motion area and the pre-agreed area.

[0063] Step 505: Based on the position comparison information and / or size comparison information, call the corresponding region matching algorithm to match each target motion region to the corresponding pre-agreed region.

[0064] Specifically, in practical applications, the target motion area may be larger or smaller than a pre-defined area, or it may be offset from the position of the pre-defined area. In this embodiment, the target motion area is matched to the corresponding pre-defined area, that is, the size and position of the target motion area are adjusted to match the pre-defined area so that the event image meets the subsequent target recognition requirements.

[0065] Step 506: Reorder the sub-images corresponding to the target motion regions after matching in the event image according to the image size to obtain the image output order.

[0066] Specifically, in this embodiment, the sub-images to be output are reordered according to the image size, and images with the same resolution are arranged adjacent to each other, which can effectively improve the image output efficiency.

[0067] Step 507: Output each sub-image to the terminal application platform in the order of image output.

[0068] Specifically, the terminal application platform in this embodiment may include the host of a smart device such as a smart door lock, and the terminal application platform executes image processing algorithms to perform target recognition applications.

[0069] It should be understood that the sequence number of each step in this embodiment does not imply the order in which the steps are executed. The execution order of each step should be determined by its function and internal logic, and should not constitute a unique limitation on the implementation process of this application embodiment.

[0070] Figure 6 This application provides an image processing apparatus according to a third embodiment. This image processing apparatus can be applied to the aforementioned image processing methods. For example... Figure 6 As shown, the image processing device mainly includes:

[0071] The detection module 601 is used to detect all target motion regions in the event images captured by the event camera;

[0072] Matching module 602 is used to match each target motion region to a corresponding pre-defined region according to a preset region matching algorithm;

[0073] The output module 603 is used to output the sub-images of the event image corresponding to each matched target motion region.

[0074] In some embodiments of this example, the detection module is specifically used to: project event pixels in the event image captured by the event camera in the row direction and column direction respectively to obtain event pixel projection information; and identify all target motion regions in the event image based on the event pixel projection information.

[0075] In some embodiments of this example, the matching module is specifically used to: obtain position comparison information and / or size comparison information between each target motion region and a pre-agreed region; and call the corresponding region matching algorithm according to the position comparison information and / or size comparison information to match each target motion region to the corresponding pre-agreed region.

[0076] Further, in some embodiments of this example, the matching module is specifically used to: when the size comparison information indicates that the size of the target motion region is larger than the pre-agreed region, invoke the region shrinkage transformation algorithm; wherein, the region shrinkage transformation algorithm is expressed as D(x,y)=S([x*b],[y*b]), (x,y) represents the pixel position, D(x,y) represents the pixel value at pixel position (x,y) in the pre-agreed region, S(x,y) represents the pixel value at pixel position (x,y) in the target motion region, b represents the multiple of the size of the target motion region relative to the pre-agreed region, and [] represents the rounding operation; calculate the mapping position of the target pixel position in the pre-agreed region relative to the target motion region through the region shrinkage transformation algorithm; assign the pixel value of the mapping position in the target motion region to the target pixel position in the pre-agreed region.

[0077] Furthermore, in some other embodiments of this example, the matching module is specifically used to: when the size comparison information indicates that the size of the target motion region is smaller than the pre-agreed region, invoke the region magnification transformation algorithm; wherein, the region magnification transformation algorithm is expressed as D(x,y)=S([x / a],[y / a]), (x,y) represents the pixel position, D(x,y) represents the pixel value at pixel position (x,y) in the pre-agreed region, S(x,y) represents the pixel value at pixel position (x,y) in the target motion region, a represents the multiple of the size of the pre-agreed region relative to the target motion region, and [] represents the rounding operation; calculate the mapping position of the target pixel position in the pre-agreed region relative to the target motion region through the region magnification transformation algorithm; assign the pixel value of the mapping position in the target motion region to the target pixel position in the pre-agreed region.

[0078] Furthermore, in some embodiments of this example, the position comparison information is the position offset vector between the target motion region and the same reference point in the pre-agreed region. Accordingly, the matching module is specifically used to: call the corresponding region alignment algorithm based on the position comparison information; wherein the region alignment algorithm includes a first region alignment algorithm and a second region alignment algorithm, the first region alignment algorithm is expressed as D(x,y)=S([x+m],[y+n]), the second region alignment algorithm is expressed as D(x,y)=S([xm],[yn]), (x,y) represents the pixel position, D(x,y) represents the pixel value at pixel position (x,y) in the pre-agreed region, S(x,y) represents the pixel value at pixel position (x,y) in the target motion region, m represents the position offset vector value on the x-axis, m represents the position offset vector value on the y-axis, and [] represents the rounding operation; calculate the mapping position of the target pixel position in the pre-agreed region relative to the target motion region using the region alignment algorithm; and assign the pixel value of the mapping position in the target motion region to the target pixel position in the pre-agreed region.

[0079] In some embodiments of this example, the output module is specifically used to: reorder the sub-images corresponding to each matched target motion region in the event image according to the image size to obtain the image output order; and output each sub-image in sequence according to the image output order.

[0080] It should be noted that the image processing methods in the first and second embodiments can be implemented based on the image processing device provided in this embodiment. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the image processing device described in this embodiment can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0081] According to the image processing apparatus provided in this embodiment, all target motion regions in the event image acquired by the event camera are detected; each target motion region is matched to a corresponding pre-defined region according to a preset region matching algorithm; and sub-images corresponding to each matched target motion region in the event image are output. Through the implementation of this application's solution, the detected target motion regions are pre-processed with reference to pre-defined regions to improve the detectability of each target motion region in the event image, ensuring the accuracy of subsequent target detection.

[0082] Figure 7 An electronic device is provided in the fourth embodiment of this application. This electronic device can be used to implement the image processing method described in the foregoing embodiments, and mainly includes:

[0083] The system includes a memory 701, a processor 702, and a computer program 703 stored on the memory 701 and executable on the processor 702. The memory 701 and the processor 702 are connected via communication. When the processor 702 executes the computer program 703, it implements the method described in Embodiment 1 or 2 above. The number of processors can be one or more.

[0084] The memory 701 can be a high-speed random access memory (RAM) or a non-volatile memory, such as a disk storage device. The memory 701 is used to store executable program code, and the processor 702 is coupled to the memory 701.

[0085] Furthermore, embodiments of this application also provide a computer-readable storage medium, which may be disposed in the aforementioned electronic device, and the computer-readable storage medium may be as described above. Figure 7 The memory in the illustrated embodiment.

[0086] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image processing method described in the foregoing embodiments. Furthermore, the computer-readable storage medium can also be a USB flash drive, a portable hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, or any other medium capable of storing program code.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0088] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0089] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0090] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0091] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0092] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0093] The above is a description of the image processing method, apparatus, device, and readable storage medium provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized in that, include: Detect all moving areas of the target in the event images captured by the event camera; Obtain position comparison information and / or size comparison information between each target motion region and a pre-defined region; wherein, the position comparison information is the position offset vector between the target motion region and the same reference point on the pre-defined region; Based on the position comparison information and / or the size comparison information, the corresponding region matching algorithm is invoked to match each of the target motion regions to the corresponding pre-defined regions in the event image; The sub-images corresponding to the target motion regions after each matching process in the event image are reordered according to image size, and the sub-images with the same resolution are arranged adjacently to obtain the image output order; The sub-images are output sequentially according to the image output order. The step of calling the corresponding region matching algorithm based on the position comparison information and / or the size comparison information to match each of the target motion regions to the corresponding pre-defined region includes: The corresponding region alignment algorithm is invoked based on the position comparison information; wherein, the region alignment algorithm includes a first region alignment algorithm and a second region alignment algorithm, and the first region alignment algorithm is expressed as follows: D (x,y)= S ([x+m],[y+n]), the second region alignment algorithm is expressed as: D (x,y)= S ([xm],[yn]) , Indicates pixel position, Indicates the pixel position in the pre-agreed region Pixel value at that location, Indicates the pixel position in the target motion region The pixel value at position m represents the position offset vector value on the x-axis, and n represents the position offset vector value on the y-axis. Indicates the rounding operation; The region alignment algorithm is used to calculate the mapping position of the target pixel position in the pre-agreed region relative to the target motion region; The pixel value of the mapped position in the target motion region is assigned to the target pixel position in the pre-agreed region.

2. The image processing method according to claim 1, characterized in that, The step of calling the corresponding region matching algorithm based on the position comparison information and / or the size comparison information to match each of the target motion regions to the corresponding pre-defined region includes: When the size comparison information indicates that the size of the target motion region is larger than the pre-agreed region, the region reduction transformation algorithm is invoked; wherein, the region reduction transformation algorithm is expressed as follows: , Indicates pixel position, Indicates the pixel position in the pre-agreed region Pixel value at that location, Indicates the pixel position in the target motion region The pixel value at point b represents the multiple of the size of the target motion region relative to the pre-defined region. Indicates the rounding operation; The region reduction transformation algorithm is used to calculate the mapping position of the target pixel position in the pre-agreed region relative to the target motion region; The pixel value of the mapped position in the target motion region is assigned to the target pixel position in the pre-agreed region.

3. The image processing method according to claim 1, characterized in that, The step of calling the corresponding region matching algorithm based on the position comparison information and / or the size comparison information to match each of the target motion regions to the corresponding pre-defined region includes: When the size comparison information indicates that the size of the target motion region is smaller than the pre-agreed region, the region magnification transformation algorithm is invoked; wherein, the region magnification transformation algorithm is expressed as follows: , Indicates pixel position, Indicates the pixel position in the pre-agreed region Pixel value at that location, Indicates the pixel position in the target motion region The pixel value at point 'a' represents the multiple of the size of the pre-defined region relative to the target motion region. Indicates the rounding operation; The region magnification transformation algorithm is used to calculate the mapping position of the target pixel position in the pre-agreed region relative to the target motion region; The pixel value of the mapped position in the target motion region is assigned to the target pixel position in the pre-agreed region.

4. The image processing method according to any one of claims 1 to 3, characterized in that, The step of detecting all target motion regions in the event image captured by the event camera includes: The event pixels in the event image captured by the event camera are projected in both the row and column directions to obtain the event pixel projection information. Identify all target motion regions in the event image based on the event pixel projection information.

5. An image processing apparatus, characterized in that, include: The detection module is used to detect all moving areas of the target in the event images captured by the event camera; A matching module is used to acquire position comparison information and / or size comparison information between each target motion region and a pre-agreed region; wherein, the position comparison information is a position offset vector of the target motion region and the same reference point on the pre-agreed region; and to call a corresponding region matching algorithm based on the position comparison information and / or the size comparison information to match each target motion region to the corresponding pre-agreed region; the step of calling a corresponding region matching algorithm based on the position comparison information and / or the size comparison information to match each target motion region to the corresponding pre-agreed region includes: calling a corresponding region alignment algorithm based on the position comparison information; wherein, the region alignment algorithm includes a first region alignment algorithm and a second region alignment algorithm, the first region alignment algorithm being expressed as... D (x,y)= S ([x+m],[y+n]), the second region alignment algorithm is expressed as: D (x,y)= S ([xm],[yn]) , Indicates pixel position, Indicates the pixel position in the pre-agreed region Pixel value at that location, Indicates the pixel position in the target motion region The pixel value at position m represents the position offset vector value on the x-axis, and n represents the position offset vector value on the y-axis. This indicates a rounding operation; using the region alignment algorithm, the mapping position of the target pixel position in the pre-agreed region relative to the target motion region is calculated; the pixel value of the mapping position in the target motion region is assigned to the target pixel position in the pre-agreed region; The output module is used to reorder the sub-images corresponding to the target motion regions after matching in the event image according to the image size, and arrange the sub-images with the same resolution adjacently to obtain the image output order; and output each sub-image in sequence according to the image output order.

6. An electronic device, characterized in that, Includes memory and processor, of which: The processor is used to execute computer programs stored in the memory; When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images