Image processing method, device, electronic device and storage medium

Abstract

The embodiment of the present application provides an image processing method, device, electronic device and storage medium, the method comprising: establishing a three-dimensional structure corresponding to the image to be detected; obtaining the normal vector of the center point of the target area in the image to be detected; determining the acquisition angle corresponding to the target area based on the position of the camera and the normal vector of the center point of the target area in the image to be detected; projecting the three-dimensional structure corresponding to the image to be detected with the acquisition angle directly above to obtain an image of the acquisition angle directly above. This achieves more accurate information such as the location of the boundary area of ​​the target area, thereby improving the accuracy of area detection.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and specifically to image processing methods, devices, electronic devices and storage media. Background Art

[0002] Intrusion detection technology is a technology that determines in real time whether there are objects entering the target area based on images collected by cameras. Intrusion detection technology is widely used in many scenarios such as urban security entry detection, intrusion warning in dangerous areas of factories, and crowd density analysis in shopping malls.

[0003] Due to the limitation of installation conditions, the installation position of the camera is usually outside the target area, and the acquisition angle of view is tilted relative to the target area. Some acquisition angles are even seriously tilted relative to the target area.

[0004] On the one hand, since the acquisition angle of view is inclined to the target area, the camera may only capture a part of the actual part of the object entering the target area. Correspondingly, the monitoring image only includes a part of the actual part of the object entering the target area, making it difficult to accurately determine the actual part of the object entering the target area through the monitoring image, and the accuracy of the entry detection result is low.

[0005] On the other hand, in the same actual entry situation, that is, the area of ​​the part of the object entering the same target area is the same, due to the different acquisition angles, the area of ​​the part of the object entering the entry area in the monitoring images captured with different acquisition angles is different. When different areas are compared with the same entry detection judgment standard, such as the same area threshold, different entry detection results may be obtained in the same actual entry situation, resulting in lower accuracy of the entry detection results. Summary of the invention

[0006] In order to overcome the problems existing in the related art, the present application provides an image processing method, device, electronic device and storage medium.

[0007] According to a first aspect of an embodiment of the present application, there is provided an image processing method, comprising:

[0008] Establishing a three-dimensional structure corresponding to the image to be detected;

[0009] Obtaining a normal vector of a center point of a target area in the image to be detected;

[0010] Determine the acquisition viewing angle directly above the target area based on the position and normal vector of the camera that acquires the image to be detected;

[0011] The three-dimensional structure is projected at the directly above acquisition viewing angle to obtain a directly above acquisition viewing angle image.

[0012] In some embodiments, the method further comprises:

[0013] Detecting the image captured from the upper viewing angle to determine a target bounding box corresponding to the target object;

[0014] The target detection result is determined based on the target bounding box and the target area in the image captured from the directly above perspective.

[0015] In some embodiments, determining the target detection result based on the target bounding box and the target area in the image captured from the upper perspective includes:

[0016] Calculate the intersection-over-union ratio of the target bounding box and the target area in the image captured from the perspective directly above;

[0017] When the intersection-over-union ratio is greater than a set threshold, a target detection result indicating that the target object corresponding to the target bounding box enters the target area is determined.

[0018] In some embodiments, based on the position and normal vector of the camera that collects the image to be detected, determining the upper capture angle corresponding to the target area includes:

[0019] estimating a posture transformation matrix based on the posture of the camera that collects the image to be detected and the normal vector;

[0020] According to the posture transformation matrix, the upper acquisition viewing angle corresponding to the target area is determined.

[0021] In some embodiments, obtaining the normal vector of the center point of the target area in the image to be detected includes:

[0022] Extracting ground feature information from the image to be detected, the ground feature information comprising: contour feature information of the ground in the image to be detected, and contour feature information of at least one object located on the ground in the image to be detected;

[0023] Based on the ground feature information, determining a target area in the image to be detected;

[0024] Generate a normal vector of the center point of the target area in the image to be detected.

[0025] In some embodiments, the center point of the target area in the image to be detected is the center of gravity of the target area in the image to be detected.

[0026] In some embodiments, before establishing the three-dimensional structure corresponding to the image to be detected, the method further includes:

[0027] The position and posture of the camera are determined based on a plurality of original captured viewing angle images captured by the camera.

[0028] According to a second aspect of an embodiment of the present application, there is provided an image processing apparatus, including:

[0029] An establishing unit, configured to establish a three-dimensional structure corresponding to the image to be detected;

[0030] An acquisition unit, configured to acquire a normal vector of a center point of a target area in the image to be detected;

[0031] A determination unit is configured to determine a capture angle of view directly above the target area based on a position and a normal vector of a camera that captures the image to be detected;

[0032] The projection unit is configured to project the three-dimensional structure at the directly above acquisition viewing angle to obtain an image of the directly above acquisition viewing angle.

[0033] In some embodiments, the apparatus further comprises:

[0034] The detection unit is configured to detect the image captured from the perspective of the top to determine a target bounding box corresponding to the target object; and determine a target detection result based on the target bounding box and the target area in the image captured from the perspective of the top.

[0035] In some embodiments, the detection unit is further configured to calculate an intersection-and-union ratio between the target bounding box and the target area in the image captured from the perspective directly above; when the intersection-and-union ratio is greater than a set threshold, a target detection result indicating that the target object corresponding to the target bounding box enters the target area is determined.

[0036] In some embodiments, the determination unit is further configured to estimate a posture transformation matrix based on the posture of the camera that captures the image to be detected and the normal vector; and determine the top capture viewing angle corresponding to the target area according to the posture transformation matrix.

[0037] In some embodiments, the acquisition unit is further configured to extract ground feature information from the image to be detected, the ground feature information including: contour feature information of the ground in the image to be detected, contour feature information of at least one object located on the ground in the image to be detected; based on the ground feature information, determine the target area in the image to be detected; and generate a normal vector of the center point of the target area in the image to be detected.

[0038] In some embodiments, the center point of the target area in the image to be detected is the center of gravity of the target area in the image to be detected.

[0039] In some embodiments, the posture determination unit is configured to determine the posture of the camera based on a plurality of original captured view images captured by the camera before establishing a three-dimensional structure corresponding to the image to be detected.

[0040] The image processing method and device provided in the embodiments of the present application realize determining the overhead acquisition viewing angle corresponding to the target area based on the posture of the camera that acquires the image to be detected and the normal vector of the center point of the target area in the image to be detected, which is equivalent to configuring the acquisition viewing angle of the camera as the overhead acquisition viewing angle, projecting the three-dimensional structure with the overhead acquisition viewing angle to obtain an overhead acquisition viewing angle image, and the overhead acquisition viewing angle is a bird's-eye view type of viewing angle, which is equivalent to the camera acquiring images when looking down at objects on the ground. Therefore, more accurate information such as the position of the boundary area of ​​the target area can be obtained, thereby improving the accuracy of area detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown;

[0043] Figure 2 A structural block diagram of an image processing device provided in an embodiment of the present application is shown;

[0044] Figure 3 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0046] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown, and the method includes:

[0048] Step 101: Establish a three-dimensional structure corresponding to the image to be detected.

[0049] In this application, the image captured by the camera used for detection is referred to as the original captured view angle image. The camera used for detection can be a single-purpose camera. During the entry detection, the camera used for entry detection continuously captures the original captured view angle image to form a video stream.

[0050] In the present application, the original captured viewing angle image captured by the camera for detecting whether an object enters the target area can be further referred to as an image to be detected.

[0051] The image to be detected can be selected from the video stream captured by the camera.

[0052] Each time an entry detection is performed using an image to be detected captured by a camera, steps 101-104 can be performed, except that each entry detection is performed on a different image to be detected by the camera. Therefore, steps 101-104 are steps in the process of performing an entry detection using an original captured view angle image captured by a camera.

[0053] Establishing a three-dimensional structure corresponding to the image to be detected is equivalent to performing three-dimensional reconstruction of the entry detection scene targeted by the camera that collects the image to be detected.

[0054] The three-dimensional structure corresponding to the image to be detected includes: the three-dimensional structure of each object in the image to be detected.

[0055] For example, an entry detection is performed on a target area of ​​a factory. The image to be detected includes: the ground, buildings, pedestrians, vehicles in the target area on the ground, buildings, pedestrians, vehicles and other objects in the non-target area on the ground. The three-dimensional structure corresponding to the image to be detected includes the three-dimensional structure of each object in the image to be detected. For example, when counting the flow of people entering a shopping mall, an entry detection is performed on a shopping mall, and the image to be detected includes: the ground, the shopping mall on the ground, buildings adjacent to the shopping mall, pedestrians adjacent to the shopping mall, vehicles adjacent to the shopping mall and other objects. The three-dimensional structure corresponding to the image to be detected includes the three-dimensional structure of each object in the image to be detected.

[0056] Since the building object is fixed, for a camera, the building in each image to be detected collected by the camera is the same, and the images to be detected collected by the camera are different only in the mobile objects such as pedestrians and vehicles that may be included. Therefore, after the first execution of step 101, the three-dimensional structure of the building object, such as a shopping mall or a factory, can be stored. When step 101 is executed again, there is no need to re-establish the three-dimensional structure of the building object, but only to establish the three-dimensional structure of mobile objects such as pedestrians and vehicles, and to form the three-dimensional structure of the building object that enters the detection scene together with the pre-stored three-dimensional structure, that is, the three-dimensional structure corresponding to the image to be detected.

[0057] In the present application, when establishing the three-dimensional structure corresponding to the image to be detected, a neural network for depth estimation can be used to first extract features related to depth information from the image to be detected, and then predict the depth information of the pixels in the image to be detected based on the features related to the depth information.

[0058] The neural network used for depth estimation can be a Coarse network, an FCN (Fully Convolutional Networks) network, etc.

[0059] Therefore, in addition to obtaining the pixel value of each pixel in the image to be detected in the R channel, the pixel value of the G channel, and the pixel value of the B channel according to the existing image feature information of the image to be detected, the depth information of each pixel is also obtained. The image to be detected is equivalent to an RGB-D image.

[0060] In the present application, the 3D-R2N2 network can be used to reconstruct the three-dimensional structure according to the pixel value of the R channel, the pixel value of the G channel, the pixel value of the B channel, and the depth information of each pixel in the image to be detected to obtain the three-dimensional structure corresponding to the image to be detected.

[0061] Step 102: Obtain the normal vector of the center point of the target area in the image to be detected.

[0062] The target area in the image to be detected is a representation of the actual target area in the detection scene in the image to be detected. In other words, the target area in the image to be detected may refer to an area in the image to be detected that represents the actual target area in the detection scene.

[0063] In the present application, the contour of the target area in the original captured perspective image can be annotated by an annotator. Thus, the positions of the line segments of the contour of the target area in the original captured perspective image in the original captured perspective image are determined, and the target area in the original captured perspective image is determined. Then, the center point of the target area in the image to be detected can be further determined, and the normal vector of the center point of the target area in the image to be detected can be generated.

[0064] The normal vector of the center point of the target area in the image to be detected is a normal vector perpendicular to the target area in the image to be detected and starting from the center point of the target area in the image to be detected.

[0065] Since the target regions in the images to be detected are the same, only one image to be detected needs to be labeled. After the target region in the labeled image to be detected is determined, the target regions in all other images to be detected can also be determined.

[0066] In the present application, the neural network for detecting the target area can also be trained in advance using training samples. The training samples are the original captured view images, and the contours of the target areas in the training samples are annotated. The trained neural network learns the features of the contours of the target areas and detects the target areas in the images to be detected.

[0067] In some embodiments, ground feature information is extracted from the image to be detected, and the ground feature information includes: contour feature information of the ground in the image to be detected, and contour feature information of each object located on the ground in the image to be detected; based on the ground feature information, a target area in the image to be detected is determined; and a normal vector of a center point of the target area in the image to be detected is generated.

[0068] In the present application, the boundary line of the ground in each direction in the image to be detected can be determined. The existing horizon estimation technology can be used to determine the boundary line of the ground in each direction in the image to be detected. After determining the boundary line of the ground in each direction in the image to be detected, the ground in the image to be detected can be determined.

[0069] Then, edge detection can be performed on the objects on the ground in the image to be detected to obtain the contour feature information of each object on the ground in the image to be detected. The contour feature information of the object on the ground describes the contour of the object on the ground. According to the contour feature information of the object, the contour of the object on the ground can be determined.

[0070] When the target area is the area occupied by a certain building on the ground, the contour of the target area is the contour of the building on the ground. The contour of the area occupied by a certain building on the ground is known, so it is possible to determine which of the contours of each object on the ground is the contour of the building, thereby determining the target area.

[0071] When the target area includes multiple buildings, the relative position relationship between the boundary lines on the contour of the target area and the boundary lines in corresponding directions on the contour of the object on the ground is known. Therefore, the boundary lines of the contour of the target area in various directions can be determined, and the target area can be determined.

[0072] In some embodiments, the center point of the target area in the image to be detected is the center of gravity of the target area in the image to be detected.

[0073] In the present application, the position of the center of gravity of the target area can be determined based on the ground feature information obtained from the image to be detected, thereby determining the center of gravity of the target area and taking the center of gravity of the target area as the center point of the area.

[0074] For example, the target area is a shopping mall on a plane. The shape of the outline of the shopping mall on the ground is not a regular shape. In this case, the center of gravity of the shopping mall on the ground is determined and used as the center point of the shopping mall on the ground.

[0075] Step 103, based on the position and normal vector of the camera, determine the upper capture angle corresponding to the target area.

[0076] In a security system, the installation position and viewing angle of a camera are known and can be obtained from a corresponding server in the security system. The position and viewing angle of the camera can be determined based on the installation position and viewing angle of the camera.

[0077] In some embodiments, before establishing the three-dimensional structure corresponding to the image to be detected, it also includes: determining the posture of the camera based on a plurality of original captured viewing angle images captured by the camera for determining the posture.

[0078] In the present application, before establishing the three-dimensional structure corresponding to the image to be detected, the pose of the camera can be estimated based on a plurality of original captured perspective images for determining the pose. The matching pixel points between the plurality of original captured perspective images for determining the pose can be first determined, and the corresponding relationship between the matching pixel points and the three-dimensional points in the three-dimensional space can be calculated based on the matching pixel points between the plurality of original captured perspective images, and the pose of the camera can be estimated based on the corresponding relationship between the matching pixel points and the three-dimensional points in the three-dimensional space.

[0079] In the present application, the neural network GeoNET network for estimating the camera's posture can be used to estimate the camera's posture based on multiple original captured view images for determining the posture.

[0080] In the present application, the camera's acquisition angle of view can be configured as the top acquisition angle of view based on the camera's posture and the normal vector of the center point of the target area in the image to be detected, thereby determining the top acquisition angle of view corresponding to the target area.

[0081] The acquisition angle of view of a camera can also be referred to as the shooting angle of the camera, and the acquisition angle of view is associated with the coordinate axis. The acquisition angle of view may refer to the angle between the line connecting the optical axis of the camera and the center point of the target area and the vertical coordinate axis in the world coordinate system, i.e., the Y axis. The acquisition angle of view may also refer to the angle between the line connecting the optical axis of the camera and the center point of the target area and the horizontal coordinate axis in the world coordinate system, i.e., the X axis. The acquisition angle of view may also refer to the angle between the line connecting the optical axis of the camera and the center point of the target area and the Z axis in the world coordinate system. The definition of the acquisition angle of view is determined according to actual needs and is not limited in this application.

[0082] In the present application, the acquisition angle directly above the target area may refer to the acquisition angle of the camera facing the center point of the target area in the three-dimensional space. In other words, when the acquisition angle of the camera is the acquisition angle directly above, the position of the camera is directly above the center point of the target area in the three-dimensional space.

[0083] The acquisition angle of view directly above the target area is determined according to the definition of the acquisition angle of view. For example, the acquisition angle of view refers to the angle between the line connecting the optical axis of the camera and the center point of the target area and the vertical coordinate axis in the world coordinate system, that is, the Y axis. When the acquisition angle of the camera is the directly above angle of view, the optical axis of the camera coincides with the vertical coordinate axis in the world coordinate system, and the camera faces the center point of the target area in the three-dimensional space, and the acquisition angle directly above is 0 degrees. For example, the acquisition angle of view refers to the angle between the line connecting the optical axis of the camera and the center point of the target area and the horizontal coordinate axis in the world coordinate system, that is, the X axis. When the acquisition angle of the camera is the directly above angle of view, the optical axis of the camera coincides with the vertical coordinate axis in the world coordinate system, and the acquisition angle directly above is 90 degrees.

[0084] The overhead capture perspective is a bird's-eye view, which is equivalent to the camera capturing images when it is looking down at objects on the ground. By configuring the camera's capture perspective to be the overhead capture perspective, an overhead capture perspective image is obtained, which can ensure that the overhead capture perspective image includes the entire actual part of the object entering the target area.

[0085] In some embodiments, determining the overhead capture angle corresponding to the target area based on the posture of the camera that captures the image to be detected and the normal vector of the center point of the target area includes: estimating the posture transformation matrix based on the posture of the camera that captures the image to be detected and the normal vector of the center point of the target area; determining the overhead capture angle corresponding to the target area based on the posture transformation matrix.

[0086] The posture transformation matrix can be estimated by first using the normal vector of the center point of the target area in the image to be detected and the posture of the camera. The posture transformation matrix can represent the transformation relationship between the posture of the camera when it is directly above the acquisition angle and the determined posture of the camera.

[0087] Then, the mapping relationship between the position of the 3D point of the object in the 3D space and the position of the pixel point corresponding to the 3D point in the image collected under the actual viewing angle of the camera and the posture transformation matrix can be used to determine the mapping relationship between the position of the 3D point of the object in the 3D space and the position of the pixel point corresponding to the 3D point in the image collected under the viewing angle directly above. According to the determined mapping relationship, any 3D point in the 3D space can be mapped to the corresponding pixel point in the image collected under the viewing angle directly above, so as to obtain the directly above acquisition image collected under the viewing angle directly above. Thus, the acquisition angle of the camera is configured as the directly above acquisition angle, and the directly above acquisition angle corresponding to the target area is determined.

[0088] It should be understood that configuring the camera's acquisition angle to be the acquisition angle directly above is not a physical change of the camera's actual acquisition angle to the acquisition angle directly above. The camera's actual acquisition angle is still the angle that is tilted relative to the target area due to the limitations of the installation conditions. It is only because the camera's acquisition angle is configured as the acquisition angle directly above through the camera's posture and the normal vector of the center point of the target area in the image to be detected that the camera is equivalent to having an acquisition angle directly above.

[0089] In other words, by configuring the camera's acquisition angle of view to be an upward acquisition angle of view, it is equivalent to setting a camera directly above the center point of the target area to capture images.

[0090] Step 104 , projecting the three-dimensional structure from a top acquisition perspective to obtain a top acquisition perspective image.

[0091] After the camera's acquisition viewing angle is configured as the top acquisition viewing angle, the three-dimensional structure corresponding to the image to be detected is projected at the top acquisition viewing angle to obtain a top acquisition viewing angle image.

[0092] The three-dimensional structure corresponding to the image to be detected includes the three-dimensional structure of the object on the ground entering the detection scene in three-dimensional space. The three-dimensional structure corresponding to the image to be detected is projected from the acquisition perspective directly above, and the image acquired from the acquisition perspective directly above is equivalent to the acquisition perspective image acquired when the camera shoots the object on the ground entering the detection scene from the acquisition perspective directly above. Projecting a three-dimensional structure from a perspective to obtain a two-dimensional image is an existing technology, and the projection process will not be described in detail in this application.

[0093] In some embodiments, it also includes: detecting the image captured from the perspective of the top to determine a target bounding box corresponding to the target object; and determining a target detection result based on the target bounding box and the target area in the image captured from the perspective of the top.

[0094] In the present application, the target detection network can be used to first detect each object in the image captured from the perspective of the top. At the same time, the target area in the image captured from the perspective of the top can be determined. The process of determining the target area in the image captured from the perspective of the top can refer to the process of determining the target area in the image to be detected.

[0095] The target detection neural network can be Faster RCNN, SSD, YOLO, RefineDet, RetinaNet, etc.

[0096] The image captured from the top is input into the target detection network to obtain the bounding box information of each object among all the objects detected by the target detection network output.

[0097] The bounding box information of the object includes: the position of the center point of the rectangular bounding box surrounding the object, the length of each side of the rectangular bounding box surrounding the object, and the like.

[0098] In the present application, objects whose positions in the overhead perspective image are near the boundary of the target area in the overhead perspective image, such as pedestrians and vehicles near the boundary of the target area in the overhead perspective image, can be determined and objects whose positions in the overhead perspective image are near the boundary of the target area in the overhead perspective image are taken as target objects.

[0099] In the present application, the detected object may be referred to as a target object, and the bounding box surrounding the target object may be referred to as a target bounding box corresponding to the target object.

[0100] The target area in the overhead capture perspective image is a representation of the actual target area in the detection scene in the overhead capture perspective image. In other words, the target area in the overhead capture perspective image may refer to an area in the overhead capture perspective image that represents the actual target area in the detection scene.

[0101] For each target object, the area occupied by the target bounding box corresponding to the target object in the image captured from the perspective directly above can be used as the area occupied by the target object. Correspondingly, the area of ​​the target bounding box corresponding to the target object in the image captured from the perspective directly above can be used as the area occupied by the target object.

[0102] For each target object in the image captured from the perspective of the top, the area of ​​the portion of the target object in the target area in the image captured from the perspective of the top, i.e., the area entering the target area, can be determined, and the ratio of the area of ​​the portion of the target object in the target area in the image captured from the perspective of the top to the area occupied by the target object is calculated, i.e., the quotient of the area of ​​the portion of the target object in the target area in the image captured from the perspective of the top divided by the area occupied by the target object. When the calculated ratio is greater than the ratio threshold, it can be determined that the target object has entered the target area, and an entry detection result indicating that the target object has entered the target area is determined.

[0103] In the present application, it is ensured that the image captured from the viewing angle directly above includes the entire actual part of the target object entering the target area, thereby avoiding the situation where the monitoring image only includes a part of the actual part of the target object entering the target area due to the tilt of the capturing viewing angle relative to the target area. In this way, the actual part of the target object entering the target area can be determined more accurately, thereby improving the accuracy of the entry detection results.

[0104] In any entry detection using any image to be detected, the camera's acquisition angle is uniformly configured as the top acquisition angle to obtain the top acquisition angle image. This ensures that in the same actual entry situation, the area of ​​the part of the target object in the top acquisition angle image that enters the entry area is the same, and the entry detection results obtained are consistent.

[0105] In some embodiments, determining the target detection result based on the target bounding box and the target area in the image captured from the perspective directly above includes: calculating the intersection-over-union (IOU) between the target bounding box and the target area in the image captured from the perspective directly above; when the IOU is greater than a set threshold, determining the target detection result indicating that the target object corresponding to the target bounding box has entered the target area.

[0106] For the target bounding box corresponding to each target object, the intersection ratio of the target bounding box and the target area in the image captured from the perspective directly above can be calculated. When the intersection ratio of the target bounding box and the target area in the image captured from the perspective directly above is greater than a set threshold, a target detection result indicating that the target object corresponding to the target bounding box has entered the target area can be determined.

[0107] Please refer to Figure 2 , which shows a structural block diagram of an image processing device provided by an embodiment of the present application. The device includes: an establishing unit 201, an acquiring unit 202, a determining unit 203, and a projecting unit 204.

[0108] The establishing unit 201 is configured to establish a three-dimensional structure corresponding to the image to be detected;

[0109] The acquisition unit 202 is configured to acquire a normal vector of a center point of a target area in the image to be detected;

[0110] The determination unit 203 is configured to determine the acquisition viewing angle directly above the target area based on the posture of the camera that acquires the image to be detected and the normal vector;

[0111] The projection unit 204 is configured to project the three-dimensional structure at the directly above acquisition viewing angle to obtain an image at the directly above acquisition viewing angle.

[0112] In some embodiments, the image processing apparatus further comprises:

[0113] The detection unit is configured to detect the image captured from the perspective of the top to determine a target bounding box corresponding to the target object; and determine a target detection result based on the target bounding box and the target area in the image captured from the perspective of the top.

[0114] In some embodiments, the detection unit is further configured to calculate an intersection-and-union ratio between the target bounding box and the target area in the image captured from the perspective directly above; when the intersection-and-union ratio is greater than a set threshold, a target detection result indicating that the target object corresponding to the target bounding box enters the target area is determined.

[0115] In some embodiments, the determination unit 203 is further configured to estimate a posture transformation matrix based on the posture of the camera that captures the image to be detected and the normal vector; and determine the upper capture viewing angle corresponding to the target area according to the posture transformation matrix.

[0116] In some embodiments, the acquisition unit 202 is further configured to extract ground feature information from the image to be detected, the ground feature information including: contour feature information of the ground in the image to be detected, contour feature information of at least one object located on the ground in the image to be detected; based on the ground feature information, determine the target area in the image to be detected; and generate a normal vector of the center point of the target area in the image to be detected.

[0117] In some embodiments, the center point of the target area in the image to be detected is the center of gravity of the target area in the image to be detected.

[0118] In some embodiments, the image processing apparatus further comprises:

[0119] The posture determination unit is configured to determine the posture of the camera based on a plurality of original captured view images for determining the posture captured by the camera before establishing the three-dimensional structure corresponding to the image to be detected.

[0120] Figure 33 is a block diagram of an electronic device provided by the present application. The electronic device includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions executable by the processing component 322, such as an application. The application stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 322 is configured to execute instructions to perform the above method.

[0121] The electronic device may also include a power supply component 326 configured to perform power management of the electronic device, a wired or wireless network interface 350 configured to connect the electronic device to a network, and an input / output (I / O) interface 358. The electronic device may operate based on an operating system stored in the memory 332, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™ or the like.

[0122] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by an electronic device to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0123] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0124] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Establishing a three-dimensional structure corresponding to the image to be detected; Obtaining a normal vector of a center point of a target area in the image to be detected; Determine the acquisition viewing angle directly above the target area based on the position and normal vector of the camera that acquires the image to be detected; Projecting the three-dimensional structure from the directly above acquisition viewing angle to obtain a directly above acquisition viewing angle image; Wherein, determining the acquisition viewing angle directly above the target area based on the posture of the camera that acquires the image to be detected and the normal vector includes: Based on the posture of the camera that collects the image to be detected and the normal vector, estimating a posture transformation matrix, the posture transformation matrix representing the transformation relationship between the posture of the camera when it is in the directly above collection angle of view and the determined posture of the camera, the image to be detected including an object; Determining the acquisition viewing angle directly above the target area according to the posture transformation matrix; Determine a mapping relationship between the position of the three-dimensional point of the object in the three-dimensional space and the position of the pixel point corresponding to the three-dimensional point in the image collected at the actual viewing angle of the camera, and a posture transformation matrix; According to the determined mapping relationship, any three-dimensional point in the three-dimensional space is mapped to a corresponding pixel point in the expected image captured at the directly above capture angle, so as to obtain the directly above capture image captured at the directly above capture angle, thereby configuring the camera's capture angle to be the directly above capture angle.

2. The method according to claim 1, characterized in that: The method further comprises: Detecting the image captured from the upper viewing angle to determine a target bounding box corresponding to the target object; The target detection result is determined based on the target bounding box and the target area in the image captured from the directly above perspective.

3. The method according to claim 2, characterized in that The determining of the target detection result based on the target bounding box and the target area in the image captured from the upper perspective includes: Calculate the intersection-over-union ratio of the target bounding box and the target area in the image captured from the perspective directly above; When the intersection-over-union ratio is greater than a set threshold, a target detection result indicating that the target object corresponding to the target bounding box enters the target area is determined.

4. The method according to claim 1, characterized in that: Obtaining the normal vector of the center point of the target area in the image to be detected includes: Extracting ground feature information from the image to be detected, the ground feature information comprising: contour feature information of the ground in the image to be detected, and contour feature information of at least one object located on the ground in the image to be detected; Based on the ground feature information, determining a target area in the image to be detected; Generate a normal vector of the center point of the target area in the image to be detected.

5. The method according to claim 1, characterized in that The center point of the target area in the image to be detected is the center of gravity of the target area in the image to be detected.

6. The method according to claim 1, characterized in that Before establishing the three-dimensional structure corresponding to the image to be detected, the method further includes: The position and posture of the camera are determined based on a plurality of original captured viewing angle images captured by the camera.

7. An image processing device, characterized in that: The device comprises: An establishing unit, configured to establish a three-dimensional structure corresponding to the image to be detected; An acquisition unit, configured to acquire a normal vector of a center point of a target area in the image to be detected; A determination unit is configured to determine a capture angle of view directly above the target area based on a position and a normal vector of a camera that captures the image to be detected; A projection unit is configured to project the three-dimensional structure from the directly above acquisition perspective to obtain a directly above acquisition perspective image; The determination unit is further configured to estimate a posture transformation matrix based on the posture of the camera that collects the image to be detected and the normal vector, wherein the posture transformation matrix represents the transformation relationship between the posture of the camera when it is in the directly above collection angle of view and the determined posture of the camera, and the image to be detected includes an object; and determine the directly above collection angle of view corresponding to the target area according to the posture transformation matrix; Based on the mapping relationship between the position of the three-dimensional point of the object in the three-dimensional space and the position of the pixel point corresponding to the three-dimensional point in the image captured at the actual viewing angle of the camera and the posture transformation matrix, the mapping relationship between the position of the three-dimensional point of the object in the three-dimensional space and the position of the pixel point corresponding to the three-dimensional point in the image captured at the directly above capturing viewing angle is determined; based on the determined mapping relationship, any three-dimensional point in the three-dimensional space is mapped to a corresponding pixel point in the image captured at the directly above capturing viewing angle that is expected to be obtained, and the directly above capturing image captured at the directly above capturing viewing angle is obtained, thereby configuring the capturing viewing angle of the camera to be the directly above capturing viewing angle.

8. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 6. 9 . A storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 6.

Images