A method for eliminating blind areas of panoramic imaging targets fused with object detection algorithms

Abstract

A panoramic imaging target blind area elimination method integrated with a target detection algorithm, which includes connecting the coordinate information of the corner points of the area occupied by the vehicle closest to the world coordinates of the target on the ground, and calculating the comparison between the angle less than 90 degrees formed by this connection line and the central axis and the angle formed by the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle formed by the connection line and the central axis is greater than the angle formed by the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis direction; if the angle formed by the connection line and the central axis is less than the angle formed by the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis direction.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection of vehicle-mounted panoramic imaging systems, and in particular to a method for eliminating blind spots of panoramic imaging targets fused with a target detection algorithm. Background Art

[0002] A vehicle-mounted panoramic imaging system is mainly based on multiple cameras installed on the vehicle. By calibrating the cameras, the images of multiple cameras are stitched and synthesized and displayed on the screen. At present, the vehicle-mounted panoramic imaging system has been mature and widely used. Since the installation points of the cameras are not the same point in the real world, it can be seen that the system can only achieve seamless stitching of a certain key plane. When seamless stitching with the ground is required, there will be a blind spot phenomenon for three-dimensional objects on the stitching line of two cameras.

[0003] Therefore, if a target three-dimensional object happens to stand near the stitching line of two cameras, it cannot be observed through the panoramic system, and it is very likely to go unnoticed by the driver, causing secondary threats and potential safety hazards. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for eliminating blind spots of panoramic imaging targets fused with a target detection algorithm to prevent accidents.

[0005] The method for eliminating blind spots of panoramic imaging targets fused with the target detection algorithm according to the present invention includes the following steps:

[0006] S1: Calibrate the area ranges of four cameras and the vehicle body through the panoramic system to obtain the coordinate information of the four corner points of the area occupied by the vehicle on the ground;

[0007] S2: According to the coordinate information of the four corner points of the area occupied by the vehicle on the ground, connect the coordinate information of the two corner points at the front of the vehicle to obtain the midpoint coordinates of the two corner points at the front of the vehicle, and connect the coordinate information of the two corner points at the rear of the vehicle to obtain the midpoint coordinates of the two corner points at the rear of the vehicle;

[0008] S3: Connect the midpoint coordinates of the two corner points at the front of the vehicle and the midpoint coordinates of the two corner points at the rear of the vehicle to obtain the central axis of the area occupied by the vehicle on the ground;

[0009] S4: Extend the connection lines of the coordinates of the two corner points at the rear of the vehicle, the connection lines of the coordinates of the two corner points at the front of the vehicle, and the connection lines of the coordinates of the corner points on the same side of the front and rear of the vehicle respectively to obtain the area ranges of the blind spots of three-dimensional objects in the panoramic imaging pictures of adjacent two cameras at each corner point;

[0010] S5: Detect the images of each camera through the target detection algorithm to obtain the area range where the target exists;

[0011] S6: Calculate the world coordinates where the target stands according to the regional range where the target exists;

[0012] S7: According to the calculated world coordinates of the target, determine whether the target is within the regional range of the blind area of the three-dimensional object. If it is not within this range, return to S5. If it is within this range, execute S8;

[0013] S8: Connect the coordinate information of the corner points of the area occupied by the vehicle closest to the obtained world coordinates of the target on the ground, and calculate the angle less than 90 degrees between this connection line and the central axis;

[0014] S9: Compare the angle between this connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis. If the angle between this connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis.

[0015] As a preferred solution of the present invention, in S9, a first limit angle value and a second limit angle value are preset in advance. Compare the angle between this connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis to a position where the angle with the central axis becomes the first limit angle value. If the angle between this connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis to a position where the angle with the central axis becomes the second limit angle value.

[0016] As a preferred solution of the present invention, if the target is detected for the first time before S9, then in S9: Compare the angle between this connection line and the central axis with the default value of 45 degrees of the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle between this connection line and the central axis is greater than the default value of 45 degrees of the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis to a position where the angle with the central axis becomes the first limit angle value. If the angle between this connection line and the central axis is less than the default value of 45 degrees of the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis to a position where the angle with the central axis becomes the second limit angle value;

[0017] If the target cannot be detected within the accumulated preset time, the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras is adjusted back to the state of the customer default configuration.

[0018] As a preferred solution of the present invention, if the target is not detected again before S9, then in S9: compare the angle between this connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis and greater than 60 degrees, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis direction to a position where the angle with the central axis becomes the first limit angle value; if the angle between this connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis and less than 30 degrees, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis direction to a position where the angle with the central axis becomes the second limit angle value; if the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis and is between 30 - 60 degrees, then do not adjust the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis;

[0019] If the target cannot be detected within the accumulated preset time, the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras is adjusted back to the state of the customer default configuration.

[0020] As a preferred solution of the present invention, the first limit angle value is 5 degrees; the second limit angle value is 85 degrees.

[0021] As a preferred solution of the present invention, the target is a pedestrian or a three - dimensional object.

[0022] The beneficial effects of the present invention:

[0023] The panoramic imaging target blind area elimination method integrated with the target detection algorithm according to the present invention connects the coordinate information of the corner points of the area occupied by the vehicle closest to the world coordinates of the target on the ground, and calculates the comparison between the angle less than 90 degrees formed by the connection line and the central axis and the angle formed by the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle formed by the connection line and the central axis is greater than the angle formed by the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras is swung towards the central axis. If the angle formed by the connection line and the central axis is less than the angle formed by the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras is swung away from the central axis, which can effectively avoid the target and prevent the target from exactly appearing on or near the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras, thereby eliminating the panoramic imaging target blind area and solving the safety hazard caused by the panoramic blind area during driving to prevent accidents from occurring. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic flowchart of a panoramic imaging target blind area elimination method integrated with a target detection algorithm provided by an embodiment of the present application;

[0025] Figure 2 is a schematic flowchart of the process of initially detecting a pedestrian provided by an embodiment of the present application;

[0026] Figure 3 is a schematic flowchart of the process of detecting a pedestrian again provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0028] The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0029] Figure 1 A schematic flowchart of a panoramic imaging target blind area elimination method integrated with a target detection algorithm provided by an embodiment of the present invention. As Figure 1As shown, the method includes the following steps:

[0030] S1: Calibrate the area ranges of four cameras and the vehicle body through a panoramic system to obtain the coordinate information of the four corner points of the area occupied by the vehicle on the ground;

[0031] S2: According to the coordinate information of the four corner points of the area occupied by the vehicle on the ground, connect the coordinate information of the two corner points at the front of the vehicle to obtain the midpoint coordinates of the two corner points at the front of the vehicle, and connect the coordinate information of the two corner points at the rear of the vehicle to obtain the midpoint coordinates of the two corner points at the rear of the vehicle;

[0032] S3: Connect the midpoint coordinates of the two corner points at the front of the vehicle and the midpoint coordinates of the two corner points at the rear of the vehicle to obtain the central axis of the area occupied by the vehicle on the ground;

[0033] S4: Extend the connection lines of the coordinates of the two corner points at the rear of the vehicle, the connection lines of the coordinates of the two corner points at the front of the vehicle, and the connection lines of the coordinates of the corner points on the same side of the front and rear of the vehicle respectively to obtain the area ranges of the blind areas of three-dimensional objects in the panoramic imaging pictures of the adjacent two cameras at each corner point;

[0034] S5: Detect the pictures of each camera through a target detection algorithm to obtain the area range where the target exists;

[0035] S6: Calculate the world coordinates where the target stands according to the area range where the target exists;

[0036] S7: According to the calculated world coordinates of the target, judge whether the target is within the area range of the blind area of the three-dimensional object. If it is not within this range, return to S5. If it is within this range, execute S8;

[0037] S8: Connect the coordinate information of the corner point of the area occupied by the vehicle on the ground that is closest to the obtained world coordinates of the target, and calculate the angle less than 90 degrees between this connection line and the central axis;

[0038] S9: Compare the angle between this connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of the adjacent two cameras and the central axis. If the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of the adjacent two cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of the adjacent two cameras towards the central axis. If the angle between this connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of the adjacent two cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of the adjacent two cameras away from the central axis.

[0039] The target is a pedestrian or a three-dimensional object. In this embodiment, a pedestrian is specifically used as an example. After detecting the images of each camera through a pedestrian detection algorithm and obtaining the area range where the pedestrian exists, the world coordinates where the pedestrian stands are calculated according to the area range where the pedestrian exists. The specific implementation is as follows:

[0040] Calculate its position relative to the origin of the world coordinate system (relative to the center of the vehicle body) through the contact point between the pedestrian and the ground, and the internal and external parameters of the camera. Taking the stitching of the fisheye camera images as an example, assuming the physical coordinates in the normalized plane of the fisheye camera are (Xfish, Yfish), then where K is the known internal parameter of the fisheye camera calibration, and the pixel coordinates of the contact point between the pedestrian and the ground are Then the physical coordinates in the normalized plane of the fisheye camera The projection model of the fisheye camera is the equidistant projection model r fish = f * θ, Under the condition that f = 1 in the normalized plane, r fish and θ can be obtained through calculation, where In the ordinary linear perspective projection model, r line = f * tanθ. Under the condition that f = 1 in the normalized plane and θ has been calculated previously, it can be obtained through calculation, Then through Obtain the physical coordinates in the normalized plane of the linear perspective projection Assume that the world coordinates of a point in space are Pw(Xw, Yw, Zw), and the camera coordinates are Pc(Xc, Yc, Zc). Then the conversion formula between the fisheye camera coordinates and the world coordinates can be expressed as: Pc = R * Pw + T, where R (a 3x3 rotation matrix) and T (a 3x1 translation matrix) are the external parameter matrices of the camera calibration, and both are parameters known after the fisheye camera calibration. The conversion formula between the normalized plane coordinates and the fisheye camera is: That is,

[0041] In summary, given the internal parameter K, external parameters R and T of the fisheye camera calibration, and given the pixel coordinates of the contact point between the pedestrian and the ground and assuming that the world coordinate Zw of the point on the ground is 0, the world coordinates Pw(Xw, Yw, 0) of the pedestrian's projection on the ground can be solved according to the above conditions and formulas.

[0042] Such as Figure 2As shown, if a pedestrian is detected for the first time before S9, the first limit angle value and the second limit angle value are 5 degrees and 85 degrees respectively. At this time, the angle between the splicing dividing line of the panoramic imaging pictures of two adjacent cameras and the central axis is defaulted to 45 degrees. If no pedestrian is detected in the target area for the first time at S5, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will not swing, that is, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras is defaulted to the 45-degree angle with the central axis. When a pedestrian is detected in the target area for the first time at S5, the coordinates of the corner points of the area occupied by the vehicle closest to the world coordinates of the pedestrian will be connected, and the angle between the connection line and the central axis less than 90 degrees will be calculated and compared with the 45-degree angle. If it is greater than 45 degrees, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will be swung towards the central axis to a position where the angle with the central axis is 5 degrees. If it is less than 45 degrees, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will be swung away from the central axis to a position where the angle with the central axis is 85 degrees. If no target is detected within the preset cumulative time, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will be adjusted back to the state of the customer's default configuration, so as to ensure the integrity of the pedestrian display in the picture and further solve the safety hazards caused by the panoramic blind area during driving.

[0043] As Figure 3 shown, when the splicing dividing line of the panoramic imaging pictures of two adjacent cameras has been adjusted as described above and a pedestrian is detected in the target area again at S5, the angle between the connection line and the central axis will be compared with the angle between the splicing dividing line of the panoramic imaging pictures of two adjacent cameras and the central axis; if the angle between the connection line and the central axis is greater than the angle between the splicing dividing line of the panoramic imaging pictures of two adjacent cameras and the central axis and greater than 60 degrees, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will be swung towards the central axis to a position where the angle with the central axis is 5 degrees; if the angle between the connection line and the central axis is less than the angle between the splicing dividing line of the panoramic imaging pictures of two adjacent cameras and the central axis and less than 30 degrees, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will be swung away from the central axis to a position where the angle with the central axis is 85 degrees; if the angle between the connection line and the central axis is greater than the angle between the splicing dividing line of the panoramic imaging pictures of two adjacent cameras and the central axis and between 30 - 60 degrees, the angle between the splicing dividing line of the panoramic imaging pictures of two adjacent cameras and the central axis will not be adjusted. If no target is detected within the preset cumulative time, the splicing dividing line of the panoramic imaging pictures of two adjacent cameras will be adjusted back to the state of the customer's default configuration, which can further solve the safety hazards caused by the panoramic blind area during driving and ensure the integrity of the pedestrian display in the picture.

[0044] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural transformation made under the inventive concept of the present invention by using the content of the specification and drawings of the present invention, or any direct / indirect application in other related technical fields is included in the patent protection scope of the present invention.

Claims

1. A panoramic imaging target blind area elimination method integrated with a target detection algorithm, characterized in that, It includes the following steps: S1: Calibrate the area ranges of four cameras and the vehicle body through a panoramic system to obtain the coordinate information of the four corner points of the area occupied by the vehicle on the ground; S2: According to the coordinate information of the four corner points of the area occupied by the vehicle on the ground, connect the coordinate information of the two corner points at the front of the vehicle to obtain the midpoint coordinates of the two corner points at the front of the vehicle, and connect the coordinate information of the two corner points at the rear of the vehicle to obtain the midpoint coordinates of the two corner points at the rear of the vehicle; S3: Connect the midpoint coordinates of the two corner points at the front of the vehicle and the midpoint coordinates of the two corner points at the rear of the vehicle to obtain the central axis of the area occupied by the vehicle on the ground; S4: Extend the connection lines of the coordinates of the two corner points at the rear of the vehicle, the connection lines of the coordinates of the two corner points at the front of the vehicle, and the connection lines of the coordinates of the corner points on the same side of the front and rear of the vehicle respectively to obtain the area ranges of the three-dimensional object blind areas in the panoramic imaging pictures of the two adjacent cameras at each corner point; S5: Detect the pictures of each camera through a target detection algorithm to obtain the area range where the target exists; S6: Calculate the world coordinates where the target stands according to the area range where the target exists; S7: According to the calculated world coordinates of the target, determine whether the target is within the area range of the three-dimensional object blind area. If it is not within this range, return to S5. If it is within this range, execute S8; S8: Connect the coordinate information of the corner point of the area occupied by the vehicle on the ground that is closest to the obtained world coordinates of the target, and calculate the angle less than 90 degrees between this connection line and the central axis; S9: Compare the angle between this connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras and the central axis. If the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras towards the central axis. If the angle between this connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras away from the central axis.

2. The panoramic imaging target blind area elimination method integrated with the target detection algorithm according to claim 1, characterized in that In S9, preset a first limit angle value and a second limit angle value. Compare the angle between this connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras and the central axis. If the angle between this connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras towards the central axis to a position where the angle with the central axis becomes the first limit angle value. If the angle between this connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras and the central axis, swing the splicing demarcation line of the panoramic imaging pictures of the two adjacent cameras away from the central axis to a position where the angle with the central axis becomes the second limit angle value.

3. The panoramic imaging target blind area elimination method integrated with the target detection algorithm according to claim 2, wherein, If the target is detected for the first time before S9, then in S9: compare the angle between the connection line and the central axis with the default value of 45 degrees, which is the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle between the connection line and the central axis is greater than the default value of 45 degrees of the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis to a position where the angle with the central axis becomes the first limit angle value. If the angle between the connection line and the central axis is less than the default value of 45 degrees of the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis to a position where the angle with the central axis becomes the second limit angle value. If the target is not detected within the preset cumulative time, then adjust the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras back to the state of the customer default configuration.

4. The panoramic imaging target blind area elimination method integrated with the target detection algorithm according to claim 3, wherein, If the target is detected again before S9, then in S9: compare the angle between the connection line and the central axis with the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the angle between the connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis and is greater than 60 degrees, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras towards the central axis to a position where the angle with the central axis becomes the first limit angle value. If the angle between the connection line and the central axis is less than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis and is less than 30 degrees, then swing the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras away from the central axis to a position where the angle with the central axis becomes the second limit angle value. If the angle between the connection line and the central axis is greater than the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis and is between 30 - 60 degrees, then do not adjust the angle between the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras and the central axis. If the target is not detected within the preset cumulative time, then adjust the splicing demarcation line of the panoramic imaging pictures of two adjacent cameras back to the state of the customer default configuration.

5. The panoramic imaging target blind area elimination method integrated with the target detection algorithm according to claim 4, characterized in that, The first limit angle value is 5 degrees; the second limit angle value is 85 degrees.

6. The panoramic imaging target blind area elimination method integrated with the target detection algorithm according to claim 1, characterized in that The target is a pedestrian or a three-dimensional object.

Images