A 360-degree surround image-assisted reversing radar blind spot detection method and system

Abstract

The present application relates to the field of vehicle blind spot detection, and provides a 360-degree surround image-assisted reversing radar blind spot detection method and system, which uses a camera to capture real-time video, decodes it into image frames, and identifies ground features, based on which auxiliary lines and warning lines are preset in the monitoring screen. The auxiliary lines are set according to the overlapping boundaries of the camera's shooting range, marking the key overlapping areas where the camera's field of view intersects; the auxiliary lines are offset to the middle of the image to define potential danger warning areas. Objects are then identified and tracked in the image frame. Once the object's area is found to be increasing and entering the warning or overlapping area, it indicates that the object may enter the driving blind spot, and a warning is immediately issued in combination with the end radar data. The safety of reversing is improved, and the potential collision risk caused by blind spots is effectively avoided.

Description

Technical Field

[0001] The present application relates to the field of vehicle blind spot detection, and more specifically, to a method and system for detecting blind spots of reversing radars assisted by 360-degree surround view images. Background Art

[0002] The contents of this section merely provide background information related to the present application and may not constitute prior art.

[0003] When reversing, drivers often face a serious safety hazard, namely the blind spot problem, especially the blind spots at the four corners of the vehicle. These blind spots are located on both sides of the rear of the vehicle and in the corners near the rear of the vehicle. Due to the obstruction of the body structure, it is often difficult for the driver to directly observe these areas through the rearview mirror and the interior rearview mirror. This means that if there are pedestrians, children, pets or low obstacles near the four corners of the vehicle, they are likely to be completely out of the driver's sight, increasing the risk of collision and accidents. Even when using reversing radar or 360 reversing image assistance system, the blind spots at the four corners of the vehicle may still not be fully covered or accurately identified due to the installation position and angle limitations of the sensor or camera.

[0004] In the prior art, a Chinese patent with announcement number CN108333589A proposes an obstacle detection device for autonomous driving vehicles, which enhances the vehicle's environmental perception capabilities by integrating multiple sensors. Specifically, the device includes a vehicle body, a vehicle chassis, a first ultrasonic sensor, a first laser radar, a window frame, a second ultrasonic sensor, and a second laser radar. Among them, the first laser radar and the second laser radar are cleverly installed on the top of the front and rear outer sides of the vehicle body, respectively, to provide long-distance and high-precision environmental data. At the same time, the first ultrasonic sensor and the second ultrasonic sensor are respectively deployed on the lower part of the front and rear outer sides of the vehicle body, and their sound wave emission directions are respectively consistent with the vehicle's straight and reverse directions, aiming to use the advantages of ultrasonic sensors in close-range detection to make up for the blind spots of laser radars at certain angles and close ranges.

[0005] However, although this multi-sensor fusion strategy can significantly improve the obstacle detection capability of autonomous vehicles in complex environments in theory, it faces certain challenges in practical applications. Especially in reversing scenarios, since most existing stock vehicles use a combination of ordinary radars and cameras, and not all vehicles install expensive and technically complex lidars at the rear of the vehicle, the promotion and application of this technical solution on existing vehicles is limited. This situation results in the vehicle's ability to detect blind spots during reversing, which increases driving safety risks.

[0006] Therefore, in the field of blind spot detection when reversing, there is an urgent need for a method based on the coordinated use of ordinary radars and cameras that are already popular on existing vehicles to achieve effective monitoring of blind spots during vehicle reversing, thereby ensuring driving safety. Summary of the invention

[0007] In order to solve the above technical problems, the purpose of this application is to provide a 360-degree surround image-assisted reversing radar blind spot detection method and system, which constructs a warning area and an overlapping area based on the shooting range of adjacent cameras, and by identifying objects entering the warning area and the overlapping area, it reminds the driver that an object is about to enter the blind spot, thereby improving the safety of reversing.

[0008] The purpose of this application is achieved through the following technical solutions:

[0009] In a first aspect, the present invention provides a 360-degree surround image-assisted reversing radar blind spot detection method, comprising a plurality of cameras arranged around a vehicle and a middle radar and an end radar arranged at the rear of the vehicle, wherein the middle radar is used to detect obstacles behind the vehicle, and the end radars are arranged on both sides of the rear of the vehicle and located on the chassis below the taillights, and the end radars are used to detect obstacles in the blind spot of the vehicle;

[0010] Detection methods include:

[0011] In response to the vehicle's reversing signal, the camera located in the vehicle is activated to obtain monitoring videos of the rear and side of the vehicle, and all reversing radars on the vehicle are activated;

[0012] Decode the surveillance video into continuous image frames, select the color and texture in the lower middle area of ​​the image frame as the first feature of the ground, and identify the ground area on the image frame according to the first feature;

[0013] Auxiliary lines and warning lines are preset on the monitoring screen of each camera. The auxiliary line is the overlapping boundary between the shooting ranges of two adjacent cameras. The image boundary closest to the auxiliary line on each image frame is found, and the ground area between the auxiliary line and the image boundary is marked as the overlapping area; the warning line is obtained by offsetting the auxiliary line to the middle of the image frame by a preset distance, and the area between the warning line and the auxiliary line is marked as the warning area;

[0014] Using the identified ground area as the background, the contour information of all objects in the image frame is identified through edge detection, and the texture information, color information, shape and area of ​​each object contour are recorded;

[0015] Based on the recorded texture information, color information and object contours, the same object is tracked in multiple adjacent image frames. If the area of ​​any object is detected to be increasing and the object is located in the warning area or the overlapping area, a prompt is issued that the object is about to enter the blind spot, and the detection information of the end radar is obtained in real time.

[0016] Furthermore, auxiliary lines are established by the following steps:

[0017] A monitoring model is established based on the shooting range of each camera, the location of each camera on the vehicle, and the size of the vehicle.

[0018] According to the built-in coordinate system of the monitoring model, the overlapping area between the shooting ranges of each camera and the boundary of the overlapping area are obtained;

[0019] Record the target boundaries of other cameras within the shooting range of the target camera, and project the target boundaries into the monitoring screen of the target camera as auxiliary lines.

[0020] Furthermore, the step of projecting the target boundary onto the monitoring screen of the target camera as an auxiliary line specifically includes:

[0021] The coordinates of the target boundary are projected onto the monitoring screen through the coordinate conversion formula, and the corresponding auxiliary lines are generated; the coordinate conversion formula is:

[0022]

[0023] in, is the horizontal coordinate of the corresponding point in the monitoring screen, is the ordinate of the corresponding point in the monitoring screen, is the translation vector from the camera coordinate system to the image plane coordinate system, is the focal length of the camera, R is the rotation matrix of the coordinate system in the camera relative to the coordinate system in the monitoring model, T is the translation vector from the coordinate system in the camera to the coordinate system in the monitoring model, and Respectively represent the scaling factors in the corresponding directions, To monitor the model and the monitoring screen The corresponding three-dimensional coordinates.

[0024] Further, the step of selecting the color and texture in the lowermost middle area of ​​the image frame as the first feature of the ground specifically includes:

[0025] The color and texture in the lower middle area of ​​the first image frame are used as ground features. When the ground area identified according to the ground features in any subsequent image frame is smaller than a preset threshold, the middle radar is detected to see if there is an obstacle. If so, an obstacle approaching prompt is issued; if not, the middle area is expanded in the vertical direction, and the colors and corresponding textures with the first and second largest area proportions are selected as the first features of the ground.

[0026] Furthermore, after selecting the colors with the first and second largest area proportions and the corresponding textures as the first feature of the ground, the method further includes:

[0027] In the recognition process based on the first feature, when the proportion of any color and the corresponding texture in the middle area of ​​any subsequent image frame is 0, the color and the corresponding texture are removed from the first feature.

[0028] Furthermore, after issuing a prompt that an object is about to enter the blind spot and acquiring the detection information of the end radar in real time, it also includes:

[0029] On the vehicle's central control display screen, the image of the warning area or overlapping area at this time is enlarged, and the outline of the object with an increased area is highlighted to indicate that the object is about to enter the blind spot.

[0030] Furthermore, the process of enlarging the image of the warning area or the overlapping area specifically includes:

[0031] The image in the warning area or the overlapping area is distorted and corrected using the intrinsic and extrinsic parameters of the corresponding camera to obtain an image that conforms to the real world; the intrinsic and extrinsic parameters are obtained by calibration of the corresponding camera.

[0032] Furthermore, the auxiliary line includes a plurality of sub-auxiliary lines connected end to end in sequence, and each sub-auxiliary line corresponds to a distance between a different position and the blind spot.

[0033] In a second aspect, the present invention provides a 360-degree surround view image-assisted reversing radar blind spot detection system, comprising:

[0034] The starting module, in response to the vehicle's reversing signal, starts the camera located in the vehicle to obtain monitoring videos of the rear and side of the vehicle, and starts all reversing radars on the vehicle;

[0035] A ground feature acquisition module is used to decode the surveillance video into continuous image frames, select the color and texture in the lower middle area of ​​the image frame as the first feature of the ground, and identify the ground area on the image frame according to the first feature;

[0036] Auxiliary line module: auxiliary lines and warning lines are preset on the monitoring screen of each camera. The auxiliary line is the overlapping boundary between the shooting ranges of two adjacent cameras. The image boundary closest to the auxiliary line on each image frame is found, and the ground area between the auxiliary line and the image boundary is marked as the overlapping area; the warning line is obtained by offsetting the auxiliary line to the middle of the image frame by a preset distance, and the area between the warning line and the auxiliary line is marked as the warning area;

[0037] The object recognition module uses the identified ground area as the background, identifies the contour information of all objects on the image frame through edge detection, and records the texture information, color information, shape and area of ​​each object contour;

[0038] The tracking prompt module is used to track the same object in multiple adjacent image frames based on the recorded texture information, color information and object contour. If the area of ​​any object is detected to be increasing and the object is located in the warning area or the overlapping area, a prompt is issued that the object is about to enter the blind spot, and the detection information of the end radar is obtained in real time.

[0039] In a third aspect, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps corresponding to the first aspect when executing the computer program.

[0040] In summary, the technical solution of the embodiment of the present application has at least the following advantages and beneficial effects:

[0041] The present invention arranges multiple cameras around the vehicle and adds middle and end radars at the rear of the vehicle to construct a 360-degree surround image system, which is specifically used to assist in detecting radar blind spots when reversing. When the reversing signal is activated, the camera and radar are immediately started, the monitoring video is converted into an image frame, and the ground is automatically identified as the background. Subsequently, auxiliary lines and warning lines are preset in the monitoring screen of each camera: the auxiliary lines are set according to the overlapping boundaries of the shooting ranges of adjacent cameras to mark the overlapping areas; the warning lines are formed after the auxiliary lines are offset by a specific distance to the center of the image, aiming to define the range that may enter the blind spot. Through edge detection technology, the system can identify the contour, color and texture features of objects and track these objects in continuous image frames. If the area of ​​an object is detected to be gradually increasing, it indicates that it is approaching the vehicle, and if the object is located in the warning or overlapping area, it is considered to be about to enter the blind spot, and the system will immediately issue a warning. At the same time, combined with the detection data of the end radar, the accuracy and safety of judging objects in the blind spot are enhanced in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a method for detecting blind spots of a reversing radar assisted by a 360-degree surround view image provided by the present invention;

[0043] Figure 2 is a distribution diagram of radar in the present invention;

[0044] Figure 3 A top view of the monitoring model in the present invention;

[0045] Figure 4 It is a schematic diagram of projecting the warning line and the auxiliary line onto a two-dimensional image in the present invention;

[0046] Figure 5 A schematic structural diagram of a 360-degree surround view image-assisted reversing radar blind spot detection system provided by the present invention.

[0047] Icons: 1. Central radar; 2. End radar; 3. Side shooting range; 4. Rear shooting range; 5. Blind spot; 6. Auxiliary line; 7. Warning line; 8. Warning area; 9. Overlapping area. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0049] Please refer to Figure 1 The embodiment of the present application proposes a 360-degree surround image-assisted reversing radar blind spot detection method, comprising a plurality of cameras arranged around the vehicle and a middle radar 1 and an end radar 2 arranged at the rear of the vehicle, wherein the middle radar 1 is used to detect obstacles behind the vehicle, and the end radar 2 is arranged on both sides of the rear of the vehicle and located on the chassis below the taillights, such as Figure 2 As shown, the end radar 2 is set at a lower position to identify shorter people or objects, while the two sides located at the rear of the vehicle are mainly used to detect obstacles in the blind spot 5 of the vehicle;

[0050] The test method includes:

[0051] S101, in response to a reversing signal of the vehicle, a camera located in the vehicle is activated to obtain monitoring videos of the rear and sides of the vehicle, and all reversing radars on the vehicle are activated.

[0052] Among them, the system automatically activates cameras distributed around the vehicle, including cameras located on both sides, the front end and the rear end of the vehicle. This embodiment mainly uses cameras on both sides and the rear end of the vehicle, which are responsible for capturing real-time monitoring videos of the rear and sides of the vehicle, and at the same time starting all reversing radars equipped on the vehicle.

[0053] S102, decoding the surveillance video into continuous image frames, selecting the color and texture in the lowermost middle area of ​​the image frame as a first feature of the ground, and identifying the ground area on the image frame according to the first feature;

[0054] Among them, the captured surveillance video is decoded into continuous image frames, which is the basis for subsequent image processing. In order to accurately distinguish the ground from other objects, the system selects the color and texture of the lower middle area of ​​the image frame as the first feature of the ground. Since the vehicle is driving on the road, the ground area near the rear of the vehicle becomes the most intuitive and stable reference point. By comparing and analyzing the color and texture features of this area, the system can effectively identify the ground area in the image frame, which provides an important basis for distinguishing the ground from obstacles in the subsequent steps.

[0055] Furthermore, the step of selecting the color and texture in the lowermost middle area of ​​the image frame as the first feature of the ground specifically includes: using the color and texture in the lowermost middle area of ​​the first image frame as the ground feature, and when the ground area identified according to the ground feature in any subsequent image frame is less than a preset threshold, detecting whether there is an obstacle in the middle radar 1, and if so, issuing an obstacle approaching prompt; if not, expanding the middle area in the vertical direction, and selecting the first and second colors and corresponding textures with the largest area proportion as the first features of the ground.

[0056] Specifically, the system only relies on the color and texture in the middle area at the bottom of the first image frame as the ground feature. However, this method may encounter challenges when facing changes in ground material (for example, reversing from asphalt road to concrete or epoxy floor in garage), because the color and texture features of different grounds are significantly different, which may lead to ground recognition errors or failures. To solve this problem, the system uses the color and texture in the middle area at the bottom of the first image frame as the initial ground feature. When the ground area of ​​the middle area is less than the preset threshold in any subsequent image frame based on this ground feature, it is determined that a change in ground material may have occurred. At this time, the system will not immediately determine that the recognition has failed, but will further detect whether there is an obstacle in the middle radar 1. If the middle radar 1 detects an obstacle, the system will immediately issue a prompt that the obstacle is approaching to remind the driver to pay attention. If the middle radar 1 does not detect an obstacle, it means that the current ground change has not led to the misjudgment of the actual obstacle, then the system will dynamically expand the range of the middle area in the vertical direction to include more possible ground feature information.

[0057] The system will reselect the colors with the largest and second largest area and their corresponding textures as new ground features in the expanded middle area. This strategy allows the system to adaptively adjust the recognition criteria of ground features when encountering changes in ground material, thereby improving the continuity and accuracy of ground recognition.

[0058] Furthermore, after selecting the colors and corresponding textures with the first and second largest area proportions as the first feature of the ground, it also includes: in the recognition process based on the first feature, when the proportion of any color and corresponding texture in the middle area of ​​any subsequent image frame is 0, the color and corresponding texture are removed from the first feature.

[0059] In actual applications, as the vehicle travels and the surrounding environment changes, some colors and textures that were originally identified as ground features may gradually disappear or become less significant. If the system continues to retain these no longer representative features for identification, it will not only waste computing resources, but may also reduce the accuracy and efficiency of identification. Therefore, after selecting new ground features, the system adds a mechanism to dynamically adjust the feature set. When the proportion of a certain color and corresponding texture in the middle area in any subsequent image frame is 0, the system believes that this feature has lost its representativeness in the current environment, so it will remove it from the ground feature set. This reduces the number of features during identification, thereby reducing the amount of calculation and improving the real-time and accuracy of recognition.

[0060] Specifically, the system checks each feature in the ground feature set in each frame. If the proportion of a feature is 0 and this state lasts for multiple frames (to avoid misjudgment due to short-term lighting changes or occlusion), the system will remove it from the feature set. In this way, the system can use limited computing resources more efficiently and focus on features that are still representative in the current environment, thereby improving recognition accuracy and efficiency.

[0061] S103, auxiliary lines 6 and warning lines 7 are preset on the monitoring screen of each camera, such as Figure 4 As shown, the auxiliary line 6 is the overlapping boundary between the shooting ranges of two adjacent cameras (i.e., the rear camera located at the rear of the vehicle and any camera located at the side), the image boundary closest to the auxiliary line 6 on each image frame is found, and the ground area between the auxiliary line 6 and the image boundary is marked as the overlapping area 9; the warning line 7 is obtained by offsetting the auxiliary line 6 to the middle of the image frame by a preset distance, and the area between the warning line 7 and the auxiliary line 6 is marked as the warning area 8;

[0062] Specifically, in order to improve the accuracy and efficiency of blind spot 5 detection, auxiliary lines 6 and warning lines 7 are pre-set on the monitoring screen of each camera. Auxiliary lines 6 mark the overlapping boundary between the shooting ranges of two adjacent cameras. By finding the image boundary closest to auxiliary line 6 on each image frame, the system can mark the ground area between auxiliary line 6 and the boundary as overlapping area 9. This area is a key area where the camera fields of view intersect, and it is also a high-risk area where objects may disappear from the field of view of one camera and enter the blind spot 5 on the other side. Warning line 7 is obtained by offsetting a certain distance from auxiliary line 6 to the middle of the image frame. The area within this line and between auxiliary line 6 is marked as warning area 8, which is used to warn objects that may enter blind spot 5 in advance. The length of auxiliary line 6 and warning line 7 is set to correspond to a distance of 2 to 3 meters in the real world to avoid false alarms and unnecessary calculation burden.

[0063] Among them, auxiliary line 6 is established by the following steps:

[0064] S201, establishing a monitoring model according to the shooting range of each camera, the position of each camera in the vehicle, and the size of the vehicle, such as Figure 3 The side shooting range 3 and the rear shooting range 4 have an overlapping area 9, and the two sides of the overlapping area 9 are auxiliary lines 6. The auxiliary lines 6 are offset to obtain two corresponding warning lines 7 and corresponding warning areas 8, thereby completing the use of the overlapping area 9 and the warning area 8 to cover the position entering the blind spot 5.

[0065] Specifically, the system builds a three-dimensional monitoring model based on the specific parameters of each camera (such as focal length, viewing angle, etc.), the camera installation position on the vehicle, and the vehicle size information. This model not only simulates the spatial position relationship between the camera and the vehicle, but also can obtain their shooting ranges and the overlapping areas between these ranges by simulating the shooting angles of different cameras at different positions9.

[0066] S202, obtaining the overlapping area 9 between the shooting ranges of each camera and the boundary of the overlapping area 9 according to the built-in coordinate system of the monitoring model.

[0067] S203, record the target boundary of other cameras within the shooting range of the target camera, and project the target boundary into the monitoring screen of the target camera as an auxiliary line 6, such as Figure 4 shown.

[0068] The implementation of this step relies on a complex coordinate conversion process: first, the system needs to convert the coordinates in the monitoring model coordinate system to the coordinate system of the target camera; then, using the internal parameters of the camera (such as focal length, etc.), the coordinates in the camera coordinate system are further converted to the image coordinate system, and finally the corresponding auxiliary lines 6 are generated in the monitoring screen. The existence of these auxiliary lines 6 enables the driver to intuitively see the boundaries between the shooting ranges of different cameras, so as to more accurately judge the environmental conditions around the vehicle.

[0069] Specifically, the specific calculation process of the projection process is as follows:

[0070] Assume that the translation vector is T, where T represents the position of the camera in the monitoring model coordinate system; Assume that the rotation matrix of the monitoring model coordinate system relative to the camera coordinate system is R, and any point of the target boundary on the monitoring model is expressed as , The coordinates in the camera coordinate system are .

[0071] First, calculate the conversion from the monitoring model coordinate system to the camera coordinate system. The specific calculation formula is as follows:

[0072]

[0073] Then, calculate the transformation from the camera coordinate system to the image plane coordinate system. The specific calculation formula is as follows:

[0074]

[0075] in, for The position on the image plane, is the focal length of the camera (i.e. the distance from the camera to the image plane).

[0076] Since the point in the image plane coordinate system is in millimeters, while points in pixel coordinates are The units are pixels, so scaling and translation are required during the conversion. Assume that the scaling factor in the x direction is , the scaling factor in the y direction is , the origin of the image plane (usually the center of the image) is located in the pixel coordinate system .

[0077] Then the translation vector from the camera coordinate system to the image plane coordinate system is for:

[0078]

[0079] Corresponding points in pixel coordinate system The calculation formula is:

[0080]

[0081] Combining formula (1) to formula (7), we get the coordinate transformation formula:

[0082]

[0083] in, is the horizontal coordinate of the corresponding point in the monitoring screen (or pixel coordinate system), is the ordinate of the corresponding point in the monitoring screen (or pixel coordinate system). Based on the above coordinate conversion formula, for existing vehicles with 360-degree panoramic cameras and reversing radars, the present invention can be used to divide the auxiliary line 6 and the warning line 7, thereby improving convenience.

[0084] S104, using the identified ground area as the background, identifying the contour information of all objects on the image frame through edge detection, and recording the texture information, color information, shape and area of ​​each object contour. The recorded information is intended to provide data support for subsequent object tracking and blind spot 5 judgment.

[0085] S105, based on the recorded texture information, color information and object contour, the same object is tracked in multiple adjacent image frames. If it is detected that the area of ​​any object is increased and the object is located in the warning area 8 or the overlapping area 9, a prompt is issued that the object is about to enter the blind area 5, and the detection information of the end radar 2 is obtained in real time.

[0086] Among them, according to the principles of visual perception and geometric optics, as the position between the object and the camera changes, the farther the object is, the smaller the area of ​​the object seen, and the closer the object is, the larger the area of ​​the object seen. Therefore, if the area of ​​any object is found to be gradually increasing, it means that the object is approaching the vehicle, and if the object is located in the warning area 8 or the overlapping area 9, it means that it may be about to enter the blind spot 5 of the reversing. At this time, the system will immediately issue a prompt that the object is about to enter the blind spot 5, and obtain the detection information of the end radar 2 in real time to double confirm the existence and position of the obstacle, so as to ensure that the driver can take avoidance measures in time and effectively avoid the occurrence of collision accidents.

[0087] Furthermore, after issuing a prompt that an object is about to enter the blind spot 5 and acquiring the detection information of the end radar 2 in real time, it also includes: on the vehicle's central control display screen, enlarging the image of the warning area 8 or the overlapping area 9 at this time, and highlighting the outline of the object with an increased area to prompt that the object is about to enter the blind spot 5.

[0088] Specifically, considering that the 360-degree panoramic imaging system commonly equipped in modern cars has significant advantages in providing an all-round field of view, it is limited by the physical size of the central control display screen. Some areas on the screen, especially the warning area 8 and the overlapping area 9 located at the four corners of the vehicle, may appear relatively small due to the display ratio problem. This size limitation often causes the driver to easily ignore these key areas when observing the screen, thereby increasing potential safety hazards. Therefore, this embodiment takes measures to enlarge the image of the warning area 8 or the overlapping area 9 on the central control display screen. Based on the natural tendency of human visual attention, the enlarged image is more likely to attract the attention of the observer. The partial enlargement of this area by technical means can ensure that the driver can clearly see the object that is about to enter the blind spot 5 even when the screen is small.

[0089] In addition, in order to further enhance the recognition effect, the function of highlighting the outline of an object with an enlarged display area has been added. On the basis of image magnification, the system will also perform edge recognition and outline extraction on the detected object, and then highlight its outline on the screen. This highlighting not only makes the object stand out more in a complex background, but also helps the driver to more quickly judge the type, size and direction of movement of the object, so as to make more accurate driving decisions.

[0090] In the process of magnifying the image of the warning area 8 or the overlapping area 9, the image of the warning area 8 or the overlapping area 9 is distorted and corrected using the internal and external parameters of the corresponding camera to obtain an image that conforms to the real world, thereby facilitating observation by the driver. The internal and external parameters are obtained by calibration of the corresponding camera.

[0091] Furthermore, the auxiliary line 6 includes a plurality of sub-auxiliary lines connected end to end in sequence, and each sub-auxiliary line corresponds to the distance between a different position and the blind spot 5, thereby providing a distance reference for the driver and improving convenience.

[0092] Based on the same inventive concept, Figure 5 As shown, the present invention proposes a 360-degree surround image-assisted reversing radar blind spot detection system for implementing the above detection method, including:

[0093] The start module 201 starts the camera located in the vehicle to obtain the surveillance video of the rear and side of the vehicle in response to the reversing signal of the vehicle, and starts all the reversing radars on the vehicle;

[0094] The ground feature acquisition module 202 is used to decode the surveillance video into continuous image frames, select the color and texture in the lower middle area of ​​the image frame as the first feature of the ground, and identify the ground area on the image frame according to the first feature;

[0095] Auxiliary line module 203, each camera's monitoring screen is preset with auxiliary lines 6 and warning lines 7, the auxiliary line 6 is the overlapping boundary between the shooting ranges of two adjacent cameras, find the image boundary closest to the auxiliary line 6 on each image frame, and mark the ground area between the auxiliary line 6 and the image boundary as the overlapping area 9; the warning line 7 is obtained by offsetting the auxiliary line 6 to the middle of the image frame by a preset distance, and the area between the warning line 7 and the auxiliary line 6 is marked as the warning area 8;

[0096] The object recognition module 204 uses the recognized ground area as the background, recognizes the contour information of all objects on the image frame through edge detection, and records the texture information, color information, shape and area of ​​each object contour;

[0097] The tracking prompt module 205 is used to track the same object in multiple adjacent image frames based on the recorded texture information, color information and object contour. If it is detected that the area of ​​any object is increased and the object is located in the warning area 8 or the overlapping area 9, a prompt is issued that the object is about to enter the blind area 5, and the detection information of the end radar 2 is obtained in real time.

[0098] Based on the same inventive concept, the present invention proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor implements the steps corresponding to the above-mentioned 360-degree surround view image-assisted reversing radar blind spot detection method.

[0099] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting blind spots of a reversing radar assisted by a 360-degree surround image, characterized in that: It includes a plurality of cameras arranged around the vehicle, a middle radar and an end radar arranged at the rear of the vehicle, wherein the middle radar is used to detect obstacles behind the vehicle, and the end radars are arranged on both sides of the rear of the vehicle and located on the chassis below the taillights, and the end radars are used to detect obstacles in the blind spot of the vehicle; The detection method comprises: In response to a reversing signal of the vehicle, a camera located in the vehicle is activated to obtain surveillance videos of the rear and sides of the vehicle, and all reversing radars on the vehicle are activated; Decoding the surveillance video into continuous image frames, selecting the color and texture in the lowermost middle area of ​​the image frame as the first feature of the ground, and identifying the ground area on the image frame according to the first feature; Auxiliary lines and warning lines are preset on the monitoring screen of each camera. The auxiliary line is the overlapping boundary between the shooting ranges of two adjacent cameras. The image boundary closest to the auxiliary line on each image frame is found, and the ground area between the auxiliary line and the image boundary is marked as the overlapping area; the warning line is obtained by offsetting the auxiliary line to the middle of the image frame by a preset distance, and the area between the warning line and the auxiliary line is marked as the warning area; Using the identified ground area as the background, identifying the contour information of all objects on the image frame through edge detection, and recording the texture information, color information, shape and area of ​​each object contour; According to the recorded texture information, color information and object contour, the same object is tracked in multiple adjacent image frames. If the area of ​​any object is detected to be increased and the object is located in the warning area or the overlapping area, a prompt is issued that the object is about to enter the blind spot, and the detection information of the end radar is obtained in real time.

2. The detection method according to claim 1, characterized in that: The auxiliary line is established by the following steps: A monitoring model is established according to the shooting range of each camera, the position of each camera on the vehicle, and the size of the vehicle. According to the built-in coordinate system of the monitoring model, obtaining the overlapping area between the shooting ranges of each camera and the boundary of the overlapping area; The target boundaries of other cameras within the shooting range of the target camera are recorded, and the target boundaries are projected into the monitoring picture of the target camera as auxiliary lines.

3. The detection method according to claim 2, characterized in that: The step of projecting the target boundary onto the monitoring screen of the target camera as an auxiliary line specifically includes: The coordinates of the target boundary are projected into the monitoring screen through a coordinate conversion formula, and corresponding auxiliary lines are generated; the coordinate conversion formula is: in, is the horizontal coordinate of the corresponding point in the monitoring screen, is the ordinate of the corresponding point in the monitoring screen, is the translation vector from the camera coordinate system to the image plane coordinate system, is the focal length of the camera, R is the rotation matrix of the coordinate system in the camera relative to the coordinate system in the monitoring model, T is the translation vector from the coordinate system in the camera to the coordinate system in the monitoring model, and Respectively represent the scaling factors in the corresponding directions, To monitor the model and the monitoring screen The corresponding point position.

4. The detection method according to claim 1, characterized in that: The step of selecting the color and texture in the lowermost middle area of ​​the image frame as the first feature of the ground specifically includes: The color and texture in the lowermost middle area of ​​the first image frame are used as ground features. When the ground area identified according to the ground features in any subsequent image frame is smaller than a preset threshold, the middle radar is detected to see if there is an obstacle. If so, an obstacle approaching prompt is issued; if not, the middle area is expanded in the vertical direction, and the colors and corresponding textures with the first and second largest area proportions are selected as the first features of the ground.

5. The detection method according to claim 4, characterized in that: After selecting the first and second colors and the corresponding textures with the largest and smallest area proportions as the first feature of the ground, the method further includes: In the recognition process based on the first feature, when the proportion of any color and corresponding texture in the middle area in any subsequent image frame is 0, the color and corresponding texture are removed from the first feature.

6. The detection method according to claim 1, characterized in that: After issuing a prompt that the object is about to enter the blind spot and acquiring the detection information of the end radar in real time, the method further includes: On the central control display screen of the vehicle, the image of the warning area or the overlapping area at this time is enlarged, and the outline of the object with an increased area is highlighted to prompt that the object is about to enter the blind spot.

7. The detection method according to claim 6, characterized in that: The process of enlarging the image of the warning area or the overlapping area specifically includes: The image of the warning area or the overlapping area is distorted by using the internal and external parameters of the corresponding camera to obtain an image that conforms to the real world; the internal and external parameters are obtained by calibration of the corresponding camera.

8. The detection method according to claim 1, characterized in that: The auxiliary line includes a plurality of sub-auxiliary lines connected end to end in sequence, and each sub-auxiliary line corresponds to a distance between a different position and a blind area.

9. A 360-degree surround image-assisted reversing radar blind spot detection system, used to implement the method as claimed in claim 1, characterized in that: include: A start module, in response to a reversing signal of the vehicle, starts a camera located in the vehicle to obtain surveillance videos of the rear and side of the vehicle, and starts all reversing radars on the vehicle; A ground feature acquisition module, used for decoding the surveillance video into continuous image frames, selecting the color and texture in the lowermost middle area of ​​the image frame as the first feature of the ground, and identifying the ground area on the image frame according to the first feature; Auxiliary line module, each camera's monitoring screen is preset with auxiliary lines and warning lines, the auxiliary line is the overlapping boundary between the shooting ranges of two adjacent cameras, finds the image boundary closest to the auxiliary line on each image frame, and marks the ground area between the auxiliary line and the image boundary as the overlapping area; The warning line is obtained by offsetting the auxiliary line toward the middle of the image frame by a preset distance, and the area between the warning line and the auxiliary line is marked as a warning area; The object recognition module uses the recognized ground area as the background, recognizes the contour information of all objects on the image frame through edge detection, and records the texture information, color information, shape and area of ​​each object contour; The tracking prompt module is used to track the same object in multiple adjacent image frames based on the recorded texture information, color information and object contour. If the area of ​​any object is detected to be increased and the object is located in the warning area or the overlapping area, a prompt is issued that the object is about to enter the blind spot, and the detection information of the end radar is obtained in real time.

10. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps corresponding to the method according to any one of claims 1 to 8 when executing the computer program.

Images