Obstacle recognition method based on monocular camera and recognition device and vehicle thereof

Abstract

The application provides a monocular camera-based obstacle identification method and an identification device and a vehicle thereof, and solves the technical problem that a vehicle-mounted system causes unnecessary safety accidents when the road environment is poor or even bad in the prior art. The monocular camera-based obstacle identification method provided by the application calibrates a calibration object with a known calibration position of a vehicle in a calibration image to obtain a corresponding relationship between the calibration position of the calibration object and a calibration pixel position of the calibration object in the calibration image, determines an actual position relationship between the obstacle and the vehicle according to a pixel position of the obstacle in an actual image and the corresponding relationship, and determines whether the obstacle threatens the vehicle according to the actual position relationship between the obstacle and the vehicle, so as to control the vehicle. The monocular camera with a lower price not only reduces the cost, but also improves the accuracy of obstacle budgeting, can detect any obstacle, and improves the detection rate of the convex and concave ground.

Description

Technical Field

[0001] This application relates to the field of automatic detection technology, specifically to an obstacle recognition method and device based on a monocular camera, and a vehicle. Background Technology

[0002] In the field of autonomous driving, the accuracy of obstacle detection is crucial for achieving driverless operation and is of great significance. LiDAR (Light Detection and Ranging) can generate three-dimensional information with high ranging accuracy, precisely determining the target location and effectively improving obstacle detection. Therefore, depth cameras can detect the depth of field in the shooting space. By acquiring the distance of each point in the image from the camera and adding its two-dimensional coordinates in the 2D image, the three-dimensional spatial coordinates of each point in the image are obtained, thereby determining obstacles in the three-dimensional space surrounding the vehicle. Therefore, current in-vehicle systems typically use laser scanners and depth cameras to identify obstacles.

[0003] When road conditions are poor or even severe, the vehicle system is prone to misjudging the road and obstacles on it, which can lead to unnecessary safety accidents. Summary of the Invention

[0004] In view of this, this application provides an obstacle recognition method, recognition device, and vehicle based on a monocular camera, which solves the technical problem in the prior art that when the road environment is poor or even severe, the vehicle system is prone to misjudging the road and obstacles on the road, causing unnecessary safety accidents.

[0005] As a first aspect of this application, this application provides an obstacle recognition method based on a monocular camera, comprising: calibrating on a calibration image based on the calibration positions of a calibration object and a vehicle, to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image, wherein the calibration image is an image captured by the monocular camera that is located within a preset range of the vehicle and includes the calibration object; acquiring an actual image of the vehicle located within the preset range during actual operation, wherein the actual image is an image captured by the monocular camera that is located within the preset range of the vehicle; determining, based on the actual image, whether an obstacle exists in the actual image; when an obstacle exists in the actual image, determining the pixel position of the obstacle in the actual image; determining the actual positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the correspondence; and generating a warning message when the actual positional relationship between the obstacle and the vehicle is within a preset warning range.

[0006] In one possible implementation of this application, the calibration object includes a calibration object and a preset region; wherein, calibration is performed on the calibration image based on the calibration position of the calibration object and the vehicle to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image, including: calibration is performed on the calibration image based on the calibration position relationship between the calibration object and the vehicle to determine a first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image; and calibration is performed on the calibration image based on the preset region to determine a second correspondence between the preset region and the calibration region of the preset region in the calibration image.

[0007] In one possible implementation of this application, determining the actual positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the correspondence includes: determining a first positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the first correspondence; and determining a second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence; wherein, when the actual positional relationship between the obstacle and the vehicle is within a preset warning range, generating warning information includes: generating warning information when the first positional relationship is within a first preset range and the second positional relationship is within a second preset range.

[0008] In one possible implementation of this application, the number of calibration objects is multiple, and the calibration positions of the multiple calibration objects and the vehicle are all different; wherein, calibration is performed on a calibration image based on the calibration position relationship between the calibration objects and the vehicle to determine a first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image, including: selecting a reference point in the calibration image and determining the reference position of the reference point; determining the reference pixel position of the reference point on the calibration image; and calculating the first correspondence based on the calibration pixel position of the calibration object on the calibration image, the reference pixel position, and the actual distance between the reference position and the calibration object.

[0009] In one possible implementation of this application, the calibration object includes a first calibration object and a second calibration object, wherein the calibration position of the first calibration object relative to the vehicle is a stop position, and the calibration position of the second calibration object relative to the vehicle is a deceleration position.

[0010] In one possible implementation of this application, generating warning information when the first positional relationship is within a first preset range and the second positional relationship is within a second preset range includes: generating braking warning information when the first positional relationship is within the stop position and the second positional relationship is within the second preset range, wherein the braking warning information is used to control the vehicle to stop running; or generating deceleration warning information when the first positional relationship is between the stop position and the deceleration position and the second positional relationship is within the second preset range, wherein the deceleration warning information is used to control the vehicle to decelerate.

[0011] In one possible implementation of this application, determining the second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence includes: calculating the area of ​​the obstacle based on the actual image; determining the intersection area between the obstacle and the preset area based on the actual image; calculating the area of ​​the intersection area; and calculating the second positional relationship based on the area of ​​the intersection area and the area of ​​the obstacle.

[0012] In one possible implementation of this application, determining whether an obstacle exists in the actual image based on the actual image includes: identifying original obstacles in the actual image based on the actual image; determining original obstacles that conform to the preset obstacle database as valid obstacles based on the original obstacles and the preset obstacle database; and determining that an obstacle exists in the actual image when the valid obstacle exists in the actual image.

[0013] As a second aspect of this application, this application also provides an obstacle recognition device based on a monocular camera, comprising: a calibration unit, configured to perform calibration on a calibration image based on the calibration position of a calibration object and a vehicle, to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image, wherein the calibration image is an image captured by the monocular camera that is located within a preset range of the vehicle and includes the calibration object; an obstacle recognition unit, configured to acquire an actual image of the vehicle located within the preset range during actual operation, wherein the actual image is an image captured by the monocular camera that is located within the preset range of the vehicle; and to determine whether an obstacle exists in the actual image based on the actual image; a calculation unit, configured to determine the pixel position of the obstacle in the actual image when an obstacle exists in the actual image; and to determine the actual positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the correspondence; and a warning unit, configured to generate warning information when the actual positional relationship between the obstacle and the vehicle is within a preset warning range.

[0014] As a third aspect of this application, this application also provides a vehicle, including: a vehicle body; a monocular camera disposed on the vehicle body; and the aforementioned obstacle recognition device based on the monocular camera; wherein the monocular camera is communicatively connected to the obstacle recognition device based on the monocular camera.

[0015] The obstacle recognition method based on a monocular camera provided in this application uses a calibration object with a known calibration position relative to the vehicle to calibrate in a calibration image, thereby obtaining the correspondence between the calibration position of the calibration object and its calibration pixel position in the calibration image. When the same monocular camera captures actual images within the same preset range, if an obstacle exists in the actual image, the pixel position of the obstacle in the actual image is determined. Based on the pixel position and the correspondence, the actual positional relationship between the obstacle and the vehicle is determined. Based on this actual positional relationship, it can be determined whether the obstacle poses a threat to the vehicle, thus enabling vehicle control. Monocular cameras are cheaper than binocular cameras, reducing costs. Furthermore, in the calculation of the actual positional relationship between the obstacle and the vehicle, only the pixel positions of the captured image are used, resulting in lower image quality requirements. Therefore, even if the vehicle's outdoor working environment is poor (e.g., poor sunlight), the impact on the calculation of the actual positional relationship between the obstacle and the vehicle is minimal. Therefore, compared to existing binocular cameras, this application not only reduces costs but also improves the accuracy of obstacle estimation. In addition, compared with the existing technology of lidar for obstacle detection, using a monocular camera can detect some obstacles that lidar cannot detect, and improves the detection rate of ground bumps and depressions. Attached Figure Description

[0016] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 The diagram shown is a flowchart illustrating an obstacle recognition method based on a monocular camera according to an embodiment of this application.

[0018] Figure 2 The diagram shows the positional relationship between the vehicle and the monocular camera.

[0019] Figure 3 The diagram shown is a flowchart illustrating an obstacle recognition method based on a monocular camera, according to another embodiment of this application.

[0020] Figure 4 The diagram shown is a flowchart illustrating an obstacle recognition method based on a monocular camera, according to another embodiment of this application.

[0021] Figure 5 The diagram shown is a flowchart illustrating an obstacle recognition method based on a monocular camera, according to another embodiment of this application.

[0022] Figure 6 The diagram shown is a flowchart illustrating an obstacle recognition method based on a monocular camera, according to another embodiment of this application.

[0023] Figure 7 The diagram shown illustrates the working principle of an obstacle recognition device based on a monocular camera according to an embodiment of this application.

[0024] Figure 8 The diagram shown is a schematic diagram of the working principle of an electronic device provided in an embodiment of this application.

[0025] Figure label:

[0026] 1-Vehicle body; 2-Monocular camera; 3-First calibration object; 4-Second calibration object;

[0027] 10 - Calibration unit; 20 - Obstacle recognition unit; 30 - Calculation unit; 40 - Early warning unit;

[0028] 600 - Electronic device; 601 - Processor; 602 - Memory; 603 - Input device; 604 - Output device. Detailed Implementation

[0029] In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, top, bottom, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movement of the components in a specific posture (as shown in the figures). If the specific posture changes, the directional indication will also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0030] Furthermore, the reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] Application Overview

[0032] When road conditions are poor or even severe, onboard systems are prone to misjudging roads and obstacles, leading to unnecessary safety accidents. For example, LiDAR cannot detect reflective obstacles such as glass, and its accuracy is low on uneven surfaces. Depth cameras are expensive, and their image quality is easily affected by outdoor sunlight; good outdoor lighting results in good image quality, and vice versa. Image quality directly affects parameters such as depth of field in the depth-sensing space, making the detection rate significantly influenced by outdoor conditions when determining the presence of obstacles.

[0033] Therefore, this application provides an obstacle recognition method and device based on a monocular camera, as well as a vehicle. The method uses a calibration object with a known calibration position relative to the vehicle in a calibration image to obtain the correspondence between the calibration object's calibration position and its corresponding pixel position in the calibration image. When the same monocular camera captures images of the same preset range, if an obstacle exists in the actual image, its pixel position is determined. Based on the pixel position and correspondence, the actual positional relationship between the obstacle and the vehicle is determined. This actual positional relationship allows for the determination of whether the obstacle poses a threat to the vehicle, thus enabling vehicle control. Monocular cameras are cheaper than binocular cameras, reducing costs. Furthermore, the calculation of the actual positional relationship between the obstacle and the vehicle only uses the pixel positions of the captured images, requiring lower image quality. Therefore, even poor outdoor operating environments (e.g., poor sunlight) have a smaller impact on the calculation of the actual positional relationship between the obstacle and the vehicle. Thus, compared to binocular cameras in the prior art, this application not only reduces costs but also improves the accuracy of obstacle estimation. In addition, compared with the existing technology of lidar for obstacle detection, using a monocular camera can detect some obstacles that lidar cannot detect, and improves the detection rate of ground bumps and depressions.

[0034] The technical methods described below, with reference to the accompanying drawings of the embodiments of this application, will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0035] As a first aspect of this application, this application provides an obstacle recognition method based on a monocular camera. Figure 1 The diagram shown is a flowchart illustrating an obstacle recognition method based on a monocular camera according to an embodiment of this application. Figure 1 As shown, the obstacle recognition method based on a monocular camera includes the following steps:

[0036] S10: Based on the calibration position of the calibration object and the vehicle, calibration is performed on the calibration image to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image. The calibration image is an image captured by a monocular camera that is located within a preset range of the vehicle and includes the calibration object.

[0037] Specifically, the number of monocular cameras can be one or more. The preset range refers to a specific area around the vehicle. The preset range can refer to the area that the monocular camera can capture after installation; that is, the preset range is related to both the installation location and the camera's field of view. For example... Figure 2 As shown, when a monocular camera is installed behind the vehicle body, that is, while the vehicle is in motion, the monocular camera can capture images of the area Q1 behind the vehicle (area Q1 includes the road on which the vehicle is traveling), then the preset range refers to area Q1. For example, as... Figure 2 As shown, when the monocular camera is installed in front of the vehicle body, that is, while the vehicle is in motion, the monocular camera can capture images of the area Q2 in front of the vehicle (this area Q2 includes the road on which the vehicle is traveling), then the preset range is area Q2. For example, when the monocular camera is installed on the left side of the vehicle body, it can capture images of the area to the left of the vehicle; similarly, when the monocular camera is installed on the right side of the vehicle body, it can capture images of the area to the right of the vehicle.

[0038] Optionally, when setting the installation location of the monocular camera, the installation location and the number of monocular cameras can be set so that multiple preset ranges can form a 360° range surrounding the vehicle. That is, within any range of 360° around the vehicle, the corresponding image can be captured by the monocular camera, so as to monitor the vehicle's surroundings without blind spots, further improving the accuracy of obstacle recognition and reducing the probability of accidents caused by blind spots.

[0039] Specifically, a monocular camera refers to a camera with only one camera. It captures and processes images through a single camera, meaning that a monocular camera can capture 2D images.

[0040] Specifically, the calibration objects include the calibration object and the calibration area. When calibrating the calibration object, the calibration position of the calibration object relative to the vehicle is predetermined (for example, a straight-line distance of 5 meters from the vehicle is the calibration position of the calibration object). Figure 2 As shown, the first marker is placed on the road behind the vehicle, 5 meters away from the vehicle (l1 = 5 meters), and the second marker is placed on the road behind the vehicle, 10 meters away from the vehicle (l1 = 5 meters).

[0041] It should be noted that the marker can be an obstacle or other non-obstacle objects.

[0042] Then, the monocular camera located behind the vehicle body is activated. The monocular camera captures a calibration image within area Q1. The calibration image includes a first calibration object. Then, calibration is performed in the calibration image based on the calibration position of the first calibration object to determine the correspondence between the calibration position of the first calibration object and the calibration pixel position of the first calibration object on the calibration image.

[0043] Similarly, when the calibration object is a calibration area, the calibration area is calibrated to obtain the correspondence between the calibration position of the calibration area and its calibration pixels on the calibration image.

[0044] In summary, S10 determines the correspondence between the calibration position of the calibration object (i.e., the position of the calibration object in the real scene) and its calibration pixel position in the calibration image. This establishes a correspondence between the position of the calibration object in the real scene and its pixel position in the calibration image.

[0045] S20: Acquire actual images within a preset range during the actual operation of the vehicle. The actual images are images captured by a monocular camera within the preset range of the vehicle.

[0046] Specifically, the monocular camera that captures the actual image within the damaged area in S20 is the same monocular camera that captures the calibration image within the preset range in S10. This ensures that the calibration image in S10 and the actual image in S20 can capture images within the same preset range.

[0047] S30: Determine whether there are obstacles in the actual image;

[0048] Determine whether there is an obstacle in the actual image. When the judgment result of S30 is yes, that is, when it is determined that there is an obstacle in the image, it can be determined that there is an obstacle within the preset range of the vehicle. It is necessary to continue to determine whether the obstacle can affect the operation of the vehicle based on the positional relationship between the obstacle and the vehicle, that is, to execute S40-S60.

[0049] When the result of S30 is negative, that is, it is determined that there is no obstacle in the image, and the monitoring continues, that is, the actual image of the vehicle within the preset range during actual operation (e.g., during operation) is acquired, that is, S20 is executed.

[0050] S40: When there are obstacles in the actual image, determine the pixel position of the obstacles in the actual image;

[0051] When it is determined that an obstacle exists in the actual image in step 30, the pixel position of the obstacle in the actual image is determined.

[0052] S50: Determine the actual positional relationship between the obstacle and the vehicle based on the pixel position and correspondence of the obstacle in the actual image;

[0053] Once the pixel position of the obstacle in the actual image is determined, the actual positional relationship between the obstacle and the vehicle can be determined based on the correspondence between the calibration position of the calibration object determined in S20 and its calibration pixel position in the calibration image.

[0054] For example, when the calibration object is a calibration material, such as Figure 2 As shown, the calibration position l1 between the calibration object and the vehicle is 5 meters, and the calibration position is denoted as Y. The calibration pixel position of the calibration object in the calibration image is denoted as X. Then, the correspondence can be determined according to Y and X as Y1 = K1X1. K1 can be calculated based on the calibration position Y and the calibration pixel position X.

[0055] Once the correspondence Y1 = K1X1 is determined, the actual positional relationship between the obstacle and the vehicle can be calculated based on the pixel position of the obstacle in the actual image, such as the actual distance between the obstacle and the vehicle.

[0056] S60: Determine whether the actual positional relationship between the obstacle and the vehicle is within the preset warning range.

[0057] Specifically, the preset warning range can include: whether it is within a preset distance, and / or whether it is within a preset area. For example, whether the distance between the obstacle and the vehicle is within a preset distance, i.e., whether it is less than a preset distance. Or whether the obstacle is within a preset area around the vehicle; if so, then a warning message is generated.

[0058] Once S50 determines the actual positional relationship between the obstacle and the vehicle, that is, after determining the specific location of the obstacle around the vehicle, S60 can compare the actual positional relationship with the preset warning range. When the judgment result of S60 is yes, it means that the actual positional relationship between the obstacle and the vehicle is within the preset warning range, that is, the obstacle has affected the driving of the vehicle, and a warning message is generated, that is, S70 is executed.

[0059] When S60 determines the result as negative, meaning the obstacle currently has no impact on the vehicle and no warning is required, monitoring continues, i.e., the vehicle continues to acquire actual images of the vehicle within the preset range during actual operation (e.g., during operation), i.e., S20 is executed.

[0060] S70: Generate early warning information.

[0061] Specifically, the warning information is used to indicate that there are obstacles around the vehicle, and the vehicle is controlled according to the warning information. For example, when the obstacle is too close to the vehicle, the warning information indicates that the obstacle is too close and the vehicle needs to be braked. The vehicle's control system then brakes the vehicle according to the warning information to bring the vehicle to a stop.

[0062] The obstacle recognition method based on a monocular camera provided in this application uses a calibration object with a known calibration position relative to the vehicle to calibrate in a calibration image, thereby obtaining the correspondence between the calibration position of the calibration object and its calibration pixel position in the calibration image. When the same monocular camera captures actual images within the same preset range, if an obstacle exists in the actual image, the pixel position of the obstacle in the actual image is determined. Based on the pixel position and the correspondence, the actual positional relationship between the obstacle and the vehicle is determined. Based on this actual positional relationship, it can be determined whether the obstacle poses a threat to the vehicle, thus enabling vehicle control. Monocular cameras are cheaper than binocular cameras, reducing costs. Furthermore, in the calculation of the actual positional relationship between the obstacle and the vehicle, only the pixel positions of the captured image are used, resulting in lower image quality requirements. Therefore, even if the vehicle's outdoor working environment is poor (e.g., poor sunlight), the impact on the calculation of the actual positional relationship between the obstacle and the vehicle is minimal. Therefore, compared to existing binocular cameras, this application not only reduces costs but also improves the accuracy of obstacle estimation. In addition, compared with the existing technology of lidar for obstacle detection, using a monocular camera can detect some obstacles that lidar cannot detect, and improves the detection rate of ground bumps and depressions.

[0063] In one possible implementation of this application, such as Figure 2 As shown, the calibration object includes the calibration object and a preset area. Optionally, the preset area can be the vehicle's track area. That is, in this application, when determining whether an obstacle is within the preset warning range, not only the distance between the obstacle and the vehicle is considered, but also whether the obstacle is within the preset area. When the distance between the obstacle and the vehicle is within the preset distance and the preset area, it is determined that the obstacle affects the vehicle's operation, and a warning is required. Throughout the warning process, the accuracy of warnings is improved for obstacles that do not affect the vehicle's operation (e.g., obstacles that are close to the vehicle but not within a certain area and do not affect the vehicle). Figure 3 As shown, specifically:

[0064] S10 (Based on the calibration positions of the calibration object and the vehicle, calibration is performed on the calibration image to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image), specifically including the following steps:

[0065] S101: Based on the calibration position relationship between the calibration object and the vehicle, calibration is performed on the calibration image to determine the first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image;

[0066] Specifically, the positional relationship between the calibration object and the vehicle is a distance; that is, the calibration position of the calibration object is the calibration distance between the calibration object and the vehicle. For example, ... Figure 2 As shown, the calibration distance between the calibration object and the vehicle is 5 meters or 10 meters.

[0067] After determining the calibration pixel position of the calibration object on the calibration image, the first correspondence between the calibration distance and the calibration pixel position can be determined. For example, when the calibration distance between the calibration object and the vehicle is 5 meters, after determining the calibration pixel position of the calibration object on the calibration image, the first correspondence can be determined based on the calibration distance of 5 meters and the calibration pixel position.

[0068] S102: Based on the preset region, calibration is performed on the calibration image to determine the second correspondence between the preset region and the preset region in the calibration image.

[0069] When the calibration object is a preset area, calibration is performed on the calibration image according to the preset area, and then the calibration area corresponding to the preset area is determined. The second correspondence between the preset area and the calibration area can be determined according to the pixel position of the calibration area. The second correspondence can be the correspondence between the actual position of the preset area and the pixel position of the calibration area.

[0070] Once the monocular camera acquires an actual image including the obstacle, the second correspondence is directly mapped onto the actual image, and the preset area can be displayed on the actual image. Then, it can be determined whether the obstacle is within the preset area based on the obstacle.

[0071] In this case, S50 (determining the actual positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the corresponding relationship) specifically includes the following steps:

[0072] S501: Determine the first positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the first correspondence;

[0073] Once the first correspondence is determined in S101, the first positional relationship between the obstacle and the vehicle can be determined based on the pixel position of the obstacle in the actual image and the first correspondence, such as the distance between the obstacle and the vehicle, or the angle between the obstacle and the vehicle.

[0074] For example, such as Figure 2 As shown, the calibration position l1 between the calibration object and the vehicle is 5 meters, and the calibration position is denoted as Y. The calibration pixel position of the calibration object in the calibration image is denoted as X. Then, the correspondence can be determined according to Y and X as Y1 = K1X1. K1 can be calculated based on the calibration position Y and the calibration pixel position X.

[0075] Once the correspondence Y1 = K1X1 is determined, the actual positional relationship between the obstacle and the vehicle can be calculated based on the pixel position of the obstacle in the actual image, such as the actual distance between the obstacle and the vehicle.

[0076] S502: Determine the second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence;

[0077] Once the second correspondence is determined in S102, the preset area can be mapped to the actual image based on the second correspondence. Based on the pixel position of the obstacle in the actual image and the second correspondence, the second positional relationship between the obstacle and the vehicle can be determined, such as whether the obstacle is within the preset area; or whether the obstacle is not within the preset area but overlaps with the preset area.

[0078] S60 (determining whether the actual positional relationship between the obstacle and the vehicle is within the preset warning range) specifically includes the following steps:

[0079] S601: Determine whether the first positional relationship is within the first preset range;

[0080] After determining the actual distance between the obstacle and the vehicle in S501, it is determined whether the actual distance is within the first preset range, such as whether the actual distance is less than the first preset value. If the determination result in S601 is yes, it can be said that the first positional relationship is within the first preset range. For example, if the actual distance between the obstacle and the vehicle is less than the first preset value, it may affect the operation of the vehicle. Therefore, it is necessary to further determine whether the obstacle is within the preset area, that is, to execute S602.

[0081] When the judgment result in S601 is negative, that is, the first positional relationship is not within the first preset range, for example, the actual distance between the obstacle and the vehicle is greater than or equal to the first preset value, then the obstacle currently has no impact on the operation of the vehicle, and monitoring continues, that is, the actual image of the vehicle within the preset range during actual operation (e.g., during operation) continues to be acquired, that is, S20 is executed.

[0082] S602: Determine whether the second positional relationship is within the second preset range.

[0083] After determining the second correspondence in S502, the mapping is performed in the actual image according to the second correspondence. The preset area can be mapped to the actual image, and the positional relationship between the obstacle and the preset area can be determined. For example, the preset area can be the road area where the vehicle travels. Then the relationship between the obstacle and the road area where the vehicle travels can be determined (i.e., the second positional relationship).

[0084] Specifically, the second preset range can be a preset overlap rate, which is the preset overlap rate between the obstacle and the preset area. For example, when the second positional relationship is that the overlap rate between the obstacle and the preset area is 100%, the second preset range means that the overlap rate is greater than the preset overlap rate (e.g., 30%).

[0085] When the judgment result in S106 is yes, it can be said that the second positional relationship between the obstacle and the preset area is within the second preset range. For example, if the overlap rate between the obstacle and the preset area is greater than the preset overlap rate, it can be said that the obstacle is within the preset area and may affect the operation of the vehicle. Then, a warning message is generated, i.e., S70 is executed.

[0086] If the judgment result in S106 is negative, it means that the second positional relationship between the obstacle and the preset area is not within the second preset range. For example, if the overlap rate between the obstacle and the preset area is less than or equal to the preset overlap rate, it means that the obstacle is not within the preset area and will not affect the operation of the vehicle. Then, the monitoring continues, that is, the actual image of the vehicle within the preset range during actual operation (e.g., during operation) is acquired, that is, S20 is executed.

[0087] In one possible implementation of this application, such as Figure 4 As shown, when there are multiple calibration objects, and the calibration positions of the multiple calibration objects and the vehicle are different, for example, there are two calibration objects, namely the first calibration object and the second calibration object. The distance between the first calibration object and the vehicle is 5 meters (that is, the calibration distance of the first calibration object is 5 meters), and the distance between the second calibration object and the vehicle is 10 meters (that is, the calibration distance of the second calibration object is 10 meters).

[0088] S101 (Based on the calibration position relationship between the calibration object and the vehicle, calibration is performed on the calibration image to determine the first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image), that is, the specific calibration method of the calibration object specifically includes the following steps:

[0089] S1011: Select a reference point in the calibration image and determine the reference position of the reference point;

[0090] Specifically, the reference point is arbitrarily selected from the calibration image, preferably a physical object that is easily identifiable in the calibration image. For example, a calibration object or a landmark in the calibration image can be selected as the reference point. After selecting the reference point, its reference position in the real scene is determined based on the reference point.

[0091] Specifically, the actual distance between the reference point and the calibration object corresponds to the calibration position between the calibration object and the vehicle. For example, if the calibration position between the calibration object and the vehicle is 5 meters, that is, the calibration distance between the calibration object and the vehicle is 5 meters, then the actual distance between the reference point and the calibration object is 5 meters.

[0092] For example, if there are two calibration objects, namely the first calibration object and the second calibration object, and the calibration distances between the first calibration object and the second calibration object and the vehicle are 5 meters and 10 meters respectively, then the actual distances between the reference point and the first calibration object and the second calibration object are 5 meters and 10 meters respectively.

[0093] S1012: Determine the reference pixel position of the reference point on the calibration image;

[0094] Once the reference point is determined, its position as the reference pixel in the calibration image is determined.

[0095] S1013: Calculate the first correspondence based on the calibration pixel position of the calibration object on the calibration image, the reference pixel position, and the actual distance between the reference position and the calibration object.

[0096] Specifically, when there are two calibration objects, namely the first calibration object and the second calibration object, the distance between the first calibration object and the vehicle is 5 meters, and the distance between the second calibration object and the vehicle is 10 meters, then the distance between the reference position of the selected reference point and the first calibration object and the second calibration object is 5 meters and 10 meters, respectively.

[0097] In S1013, the first correspondence is calculated based on the reference position of the reference point, the reference pixel position, the calibration pixel position of the calibration object, and the actual distance (e.g., 5 meters) between the calibration object and the reference position.

[0098] For example, firstly, based on the actual distance of 5 meters and the calibrated pixel position of the calibration object, the correspondence between the actual distance and the pixel position is calculated when the actual distance is between 0 and 5 meters.

[0099] Furthermore, based on the actual distance of 5-10 meters and the calibrated pixel position, the correspondence between the actual distance and the pixel position when the actual distance is 5-10 meters is calculated.

[0100] The first correspondence includes the correspondence between the actual distance and the pixel position when the actual distance is 0-5 meters, and the correspondence between the actual distance and the pixel position when the actual distance is 5-10 meters.

[0101] It should be noted that the number of calibrators, n, can be greater than 2. When the number of calibrators, n, is greater than 2, the first correspondence includes n correspondences, namely: the correspondence between actual distance and pixel position when the actual distance is between 0 and L1; the correspondence between actual distance and pixel position when the actual distance is between L1 and L2; ...; the correspondence between actual distance and pixel position when the actual distance is between L1 and L2. n-2 -L n-1 At that time, the correspondence between the actual distance and the pixel position; the actual distance in L n-1 -L n At that time, the correspondence between the actual distance and the pixel position. Where L... n The calibration position (e.g., calibration distance) between the nth calibration point and the vehicle.

[0102] Optionally, when the first correspondence includes multiple correspondences as described above, then in SS501 (determining the first positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the first correspondence), that is, calculating the first positional relationship (e.g., actual distance) between the obstacle and the vehicle according to the first correspondence, the calculation method can be:

[0103] (1) Calculate the actual distance between the obstacle and the vehicle based on the correspondence between the actual distance and the pixel position when the actual distance is between 0 and L1.

[0104] (2) When the actual distance between the obstacle and the vehicle calculated in (1) is between 0 and L1, the actual distance between the obstacle and the vehicle can be determined as the actual distance calculated in (1).

[0105] (3) When the actual distance between the obstacle and the vehicle calculated in (1) is not in the range of 0-L1, but in the range of L1-L2, the correspondence between the actual distance and the pixel position when the actual distance is in L1-L2 is selected as the basis for calculation to recalculate the actual distance between the obstacle and the vehicle.

[0106] The actual distance between the obstacle and the vehicle is determined by recalculating (3).

[0107] In other words, when using one of multiple correspondences to calculate the actual distance between the obstacle and the vehicle, the actual distance is then used to determine the calibrated distance interval (i.e., 0-L1, L1-L2...L...). n-2 -L n-1 L n-1 -L nThen, based on the corresponding calibration distance interval, a new correspondence is selected, and the actual distance between the obstacle and the vehicle is recalculated based on the newly selected correspondence. This multi-step calculation improves the accuracy of the calculated actual distance between the obstacle and the vehicle.

[0108] Optionally, there are two calibration objects: a first calibration object and a second calibration object. The first calibration object is 5 meters away from the vehicle (i.e., the calibration distance of the first calibration object is 5 meters), and the second calibration object is 10 meters away from the vehicle (i.e., the calibration distance of the second calibration object is 10 meters). The calibration position of the first calibration object is the stop position, and the calibration position of the second calibration object is the deceleration position, i.e., when Zhang...

[0109] Optionally, the calibration objects include a first calibration object and a second calibration object. The calibration position of the first calibration object relative to the vehicle is the stop position, and the calibration position of the second calibration object relative to the vehicle is the deceleration position. In this case, S601 (determining whether the first positional relationship is within a first preset range) specifically includes:

[0110] Determine whether the first position relationship is within the stop position. If the first position relationship is within the stop position, generate braking warning information. The braking warning information is used to control the vehicle to stop running.

[0111] If the first position relationship is not within the stop position, determine whether the first position relationship is within the second position relationship. If the first position relationship is within the second position relationship, generate a deceleration warning message. The deceleration warning message is used to control the vehicle to decelerate.

[0112] This means that different markers are set to indicate the level of warning, so as to control the vehicle in a timely manner.

[0113] In one possible implementation of this application, such as Figure 5 As shown, S502 (determining the second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence) includes the following steps in the specific calculation method for calculating the positional relationship between the obstacle and the preset area:

[0114] S5021: Calculate the area of ​​obstacles based on actual images;

[0115] First, based on the second correspondence, the preset region is mapped to the actual image so that a mark about the preset region appears in the actual image.

[0116] Then, the area of ​​the obstacle is calculated based on its pixel position in the actual image.

[0117] S5022: Determine the intersection area between the obstacle and the preset area based on the actual image;

[0118] Specifically, obstacles and preset areas can have intersecting areas (i.e., the obstacle intersects with the preset area or the obstacle is within the preset area), or they can have no intersecting areas (i.e., the obstacle is outside the preset area).

[0119] Once the preset area is mapped onto the actual image, the intersection area between the obstacle and the preset area can be determined based on the pixel position of the preset area in the actual image.

[0120] S5023: Calculate the area of ​​the intersecting region;

[0121] Specifically, when the obstacle does not intersect with the preset area, the area of ​​the intersecting area is 0.

[0122] When an obstacle intersects with a preset area but the obstacle is not within the preset area, the area of ​​the intersecting area is the area of ​​the intersection between the obstacle and the preset area.

[0123] When an obstacle is within a preset area, the area of ​​the intersecting area is equal to the area of ​​the obstacle.

[0124] S5024: Calculate the second positional relationship based on the area of ​​the intersecting region and the area of ​​the obstacle.

[0125] Specifically, the second positional relationship can be the overlap rate: overlap rate = area of ​​the intersecting region / area of ​​the obstacle.

[0126] Correspondingly, the second preset range in S602 (determining whether the second positional relationship is within the second preset range) can be a preset overlap rate, for example, a preset overlap rate of 20%.

[0127] When the overlap rate is 100%, it means that the obstacle is within the preset area. At this time, the obstacle will inevitably affect the vehicle, and a warning message will be generated.

[0128] When the overlap rate is 30%, it means that although the obstacle is not in the preset area, it may still have an impact, and a warning message still needs to be generated to improve vehicle safety.

[0129] When the overlap rate is 10%, it means that only a small part of the obstacle is within the preset area, and its impact on the vehicle is negligible, so no warning information is generated.

[0130] In one possible implementation of this application, such as Figure 7 As shown, S30 (determining whether there are obstacles in the actual image) specifically includes the following steps:

[0131] S301: Based on the actual image, identify the original obstacles in the actual image;

[0132] Specifically, the method for identifying original obstacles in a real image can be any image recognition algorithm. For example, the YoloV7 (You Only Look Once) algorithm is a fast object detection algorithm used to locate and identify objects in an image. The YoloV7 algorithm works by segmenting the input image into multiple feature maps, then using a multi-layered convolutional network learned by a neural network to transform the feature maps into bounding box positions and class probabilities, ultimately forming the detection result.

[0133] S302: Based on the original obstacles and the preset obstacle database, determine the original obstacles that match the preset obstacle database as valid obstacles;

[0134] The default obstacle database includes obstacles that pose a threat to vehicles, such as cars, people, and traffic cones. However, the default obstacle database does not include obstacles that do not affect vehicles, such as small bricks and tree branches.

[0135] If a preset obstacle that matches the original obstacle is found in the preset obstacle database, then the obstacle is determined to be a valid obstacle. If no preset obstacle is found in the preset obstacle database, then the obstacle is determined to be an invalid obstacle, meaning that the obstacle does not affect the operation of the vehicle and is filtered out.

[0136] S303: Are there any valid obstacles in the actual image?

[0137] When the judgment result in S303 is yes, that is, there is a valid obstacle in the actual image, then S40 is executed.

[0138] If the judgment result in S303 is negative, that is, there is no valid obstacle in the actual image, it can be concluded that there is no obstacle in the actual image, and monitoring continues, that is, the actual image of the vehicle within the preset range during actual operation (e.g., during operation) is acquired, that is, S20 is executed.

[0139] This application improves the accuracy of warnings by setting up a preset obstacle database and filtering out obstacles that do not affect vehicle operation before issuing a warning.

[0140] As a second aspect of this application, this application also provides an obstacle recognition device based on a monocular camera, such as... Figure 7 As shown, the obstacle recognition device based on a monocular camera includes:

[0141] Calibration unit 10: used to calibrate the calibration position of the calibration object and the vehicle on the calibration image, so as to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image. The calibration image is an image captured by a monocular camera that is located within a preset range of the vehicle and includes the calibration object.

[0142] That is, the calibration unit performs S10 in the obstacle recognition method based on a monocular camera described above.

[0143] Obstacle recognition unit 20: used to acquire actual images of the vehicle within a preset range during actual operation, the actual images being images captured by a monocular camera within the preset range of the vehicle; and based on the actual images, to determine whether there are obstacles in the actual images;

[0144] That is, the obstacle recognition unit 20 is used to execute S20 and S30 in the obstacle recognition method based on a monocular camera described above.

[0145] The calculation unit 30 is used to determine the pixel position of the obstacle in the actual image when there is an obstacle in the actual image; and to determine the actual positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the correspondence.

[0146] That is, the computing unit is used to execute S40 and S50 in the obstacle recognition method based on a monocular camera described above.

[0147] The warning unit 40 is used to generate warning information when the actual positional relationship between the obstacle and the vehicle is within the preset warning range.

[0148] That is, the early warning unit is used to execute S60 and 70 in the obstacle recognition method based on a monocular camera described above.

[0149] As a third aspect of this application, this application also provides a vehicle, including:

[0150] Vehicle body;

[0151] A monocular camera mounted on the vehicle body; specifically, the installation location of the monocular camera on the vehicle body can be as follows: Figure 2 As shown.

[0152] The obstacle recognition device based on a monocular camera described above;

[0153] The monocular camera is connected in communication with the obstacle recognition device based on the monocular camera.

[0154] Exemplary electronic devices

[0155] Below, for reference Figure 8 This describes an electronic device according to embodiments of the present application. Figure 8 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application.

[0156] like Figure 8 As shown, the electronic device 600 includes one or more processors 601 and memory 602.

[0157] The processor 601 may be a central processing unit (CPU) or other form of processing unit with information processing and / or information execution capabilities, and may control other components in the electronic device 600 to perform desired functions.

[0158] The memory 601 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program information may be stored on the computer-readable storage medium, and the processor 601 may run the program information to implement the obstacle recognition method based on a monocular camera in the various embodiments of this application described above, or other desired functions.

[0159] In one example, the electronic device 600 may also include an input device 603 and an output device 604, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0160] The input device 603 may include, for example, a keyboard, a mouse, etc.

[0161] The output device 604 can output various information to the outside. The output device 604 may include, for example, a display, a communication network, and remote output devices connected thereto.

[0162] Of course, for the sake of simplicity, Figure 8 Only some of the components of the electronic device 600 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 600 may include any other suitable components depending on the specific application.

[0163] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program information that, when run by a processor, causes the processor to perform the steps in the obstacle recognition method based on a monocular camera according to various embodiments of this application as described in this specification.

[0164] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0165] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program information thereon, which, when run by a processor, causes the processor to perform the steps in the obstacle recognition method based on a monocular camera according to various embodiments of this application.

[0166] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0167] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0168] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0169] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. Such disassembly and / or recombination should be considered as equivalent to the present application.

[0170] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features of the invention herein.

[0171] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Any modifications or equivalent substitutions made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. An obstacle recognition method based on a monocular camera, characterized in that, include: The calibration is performed on the calibration image based on the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image. The calibration object includes a calibration object and a preset area. The calibration image is an image captured by a monocular camera that is located within a preset range of the vehicle and includes the calibration object. The preset range refers to the area that the monocular camera can capture after installation. The calibration is performed on the calibration image based on the calibration positions of the calibration object and the vehicle to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image. This includes: performing calibration on the calibration image based on the calibration position relationship between the calibration object and the vehicle to determine a first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image; and performing calibration on the calibration image based on the preset region to determine a second correspondence between the preset region and the calibration region of the preset region in the calibration image. Acquire actual images of the vehicle within the preset range during actual operation, wherein the actual images are images captured by the monocular camera within the preset range of the vehicle; Based on the actual image, determine whether there are obstacles in the actual image; When an obstacle exists in the actual image, determine the pixel position of the obstacle in the actual image; Determining the actual positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the correspondence includes: determining a first positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and the first correspondence; determining a second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence; and When the first positional relationship is within a first preset range and the second positional relationship is within a second preset range, a warning message is generated; the first preset range is a preset distance range that affects vehicle operation; the second preset range is a preset overlap rate range between the obstacle and the preset area.

2. The obstacle recognition method according to claim 1, characterized in that, The number of calibration objects is multiple, and the calibration positions of the multiple calibration objects and the vehicle are all different; Specifically, calibration is performed on the calibration image based on the calibration position relationship between the calibration object and the vehicle to determine a first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image, including: Select a reference point in the calibration image and determine the reference position of the reference point; Determine the reference pixel position of the reference point on the calibration image; A first correspondence is calculated based on the calibration pixel position of the calibration object on the calibration image, the reference pixel position, and the actual distance between the reference position and the calibration object.

3. The obstacle recognition method according to claim 2, characterized in that, The calibration objects include a first calibration object and a second calibration object. The calibration position of the first calibration object relative to the vehicle is a stop position, and the calibration position of the second calibration object relative to the vehicle is a deceleration position.

4. The obstacle recognition method according to claim 3, characterized in that, When the first positional relationship is within a first preset range and the second positional relationship is within a second preset range, a warning message is generated, including: When the first positional relationship is within the stopped position and the second positional relationship is within a second preset range, braking warning information is generated, and the braking warning information is used to control the vehicle to stop running; or When the first positional relationship is between the stop position and the deceleration position, and the second positional relationship is within a second preset range, deceleration warning information is generated, and the deceleration warning information is used to control the vehicle to decelerate.

5. The obstacle recognition method according to claim 1, characterized in that, Determining the second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence includes: Calculate the area of ​​the obstacle based on the actual image; The intersection area between the obstacle and the preset area is determined based on the actual image; Calculate the area of ​​the intersecting region; and The second positional relationship is calculated based on the area of ​​the intersecting region and the area of ​​the obstacle.

6. The obstacle recognition method according to claim 1, characterized in that, Based on the actual image, determining whether there are obstacles in the actual image includes: Based on the actual image, identify the original obstacles in the actual image; Based on the original obstacles and a preset obstacle database, original obstacles that match the preset obstacle database are identified as valid obstacles; and When the effective obstacle exists in the actual image, it is determined that the obstacle exists in the actual image.

7. An obstacle recognition device based on a monocular camera, characterized in that, include: A calibration unit is used to perform calibration on a calibration image based on the calibration positions of the calibration object and the vehicle, to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image. The calibration object includes a calibration object and a preset area. The calibration image is an image captured by a monocular camera that is located within a preset range of the vehicle and includes the calibration object. The preset range refers to the area that the monocular camera can capture after installation. The calibration on the calibration image based on the calibration positions of the calibration object and the vehicle, to determine the correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image, includes: performing calibration on the calibration image based on the calibration position relationship between the calibration object and the vehicle, to determine a first correspondence between the calibration position of the calibration object and the calibration pixel position of the calibration object on the calibration image; and performing calibration on the calibration image based on the preset area, to determine a second correspondence between the preset area and the calibration area of ​​the preset area in the calibration image. An obstacle recognition unit is used to acquire actual images of the vehicle within the preset range during actual operation, wherein the actual images are images captured by the monocular camera within the preset range of the vehicle; and based on the actual images, to determine whether there are obstacles in the actual images. A calculation unit is configured to determine the pixel position of an obstacle in the actual image when an obstacle exists in the actual image, including: determining a first positional relationship between the obstacle and the vehicle based on the pixel position of the obstacle in the actual image and a first correspondence; determining a second positional relationship between the obstacle and the preset area based on the pixel position of the obstacle in the actual image and the second correspondence; and The warning unit generates warning information when the first positional relationship is within a first preset range and the second positional relationship is within a second preset range; the first preset range is a preset distance range that affects vehicle operation; the second preset range is a preset overlap rate range between the obstacle and the preset area.

8. A vehicle, characterized in that, include: Vehicle body; A monocular camera mounted on the vehicle body; as well as The obstacle recognition device based on a monocular camera as described in claim 7; The monocular camera is communicatively connected to the obstacle recognition device based on the monocular camera.

Images