Obstacle perception method and device based on visual parking

Abstract

The present invention discloses an obstacle perception method and device based on visual parking, the method comprising: obtaining a video stream of a fisheye camera and a video stream of a linear camera; outputting first identification information of a target obstacle based on the video stream of the fisheye camera, and outputting second identification information of the target obstacle based on the video stream of the linear camera; dynamically adjusting the confidence of the first identification information and the confidence of the second identification information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera; and using at least one of the first identification information or the second identification information as the output result of the perception information according to the confidence of the first identification information and the confidence of the second identification information. The present invention can complete the accurate positioning and perception of obstacles around the vehicle during parking, ensuring safer and more efficient parking.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic parking, and in particular to an obstacle perception method and device based on vision parking. Background Art

[0002] Driverless parking is automatic parking based on computer vision. Compared with manual parking, automatic parking technology has a more precise parking path, simpler parking operation, and can reduce scratches and collisions, making the parking process safer and more efficient.

[0003] Intelligent driving technology for parking scenarios mainly relies on ultrasonic radar for obstacle perception. In recent years, with the development of computer vision technology, technical solutions that rely on computer vision have also attracted much attention. Since fisheye cameras have a wider viewing angle than pinhole cameras, surround-view fisheye cameras are more commonly used in parking scenarios to perceive obstacles around the vehicle. However, a simple fisheye camera system has problems such as difficulty in short- and medium-range distance measurement and difficulty in locating obstacles in a large distortion background. Summary of the invention

[0004] In response to the above technical problems, the present invention provides an obstacle perception method and device based on visual parking to improve the obstacle perception capability.

[0005] A first aspect of the present invention provides an obstacle perception method based on vision parking, comprising:

[0006] Get the video stream of the fisheye camera and the video stream of the linear camera;

[0007] Outputting first identification information of the target obstacle based on the video stream of the fisheye camera, and outputting second identification information of the target obstacle based on the video stream of the linear camera;

[0008] Dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera;

[0009] At least one of the first identification information or the second identification information is used as an output result of the perception information according to the confidence of the first identification information and the confidence of the second identification information.

[0010] In an optional implementation, the outputting first identification information of the target obstacle based on the video stream of the fisheye camera and the outputting second identification information of the target obstacle based on the video stream of the linear camera include:

[0011] Outputting first identification information of a target obstacle based on a video stream of the fisheye camera using a first neural network model;

[0012] A second neural network model is used to output second recognition information of the target obstacle based on the video stream of the linear camera; wherein the first recognition information and the second recognition information both include a bounding box of the target obstacle.

[0013] In an optional implementation manner, dynamically adjusting the confidence of the first recognition information and the second recognition information according to the distance between the target obstacle and the vehicle includes:

[0014] When the visual areas of the fisheye camera and the linear camera overlap, calculating the distance between the target obstacle and the ego vehicle based on the video stream of the linear camera;

[0015] If the distance between the target obstacle and the vehicle is greater than a preset distance, the confidence level of the second recognition information is dynamically adjusted to be higher than the confidence level of the first recognition information.

[0016] In an optional implementation, dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the position of the target obstacle in the fisheye camera includes:

[0017] When the visual areas of the fisheye camera and the linear camera overlap, the viewing angle area of ​​the fisheye camera is divided, and the viewing angle area includes a middle area and other areas;

[0018] If the target obstacle is in the middle area photographed by the fisheye camera, the confidence level of the first recognition information is dynamically adjusted to be higher than the confidence level of the second recognition information.

[0019] In an optional implementation, the obstacle perception method based on vision parking further includes: when the vehicle is in the process of moving forward, adjusting the confidence level of the second recognition information to be higher than the confidence level of the first recognition information.

[0020] In an optional implementation, the obstacle perception method based on vision parking further includes: calculating obstacle key points based on the video stream of the fisheye camera, and using the obstacle key points as part of the output result of the perception information.

[0021] In an optional implementation, the key points of the obstacle include the grounding point of the obstacle and the ground positioning point of the obstacle close to the vehicle, and the key points of the obstacle are marked on the second identification information.

[0022] A second aspect of the present invention provides an obstacle sensing device based on vision parking, comprising:

[0023] An information acquisition module, used to obtain the video stream of the fisheye camera and the video stream of the linear camera;

[0024] a model detection module, configured to output first identification information of a target obstacle based on the video stream of the fisheye camera, and output second identification information of the target obstacle based on the video stream of the linear camera;

[0025] A confidence adjustment module, used for dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera;

[0026] A perception result output module is used to output at least one of the first recognition information or the second recognition information as the output result of the perception information according to the confidence of the first recognition information and the confidence of the second recognition information.

[0027] A third aspect of the present invention provides a vision-based parking method, including the vision-based parking obstacle perception method described in the first aspect of the present invention.

[0028] A fourth aspect of the present invention provides a vehicle, comprising:

[0029] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in the first aspect of the embodiment of the present invention or the second aspect of the present invention.

[0030] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the method according to the first aspect of the embodiment of the present invention or the second aspect of the present invention is executed.

[0031] The present invention is based on the video stream of a fisheye camera and a linear camera, and can take into account both the recognition and perception of obstacles at medium and long distances and the perception of obstacles at close range, while being conducive to the calculation of incentives for obstacles; by utilizing the dynamic adjustment of confidence, accurate positioning and perception of obstacles around the vehicle can be completed during the parking process, ensuring safer and more efficient parking. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The figure is a flow chart of an obstacle perception method based on vision parking in an embodiment of the present invention.

[0033] Figure 2 It is a schematic diagram of the output after the recognition of the image taken by the linear camera in an embodiment of the present invention.

[0034] Figure 3 It is a schematic diagram of the output after the recognition of an image taken by a fisheye camera in an embodiment of the present invention.

[0035] Figure 4Schematic diagram of another obstacle perception method based on vision parking in an embodiment of the present invention.

[0036] Figure 5 It is a schematic diagram of identifying the division of the shooting area of ​​a fisheye camera in an embodiment of the present invention.

[0037] Figure 6 4 is a module diagram of an obstacle perception device based on vision parking in an embodiment of the present invention.

[0038] Figure 7 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0040] It should be understood that the terms "first", "second", and "third" in the claims, specifications, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0041] Linear cameras and fisheye cameras are usually set around the vehicle to obtain video information around the vehicle. The fisheye camera has a large viewing angle, large distortion, and large ranging and positioning errors; the linear camera has a small viewing angle, a long viewing distance, small distortion, and more accurate ranging and positioning. The linear camera uses a forward-looking linear camera, and fisheye cameras are set at the front, back, left, and right positions of the vehicle.

[0042] Since linear cameras are more accurate in ranging and positioning, they can be used to measure the distance to obstacles when the vehicle is parking to ensure safe driving. Fisheye cameras can be used to sense obstacles when parking, and their wide viewing angle can be used to sense more obstacles to ensure the safety of the vehicle when parking.

[0043] The present invention proposes that, during the parking process, attention should be paid not only to the recognition of obstacles in the middle and long distances ahead to ensure the normal driving of the vehicle, but also to the repeated obstacles perceived at close range during the parking process. By assigning different confidence levels to the perception results of the video streams collected by the linear camera and the fisheye camera, accurate obstacle perception output can be achieved, thereby improving the perception accuracy and achieving the purpose of accurate ranging.

[0044] See also Figure 1 The present invention provides an obstacle perception method based on visual parking, comprising:

[0045] Step 100: Obtain the video stream of the fisheye camera and the video stream of the linear camera.

[0046] Different fisheye cameras obtain video streams separately, and linear cameras obtain video streams as separate devices.

[0047] In some embodiments, four fisheye cameras disposed at the front, rear, left, and right sides of the vehicle respectively obtain video streams at the front, rear, left, and right sides of the vehicle. The fisheye camera located in front of the vehicle may overlap the perception area with the linear camera, that is, recognize the same obstacle. In order to improve the perception accuracy, the output result of the perception can be adjusted based on the confidence level, which will be described in detail later.

[0048] Step 200: Outputting first identification information of a target obstacle based on the video stream of the fisheye camera, and outputting second identification information of the target obstacle based on the video stream of the linear camera.

[0049] Different perception models can be matched to different cameras and different imaging characteristics to accurately output the perception output results. The perception of obstacles can be realized based on artificial intelligence AI models, which can usually output the 2D full bounding box or 3D information box of the target obstacle. The perception of obstacles can also complete the identification and positioning of key points.

[0050] In some embodiments, the first identification information includes multiple different first identification information. For example, the video stream acquired by a fisheye camera in front of the vehicle corresponds to the first identification information of the perception result, and the video stream acquired by a fisheye camera behind the vehicle corresponds to the first identification information of the perception result. Different fisheye cameras may correspond to different identification information, which are collectively referred to as first identification information.

[0051] The linear camera is used as a forward-looking camera, and its corresponding recognition result is the second recognition information, which is used as the information basis during the forward movement of the vehicle.

[0052] It is understandable that during the parking process, the vehicle will move forward, backward, left or right many times, and there are often other vehicles or other buildings on both sides of the lane. If the fisheye camera is used to measure the distance, errors and inaccurate positioning will occur; especially when the vehicle moves forward, scratches and collisions may occur. However, the above-mentioned first identification information and second identification information can avoid the above-mentioned problems.

[0053] Step 300: dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera.

[0054] The fisheye camera has a certain degree of reliability in measuring distance at a relatively close distance, and the distance measurement of obstacles in the area with small distortion of the fisheye camera is also reliable. However, the distance measurement is inaccurate in the area with large distortion, and the vehicle may also be scratched or collided due to inaccurate distance measurement in this area.

[0055] Therefore, the accurate distance measurement of linear cameras can be used to solve the inaccuracy of fisheye cameras. Through dynamic adjustment of confidence, the switching of obstacle perception information can be adjusted according to the actual parking scene.

[0056] Based on the above, it can be seen that when the distance between the target obstacle and the vehicle is far, the second recognition information is used for perception and ranging, and the confidence value of the second recognition information is assigned a higher value; or when the target obstacle is within the fisheye camera ranging calibration area, the first recognition information of the fisheye camera is directly used, and the confidence value of the first recognition information is assigned a higher value.

[0057] Step 400: at least one of the first identification information or the second identification information is used as an output result of the perception information according to the confidence level of the first identification information and the confidence level of the second identification information.

[0058] In combination with the aforementioned steps 100 to 300, it can be seen that the confidence values ​​of the first identification information and the second identification information are assigned different values. During the parking process, parking can be completed entirely based on the first identification information, that is, parking can be completed using the video stream of the fisheye camera; or parking can be completed based on the first identification information and the second identification information, such as scenarios such as parking the front of the vehicle into a warehouse.

[0059] When the linear camera and the fisheye camera obtain the video stream for identifying obstacles, a forward overlapping area will be generated. When both cameras identify the same obstacle, it can be decided which video stream information to use based on the confidence value, and then the first identification information or the second identification information is switched as the output identification result according to the confidence value.

[0060] Because confidence can ensure accurate ranging, avoid ranging errors, and locate obstacles more accurately; therefore, the output results of the perception information can be accurately determined.

[0061] The present invention is based on the video stream of a fisheye camera and a linear camera, and can take into account both the recognition and perception of obstacles at medium and long distances and the perception of obstacles at close range, while being conducive to the calculation of incentives for obstacles; by utilizing the dynamic adjustment of confidence, accurate positioning and perception of obstacles around the vehicle can be completed during the parking process, ensuring safer and more efficient parking.

[0062] Furthermore, in the above step 200, the first identification information of the target obstacle output based on the video stream of the fisheye camera and the second identification information of the target obstacle output based on the video stream of the linear camera include:

[0063] A first neural network model is used to output first identification information of a target obstacle based on the video stream of the fisheye camera; and a second neural network model is used to output second identification information of the target obstacle based on the video stream of the linear camera.

[0064] The first neural network module and the second neural network module are trained using different training sets, for example, the first neural network module is trained using images taken by a fisheye camera using a convolutional neural network, and the second neural network model is trained using images taken by a linear camera using a convolutional neural network module. The model training process can refer to the prior art and will not be described in detail.

[0065] The first identification information and the second identification information both include a bounding box of the target obstacle. Figure 2 , Figure 3 Bounding boxes of obstacles recorded in the parking scene shown in . Figure 2 is the obstacle image taken by the first camera, and the bounding box in the figure is the second recognition information output. Figure 2 It can be seen that four building columns and a car are identified on the left and right sides of the picture respectively. Figure 3 The obstacle image taken by the fisheye camera is a car obstacle on the left side of the vehicle.

[0066] Furthermore, for example Figure 3 As shown, the key point positions of obstacles can also be identified. Figure 3 The obstacle's grounding point and ground positioning point near the vehicle are shown in Figure 3 The position of the two tires and the line connecting the ground contact points between the two tires.

[0067] Therefore, based on the above steps, the present invention can also calculate the key points of obstacles based on the video stream of the fisheye camera, and use the key points of obstacles as part of the output result of the perception information. The key points of obstacles include the grounding point of obstacles and the ground positioning point of obstacles close to the vehicle, and the key points of obstacles are marked on the second identification information. The identification of key points is more conducive to parking, preventing scratches and collisions, and can accurately complete the distance measurement.

[0068] In some embodiments, a key point detection network may be added to the first neural network model to locate key points of the vehicle position, such as key point detection algorithm, ROI algorithm (region of interest), etc. Of course, a key point detection network may also be added to the second neural network model to locate key points of the vehicle position.

[0069] See also Figure 4 The present invention also provides an obstacle perception method based on visual parking, which includes the following steps:

[0070] Step 410: Obtain the video stream of the fisheye camera and the video stream of the linear camera.

[0071] The video stream of the fisheye camera located at the front of the vehicle is obtained, or the video stream of the fisheye camera located on the left or right side of the front of the vehicle is obtained. A linear camera is set. The visual areas of the aforementioned fisheye camera and the linear camera may overlap. When the same obstacle is identified, the fisheye camera whose visual area may overlap with the linear camera is selected as the target fisheye camera.

[0072] Step 420: When the vehicle is moving forward, adjust the confidence level of the second recognition information to be higher than the confidence level of the first recognition information.

[0073] When the vehicle is moving forward and searching for a storage space, it has high requirements for obstacle recognition. Therefore, the video stream obtained by the linear camera is processed into second recognition information by a neural network model, and the second recognition information is directly used for automatic driving. The parking system can identify vacant storage spaces based on the second recognition information. Usually, the corner points of the storage space in the second recognition information (such as the four opposite corners of a rectangle) can be identified, and whether the storage space is vacant and whether it meets the parking requirements can be determined based on whether there are obstacles or the distance between the obstacles.

[0074] Step 430: When the visual areas of the fisheye camera and the linear camera overlap, the distance between the target obstacle and the ego vehicle is calculated based on the video stream of the linear camera.

[0075] The linear camera obtains the video stream in the front direction of the vehicle, and the fisheye camera also obtains the video stream in the front direction of the vehicle. The visual areas obtained by the two overlap, and of course the same obstacle will be identified in the overlapping area. At this time, the video streams of the fisheye camera and the linear camera can be judged to be more reliable based on the distance between the vehicle and the obstacle. The linear camera has a more accurate range measurement, and the linear camera can be used to calculate the distance between the target obstacle and the vehicle.

[0076] The distance calculation formula is: ;

[0077] hpixel represents the pixel height value of the minimum bounding box of the target, H is the real vehicle height, fy is the focal length of the camera, and D is the approximate distance of the vehicle in the real world coordinate system.

[0078] Step 440: If the distance between the target obstacle and the vehicle is greater than a preset distance, dynamically adjust the confidence level of the second recognition information to be higher than the confidence level of the first recognition information.

[0079] For example, if the target obstacle is a car and the distance between the car and the vehicle is greater than a preset distance, the video stream of the linear camera is used as the perception basis to dynamically assign a higher confidence value to the second recognition information. The preset distance can be a distance limit with better accuracy of fisheye camera distance recognition.

[0080] If the confidence level of the second recognition information is higher than that of the first recognition information, the autonomous driving system will adopt the second recognition information for parking and discard the first recognition information of the fisheye camera whose visual area overlaps with that of the linear camera.

[0081] Step 450: When the visual areas of the fisheye camera and the linear camera overlap, the viewing angle area of ​​the fisheye camera is divided, and the viewing angle area includes a middle area and other areas.

[0082] In some other embodiments, when the visual area overlaps, the viewing area of ​​the fisheye camera is divided into a middle area, a ground line and other areas, see Figure 5 The middle area photographed by the fisheye camera is the area surrounded by the two middle curves. The ground line is marked by a curve placed horizontally in the middle of the area. The area outside the middle area is the other area, that is, the area with severe distortion, which is not suitable for distance calculation.

[0083] Step 460: If the target obstacle is in the middle area captured by the fisheye camera, dynamically adjust the confidence level of the first recognition information to be higher than the confidence level of the second recognition information.

[0084] In some embodiments, it is possible to determine whether the target obstacle is in the middle area captured by the fisheye camera by identifying the coordinate area of ​​the target obstacle in the image. Alternatively, it is possible to determine whether the target obstacle is in the middle area captured by the fisheye camera by other methods, which are prior arts and will not be described in detail.

[0085] As for how to identify the target obstacle between the linear camera and the fisheye camera, the target obstacle can be projected into the world coordinate system. If the coordinates of the obstacles identified by the two cameras are consistent or roughly consistent, they are the same target obstacle. This can be achieved by combining the SLAM self-positioning module in autonomous driving. The SLAM self-positioning module combines the video stream transmitted by the fisheye camera with the wheel speed meter and other information to provide the vehicle body positioning and posture information. SLAM is a known technology and will not be described in detail.

[0086] Then, according to the recognition result, the confidence of the first recognition information is dynamically adjusted and assigned a high value.

[0087] Step 470: Based on the confidence level of the first recognition information and the confidence level of the second recognition information, at least one of the first recognition information or the second recognition information is used as an output result of the perception information.

[0088] See also Figure 6 The present invention provides an obstacle sensing device based on visual parking, comprising:

[0089] An information acquisition module 61 is used to obtain the video stream of the fisheye camera and the video stream of the linear camera;

[0090] A model detection module 62, configured to output first identification information of a target obstacle based on the video stream of the fisheye camera, and output second identification information of the target obstacle based on the video stream of the linear camera;

[0091] A confidence adjustment module 63, configured to dynamically adjust the confidence of the first recognition information and the confidence of the second recognition information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera;

[0092] The perception result output module 64 is used to output at least one of the first recognition information or the second recognition information as the output result of the perception information according to the confidence of the first recognition information and the confidence of the second recognition information.

[0093] Furthermore, the above-mentioned model detection module 62 is also used to use the first neural network model to output first recognition information of the target obstacle based on the video stream of the fisheye camera; and use the second neural network model to output second recognition information of the target obstacle based on the video stream of the linear camera; wherein the first recognition information and the second recognition information both include the bounding box of the target obstacle.

[0094] The confidence adjustment module 63 is also used to calculate the distance between the target obstacle and the vehicle based on the video stream of the linear camera when the visual areas of the fisheye camera overlap with the linear camera; if the distance between the target obstacle and the vehicle is greater than a preset distance, dynamically adjust the confidence of the second recognition information to be higher than the confidence of the first recognition information; and to: when the visual areas of the fisheye camera and the linear camera overlap, divide the viewing area of ​​the fisheye camera, the viewing area includes the middle area and other areas; if the target obstacle is in the middle area photographed by the fisheye camera, dynamically adjust the confidence of the first recognition information to be higher than the confidence of the second recognition information. Wherein, when the vehicle is in the process of moving forward, the confidence of the second recognition information is adjusted to be higher than the confidence of the first recognition information.

[0095] Furthermore, the model detection module 62 is further used to calculate the key points of the obstacle based on the video stream of the fisheye camera, and use the key points of the obstacle as part of the output result of the perception information. The key points of the obstacle include the grounding point of the obstacle and the ground positioning point of the obstacle close to the vehicle, and the key points of the obstacle are marked on the second identification information.

[0096] In addition, the present invention provides a vision-based parking method, comprising the above Figures 1 to 5 The obstacle perception method based on visual parking is described; then the visual SLAM technology is used to map and locate the vehicle and the surrounding environment, and the valid storage position is detected at the same time; and after releasing the valid storage position information, the self-positioning information is used to adjust the vehicle body posture, and the camera is used to perform high-precision re-detection and matching of the valid storage position, so as to ensure that the vehicle can be parked smoothly in the storage position.

[0097] like Figure 7 As shown, the present invention also provides a vehicle, comprising:

[0098] At least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the above-mentioned obstacle perception method based on visual parking.

[0099] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned obstacle perception method based on visual parking is implemented.

[0100] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. A computer program includes a computer program code. The computer program code may be in source code form, object code form, an executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0101] In certain embodiments of the present invention, the device may include a controller, which is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to a processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0102] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0103] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An obstacle perception method based on visual parking, characterized in that: include: Get the video stream of the fisheye camera and the video stream of the linear camera; Outputting first identification information of the target obstacle based on the video stream of the fisheye camera, and outputting second identification information of the target obstacle based on the video stream of the linear camera; Dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera; At least one of the first identification information or the second identification information is used as an output result of the perception information according to the confidence of the first identification information and the confidence of the second identification information.

2. The obstacle perception method based on visual parking according to claim 1, characterized in that: The first identification information of the target obstacle output based on the video stream of the fisheye camera and the second identification information of the target obstacle output based on the video stream of the linear camera include: Outputting first identification information of a target obstacle based on a video stream of the fisheye camera using a first neural network model; A second neural network model is used to output second recognition information of the target obstacle based on the video stream of the linear camera; wherein the first recognition information and the second recognition information both include a bounding box of the target obstacle.

3. The obstacle perception method based on visual parking according to claim 1, characterized in that: The dynamically adjusting the confidence of the first recognition information and the second recognition information according to the distance between the target obstacle and the vehicle includes: When the visual areas of the fisheye camera and the linear camera overlap, calculating the distance between the target obstacle and the ego vehicle based on the video stream of the linear camera; If the distance between the target obstacle and the vehicle is greater than a preset distance, the confidence level of the second recognition information is dynamically adjusted to be higher than the confidence level of the first recognition information.

4. The obstacle perception method based on visual parking according to claim 1, characterized in that: The dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the position of the target obstacle in the fisheye camera includes: When the visual areas of the fisheye camera and the linear camera overlap, the viewing angle area of ​​the fisheye camera is divided, and the viewing angle area includes a middle area and other areas; If the target obstacle is in the middle area photographed by the fisheye camera, the confidence level of the first recognition information is dynamically adjusted to be higher than the confidence level of the second recognition information.

5. The obstacle perception method based on visual parking according to claim 1, characterized in that: Also includes: When the vehicle is in the process of moving forward, the confidence level of the second recognition information is adjusted to be higher than the confidence level of the first recognition information.

6. The obstacle perception method based on visual parking according to claim 1, characterized in that: Also includes: Obstacle key points are calculated based on the video stream of the fisheye camera, and the obstacle key points are used as part of the output result of the perception information.

7. The obstacle perception method based on visual parking according to claim 6, characterized in that: The key points of the obstacle include the grounding point of the obstacle and the ground positioning point of the obstacle close to the vehicle, and the key points of the obstacle are marked on the second identification information.

8. An obstacle sensing device based on visual parking, characterized in that: include: An information acquisition module, used to obtain the video stream of the fisheye camera and the video stream of the linear camera; a model detection module, configured to output first identification information of a target obstacle based on the video stream of the fisheye camera, and output second identification information of the target obstacle based on the video stream of the linear camera; A confidence adjustment module, used for dynamically adjusting the confidence of the first recognition information and the confidence of the second recognition information according to the distance between the target obstacle and the vehicle or the position of the target obstacle in the fisheye camera; A perception result output module is used to output at least one of the first recognition information or the second recognition information as the output result of the perception information according to the confidence of the first recognition information and the confidence of the second recognition information.

9. A vision-based parking method, characterized in that: The obstacle perception method based on visual parking includes any one of claims 1 to 7.

10. A vehicle, characterized in that: include: at least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method as claimed in any one of claims 1 to 7 or claim 9.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 or claim 9 is executed.

Images