Detection method and apparatus, and vehicle

Abstract

The present application provides a detection method and apparatus, and a vehicle, and can be applied to the field of intelligent driving. The method comprises: acquiring an image of the surroundings of a vehicle, wherein the image comprises an obstacle and information of a lane line associated with the obstacle, and a lane where the vehicle is located is adjacent to a lane where the obstacle is located; inputting the image into a prediction model to obtain the distance between the obstacle and the lane line, wherein the prediction model is trained by using a sample data set, and the sample data set comprises an image of a sample obstacle, an image of a lane line associated with the sample obstacle and the distance between the sample obstacle and the lane line; and on the basis of the distance between the obstacle and the lane line, determining whether the obstacle is to change to the lane where the vehicle is located. Embodiments of the present application can be applied to intelligent vehicles or electric vehicles, and can avoid missing detection or false detection during the cut-in of an obstacle to a lane where a vehicle is located, thereby facilitating the improvement of the driving and riding safety of users.

Description

Detection method, device and vehicle

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on December 28, 2023, with application number 202311836981.9 and application name “Detection Method, Device and Vehicle”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of intelligent driving, and more specifically, to a detection method, device and vehicle. Background Art

[0003] Autonomous driving perception technology uses sensors such as lidar, cameras, and millimeter-wave radar to perceive the vehicle's surroundings while driving. In current lane-changing (cutin) scenarios, the vehicle needs to detect whether an obstacle in an adjacent lane is cutting into the vehicle's lane, thereby determining whether to slow down and yield.

[0004] Existing solutions use the absolute position of the obstacle and the lane marking to determine their relative position. This approach relies on the accuracy of calculating the absolute positions of the obstacle and lane marking. If there's an error in either of these calculations, it can lead to missed or false detections of obstacles when the vehicle changes lanes. This can cause the vehicle to behave unexpectedly, impacting the driving experience. Summary of the Invention

[0005] The present application provides a detection method, device, and vehicle, which can avoid missed detection or false detection of obstacles when the vehicle changes lanes and cuts into its lane, thereby helping to improve user driving safety.

[0006] In a first aspect, a detection method is provided, comprising: acquiring a first image of a vehicle surrounding a vehicle located in a first lane, the first image including information about an obstacle and a first lane line associated with the obstacle, the first lane line being the lane line of a second lane, the first lane and the second lane being adjacent; inputting the first image into a prediction model to obtain a distance between the obstacle and the first lane line, the prediction model being trained using a sample dataset including an image of a sample obstacle, an image of a second lane line associated with the sample obstacle, and the distance between the sample obstacle and the second lane line; and determining, based on the distance between the obstacle and the first lane line, whether the obstacle is changing lanes into the first lane.

[0007] Based on the above technical solution, by inputting images of the vehicle's surroundings into the prediction model, the relative positional relationship between obstacles and lane lines can be predicted. This eliminates the need to determine the relative positional relationship between the obstacle and lane lines based on their absolute positions, helping to reduce the accuracy requirements for calculating the absolute positions of the obstacle and lane lines, and also helps reduce the complexity of calculating the relative positional relationship between the obstacle and lane lines. Furthermore, by using the predicted relative positional relationship between the obstacle and lane lines, the vehicle can accurately determine whether the obstacle is changing lanes to cut into the vehicle's lane, avoiding missed or false detections of obstacles changing lanes to cut into the vehicle's lane, and helping to improve the user's driving safety and experience.

[0008] In some possible implementations, the obstacle may be a vehicle in the second lane.

[0009] The above detection method uses the example of inputting an image into a prediction model to obtain the distance between the obstacle and the first lane line, but the embodiments of the present application are not limited to this. For example, data collected by a detection device (such as a lidar or millimeter-wave radar) can also be input into the prediction model to obtain the distance between the obstacle and the first lane line.

[0010] In some possible implementations, the distance between the obstacle and the first lane line may be a real physical distance between the obstacle and the first lane line, or may be a pixel distance.

[0011] The actual physical distance between the obstacle and the first lane line can also be referred to as the 3D distance between the obstacle and the first lane line, and the pixel distance between the obstacle and the lane line can also be referred to as the 2D distance between the obstacle and the first lane line.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: obtaining a first detection frame of the obstacle and information about the first lane line; wherein, determining whether the obstacle changes lanes to the first lane based on the distance between the obstacle and the first lane line includes: correcting the first detection frame based on the distance between the obstacle and the first lane line to obtain a second detection frame; and determining whether the obstacle changes lanes to the first lane based on the second detection frame and information about the first lane line.

[0013] In some possible implementations, the first detection box is determined by a first perception network and the information of the first lane line is determined by the second perception network.

[0014] In some possible implementations, the first perception network and the second perception network are each connected to the regulation and control module. For example, the first perception network may send the corrected second detection box information to the regulation and control module, and the second perception network may send the calculated lane line information to the regulation and control module.

[0015] In some possible implementations, a connection may also be established between the first perception network and the second perception network. The first perception network may receive the first lane line information sent by the second perception network.

[0016] Exemplarily, the first perception network may be a dynamic perception network, which may be used to detect moving targets around the vehicle.

[0017] Exemplarily, the second perception network may be a static perception network, which may be used to detect stationary targets around the vehicle.

[0018] Currently, the perception results of the absolute position of the obstacle (which can be represented by the first detection frame mentioned above) and the absolute position of the lane line come from two different perception networks. In order to correctly detect the relative position relationship between the obstacle and the lane line, these two perception networks need to accurately detect the absolute position of the obstacle and the absolute position of the lane line respectively. This places high precision requirements on both perception networks. Once the perception results of a certain perception network are less accurate, it will lead to inaccurate calculation of the relative position relationship between the obstacle and the lane line, which will in turn cause the regulation and control module to perform operations that affect the user experience and affect the user's driving experience.

[0019] Based on the above technical solution, after the prediction model predicts the relative positional relationship between the obstacle and the lane line, the first detection frame can be corrected based on this relative positional relationship. The relative positional relationship between the obstacle and the lane line is then determined using the corrected second detection frame and lane line information. This prevents misjudgments by the regulatory control module, helps the module make more accurate decisions, and ultimately improves the user's driving experience. Furthermore, this approach does not require modifications to the regulatory control module; the information it receives remains the same as the detection frame and lane line information.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the first detection frame is corrected based on the distance between the obstacle and the first lane line, including: when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, the first detection frame is corrected based on the distance between the obstacle and the first lane line.

[0021] Based on the above technical solution, the absolute position of the first detection frame can be corrected by using the distance between the obstacle and the first lane line, as well as the distance between the first detection frame and the first lane line, output by the prediction model. This can avoid false detections and missed detections of obstacles when the vehicle changes lanes, helping to improve the user's driving experience and safety.

[0022] In the above technical solution, the absolute position of the first detection frame is corrected, and then the relative position relationship between the obstacle and the first lane line is determined based on the corrected second detection frame and the absolute position of the first lane line. The embodiments of the present application are not limited to this.

[0023] Exemplarily, before correcting the first detection frame, the method further includes: controlling the display device to display a first 3D model of the obstacle based on the absolute position of the first detection frame; and after correcting the first detection frame, controlling the display device to display a second 3D model of the obstacle based on the absolute position of the second detection frame. In this way, in scenarios where the user is manually driving the vehicle, the user can use the information displayed on the display device to determine the distance between the obstacle and the vehicle, or whether the obstacle is changing lanes and cutting into the vehicle's lane.

[0024] In some possible implementations, when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: when the first distance between the obstacle and the first lane line is greater than the second distance between the first detection frame and the first lane line and the difference between the first distance and the second distance is greater than a preset distance difference, the position of the first detection frame is corrected in a direction away from the lane line.

[0025] Based on the above technical solution, when the distance between the obstacle and the lane line obtained by the prediction model is relatively far, but the distance between the obstacle and the lane line is relatively close as determined by the absolute position of the obstacle and the absolute position of the lane line, the absolute position of the obstacle can be corrected to make the absolute position of the obstacle away from the lane line, thereby avoiding sudden braking caused by the vehicle's false detection of an obstacle changing lanes, which helps to improve the user's driving experience.

[0026] In some possible implementations, when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: when the first distance between the obstacle and the first lane line is less than the second distance between the first detection frame and the first lane line and the difference between the first distance and the second distance is greater than a preset distance difference, the position of the first detection frame is corrected toward the direction close to the lane line.

[0027] Based on the above technical solution, when the distance between the obstacle and the lane line obtained by the prediction model is close, but the distance between the obstacle and the lane line determined by the absolute position of the obstacle and the absolute position of the lane line is far, the absolute position of the obstacle can be corrected to make the absolute position of the obstacle close to the lane line, avoiding collision accidents caused by the vehicle's missed detection of the obstacle when changing lanes, which helps to improve user driving safety.

[0028] In combination with the first aspect, in certain implementations of the first aspect, the first detection frame is corrected based on the distance between the obstacle and the first lane line, including: determining the driving trajectory of the obstacle based on the distance between the obstacle and the first lane line; and correcting the head orientation of the first detection frame based on the driving trajectory of the obstacle.

[0029] Based on the above technical solution, the distance between the obstacle and the first lane line obtained by the prediction model can be used to determine the obstacle's trajectory. Based on this trajectory, the head orientation of the first detection frame can be corrected. This can avoid false detections and missed detections of obstacles entering the vehicle's lane during lane changes, helping to improve the user's driving experience and safety.

[0030] In combination with the first aspect, in certain implementations of the first aspect, the head orientation of the first detection frame is corrected according to the driving trajectory of the obstacle, including: when the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, and the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, the head orientation of the first detection frame is corrected according to the distance between the obstacle and the first lane line.

[0031] Based on this technical solution, when the difference between the slope of the obstacle's trajectory and the slope of the lane marking is small, but the difference between the head orientation of the first detection frame and the slope of the lane marking is large, the prediction model output indicates that the obstacle is not changing lanes into the vehicle's lane, while the perception results of the two perception networks indicate that the obstacle is changing lanes into the vehicle's lane. This allows the head orientation of the first detection frame to be corrected, avoiding the vehicle's sudden braking caused by false detection of an obstacle changing lanes, thereby improving the user's driving experience.

[0032] In some possible implementations, the first preset difference is smaller than the second preset difference.

[0033] In combination with the first aspect, in certain implementations of the first aspect, the head orientation of the first detection frame is corrected according to the driving trajectory of the obstacle, including: when the slope of the driving trajectory is greater than the slope of the head orientation, the head orientation of the first detection frame is corrected according to the distance between the obstacle and the first lane line.

[0034] Based on the above technical solution, when the slope of the driving trajectory is greater than the slope of the obstacle's head orientation, the head orientation of the first detection frame can be corrected toward the lane line. This can avoid false detections and missed detections of obstacles during lane changes, helping to improve the user's driving experience and safety.

[0035] In some possible implementations, when the slope of the driving trajectory is greater than the slope of the head orientation of the obstacle, the head orientation of the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: when the slope of the driving trajectory is monotonic and the slope of the driving trajectory is greater than the slope of the head orientation, the head orientation of the first detection frame is corrected according to the distance between the obstacle and the first lane line.

[0036] In a second aspect, a detection device is provided, comprising: an acquisition unit for acquiring a first image around a vehicle, the vehicle being located in a first lane, the first image including information about an obstacle and a first lane line associated with the obstacle, the first lane line being the lane line of a second lane, the first lane and the second lane being adjacent; a prediction unit for inputting the first image into a prediction model to obtain a distance between the obstacle and the first lane line, the prediction model being trained using a sample data set including an image of a sample obstacle, an image of a second lane line associated with the sample obstacle, and the distance between the sample obstacle and the second lane line; and a determination unit for determining, based on the distance between the obstacle and the first lane line, whether the obstacle is changing lanes into the first lane.

[0037] In combination with the second aspect, in certain implementations of the second aspect, the device further includes a correction unit, and the acquisition unit is further used to obtain a first detection frame of the obstacle and information about the first lane line; the correction unit is used to: correct the first detection frame according to the distance between the obstacle and the first lane line to obtain a second detection frame; the determination unit is used to: determine whether the obstacle changes lanes to the first lane based on the second detection frame and information about the first lane line.

[0038] In combination with the second aspect, in certain implementations of the second aspect, the correction unit is used to: when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, correct the first detection frame according to the distance between the obstacle and the first lane line.

[0039] In combination with the second aspect, in certain implementations of the second aspect, the determination unit is further used to determine the driving trajectory of the obstacle based on the distance between the obstacle and the first lane line; the correction unit is used to correct the head orientation of the first detection frame based on the driving trajectory of the obstacle.

[0040] In combination with the second aspect, in certain implementations of the second aspect, the correction unit is used to: when the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, and the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, correct the head orientation of the first detection frame according to the driving trajectory.

[0041] In combination with the second aspect, in certain implementations of the second aspect, the correction unit is used to: when the slope of the driving trajectory is greater than the slope of the head orientation of the obstacle, correct the head orientation of the first detection frame according to the driving trajectory.

[0042] In combination with the second aspect, in certain implementations of the second aspect, the first detection frame is determined by a first perception network and the information of the first lane line is determined by the second perception network.

[0043] In a third aspect, a detection device is provided, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to perform any possible detection method in the first aspect.

[0044] In a fourth aspect, a detection system is provided, which includes a perception system and a computing platform, and the computing platform includes any possible detection device in the second aspect or the third aspect.

[0045] In a fifth aspect, the present application provides a vehicle, which includes any possible detection device in the second aspect or the third aspect, or includes the detection system described in the fourth aspect.

[0046] In a sixth aspect, the present application provides a computer program product, comprising: a computer program code, which, when executed on a computer, enables the computer to execute any possible detection method according to the first aspect.

[0047] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately packaged with the processor, and the embodiments of the present application do not specifically limit this.

[0048] In a seventh aspect, the present application provides a computer-readable medium storing a program code, which enables the computer to execute any possible detection method in the first aspect when the computer program code is run on the computer.

[0049] In an eighth aspect, the present application provides a chip comprising a circuit for executing any possible detection method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] FIG1 is a functional block diagram of a vehicle provided in an embodiment of the present application.

[0051] FIG2 is a schematic diagram of the system architecture provided in an embodiment of the present application.

[0052] FIG3 is a schematic diagram of the distance between an obstacle and a lane line provided in an embodiment of the present application.

[0053] FIG4 is a schematic diagram of an object-lane line association network OLA network provided in an embodiment of the present application.

[0054] FIG5 is a schematic flowchart of a method for correcting a detection frame provided in an embodiment of the present application.

[0055] FIG6 is a schematic diagram of correcting a detection frame provided in an embodiment of the present application.

[0056] FIG7 is another schematic diagram of correcting a detection frame provided by an embodiment of the present application.

[0057] FIG8 is another schematic diagram of correcting a detection frame provided by an embodiment of the present application.

[0058] FIG9 shows the timestamps of lane change cut-in detected without using the OLA network and the timestamps of lane change cut-in detected using the OLA network, provided by an embodiment of the present application.

[0059] FIG10 is a schematic flow chart of the detection method provided in an embodiment of the present application.

[0060] FIG11 is a schematic block diagram of a detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. For example, "at least one of A and B" is similar to "A and / or B", describing the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0062] In the embodiments of the present application, prefixes such as "first" and "second" are used only to distinguish different description objects and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present application does not constitute a restriction on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary restriction. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0063] Figure 1 is a functional block diagram of a vehicle 100 provided in an embodiment of the present application. The vehicle 100 may include a perception system 110, a computing platform 120 and a display device 130, wherein the perception system 110 may include one or more sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 110 may include a positioning system, and the positioning system may be a global positioning system (GPS), or a BeiDou system or other positioning systems. For another example, the perception system 110 may include one or more of an inertial measurement unit (IMU), an acceleration sensor, a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device. Exemplarily, the acceleration sensor may include a sensor for detecting the acceleration signal of an air suspension system, or may also include a sensor for the acceleration signal of an ESC.

[0064] Some or all functions of the vehicle 100 may be controlled by a computing platform 120. The computing platform 120 may include one or more processors, such as processors 121 to 12n (n is a positive integer). A processor is a circuit capable of processing signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationships of a hardware circuit. The logical relationships of the hardware circuit may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration file to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, the processor may also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 120 may also include a memory for storing instructions, and some or all of the processors 121 to 12n may call the instructions in the memory to implement corresponding functions.

[0065] The display devices 130 in the cockpit are mainly divided into two categories: the first is the vehicle-mounted display screen; the second is a projection display screen, such as a head-up display (HUD). The vehicle-mounted display screen is a physical display screen and a key component of the in-vehicle infotainment system. The cockpit can be equipped with multiple displays, such as the digital instrument panel, the central control screen, the display in front of the front passenger (also known as the front passenger), the display in front of the left rear passenger, and the display in front of the right rear passenger. Even the vehicle windows can serve as display screens. A head-up display, also known as a head-up display system, is primarily used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's gaze shift time, avoids pupil changes caused by the driver's gaze shift, and improves driving safety and comfort. HUDs include, for example, combiner-HUD (C-HUD), windshield-HUD (W-HUD), and augmented reality HUD (AR-HUD). It should be understood that other types of HUD systems may appear as technology evolves, and this application is not limited to this.

[0066] The above display device 130 is described by taking a vehicle-mounted display screen and a projection display screen as examples, and the embodiments of the present application are not limited thereto. For example, the display device 130 can also be a light display screen or a projection screen.

[0067] Vehicle 100 may include an advanced driving assistance system (ADAS). The ADAS utilizes various onboard sensors (including but not limited to lidar, millimeter-wave radar, cameras, ultrasonic sensors, global positioning systems, and inertial measurement units) to acquire information from the vehicle's surroundings. The ADAS analyzes and processes this information to implement functions such as obstacle detection, target recognition, vehicle positioning, path planning, and driver monitoring / alerts, thereby enhancing the safety, automation, and comfort of vehicle driving. Logically, an ADAS system generally includes three main functional modules: a perception module, a regulation and control module, and an execution module. The perception module uses sensors to perceive the vehicle's surroundings and inputs real-time data to a decision-making processing center. The perception module primarily includes an onboard camera, ultrasonic radar, millimeter-wave radar, or lidar. The regulation and control module uses computing devices and algorithms to make decisions based on the information acquired by the perception module. Upon receiving the decision signal from the regulation and control module, the execution module takes appropriate actions, such as driving, changing lanes, steering, braking, and issuing warnings.

[0068] Currently, the absolute positions of obstacles and lane markings are perceived by two different perception modules: a dynamic perception network and a static perception network. Dynamic objects (such as vehicles and pedestrians) can be perceived by the dynamic perception network. The dynamic perception network represents obstacles in the three-dimensional world using detection boxes (e.g., 3D rectangular boxes, cubes (3D bounding boxes, 3D bboxes), or polygonal outlines). Static objects (such as lane markings and traffic lights) can be perceived by the static perception network. The static perception network represents lane markings as a series of 3D points. To accurately detect the relative positional relationship between obstacles and lane markings, both the dynamic and static perception networks must accurately detect the 3D bounding boxes of obstacles and the 3D points of lane markings, respectively. This places high precision requirements on both the dynamic and static perception networks. If the perception results of one perception network are inaccurate, the relative positional relationship between obstacles and lane markings will be inaccurately calculated, which in turn can cause the control module to make decisions that affect the user's driving experience.

[0069] For example, both lidar and cameras are subject to noise, which can cause issues like positional jumps and angular jitter in the 3D bounding boxes detected by the dynamic perception network. Furthermore, for the control module, the accuracy of the relative position relationship between obstacles and lane markings is more important than the accuracy of their absolute positions.

[0070] For example, if a truck traveling normally in an adjacent lane experiences slight angular jitter in its 3D bbox detection, it could mistakenly appear to cross a lane line. The control module would then interpret the truck as attempting to cross into the vehicle's lane and decelerate the vehicle. This misjudgment can significantly impact the user experience.

[0071] For example, if the 3D bbox doesn't detect the angle deviation of a truck changing lanes in time, it will cross the line too late, preventing the control module from making a timely braking decision. This can be fatal at high speeds, raising concerns about the safety of autonomous driving.

[0072] The embodiments of the present application provide a detection method, device, and vehicle. By inputting data collected by sensors into a prediction model, the relative position relationship between the obstacle and the lane line can be directly obtained. This avoids the process of first obtaining the absolute position of the obstacle and the absolute position of the lane line through a dynamic perception network and a static perception network respectively, and then determining the relative position relationship between the obstacle and the lane line based on the absolute position of the obstacle and the absolute position of the lane line. This solves the problem of misjudgment of the regulation and control module due to perception accuracy errors, thereby helping to improve the user's driving experience and driving safety.

[0073] In the embodiments of the present application, the obstacle is other vehicles in adjacent lanes, but the embodiments of the present application are not limited thereto. For example, in a scenario where a vehicle is traveling at low speed through a downtown area, the obstacle may also be a pedestrian, a user riding an electric scooter or an electric bike, etc.

[0074] Figure 2 shows a schematic diagram of a system architecture 200 provided in an embodiment of the present application. The system architecture 200 includes a dynamic perception network 210, a static perception network 220 and a regulation and control module 230. The dynamic perception network 210 can determine the detection frame (3D bbox or polygon) of the obstacle, or the absolute position of the obstacle based on the data collected by the sensor. The dynamic perception network 210 includes a prediction model 211. The dynamic perception network 210 can input the data collected by the sensor (for example, the image collected by the camera) into the prediction model 211 to obtain the relative position relationship between the obstacle and the lane line. The dynamic perception network 210 can correct the detection frame of the obstacle based on the relative position relationship between the obstacle and the lane line output by the prediction model 211. The corrected detection frame can then be sent to the regulation and control module 230.

[0075] In this embodiment of the present application, a prediction model 211 is added to the dynamic perception network 210. This allows the dynamic perception network 210 to modify the detection frame determined by the dynamic perception network 210 based on the relative positional relationship between obstacles and lane markings output by the prediction model 211. This enables the regulation and control module 230 to make accurate regulation and control decisions, thereby improving the user's driving experience and safety.

[0076] For example, the dynamic perception network 210 can determine the relative position relationship 1 between the obstacle and the lane line based on the obstacle detection frame determined by the dynamic perception network 210 and the 3D points of the lane line determined by the static perception network 220. The dynamic perception network 210 can correct the detection frame based on the relative position relationship 1 and the relative position relationship 2 between the obstacle and the lane line output by the prediction model 211.

[0077] The above prediction model 211 can be a neural network (NN), such as a transformer, a multi-layer perceptron (MLP), a residual neural network (ResNet), or a recurrent neural network (RNN); or it can be other machine learning algorithms, such as a support vector machine (SVM) or a decision tree.

[0078] The following description will be made by taking the prediction model 211 as an object lane association (OLA) network as an example.

[0079] To accurately determine whether a vehicle has crossed a lane line, as seen from a camera, using 2D image information, an OLA network is employed in this embodiment. This OLA network can detect the distance between obstacles and lane lines. The OLA network's task is to directly predict the distance between the obstacle and the lane line. This distance can be the actual physical distance between the wheel contact point and its associated lane line. The associated lane line is selected based on the target's lane location, whether the vehicle has crossed the lane line, and the distance between the two objects.

[0080] The above OLA network can be trained by a sample data set, which may include sample images (the sample images include images of sample obstacles and images of lane lines associated with the sample obstacles) and the real physical distance between the wheels of the sample obstacles and the lane lines.

[0081] For example, the driving trajectory of the sample vehicle and the driving trajectory of the sample obstacle can be planned in advance, wherein the actual trajectory of the sample obstacle can include the distance change information between the obstacle and the lane line. In this way, the real physical distance between the obstacle and the lane line at different times can be marked in advance. During the driving process of the sample vehicle, a sample image captured by the camera of the sample vehicle can be obtained, and the sample image includes information about the obstacle and the lane line. In this way, since the driving trajectory of the obstacle has been planned in advance, a sample training data set can be established. The sample training data set includes the information of the sample obstacle in the sample image, the information of the lane line in the sample image, and the correspondence between the real physical distance between the sample obstacle and the lane line. Through the sample training data set, the OLA network can be trained.

[0082] The above training of the OLA network can be implemented on the cloud server.

[0083] For example, the distance between the above obstacle and its associated lane line may satisfy one or more of the following conditions:

[0084] (1) Ensure the continuity of the distance between obstacles and lane lines.

[0085] For example, the distance between the obstacle and the lane line can be periodically acquired through the OLA network. Each frame of image input into the OLA model can obtain a distance between the obstacle and the lane line. The distances between the obstacle and the lane line corresponding to two adjacent frames of image are continuous, or the difference between the distances between the obstacle and the lane line corresponding to two adjacent frames of image are less than a preset difference.

[0086] (2) A positive distance indicates that the obstacle is not pressing the lane line, and a negative distance indicates that the obstacle is pressing the lane line.

[0087] For example, FIG3 shows a schematic diagram of the distance between an obstacle and a lane line provided in an embodiment of the present application.

[0088] For example, if vehicle 1 is the ego vehicle and is located in lane 1, and the obstacle is vehicle 2 and is located in lane 2 (lane 2 includes lane line 1), the range of the distance between vehicle 2 and lane line 1 can be [-W 车辆2 / 2,W 车道 / 2], where -W 车辆2 / 2 means that vehicle 2 is driving on lane line 1 and the distance between the wheels of vehicle 2 and lane line 1 is half the width of vehicle 2. 车道 / 2 means that vehicle 2 is not running on lane line 1 and the distance between the wheels of vehicle 2 and lane line 1 is half the width of lane 2.

[0089] (3) Select the obstacle by the distance between the lane line and the obstacle's wheel

[0090] For example, as shown in Figure 3, lanes 1 and 2 are adjacent lanes and both lanes 1 and 2 include lane line 1. Vehicle 1 can select a target obstacle in lane 2 based on the distance between the obstacle's wheel and lane line 1. For example, the distance between the right wheel of vehicle 2 in lane 2 and lane line 1 is between [-W 车辆2 / 2,W 车道 / 2], it can be determined that vehicle 2 is the target obstacle.

[0091] For example, FIG4 shows a schematic diagram of an OLA network provided in an embodiment of the present application.

[0092] The input of the OLA network can be an image captured by the vehicle's camera, and the output of the OLA network can be a 2D detection target, a point pair existence probability, and a 3D distance between the obstacle and the associated lane line. Among them, the 2D detection target includes information about the obstacle and lane line in the image, and the point pair existence probability can include the degree of association between the obstacle and the lane line. The above point pair existence probability can be determined by the lane where the obstacle is located and the distance between the obstacle and the lane line. For example, as shown in Figure 3, vehicle 1 is in lane 1, and the OLA network can output a 2D image of the obstacle in lane 1, a 2D image of the obstacle in the left lane (lane 2) adjacent to lane 1, and a 2D image of the obstacle in the right lane (lane 3, not shown in the figure) adjacent to lane 1. The dynamic perception network can first filter out the 2D image of the obstacle in lane 1. Then select the target obstacle based on the distance between the obstacle in the adjacent lane and the lane line. For example, when determining that the distance between vehicle 2 in lane 2 and lane line 1 is between [-W 车辆2 / 2,W 车道 / 2], it can be determined that vehicle 2 is the target obstacle and the lane line associated with vehicle 2 is lane line 1.

[0093] With the OLA network output, an algorithm can be designed to address false or missed detections in cutin scenarios. First, the target obstacles in the left and right adjacent lanes are identified and their multi-frame OLA information (including the 3D distance between the obstacle and the lane markings, as output by the OLA network) is extracted. This information is used to fit a trajectory of the obstacle relative to the lane markings and compare it with the position of the lane markings. If the position of the detected obstacle's 3D bounding box does not match the position determined by the multi-frame OLA information, the trajectory is used to correct the position of the 3D bounding box.

[0094] For example, FIG5 shows a schematic flow chart of a method 500 for correcting a detection frame provided in an embodiment of the present application. The correction method 500 can be performed by the dynamic perception network or the regulation and control module. The correction method 500 includes:

[0095] S501, obtaining multi-frame OLA information output by the prediction model.

[0096] Exemplarily, the multi-frame OLA information includes the distances between obstacles and lane lines output by the OLA model at multiple moments.

[0097] S502: Determine the driving trajectory of the obstacle according to the multi-frame OLA information.

[0098] S503: Determine whether the slope of the driving trajectory is monotonic.

[0099] Optionally, determining whether the slope of the driving trajectory is monotonic includes: determining whether the slope of the driving trajectory is monotonic based on a lane line and the driving trajectory.

[0100] The slope of the above driving trajectory can be determined by the angle between the tangent direction of the trajectory point and the preset direction. The larger the angle between the tangent direction of the trajectory point and the preset direction, the greater the slope of the driving trajectory.

[0101] Exemplarily, the preset direction may be a tangent direction of the lane line, or may be a direction in an odom coordinate system or a world coordinate system.

[0102] The above determination of whether the slope of the driving trajectory is monotonic can be understood as determining whether the slope of the driving trajectory is gradually increasing (indicating that the obstacle is gradually changing lanes and cutting into the lane where the vehicle is located), or it can also be understood as determining whether the angle between the tangent direction of the trajectory point in the driving trajectory and the tangent direction of the lane line is gradually increasing.

[0103] If the slope of the driving trajectory is monotonic, execute S504-506; otherwise, execute S507-S511.

[0104] S504: Determine that an obstacle is changing lanes and cutting into the lane where the vehicle is located.

[0105] S505 , determining whether the slope of the driving trajectory is greater than the slope of the head orientation of the detection frame.

[0106] Optionally, determining whether the slope of the driving trajectory is greater than the slope of the head orientation of the detection frame includes: determining whether the slope of the driving trajectory is greater than the slope of the head orientation of the detection frame and whether the difference between the slope of the driving trajectory and the slope of the head orientation is greater than or equal to a preset difference.

[0107] The slope of the driving trajectory can be determined by the angle between the tangent direction of the trajectory point and the tangent direction of the lane line. The slope of the head orientation can be determined by the angle between the head orientation and the tangent direction of the lane line. A larger angle indicates a larger slope.

[0108] Optionally, determining whether the slope of the driving trajectory is greater than the slope of the head orientation of the detection frame includes: determining whether a first angle is greater than a second angle, the first angle may be the angle between the tangent direction of the trajectory point on the driving trajectory and the tangent direction of the lane line, and the second angle may be the angle between the head orientation and the tangent direction of the lane line.

[0109] The first angle and the second angle mentioned above are both acute angles.

[0110] Optionally, determining whether the difference between the slope of the driving trajectory and the slope of the head orientation is greater than or equal to a preset difference includes: determining whether the difference between the first angle and the second angle is greater than or equal to a preset angle difference.

[0111] Exemplarily, when the first angle is greater than the second angle and the difference between the first angle and the second angle is greater than or equal to a preset angle difference, it can be determined that the head orientation is incorrect.

[0112] If the first angle is greater than the second angle and the difference between the first angle and the second angle is greater than the preset angle difference, execute S506; if the first angle is equal to the second angle, or the difference between the first angle and the second angle is less than or equal to the preset angle difference, the detection frame is not corrected.

[0113] S506: Determine that the head orientation of the detection frame is incorrect, and correct the head orientation in the detection frame.

[0114] Exemplarily, the correcting the head orientation of the detection frame includes: rotating the head orientation of the detection frame toward the direction of the lane line, so that the angle between the head orientation of the detection frame and the direction of the lane line after correction is greater than the angle between the head orientation of the detection frame and the direction of the lane line before correction.

[0115] For example, FIG6 shows a schematic diagram of correcting a detection frame provided in an embodiment of the present application.

[0116] As shown in Figure 6 (a), the vehicle's driving trajectory can be obtained from the output of the OLA network. This driving trajectory includes trajectory points at times T1-T4. At times T1, T2, and T3, the difference between the first angle and the second angle is less than or equal to the preset angle difference. At time T4, the first angle is greater than the second angle, and the difference between the first angle and the second angle is greater than the preset angle difference. Therefore, it can be determined that the head orientation determined by the dynamic perception network at time T4 is incorrect, and the dynamic perception network can correct the head orientation.

[0117] As shown in Figure 6(a), the slope of the driving trajectory at time T4 is significantly greater than the slope of the head orientation, or the first angle at time T4 is significantly greater than the second angle. This indicates that the output from the OLA network indicates that the obstacle is changing lanes and cutting into the vehicle's lane, while the detection box output by the dynamic perception network and the lane marking information output by the static perception network indicate that the obstacle is not changing lanes and cutting into the vehicle's lane.

[0118] As shown in Figure 6(b), the dynamic perception network can correct the head orientation of the detection frame at time T4 based on the driving trajectory. For example, the dynamic perception network can correct the head orientation of the detection frame based on the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line. For example, the angle between the corrected head orientation of the detection frame and the tangent direction of the lane line can be equal to the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line.

[0119] In this way, when the slope of the driving trajectory is greater than the slope of the head orientation, the driving trajectory can be used to correct the head orientation. This can avoid the vehicle from missing the detection of obstacles when changing lanes, which helps to improve the user's driving safety.

[0120] S507 , obtaining information on the obstacle's driving trajectory, lane line, and detection frame in the first coordinate system.

[0121] For example, the information of the detection frame includes the vehicle's front direction in the detection frame. The lane line information can be obtained from a static perception network, and the detection frame can be determined by a dynamic perception network based on data collected by sensors.

[0122] The above first coordinate system can be a world coordinate system or an odom coordinate system.

[0123] S508 , determining whether the distance between the obstacle and the lane line indicated by the driving trajectory matches the distance between the detection frame and the lane line.

[0124] The above judgment of whether the distance between the obstacle and the lane line indicated by the driving trajectory matches the distance between the detection frame and the lane line can be understood as judging whether the difference between the first distance between the obstacle and the lane line indicated by the driving trajectory and the second distance between the detection frame and the lane line is greater than or equal to the preset distance difference.

[0125] Exemplarily, the preset distance difference is 0.1 meter.

[0126] If the first distance does not match the second distance, execute S509; otherwise, do not modify the detection frame.

[0127] S509: Determine that the position of the detection frame is incorrect, and correct the horizontal position of the detection frame.

[0128] For example, FIG7 shows a schematic diagram of correcting a detection frame provided in an embodiment of the present application.

[0129] As shown in (a) of Figure 7, the vehicle's driving trajectory can be obtained through the results output by the OLA network. The driving trajectory includes trajectory points at time T1-T4. Among them, at time T1, the difference between the distance L1 between the obstacle and the lane line and the distance between the detection frame and the lane line at time T1 is less than or equal to the preset distance difference (for example, 0.1 meters); at time T2, the difference between the distance L2 between the obstacle and the lane line and the distance between the detection frame and the lane line at time T2 is less than or equal to the preset distance difference; at time T3, the difference between the distance L3 between the obstacle and the lane line and the distance L5 between the detection frame and the lane line at time T3 is greater than the preset distance difference (for example, 0.1 meters). In this way, the dynamic perception network can determine that the position of the output detection frame at time T3 is incorrect and needs to be corrected.

[0130] As shown in Figure 7(b), the dynamic perception network can correct the position of the detection frame at time T3 based on the driving trajectory. For example, the distance between the corrected detection frame and the lane line can be equal to L3.

[0131] In this way, when the result output by the OLA network indicates that the distance between the obstacle and the lane line is close, and the absolute position of the obstacle output by the dynamic perception network and the absolute position of the lane line output by the static perception network determine that the distance between the obstacle and the lane line is far, the absolute position of the obstacle can be corrected so that the absolute position of the obstacle is close to the lane line, avoiding collision accidents caused by the vehicle's missed detection of the obstacle when changing lanes, which helps to improve the user's driving safety.

[0132] S510, determine whether the difference between the slope of the driving trajectory and the slope of the lane line is less than or equal to a first preset difference, and whether the difference between the slope of the head direction of the detection frame and the slope of the lane line is greater than or equal to a second preset difference.

[0133] The slope of the lane line can be determined by the angle between the lane line and a preset direction. The slope of the driving track can be determined by the angle between the tangent direction of a track point on the driving track and the preset direction (e.g., the third angle). The slope of the head orientation can be determined by the angle between the head orientation and the preset direction (e.g., the fourth angle). A larger angle indicates a larger slope.

[0134] The following description uses the preset direction as the tangent direction of the lane line as an example. In this case, the slope of the lane line can be 0, the slope of the driving trajectory can be determined by the third angle, and the slope of the head direction can be determined by the fourth angle.

[0135] The third angle and the fourth angle mentioned above are both acute angles.

[0136] Optionally, the difference between the slope of the driving trajectory and the slope of the lane line is less than or equal to a first preset difference, including: the third angle is less than or equal to the first preset angle. The third angle being less than or equal to the first preset angle indicates that the tangent direction of the trajectory point is substantially parallel to the tangent direction of the lane line, indicating that the obstacle has not changed lanes into the vehicle's lane.

[0137] Optionally, the difference between the slope of the head orientation and the slope of the lane marking is greater than or equal to a second preset difference, including: a fourth angle being greater than the second preset angle. The fourth angle being greater than the second preset angle indicates that the angle between the head orientation and the tangent direction of the lane marking is too large, indicating that an obstacle is changing lanes and cutting into the lane where the vehicle is located.

[0138] Optionally, the second preset angle is greater than the first preset angle.

[0139] Exemplarily, the first preset angle is 2°, and the second preset angle is 5°.

[0140] Optionally, in a certain frame, when the third angle is less than or equal to the first preset angle and the fourth angle is greater than the second preset angle, the dynamic perception network can immediately correct the head orientation in the corresponding frame. In a certain frame, when the third angle is less than or equal to the first preset angle and the fourth angle is greater than the first preset angle and less than or equal to the second preset angle, the dynamic perception network may not correct the head orientation first, and may continue to detect the angle between the tangent direction of the trajectory point of the driving trajectory and the tangent direction of the lane line in the next frame (or the next few frames), and the change in the fourth angle. For example, if the angle between the tangent direction of the trajectory point and the tangent direction of the lane line in the next frame is less than or equal to the first preset angle, and the fourth angle is greater than the first preset angle and less than or equal to the second preset angle, the head orientation can be corrected.

[0141] When the difference between the slope of the driving trajectory and the slope of the lane line is less than or equal to the first preset difference, and the difference between the slope of the head direction of the detection frame and the slope of the lane line is greater than or equal to the second preset difference, execute S511; otherwise, the detection frame is not corrected.

[0142] S511: Determine that the head orientation of the detection frame is incorrect, and correct the head orientation of the detection frame.

[0143] For example, FIG8 shows a schematic diagram of correcting a detection frame provided in an embodiment of the present application.

[0144] As shown in (a) of Figure 8 , the vehicle's driving trajectory can be obtained from the output of the OLA network. This driving trajectory includes trajectory points at times T1-T4. At times T1, T2, and T3, the third angle is less than or equal to the first preset angle, and the fourth angle is less than or equal to the first preset angle. At time T4, the third angle is less than or equal to the first preset angle, and the fourth angle is greater than the second preset angle. This indicates that the head orientation determined by the dynamic perception network at time T4 is incorrect, and the dynamic perception network can correct the head orientation.

[0145] As can be seen from (a) in Figure 8, at time T4, the third angle is less than or equal to the first preset angle and the fourth angle is greater than the second preset angle. This means that the result output by the OLA network indicates that the obstacle has not changed lanes and cut into the lane where the vehicle is located, while the detection box output by the dynamic perception network and the lane line information output by the static perception network indicate that the obstacle is changing lanes and cutting into the lane where the vehicle is located.

[0146] As shown in Figure 8(b), the dynamic perception network can correct the head orientation of the detection frame at time T4 based on the driving trajectory. For example, the dynamic perception network can rotate the head orientation clockwise based on the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line to obtain a corrected detection frame. For another example, the fourth angle between the head orientation of the corrected detection frame and the tangent direction of the lane line can be equal to the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line.

[0147] In this way, when the difference between the slope of the driving trajectory and the slope of the lane marking is small, but the difference between the slope of the head orientation of the first detection frame and the slope of the lane marking is large, the prediction model output indicates that the obstacle is not changing lanes into the vehicle's lane, while the perception results of the dynamic perception network and the static perception network indicate that the obstacle is changing lanes into the vehicle's lane. In this way, the dynamic perception network can correct the head orientation, avoiding the vehicle's sudden braking caused by false detection of an obstacle changing lanes, which helps improve the user's driving experience.

[0148] S512: Obtain a corrected detection frame.

[0149] In the above embodiment, the dynamic perception network uses the distance between the obstacle and the lane line output by the OLA network to correct the detection frame as an example. The dynamic perception network can send the corrected detection frame to the regulation and control module, so that the regulation and control module can determine the relative position relationship (for example, 3D distance) between the obstacle and the lane line based on the corrected detection frame output by the dynamic perception network and the lane line information output by the static perception network. In this way, there is no need for improvement for the regulation and control module, and the embodiments of the present application are not limited to this.

[0150] For example, the distance between the obstacle and the lane line output by the above OLA network can also be sent directly to the regulation and control module, so that the regulation and control module can directly correct the detection box output by the dynamic perception network based on the distance between the obstacle and the lane line.

[0151] For example, the distance between the obstacle and the lane line output by the above OLA network can also be sent directly to the regulation and control module. The regulation and control module can directly determine whether the obstacle is changing lanes and cutting into the lane where the vehicle is located based on the distance between the obstacle and the lane line output by the OLA network.

[0152] In an embodiment of the present application, the absolute position of the obstacle (for example, the detection frame) can be corrected by predicting the relative position relationship between the obstacle and the lane line output by the model, thereby avoiding missed detection or false detection of the obstacle when changing lanes.

[0153] For example, when a vehicle's sensors are noisy, the introduction of the aforementioned OLA network can predict the time at which an obstacle will be cut in before a lane change. Figure 9 shows the timestamps of lane changes detected without the use of the OLA network (the cutin timestamp before correction in the figure) and using the OLA network (the cutin timestamp after correction in the figure), as provided in an embodiment of the present application. It can be seen that the time at which a lane change is detected is approximately 250 milliseconds (ms) earlier when the OLA network is used, compared to when the OLA network is not used.

[0154] FIG10 shows a schematic flow chart of a detection method 1000 provided in an embodiment of the present application. The method 1000 may be executed by the vehicle 100, or the computing platform 120, or the system consisting of the computing platform 120 and the perception system 110, or the system-on-a-chip (SoC) in the computing platform 120, or the processor, chip, or circuit in the computing platform 120. The method 1000 includes:

[0155] S1010, acquiring a first image around a vehicle, where the vehicle is located in a first lane. The first image includes information about an obstacle and a first lane line associated with the obstacle, where the first lane line is a lane line of a second lane, and the first lane and the second lane are adjacent.

[0156] Alternatively, the obstacle may be a vehicle in the second lane.

[0157] S1020: Input the first image into a prediction model to obtain the distance between the obstacle and the first lane line. The prediction model is trained by a sample data set, and the sample data set includes an image of a sample obstacle, an image of a second lane line associated with the sample obstacle, and the distance between the sample obstacle and the second lane line.

[0158] The above detection method uses an example of inputting an image into a prediction model to predict the distance between the obstacle and the first lane line, but the embodiments of the present application are not limited to this. For example, data collected by a detection device (such as a lidar or millimeter-wave radar) can also be input into the prediction model to obtain the distance between the obstacle and the first lane line.

[0159] Optionally, the distance between the obstacle and the first lane line may be a real physical distance between the obstacle and the lane line, or may be a pixel distance.

[0160] For example, if the output result is the actual physical distance between the obstacle and the lane line, the position of the detection frame and the head orientation can be corrected according to the actual physical distance between the obstacle and the lane line.

[0161] For example, if the output result is the pixel distance between the obstacle and the lane line, the head orientation of the detection frame can be corrected according to the pixel distance between the obstacle and the lane line.

[0162] The actual physical distance between the obstacle and the lane line can also be called the 3D distance between the obstacle and the lane line, and the pixel distance between the obstacle and the lane line can also be called the 2D distance between the obstacle and the lane line.

[0163] S1030: Determine whether the obstacle changes lanes toward the first lane based on the distance between the obstacle and the first lane line.

[0164] Optionally, the method 1000 further includes: obtaining a first detection frame of the obstacle and information about the first lane line; wherein, determining whether the obstacle changes lanes to the first lane based on the distance between the obstacle and the first lane line includes: correcting the first detection frame based on the distance between the obstacle and the first lane line to obtain a second detection frame; and determining whether the obstacle changes lanes to the first lane based on the second detection frame and information about the first lane line.

[0165] Optionally, the first detection frame is determined by a first perception network and the information of the first lane line is determined by the second perception network.

[0166] Optionally, the first perception network and the second perception network are connected to the regulation and control module respectively. For example, the first perception network can send the calculated obstacle detection box information to the regulation and control module, and the second perception network can send the calculated lane line information to the regulation and control module.

[0167] Exemplarily, the first perception network may be the above-mentioned dynamic perception network, which may be used to detect moving targets around the vehicle.

[0168] Exemplarily, the second perception network may be the aforementioned static perception network, which may be used to detect stationary targets around the vehicle.

[0169] Optionally, the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, the first detection frame is corrected according to the distance between the obstacle and the first lane line.

[0170] Optionally, when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: when the distance between the obstacle and the first lane line is greater than the distance between the first detection frame and the first lane line and the difference between the distance between the obstacle and the first lane line and the distance between the first detection frame and the first lane line is greater than a preset difference, correcting the position of the first detection frame in a direction away from the lane line.

[0171] Optionally, when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: when the distance between the obstacle and the first lane line is less than the distance between the first detection frame and the first lane line and the difference between the distance between the obstacle and the first lane line and the distance between the first detection frame and the first lane line is greater than a preset difference, the position of the first detection frame is corrected in a direction close to the lane line.

[0172] For example, as shown in (a) in Figure 7, at time T3, when L3 is less than L5 and the distance difference between L3 and L5 is greater than a preset distance difference (for example, 0.1 meters), the position of the detection frame at time T3 can be corrected toward the direction close to the lane line, thereby obtaining a corrected detection frame as shown in (b) in Figure 7.

[0173] Optionally, the first detection frame is corrected according to the distance between the obstacle and the first lane line, including: determining the driving trajectory of the obstacle according to the distance between the obstacle and the first lane line; and correcting the head orientation of the first detection frame according to the driving trajectory of the obstacle.

[0174] Exemplarily, the head orientation of the first detection frame is corrected according to the driving trajectory of the obstacle, including: correcting the head orientation of the first detection frame according to the tangent direction of the trajectory point on the driving trajectory of the obstacle, the direction of the lane line (for example, the tangent direction of the lane line) and the head orientation of the first detection frame.

[0175] Optionally, the head orientation of the first detection frame is corrected according to the driving trajectory of the obstacle, including: when the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, and the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, the head orientation of the first detection frame is corrected according to the driving trajectory.

[0176] Optionally, the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, including: the slope of the driving trajectory is determined by a third angle between the tangent direction of the trajectory point on the driving trajectory and the tangent direction of the lane line is less than or equal to the first preset angle.

[0177] Optionally, the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, including: the slope of the head orientation can be greater than or equal to a fourth angle between the head orientation and the tangent direction of the lane line.

[0178] The larger the above angle, the greater the slope.

[0179] For example, as shown in (a) and (b) in Figure 8, at time T4, the third angle is less than or equal to the first preset angle and the fourth angle is greater than the second preset angle, so that it can be determined that the head orientation determined by the dynamic perception network at time T4 is incorrect, and the dynamic perception network can correct the head orientation.

[0180] The dynamic perception network can correct the head orientation of the detection frame at time T4 based on the driving trajectory. For example, the dynamic perception network can rotate the head orientation clockwise based on the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line to obtain a corrected detection frame. For another example, the fourth angle between the head orientation of the corrected detection frame and the tangent direction of the lane line can be equal to the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line.

[0181] Optionally, the head orientation of the first detection frame is corrected according to the driving trajectory of the obstacle, including: when the slope of the driving trajectory is greater than the slope of the head orientation of the obstacle, the head orientation of the first detection frame is corrected according to the driving trajectory.

[0182] Optionally, the slope of the driving trajectory is greater than the slope of the head orientation of the first detection frame, including: the slope of the driving trajectory is greater than the slope of the head orientation and the difference between the slope of the driving trajectory and the slope of the head orientation is greater than or equal to a preset difference.

[0183] Optionally, the slope of the driving trajectory can be determined by a first angle between the tangent direction of a trajectory point on the driving trajectory and the tangent direction of the lane line, and the slope of the head orientation can be determined by a second angle between the head orientation and the tangent direction of the lane line.

[0184] Optionally, the difference between the slope of the driving trajectory and the slope of the head orientation is greater than or equal to a preset difference, including: the difference between the first angle and the second angle is greater than the preset angle difference.

[0185] For example, as shown in (a) and (b) in Figure 6, at time T4, the first angle is greater than the second angle and the difference between the first angle and the second angle is greater than the preset angle difference, so it can be determined that the head orientation determined by the dynamic perception network at time T4 is incorrect, and the dynamic perception network can correct the head orientation.

[0186] The dynamic perception network can correct the head orientation of the detection frame at time T4 based on the driving trajectory. For example, the dynamic perception network can correct the head orientation of the detection frame based on the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line. For example, the angle between the corrected head orientation and the tangent direction of the lane line can be equal to the angle between the tangent direction of the trajectory point at time T4 and the tangent direction of the lane line.

[0187] FIG11 shows a schematic block diagram of a detection device 1100 provided in an embodiment of the present application. The device 1100 includes: an acquisition unit 1110 for acquiring a first image of a vehicle located in a first lane, the first image including information about an obstacle and a first lane line associated with the obstacle, the first lane line being the lane line of a second lane adjacent to the first lane; a prediction unit 1120 for inputting the first image into a prediction model to obtain the distance between the obstacle and the first lane line, the prediction model being trained using a sample dataset including an image of a sample obstacle, an image of a second lane line associated with the sample obstacle, and the distance between the sample obstacle and the second lane line; and a determination unit 1130 for determining whether the obstacle is changing lanes into the first lane based on the distance between the obstacle and the first lane line.

[0188] Optionally, the device 1100 also includes a correction unit, and the acquisition unit 1110 is further used to obtain a first detection frame of the obstacle and information about the first lane line; the correction unit is used to: correct the first detection frame according to the distance between the obstacle and the first lane line to obtain a second detection frame; the determination unit 1130 is used to: determine whether the obstacle changes lanes to the first lane based on the second detection frame and information about the first lane line.

[0189] Optionally, the correction unit is used to correct the first detection frame according to the distance between the obstacle and the first lane line when the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line.

[0190] Optionally, the determination unit 1130 is further used to determine the driving trajectory of the obstacle based on the distance between the obstacle and the first lane line; the correction unit is used to correct the head orientation of the first detection frame based on the driving trajectory of the obstacle.

[0191] Optionally, the correction unit is used to: when the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, and the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, correct the head orientation of the first detection frame according to the distance between the obstacle and the first lane line.

[0192] Optionally, the correction unit is used to correct the head orientation of the first detection frame according to the driving trajectory when the slope of the driving trajectory is greater than the slope of the head orientation of the first detection frame.

[0193] Optionally, the first detection frame is determined by a first perception network and the information of the first lane line is determined by the second perception network.

[0194] For example, the acquisition unit 1110 may be the computing platform in FIG1 or a processing circuit, processor, or controller in the computing platform. For example, if the acquisition unit 1110 is the processor 121 in the computing platform, the processor 121 may acquire first data acquired by a sensor outside the cabin (e.g., a first image acquired by a camera outside the cabin).

[0195] For another example, the prediction unit 1120 may be the computing platform in FIG1 or a processing circuit, processor, or controller in the computing platform. For example, if the prediction unit 1120 is the processor 122 in the computing platform, the processor 122 may input the first data acquired by the processor 121 into a preset model to obtain the 3D distance between the obstacle and the lane line.

[0196] For another example, the determining unit 1130 may be the computing platform in FIG1 or a processing circuit, processor, or controller in the computing platform. For example, if the determining unit 1130 is the processor 123 in the computing platform, the processor 123 may determine whether the obstacle is changing lanes and cutting into the lane where the vehicle is located based on the distance between the obstacle and the lane line obtained by the processor 122.

[0197] The functions implemented by the above-mentioned acquisition unit 1110, the functions implemented by the prediction unit 1120, and the functions implemented by the determination unit 1130 can be implemented by different processors, or they can be implemented by the same processor, or some functions can be implemented by the same processor. The embodiments of the present application do not limit this.

[0198] It should be understood that the division of the various units in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units in the device may be implemented in the form of a processor calling software; for example, the device includes a processor connected to a memory storing instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of the various units in the device, where the processor is, for example, a general-purpose processor such as a CPU or a microprocessor, and the memory is a memory within the device or a memory external to the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented through the design of the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units may be implemented through the design of the logical relationships between the components within the circuits. In another implementation, the hardware circuit may be implemented using a PLD, such as an FPGA, which may include a large number of logic gate circuits, and the connections between the logic gate circuits may be configured using a configuration file to implement the functions of some or all of the above units. All units of the above apparatus may be implemented entirely in the form of software called by a processor, or entirely in the form of hardware circuits, or partially in the form of software called by a processor and the rest in the form of hardware circuits.

[0199] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0200] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0201] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0202] An embodiment of the present application also provides a detection device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to perform the method or steps performed in the above embodiment.

[0203] Optionally, if the device is located in a vehicle, the processing unit may be the processors 121 - 12n shown in FIG. 1 .

[0204] An embodiment of the present application further provides a detection system, which may include a computing platform and a perception system, and the computing platform may include the above-mentioned detection device 1100.

[0205] An embodiment of the present application also provides a vehicle, which may include the above-mentioned detection device 1100 or detection system.

[0206] An embodiment of the present application further provides a computer program product, which includes: computer program code, which enables the computer to execute the method in the above embodiment when the computer program code is run on a computer.

[0207] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code. When the computer program code runs on a computer, the computer executes the method in the above embodiment.

[0208] An embodiment of the present application further provides a chip, which includes a circuit, and the circuit is used to execute the method in the above embodiment.

[0209] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or a power-on erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0210] It should be understood that in the embodiment of the present application, the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.

[0211] It should also be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0212] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0213] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0215] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0217] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0218] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be covered and fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A detection method, characterized in that, Including: Obtaining a first image around a vehicle, where the vehicle is located in a first lane, and the first image includes information about an obstacle and a first lane line associated with the obstacle, the first lane line being a lane line of a second lane, and the first lane and the second lane being adjacent; Inputting the first image into a prediction model to obtain the distance between the obstacle and the first lane line, where the prediction model is trained by a sample data set, and the sample data set includes images of sample obstacles, images of second lane lines associated with the sample obstacles, and the distances between the sample obstacles and the second lane lines; Determining whether the obstacle is changing lanes into the first lane according to the distance between the obstacle and the first lane line.

2. The method according to claim 1, wherein The method further includes: Obtaining a first detection frame of the obstacle and information about the first lane line; Wherein, the determining whether the obstacle is changing lanes into the first lane according to the distance between the obstacle and the first lane line includes: Correcting the first detection frame according to the distance between the obstacle and the first lane line to obtain a second detection frame; Determining whether the obstacle is changing lanes into the first lane according to the second detection frame and the information about the first lane line.

3. The method according to claim 2, wherein The correcting the first detection frame according to the distance between the obstacle and the first lane line includes: When the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, correcting the first detection frame according to the distance between the obstacle and the first lane line.

4. The method according to claim 2, wherein The correcting the first detection frame according to the distance between the obstacle and the first lane line includes: Determining the driving trajectory of the obstacle according to the distance between the obstacle and the first lane line; Correcting the head orientation of the first detection frame according to the driving trajectory of the obstacle.

5. The method according to claim 4, characterized in that, The correcting the head orientation of the first detection frame according to the driving trajectory of the obstacle includes: When the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, and the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, correcting the head orientation according to the driving trajectory.

6. The method according to claim 4, characterized in that, The correcting the head orientation of the first detection frame according to the driving trajectory of the obstacle includes: When the slope of the driving trajectory is greater than the slope of the head orientation, correcting the head orientation according to the driving trajectory.

7. The method according to any one of claims 2 to 6, characterized in that The first detection frame is determined by a first perception network and the information about the first lane line is determined by the second perception network.

8. A detection device, characterized in that, Including: An obtaining unit configured to obtain a first image around a vehicle, where the vehicle is located in a first lane, and the first image includes information about an obstacle and a first lane line associated with the obstacle, the first lane line being a lane line of a second lane, and the first lane and the second lane being adjacent; A prediction unit, configured to input the first image into a prediction model to obtain the distance between the obstacle and the first lane line, where the prediction model is trained by a sample data set, and the sample data set includes images of sample obstacles, images of second lane lines associated with the sample obstacles, and the distances between the sample obstacles and the second lane lines; A determination unit, configured to determine whether the obstacle changes lanes towards the first lane according to the distance between the obstacle and the first lane line.

9. The device according to claim 8, characterized in that The device further includes a correction unit. The acquisition unit is further configured to acquire the first detection frame of the obstacle and the information of the first lane line. The correction unit is configured to: correct the first detection frame according to the distance between the obstacle and the first lane line to obtain a second detection frame. The determination unit is configured to: determine whether the obstacle changes lanes towards the first lane according to the second detection frame and the information of the first lane line.

10. The device according to claim 9, characterized in that, The correction unit is configured to: When the distance between the obstacle and the first lane line does not match the distance between the first detection frame and the first lane line, correct the first detection frame according to the distance between the obstacle and the first lane line.

11. The device according to claim 9, wherein the determination unit is further configured to determine the driving trajectory of the obstacle according to the distance between the obstacle and the first lane line. The correction unit is configured to correct the head orientation of the first detection frame according to the driving trajectory of the obstacle.

12. The device according to claim 11, characterized in that, The correction unit is configured to: When the difference between the slope of the driving trajectory and the slope of the first lane line is less than or equal to a first preset difference, and the difference between the slope of the head orientation of the first detection frame and the slope of the first lane line is greater than or equal to a second preset difference, correct the head orientation according to the driving trajectory.

13. The device according to claim 11, characterized in that, The correction unit is configured to: When the slope of the driving trajectory is greater than the slope of the head orientation, correct the head orientation according to the driving trajectory.

14. The device according to any one of claims 9 to 13, characterized in that The first detection frame is determined by a first perception network and the information of the first lane line is determined by the second perception network.

15. A detection device, characterized in that, including: a memory, configured to store a computer program; a processor, configured to execute the computer program stored in the memory, so that the device executes the method according to any one of claims 1 to 7.

16. A detection system, characterized in that, The control system includes a perception system and a computing platform, and the computing platform includes a detection device according to any one of claims 8 to 15.

17. A vehicle, characterized in that, including a detection device according to any one of claims 8 to 15, or including a detection system according to claim 16.

18. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

19. A chip, characterized in that, including: a circuit, and the circuit is configured to execute the method according to any one of claims 1 to 7.

20. A computer program product, characterized in that, The computer program product includes computer program code which, when run on a computer, enables the implementation of the method according to any one of claims 1 to 7.

Images