Vehicle inner wheel difference dangerous area limit detection and early warning method, device, equipment, medium and product

Abstract

The invention discloses a vehicle inner wheel difference dangerous area limit detection and early warning method, device, equipment, medium and product, and relates to the field of auxiliary driving, the method comprises the following steps: determining a front inner wheel steering angle according to information of an included angle between a tractor and a trailer collected by an angle sensor, and determining whether the vehicle steers, if yes, determining that the vehicle is not steered; the method comprises the following steps of: simultaneously capturing real-time images and point cloud data of a steering inner side of a vehicle by using a camera and a radar on the steering inner side of the vehicle, calculating an inner wheel difference of a tractor and an inner wheel difference of a trailer, and determining an inner wheel difference dangerous area limit according to the two inner wheel differences; based on a neural network model, recognizing the position and distance of a road obstacle according to the real-time image and the point cloud data; according to the position and the distance of the road surface obstacle and the inner wheel difference dangerous area limit, whether an alarm is given or not is judged, the inner wheel difference dangerous area limit can be recognized more accurately, the road condition on the inner side of the vehicle during steering can be recognized more visually, and accidents are reduced.

Description

Technical Field

[0001] The present application relates to the field of assisted driving, and in particular to a method, device, equipment, medium and product for detecting and warning the limit of a dangerous area of ​​inner wheel difference of a vehicle. Background Art

[0002] The inner wheel difference is one of the most important factors leading to traffic accidents during vehicle turning. It is an attribute that cannot be eliminated during vehicle turning. Under certain dangerous roads and driving conditions, it will cause casualties and property losses, and is a potential safety hazard that cannot be ignored. Especially in serious traffic accidents involving large vehicles, accidents caused by "inner wheel difference" account for more than 70%, and the mortality rate is more than 90%. The calculation of the inner wheel difference is related to the vehicle's wheelbase, track and turning radius.

[0003] Research shows that when large vehicles turn, the inner wheel difference area may invade the inner lane, posing a great safety threat to vehicles and pedestrians on the inner lane. For example, when a semi-trailer turns right on an urban road, due to the inner wheel difference, it may run over pedestrians and non-motor vehicles, causing an accident.

[0004] At present, there are some studies aimed at solving the problem of inner wheel difference of large trailer vehicles. The development of the surround view system of automobile trains (trailers) has become an important research direction. The blind spot monitoring system such as 360 surround view can collect images around the vehicle at the same time through 180-degree wide-angle cameras installed in the front, back, left, and right directions of the vehicle. After correction and splicing by the image processing unit, a 360-degree panoramic bird's-eye view around the vehicle is formed to help the driver better observe the situation around the vehicle, eliminate the blind spot of vision and solve the traffic problems caused by the inner wheel difference. However, since the 360 ​​surround view provides a bird's-eye view, which is quite different from the driver's perspective, it is impossible to accurately identify the inner wheel difference danger zone limit by relying solely on the camera for recognition, and the recognition accuracy is low. Therefore, it is difficult for the driver to finely control the turning angle through the 360 ​​surround view; and if the system fails, bugs may appear in harsh environments. It can be seen that the blind spot monitoring system such as 360 surround view cannot accurately identify the inner wheel difference danger zone limit, resulting in errors in the recognized road conditions inside the vehicle during turning, which is prone to accidents. Summary of the invention

[0005] The purpose of this application is to provide a vehicle inner wheel difference danger zone limit detection and early warning method, device, equipment, medium and product to solve the problem that the inner wheel difference danger zone limit cannot be accurately identified.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for detecting and warning a dangerous area limit of a vehicle inner wheel difference, comprising:

[0008] The front inner wheel steering angle is determined based on the angle information between the tractor and the trailer collected by the angle sensor; the angle sensor is arranged at the hinge point between the tractor and the trailer and at the bottom of the tractor steering point;

[0009] Determining whether the vehicle is turning according to the front inner wheel steering angle to obtain a first determination result;

[0010] If the first judgment result is yes, using the camera and radar on the inner side of the vehicle to simultaneously capture the real-time image and point cloud data of the inner side of the vehicle;

[0011] Based on the real-time image and the point cloud data, the inner wheel difference of the tractor and the inner wheel difference of the trailer are calculated, and the inner wheel difference danger zone limit is determined according to the inner wheel difference of the tractor and the inner wheel difference of the trailer;

[0012] Based on the neural network model, identifying the location and distance of road obstacles according to the real-time image and the point cloud data;

[0013] Determining whether to issue an alarm based on the location and distance of the road obstacle and the inner wheel difference danger zone limit;

[0014] If the first judgment result is no, return to the step of "determining the front inner wheel steering angle according to the angle information between the tractor and the trailer collected by the angle sensor".

[0015] In a second aspect, the present application provides a vehicle inner wheel difference dangerous area limit detection and warning device, comprising:

[0016] A front inner wheel steering angle determination module is used to determine the front inner wheel steering angle according to the angle information between the tractor and the trailer collected by the angle sensor; the angle sensor is arranged at the hinge point of the tractor and the trailer and at the bottom of the tractor steering point;

[0017] A first judgment module, used for judging whether the vehicle is turning according to the front inner wheel steering angle, and obtaining a first judgment result;

[0018] Cameras and radars to simultaneously capture real-time images of the vehicle’s inboard turn as well as point cloud data;

[0019] An inner wheel difference danger zone limit determination module, used to calculate the inner wheel difference of the tractor and the inner wheel difference of the trailer based on the real-time image and the point cloud data, and determine the inner wheel difference danger zone limit according to the inner wheel difference of the tractor and the inner wheel difference of the trailer;

[0020] A road obstacle location and distance recognition module, used to recognize the location and distance of road obstacles based on the real-time image and the point cloud data based on a neural network model;

[0021] The early warning module is used to determine whether to issue an alarm based on the position and distance of the road obstacle and the inner wheel difference danger zone limit.

[0022] The return module is used to return to the step of "determining the steering angle of the front inner wheel according to the angle information between the tractor and the trailer collected by the angle sensor".

[0023] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described vehicle inner wheel difference dangerous area limit detection and warning methods.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described vehicle inner wheel difference dangerous area limit detection and warning methods.

[0025] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned vehicle inner wheel difference dangerous area limit detection and warning methods.

[0026] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application uses radar and camera to simultaneously capture the real-time image and point cloud data of the inner side of the vehicle turning, thereby realizing dual recognition of obstacles, using radar to compensate for the inaccurate recognition problem of the camera in harsh environments, and calculating the inner wheel difference of the trailer through real-time images and point cloud data, and finally obtaining a more accurate inner wheel difference danger zone limit to accurately identify the road conditions on the inner side of the vehicle when turning. In addition, the present application also determines the location and distance of road obstacles based on a neural network model, and determines whether to issue an alarm based on the obtained inner wheel difference danger zone limit, thereby reducing the occurrence of accidents through early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1A flow chart of a method for detecting and warning the boundary of a dangerous area of ​​inner wheel difference of a vehicle provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of vehicle line connection provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a vehicle steering process provided by an embodiment of the present application;

[0031] Figure 4 A schematic diagram of the inner wheel differential danger zone provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the YOLOv5 network structure integrating a small target detection layer provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of the point cloud data and real-time image fusion process provided in an embodiment of the present application;

[0034] Figure 7 A flow chart of a method for detecting and warning the boundary of a dangerous area of ​​inner wheel difference of a vehicle provided in another embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0037] The present application embodiment provides a vehicle inner wheel difference dangerous area limit detection and early warning method, the method is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the present application embodiment, if Figure 1 As shown, the method includes the following steps.

[0038] S1: Determine the front inner wheel steering angle according to the angle information between the tractor and the trailer collected by the angle sensor; the angle sensor is arranged at the hinge point of the tractor and the trailer and at the bottom of the tractor steering point; wherein the bottom of the tractor steering point is the bottom of the tractor front wheel steering column.

[0039] S2: Determine whether the vehicle is turning according to the front inner wheel steering angle, if so, execute S3, if not, return to S1.

[0040] S3: The camera and radar on the inner side of the vehicle are used to simultaneously capture real-time images and point cloud data of the inner side of the vehicle.

[0041] S4: Based on the real-time image and the point cloud data, the inner wheel difference of the tractor and the inner wheel difference of the trailer are calculated, and the inner wheel difference danger zone limit is determined according to the inner wheel difference of the tractor and the inner wheel difference of the trailer.

[0042] S5: Based on the neural network model, identify the location and distance of road obstacles according to the real-time image and the point cloud data.

[0043] S6: Determine whether to issue an alarm based on the position and distance of the road obstacle and the inner wheel difference danger zone limit.

[0044] In an exemplary embodiment, S2 may be replaced by the following steps.

[0045] S21: Determine whether the front inner wheel steering angle is greater than or equal to 15° and whether the duration is greater than or equal to 2.5 seconds. If so, execute S22; if not, execute S23.

[0046] S22: Determine that the first judgment result is that the vehicle is turning.

[0047] S23: Determine that the first judgment result is that the vehicle is not turning.

[0048] In an exemplary embodiment, S4 may be replaced by the following steps.

[0049] S41: Fusing the real-time image and the point cloud data to determine a fused image.

[0050] S42: Calculating the inner wheel difference of the tractor and the inner wheel difference of the trailer according to the fused image.

[0051] S43: Determine the inner contour of the vehicle according to the inner wheel difference of the tractor and the inner wheel difference of the trailer.

[0052] S44: Determine a rear inner wheel steering trajectory according to the vehicle inner contour.

[0053] S45: Determine the inner wheel difference danger zone limit according to the rear inner wheel steering trajectory.

[0054] In an exemplary embodiment, S42 may be replaced by the following steps.

[0055] S421: Based on the fused image, determine the basic vehicle data and turning angle information; the basic vehicle data includes the wheelbase of the tractor, the distance between the center lines of the outer rear wheels of the tractor, and the distance between the traction pin and the rear axle of the tractor; the turning angle information includes the movement radius of the center line of the inner front wheel and the movement radius of the center line of the inner rear wheel.

[0056] S422: Calculate the inner wheel difference of the tractor and the inner wheel difference of the trailer according to the basic vehicle data and the turning angle information; the inner wheel difference is: Among them, m' is the inner wheel difference; b is the movement radius of the center line of the inner front wheel; a is the movement radius of the center line of the inner rear wheel; d2 is the distance between the center lines of the outer rear wheels of the tractor; k is the distance between the traction pin and the rear axle of the tractor; L is the wheelbase of the tractor.

[0057] In an exemplary embodiment, S422 may be replaced by the following steps.

[0058] The movement radius of the traction pin is determined according to the distance between the center lines of the outer rear wheels of the tractor, the distance between the traction pin and the rear axle of the tractor, and the movement radius of the center line of the inner rear wheels.

[0059] The moving radius of the centerline of the inner rear wheel of the trailer is determined according to the moving radius and the wheelbase of the tractor.

[0060] The inner wheel difference of the trailer is determined according to the movement radius of the inner front wheel centerline and the movement radius of the inner rear wheel centerline of the trailer.

[0061] In an exemplary embodiment, S5 may be replaced by the following steps.

[0062] S51: Input the real-time image into the neural network model, process the real-time image based on the YOLOv5 algorithm, and generate an image including a two-dimensional target detection frame; the neural network model is a YOLOv5 network.

[0063] S52: Performing Euclidean clustering processing on the point cloud data to generate a three-dimensional point cloud bounding box.

[0064] S53: Projecting the three-dimensional point cloud bounding box onto the image containing the two-dimensional target detection frame to determine a fused image.

[0065] S54: Identify the location and distance of road obstacles according to the fused image.

[0066] In order to solve the problem of traffic accidents caused by the dangerous area of ​​inner wheel difference between cars and trains, the present application enables the driver to understand the road conditions on the inner side of the vehicle more intuitively and clearly when turning, thereby reducing traffic accidents caused by inner wheel difference when turning.

[0067] In practical applications, Figure 2 Take the example to further illustrate the technical solution of the present application.

[0068] Among them, the angle sensor is installed at the hinge point of the tractor and trailer and at the bottom of the tractor's steering point to obtain the angle information between the tractor and trailer to detect whether to turn; and obtain the steering angle of the front inner wheel.

[0069] Cameras: Installed on both sides of the vehicle to capture images of the vehicle turning inward.

[0070] Display screen: Usually integrated into the vehicle's center console, dashboard or rearview mirror, used to display the image captured by the camera.

[0071] Graphics Processing Unit (GPU): Responsible for processing camera signals and converting them into a format suitable for display on the display screen. The GPU combines angle information and basic vehicle information to estimate the inner wheel difference and derive the range of the danger zone. It is responsible for controlling the radar sensor to emit ultrasonic signals, and processing the received reflected signals to calculate the distance to obstacles within the danger zone.

[0072] Connect cables: connect the camera and GPU to transmit video signals, connect the angle sensor and GPU to transmit angle signals.

[0073] Auxiliary lines: The system will add auxiliary lines to the image on the display to help the driver judge the steering trajectory of the trailer and the position of surrounding obstacles.

[0074] Digital signal processor DSP: performs high-speed real-time pre-processing on the image signals captured by the camera.

[0075] Ultrasonic sensor (probe): installed on both sides of the vehicle, used to transmit ultrasonic signals and receive reflected signals.

[0076] Buzzer: Based on the distance calculated by the controller, a sound (buzzer) is used to warn the driver.

[0077] When the car starts, the system starts as well; the system is a vehicle inner wheel difference dangerous area limit detection and warning device corresponding to the vehicle inner wheel difference dangerous area limit detection and warning method.

[0078] Detecting whether to turn: The system will transmit the angle data of the center axis of the tractor and trailer through the angle sensor and transmit it to the ECU via the CAN bus for processing to detect whether to turn. Furthermore, when the angle between the center axis of the tractor and trailer is greater than or equal to 15 degrees and the angle duration is greater than or equal to 2.5 seconds, it is determined that the vehicle is turning. If not, the system, camera and radar are on standby.

[0079] Activate Turn Camera: Once a turn is detected, the system activates the camera and radar on the inside of the car train turn to begin capturing real-time images of the inside of the vehicle turn.

[0080] like Figure 3 As shown, O1 and O2 are the turning centers of the tractor and trailer of the automobile train respectively; the turning angle information obtained by the angle sensor includes: R5 and R6 are the turning radius of the lane where the tractor and trailer are located, α and β are the angles between the midpoints of the front and rear axles of the tractor and trailer and the center of the turning circle, R1 and R3 are the movement radius of the center line of the inner rear wheels of the tractor and trailer, and R2 and R4 are the movement radius of the center line of the inner front wheels of the tractor and trailer.

[0081] The basic vehicle data include: L1 and L2 are the wheelbases of the tractor and trailer; d1 and d2 are the front and rear wheelbases of the tractor; k is the distance between the towing pin and the rear axle of the tractor.

[0082] The tractor and trailer are regarded as two rigid bodies, and their inner wheel differences are calculated independently.

[0083] Calculation formula for inner wheel difference of tractor:

[0084]

[0085] m1=R1-R2

[0086] Calculation formula for the inner wheel difference of trailer:

[0087]

[0088] m2=R3-R4

[0089] The GPU combines the provided data and calculates the inner wheel difference using the above two inner wheel difference formulas.

[0090] Determination of the danger zone: Determine the danger zone of the inner wheel difference based on the size of the inner wheel difference, the inner contour of the car and the train, and the expected steering trajectory of the rear wheels, such as Figure 4 As shown, the area between the rear wheel trajectory and the inner boundary line of the car and train is defined as the inner wheel difference danger zone, and a plane rectangular coordinate system is constructed with the inner wheel turning center as the coordinate origin; the obstacle projection is placed on the inner wheel difference danger zone layer, and an alarm is triggered if the obstacle projection coincides with the inner wheel difference danger zone.

[0091] Image and trajectory overlay: The system calls the OVERLAY layer to overlay the calculated turning trajectory of the trailer with the real-time image captured by the camera.

[0092] Semi-transparent processing: To improve safety, dynamic auxiliary lines are usually processed to be semi-transparent so as not to block the view of obstacles behind.

[0093] Display turning image: The processed image is displayed together with auxiliary lines on the vehicle's display screen to provide steering guidance for the driver.

[0094] Real-time Updates: As the car moves and turns, the auxiliary lines will update in real time to reflect the current steering trajectory.

[0095] Obtain and process image information: GPU uses neural network algorithms to filter image information, obtain obstacle information, and determine whether the obstacle is in a dangerous area.

[0096] Activate turning radar: After the GPU determines the dangerous area, it activates the radar on the inner side of the car train.

[0097] Ultrasonic signal emission: The sensors (probes) of the steering radar system emit ultrasonic signals. These signals are usually emitted by ultrasonic sensors installed on both sides of the vehicle.

[0098] Signal reflection: When ultrasonic waves encounter obstacles, they will generate reflected waves, which will be received by the same or adjacent sensors.

[0099] Signal reception and processing: After the sensor receives the reflected wave, it converts it into an electrical signal and transmits it to the GPU. The GPU uses a neural network algorithm to calculate the distance of the obstacle based on the time difference between transmission and reception.

[0100] Distance calculation: The system calculates the distance through the propagation speed of sound waves (usually 340 meters / second) and the time difference from transmission to reception, distance = (time difference × speed of sound) / 2.

[0101] Data analysis: The controller analyzes the collected data to determine the location and distance of obstacles, and determines whether an alarm needs to be issued based on the preset danger zone range.

[0102] Alarm trigger: When the system determines that the obstacle distance reaches the preset alarm distance, an alarm is sounded through a buzzer or display. The alarm may be an audible signal or a visual signal, such as a flashing LED light.

[0103] System standby: When the driver completes the turning operation, the system returns to standby state and prepares for the next steering operation.

[0104] In practical applications, the fusion process of point cloud data and real-time images specifically includes the following steps.

[0105] 1) Point cloud preprocessing and improved Euclidean clustering algorithm.

[0106] Point cloud preprocessing: According to the threshold range of the specified spatial coordinate axis (such as X, Y, Z), the straight-through filtering selects points within (or outside) the threshold range from the original point cloud data, and then generates a region of interest (ROI). This process effectively divides the point cloud into two parts: the region of interest and the region of non-interest, which facilitates further point cloud processing such as feature extraction and target recognition. After the straight-through filtering, only the distance of 8m in front of the radar, 4m on the left and right sides, and 1.5m above and below is retained.

[0107] Voxel filtering compresses a large amount of point cloud data into a smaller number to reduce the computational difficulty of subsequent algorithms while maintaining the geometric structure and features of the point cloud.

[0108] Using the radius filtering method, with each point cloud as the center, the neighboring points are searched within the set radius, and the average value within the radius is taken as the new value of the point to eliminate isolated points and noise points in the point cloud data.

[0109] Improved Euclidean clustering algorithm: First, input the lidar point cloud data to be clustered, and use KD-tree to divide the point cloud into spatial structures.

[0110] Take every 3m as an interval, calculate the Euclidean distance of each point in the interval, and use the same threshold for calculation and comparison in the same interval.

[0111] d r =μd+d0

[0112]

[0113] d0=||p m -p n ||

[0114] Among them, d r is the Euclidean distance of each point, (x, y, z) is the position coordinate of the point cloud in the spatial coordinate system with the laser radar as the origin, d is the distance from the point cloud to the laser radar, d0 is the initial clustering threshold, and the two points p closest to each other in the point cloud are clustered. m and p n Calculate, μ is the modification factor.

[0115] 2) Road obstacle recognition and detection based on YOLOv5.

[0116] The YOLOv5 network consists of an input terminal (Input), a backbone structure (Backbone), a neck structure (Neck) and a prediction output terminal (Prediction), such as Figure 5 As shown in Table 1, Table 1 is a schematic table of anchor box parameters of the YOLOv5 network.

[0117] Table 1

[0118] Feature Pyramid Anchor box size Output feature map scale P2(Add) [5,6,8,15,17,11] 160x160 P3 [10,13,16,30,33,23] 80x80 P4 [30,61,62,45,59,119] 40x40 P5 [116,90,156,198,373,326] 20x20

[0119] When an image is input, the detection image will be divided into S×S cells, and then the image feature information will be extracted through the network model. Each cell will predict the detection frame of the target object. The prediction content includes information such as the center coordinates of the small cell in the image. Finally, the predicted detection frame is compared probabilistically through the non-maximum suppression algorithm, so that the network model outputs the optimal detection frame and introduces a small target detection layer.

[0120] While keeping the original network module and training parameters unchanged, add a small target detection layer for road traffic cones and pedestrians at the output of the YOLOv5 network. Adding a small target detection layer also requires adding information extraction modules to the backbone and neck structures of the algorithm. Add a P2 detection layer at the output of the algorithm to reduce the model's receptive field size for small target obstacles, set the feature scale of the output to 160×160, so that the minimum receptive field of the network model is 4×4, and then set the anchor box size parameters of the P2 layer.

[0121] After adding the detection head at the prediction end, it is necessary to add the convolution-batch normalization-Sigmoid Linear Unit (SiLU) activation module, the cross-stage partial network (CSP) bottleneck and three convolution modules, upsampling modules and concatenation modules to the network structure of the algorithm. Specifically, the upsampling process is added to the 20th layer of the YOLOv5 network structure to improve the feature extraction scale; the concatenation layer (Concat) is added to the 21st layer to fuse the feature information obtained by upsampling with the output results of the CBS module in the Backbone, so as to increase the network's ability to extract road obstacle information, thereby more accurately identifying road information and realizing the detection of target obstacles by vehicles.

[0122] 3) Point cloud and image fusion process.

[0123] The point cloud data of the LiDAR is processed by Euclidean clustering to generate a three-dimensional bounding box, and the image data of the camera is processed by the YOLOv5 algorithm to generate a two-dimensional target detection box. The two are fused to extract obstacles, such as Figure 6 shown.

[0124] 3.1) Determination of detection frame overlap: When one detection frame completely surrounds another detection frame, it can be determined that the 3D detection frame and the 2D detection frame completely overlap, and it is determined that the same object is detected, so the obstacle feature extraction is output; however, when the two frames do not surround each other, the overlap is used to determine whether they are the same object.

[0125] To calculate the overlap, we first need to calculate the area of ​​the target detection frame. For the camera's target detection frame, the area calculation formula is:

[0126]

[0127] Among them, S represents the area of ​​the target detection box, which represents the number of pixels occupied by the target detection box in the image; v i is the volume of the i-th detection box; u i is the weight or confidence of the i-th detection box; t is the total number of detection boxes.

[0128] In actual experimental situations, the LiDAR may detect irregular objects, which will cause the detection frame projected onto the vehicle image to also present an irregular image. In order to facilitate comparison and a series of calculations, the 3D detection frame generated by the LiDAR is projected onto a plane and treated as a simplified rectangular frame.

[0129] S 3D =|Y 3D-Max -Y 3D-Min ||Z 3D-Max -Z 3D-Min |

[0130] Among them, Y and Z represent the y-axis and z-axis coordinates of the vertex of the detection box on the two-dimensional plane, respectively. This method can more intuitively compare the detection results of the laser radar and the camera; S 3D is the volume of the object in three-dimensional space; 3D-Max is the maximum value of the object on the Y axis; 3D-Min is the minimum value of the object on the Y axis; Z 3D-Max is the maximum value of the object on the Y axis; Z 3D-Min It is the minimum value of the object on the Z axis.

[0131] The basis for determining the overlap degree is set as a comparison value between the overlap area and the areas of the laser radar detection frame and the camera detection frame.

[0132]

[0133] Among them, M is the overlap of two detection frames, s μ is their overlapping area.

[0134] According to the center points of the lidar and camera detection frames, the KD-Tree nearest neighbor search algorithm is used to find the closest matching pairs. During matching, if the distance between the center points of two detection frames exceeds their maximum side length, it means that the distance between them is too far to be fused. Traverse the detection frame pairs that have been matched successfully,

[0135] Compare their overlap ranges and determine whether the overlap is greater than or equal to 50%. If there are unmatched points or the overlap is less than 50%, no subsequent fusion operation will be performed.

[0136] like Figure 7 As shown, compared with the 360-degree surround view, the driver can more intuitively and clearly understand the road conditions on the inside of the vehicle when turning and the expected driving route of the trailer, so as to make corresponding adjustments to ensure the safety of the turning process; the driver's operation is relatively simple; the cost is low; the process is simpler; the inner wheel difference danger zone obtained by algorithm fusion data is accurate and the error is small.

[0137] Based on the same inventive concept, the embodiment of the present application also provides a vehicle inner wheel difference dangerous area limit detection and warning device for implementing the vehicle inner wheel difference dangerous area limit detection and warning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more vehicle inner wheel difference dangerous area limit detection and warning device embodiments provided below can refer to the limitations of the vehicle inner wheel difference dangerous area limit detection and warning method above, and will not be repeated here.

[0138] In an exemplary embodiment, a vehicle inner wheel difference dangerous area limit detection and warning device is provided, comprising:

[0139] The front inner wheel steering angle determination module is used to determine the front inner wheel steering angle according to the angle information between the tractor and the trailer collected by the angle sensor; the angle sensor is arranged at the hinge point of the tractor and the trailer and at the bottom of the tractor steering point.

[0140] The first judgment module is used to judge whether the vehicle is turning according to the front inner wheel steering angle to obtain a first judgment result.

[0141] Camera and radar to simultaneously capture real-time images of the vehicle's inside turn as well as point cloud data.

[0142] The inner wheel difference danger zone limit determination module is used to calculate the inner wheel difference of the tractor and the inner wheel difference of the trailer based on the real-time image and the point cloud data, and determine the inner wheel difference danger zone limit according to the inner wheel difference of the tractor and the inner wheel difference of the trailer.

[0143] The road obstacle location and distance recognition module is used to recognize the road obstacle location and distance based on the neural network model according to the real-time image and the point cloud data.

[0144] The early warning module is used to determine whether to issue an alarm based on the position and distance of the road obstacle and the inner wheel difference danger zone limit.

[0145] The return module is used to return to the step of "determining the steering angle of the front inner wheel according to the angle information between the tractor and the trailer collected by the angle sensor".

[0146] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store vehicle inner wheel difference dangerous area limit detection and warning data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a vehicle inner wheel difference dangerous area limit detection and warning method is implemented.

[0147] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0148] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0149] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric random access memory (Ferroelectric RandomAccess Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0151] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0152] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0153] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A vehicle inner wheel difference dangerous area limit detection and early warning method, characterized in that: The vehicle inner wheel difference dangerous area limit detection and early warning method comprises: The front inner wheel steering angle is determined based on the angle information between the tractor and the trailer collected by the angle sensor; the angle sensor is arranged at the hinge point between the tractor and the trailer and at the bottom of the tractor steering point; According to the front inner wheel steering angle, determining whether the vehicle is turning, and obtaining a first determination result; the vehicle includes a tractor and a trailer; If the first judgment result is yes, using the camera and radar on the inner side of the vehicle to simultaneously capture the real-time image and point cloud data of the inner side of the vehicle; Based on the real-time image and the point cloud data, the inner wheel difference of the tractor and the inner wheel difference of the trailer are calculated, and the inner wheel difference danger zone limit is determined according to the inner wheel difference of the tractor and the inner wheel difference of the trailer; Based on the neural network model, identifying the location and distance of road obstacles according to the real-time image and the point cloud data; Determining whether to issue an alarm based on the location and distance of the road obstacle and the inner wheel difference danger zone limit; If the first judgment result is no, return to step "determine the front inner wheel steering angle according to the angle information between the tractor and the trailer collected by the angle sensor".

2. The vehicle inner wheel difference dangerous area limit detection and early warning method according to claim 1 is characterized in that: Judging whether the vehicle is turning according to the front inner wheel steering angle, and obtaining a first judgment result, specifically includes: Determine whether the front inner wheel steering angle is greater than or equal to 15° and whether the duration is greater than or equal to 2.5 seconds, and obtain a second determination result; If the second judgment result is yes, determining the first judgment result is that the vehicle is turning; If the second judgment result is no, it is determined that the first judgment result is that the vehicle is not turning.

3. The vehicle inner wheel difference dangerous area limit detection and early warning method according to claim 1 is characterized in that: Based on the real-time image and the point cloud data, the inner wheel difference of the tractor and the inner wheel difference of the trailer are calculated, and the inner wheel difference danger zone limit is determined according to the inner wheel difference of the tractor and the inner wheel difference of the trailer, specifically including: fusing the real-time image and the point cloud data to determine a fused image; Calculating the inner wheel difference of the tractor and the inner wheel difference of the trailer according to the fused image; Determining a vehicle inner contour according to an inner wheel difference of the tractor and an inner wheel difference of the trailer; determining a rear inner wheel steering trajectory according to the vehicle inner contour; The inner wheel difference danger zone limit is determined according to the rear inner wheel steering trajectory.

4. The vehicle inner wheel difference dangerous area limit detection and early warning method according to claim 3 is characterized in that: According to the fused image, the inner wheel difference of the tractor and the inner wheel difference of the trailer are calculated, specifically including: Based on the fused image, determine the basic vehicle data and turning angle information; the basic vehicle data includes the wheelbase L1 of the tractor, the wheelbase L2 of the trailer, the front wheelbase d1 of the tractor, the rear wheelbase d2 of the tractor, and the distance k between the towing pin and the rear axle of the tractor; the turning angle information includes the turning radius R5 of the lane where the tractor is located, the turning radius R6 of the lane where the trailer is located, the angle α between the midpoint of the tractor and the rear axle and the center of the turning circle, the angle β between the midpoint of the front axle and the rear axle of the trailer and the center of the turning circle, the movement radius R1 of the centerline of the inner rear wheel of the tractor, the movement radius R3 of the centerline of the inner rear wheel of the trailer, the movement radius R2 of the centerline of the inner front wheel of the tractor, and the movement radius R4 of the centerline of the inner front wheel of the trailer; The inner wheel difference of the tractor and the inner wheel difference of the trailer are calculated according to the vehicle basic data and the turning angle information.

5. The vehicle inner wheel difference dangerous area limit detection and early warning method according to claim 4 is characterized in that: Calculating the inner wheel difference of the tractor and the inner wheel difference of the trailer according to the basic vehicle data and the turning angle information specifically includes: The inner wheel difference of the tractor is calculated using the formula m1=R1-R2; wherein m1 is the inner wheel difference of the tractor; The inner wheel difference of the trailer is calculated using the formula m2=R3-R4; wherein m2 is the inner wheel difference of the trailer; 6. The vehicle inner wheel difference dangerous area limit detection and early warning method according to claim 1 is characterized in that: Based on the neural network model, the position and distance of the road obstacle are identified according to the real-time image and the point cloud data, specifically including: Inputting the real-time image into the neural network model, processing the real-time image based on the YOLOv5 algorithm, and generating an image containing a two-dimensional target detection frame; the neural network model is a YOLOv5 network; Performing Euclidean clustering on the point cloud data to generate a three-dimensional point cloud bounding box; Projecting the three-dimensional point cloud bounding box onto the image containing the two-dimensional target detection frame to determine a fused image; The position and distance of the road obstacle are identified according to the fused image.

7. A vehicle inner wheel difference dangerous area limit detection and warning device, characterized in that: The vehicle inner wheel difference dangerous area limit detection and early warning device comprises: A front inner wheel steering angle determination module is used to determine the front inner wheel steering angle according to the angle information between the tractor and the trailer collected by the angle sensor; the angle sensor is arranged at the hinge point of the tractor and the trailer and at the bottom of the tractor steering point; A first judgment module, used for judging whether the vehicle is turning according to the front inner wheel steering angle, and obtaining a first judgment result; Cameras and radars to simultaneously capture real-time images of the vehicle’s inboard turn as well as point cloud data; An inner wheel difference danger zone limit determination module, used to calculate the inner wheel difference of the tractor and the inner wheel difference of the trailer based on the real-time image and the point cloud data, and determine the inner wheel difference danger zone limit according to the inner wheel difference of the tractor and the inner wheel difference of the trailer; A road obstacle location and distance recognition module, used to recognize the location and distance of road obstacles based on the real-time image and the point cloud data based on a neural network model; An early warning module, used to determine whether to issue an alarm based on the location and distance of the road obstacle and the inner wheel difference danger zone limit; The return module is used to return to the step of "determining the steering angle of the front inner wheel according to the angle information between the tractor and the trailer collected by the angle sensor".

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle inner wheel difference dangerous area limit detection and warning method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle inner wheel difference dangerous area limit detection and warning method described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle inner wheel difference dangerous area limit detection and warning method described in any one of claims 1 to 6 is implemented.

Images