Trailer blind area early warning method and device, controller and trailer

Abstract

The invention provides a trailer blind area early warning method and device, a controller and a trailer. The method comprises the steps that environment data around the trailer and the turning angle of a trailer head are obtained; if a preset target object is identified from the environment data, determining the position of the target object in a vehicle body world coordinate system according to the environment data; predicting a target position of the target object at the next preset moment according to the position of the target object in the vehicle body world coordinate system; determining a dangerous area around the trailer according to the turning angle; and if the target position is in the dangerous area, outputting an alarm prompt. Through real-time environment perception, target object position prediction and dangerous area evaluation of trailer turning, the risk of blind area accidents of the trailer is reduced, the driving pressure of a driver is relieved, and the driving safety is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, controller and trailer for trailer blind spot warning. Background Art

[0002] Trailer transportation scenarios are an important part of the transportation field, involving multiple aspects such as cargo transportation, building materials transportation, express transportation and garbage transportation.

[0003] Due to the structure of the vehicle, there will be different degrees of visual blind spots in the field of vision of the trailer driver, and when the driver only observes the environmental information around the vehicle through the rearview mirror, there will be a large visual blind spot, which seriously affects driving safety.

[0004] Therefore, how to detect and warn the blind spots of trailers is an urgent problem to be solved. Summary of the invention

[0005] The method, device, controller and trailer for trailer blind spot warning provided in the present application are used to detect and warn the blind spots of the trailer to improve driving safety.

[0006] In a first aspect, the present application provides a method for trailer blind spot warning, the method comprising:

[0007] Obtain environmental data around the trailer and the turning angle of the trailer head;

[0008] If a preset target object is identified from the environmental data, determining the position of the target object in the vehicle body world coordinate system according to the environmental data;

[0009] Predicting the target position of the target object at the next preset time according to the position of the target object in the vehicle body world coordinate system;

[0010] determining a danger zone around the trailer according to the turning angle;

[0011] If the target position is within the danger zone, an alarm prompt is output, wherein the alarm prompt is used to prompt the driver to pay attention to the area around the vehicle.

[0012] Optionally, the environmental data is image data, and determining the position of the target object in the vehicle body world coordinate system according to the environmental data includes:

[0013] Determining the position of the target object in the image according to the image data;

[0014] According to the position of the target object in the image, the position of the target object in the vehicle body world coordinate system is determined.

[0015] Optionally, if the image data is collected by a camera on the front of a trailer, determining the position of the target object in the world coordinate system of the vehicle body according to the position of the target object in the image includes:

[0016] Determine the angle conversion matrix corresponding to the turning angle according to the preset mapping relationship between the turning angle and the angle conversion matrix;

[0017] The position of the target object in the vehicle body world coordinate system is determined based on the position coordinates of the target object in the image, a preset rotation matrix, a preset translation matrix, and the angle conversion matrix. The rotation matrix and the translation matrix are pre-calibrated to represent the conversion relationship between the pixel coordinate system and the vehicle body world coordinate system.

[0018] Optionally, if the image data is collected by a trailer body camera, determining the position of the target object in the vehicle body world coordinate system according to the position of the target object in the image includes:

[0019] The position of the target object in the vehicle body world coordinate system is determined according to the position coordinates of the target object in the image, a preset rotation matrix and a translation matrix, wherein the rotation matrix and the translation matrix are pre-calibrated to represent the conversion relationship between the pixel coordinate system and the vehicle body world coordinate system.

[0020] Optionally, predicting the target position of the target object at the next preset moment according to the position of the target object in the vehicle body world coordinate system includes:

[0021] According to the real-time acquired position of the target object in the vehicle body world coordinate system, the driving trajectory of the target object is predicted through Kalman filtering to obtain the target position of the target object in the vehicle body world coordinate system at the next preset moment.

[0022] Optionally, the method further includes:

[0023] Get the checkerboard pattern captured by the camera;

[0024] Obtaining the position of the target grid in the checkerboard pattern in the vehicle body world coordinate system;

[0025] Get the position of the target grid in the image taken by the camera in the pixel coordinate system;

[0026] According to the position of the target grid in the vehicle body world coordinate system and the position of the target grid in the pixel coordinate system, a calibrated rotation matrix and a calibrated translation matrix are determined.

[0027] Optionally, before determining the position of the target object in the vehicle body world coordinate system according to the environmental data, the method further includes:

[0028] Inputting the environmental data into a pre-trained deep learning-based target detection model to obtain a result of whether there is a preset target object in the environmental data;

[0029] If no preset target object is identified, new environmental data is acquired again.

[0030] In a second aspect, the present application provides a trailer blind spot warning device, the device comprising:

[0031] An acquisition module is used to acquire environmental data around the trailer and the turning angle of the trailer head;

[0032] A position determination module, configured to determine the position of a preset target object in a vehicle body world coordinate system according to the environmental data if a preset target object is identified from the environmental data;

[0033] A prediction module, used for predicting the target position of the target object at the next preset moment according to the position of the target object in the vehicle body world coordinate system;

[0034] An area determination module, used to determine a dangerous area around the trailer according to the turning angle;

[0035] The output module is used to output an alarm prompt if the target position is within the dangerous area, and the alarm prompt is used to prompt the driver to pay attention to the area around the vehicle.

[0036] In a third aspect, the present application provides a controller, including: a memory, a processor;

[0037] The memory stores computer-executable instructions;

[0038] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method as described in any one of the first aspects.

[0039] In a fourth aspect, the present application provides a trailer, comprising a controller, wherein the controller is used to execute the method as described in any one of the first aspects.

[0040] In a fifth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementations of the first aspect.

[0041] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect.

[0042] The present application provides a method, device, controller and trailer for trailer blind spot warning, the method comprising: obtaining environmental data around the trailer and the turning angle of the trailer head; if a preset target object is identified from the environmental data, then according to the environmental data, determine the position of the target object in the vehicle body world coordinate system; according to the position of the target object in the vehicle body world coordinate system, predict the target position of the target object at the next preset time; according to the turning angle, determine the dangerous area around the trailer; if the target position is in the dangerous area, output an alarm prompt, the alarm prompt is used to remind the driver to pay attention to the area around the vehicle. Through real-time environmental perception, target object position prediction and dangerous area assessment, driving safety is effectively improved, the risk of blind spot accidents is reduced, and the driving pressure of the driver is reduced. This strategy provides the driver with accurate early warning, helps to judge potential dangers and respond in time, avoids collisions and accidents, and improves driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] Figure 1 Schematic diagram of the process of the trailer blind spot warning method provided in this application Figure 1 ;

[0045] Figure 2 A schematic diagram of the trailer provided for this application;

[0046] Figure 3 Schematic diagram of the process of the trailer blind spot warning method provided in this application Figure 2 ;

[0047] Figure 4 Calibration diagram provided for this application;

[0048] Figure 5 A schematic diagram of the structure of a trailer blind spot warning device provided in this application;

[0049] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.

[0050] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0051] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0052] Trailer types include semi-trailers and full trailers. Due to the structure of the trailer, the driver will have different degrees of visual blind spots within the field of vision, which seriously affects driving safety. Due to the angle change between the body and the trailer and the length of the front and rear of the vehicle, the driver cannot directly see the traffic conditions in these areas. Especially during the turning process, the angle of the trailer front relative to the body changes, making it more difficult for the driver to perceive the rear and side traffic. Especially at large turns or roundabouts, the trailer may come into contact with other vehicles, causing traffic accidents.

[0053] In view of the above problems, the present application provides a method for trailer blind spot warning. First, the current dangerous area of ​​the trailer is determined according to the turning angle of the front of the vehicle. Sensors are placed around the trailer to detect all-round viewing angles. Then the area around the vehicle is detected by the sensor. When a vehicle or person is detected around the trailer, it is determined whether it will move to the dangerous area around the vehicle. If this happens, a warning is issued to the driver in time. This can help the driver to warn of danger.

[0054] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0055] Figure 1 Schematic diagram of the process of the trailer blind spot warning method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0056] S101, obtaining environmental data around the trailer and a turning angle of the front of the trailer.

[0057] In this step, sensors (such as cameras, radars, lidars, ultrasonic sensors, etc.) are pre-placed around the trailer to obtain surrounding environmental information. These environmental data include raw data such as the location, speed, and category of objects (such as pedestrians, vehicles, obstacles).

[0058] The turning angle of the trailer head can be obtained by any of the following methods:

[0059] In one implementation, the turning angle is obtained based on the angle sensor of the hinge plate between the front and the car body of the semi-trailer. The hinge plate angle sensor is directly installed at the hinge part between the front and the car body. The sensor can detect the rotation angle of the front relative to the car body (i.e., the turning angle) and feed the data back to the vehicle controller in real time.

[0060] In one implementation, the steering wheel angle sensor records the angle at which the driver turns the steering wheel, and calculates the turning angle of the trailer or semitrailer based on the steering ratio (ie, the proportional relationship between the steering wheel turning angle and the wheel turning angle).

[0061] In one implementation, a vehicle camera or other visual sensors (such as LiDAR, stereo vision sensors, etc.) are used to estimate the turning angle of the vehicle by analyzing the environment in real time. The visual sensor can capture image information around the vehicle and analyze the relative positions of the road, lane lines, surrounding objects, etc. through image processing technology (such as computer vision algorithms) to infer the turning angle of the vehicle.

[0062] S102: If a preset target object is identified from the environmental data, the position of the target object in the vehicle body world coordinate system is determined according to the environmental data.

[0063] In this step, object recognition is performed on the collected environmental data. Object recognition uses deep learning or traditional algorithms to determine the types of surrounding objects (such as pedestrians, other vehicles, obstacles, etc.). If a target object is recognized, the position of the object is converted to the world coordinate system of the trailer body. If no target object is recognized, the process returns to step S101 and reacquires new environmental data for the next target object recognition.

[0064] The body world coordinate system is a coordinate system relative to the trailer body, and the body coordinate system is used to accurately describe the relative position between the target object and the trailer.

[0065] If the environmental data is an image, coordinate transformation is required based on the image to determine its position in the vehicle body world coordinate system.

[0066] If the environmental data is radar data, the distance of the target object from the vehicle can be directly identified, and its position in the world coordinate system of the vehicle body can be converted.

[0067] S103 . Predicting the target position of the target object at the next preset time according to the position of the target object in the vehicle body world coordinate system.

[0068] Once the target object is identified, the current position information of the target object can be determined based on the data of the target object in the environmental data, and the position of the target object at the next preset time can be predicted based on the motion characteristics of the target object (such as speed, direction, acceleration, etc.). The prediction method can use a simple kinematic model (such as uniform linear motion) or a more complex prediction algorithm (such as Kalman filtering, neural network, etc.) to cope with the motion changes of the target object.

[0069] Among them, the "preset time" can be set to a short time window, such as 1 second, 2 seconds, 3 seconds, etc., so that the vehicle can adjust its decision in time in a shorter time.

[0070] S104: Determine a dangerous area around the trailer according to the turning angle.

[0071] The turning angle of the trailer head is used to analyze and predict the dangerous areas that the trailer may enter when turning. When turning, the angle between the front and rear of the trailer and the length of the trailer will affect the movement trajectory of the trailer, and thus affect the safety of the surrounding area.

[0072] An area within a first preset distance around the vehicle body may be set as a danger zone, and the danger zone may change as the vehicle turns.

[0073] S105: If the target position is within the dangerous area, an alarm prompt is output, where the alarm prompt is used to remind the driver to pay attention to the area around the vehicle.

[0074] Based on the previous prediction results, it is determined whether the target object is in the danger zone of the trailer at the preset time. If the target object is in the danger zone, an alarm will be issued to remind the driver of the potential risk. This alarm is usually prompted by sound or visual display (such as warning lights or display screens on the dashboard).

[0075] Optionally, the danger zone can be divided into different danger zone levels, and the closer to the trailer or the turning area of ​​the trailer, the higher the danger level, and the danger zone level is indicated in the alarm prompt. Exemplarily, the higher the danger level of the target object, the more urgent the alarm sound.

[0076] This embodiment provides a method for trailer blind spot warning, which includes: obtaining environmental data around the trailer and the turning angle of the trailer head; if a preset target object is identified from the environmental data, then according to the environmental data, determine the position of the target object in the vehicle body world coordinate system; according to the position of the target object in the vehicle body world coordinate system, predict the target position of the target object at the next preset time; according to the turning angle, determine the dangerous area around the trailer; if the target position is in the dangerous area, output an alarm prompt, and the alarm prompt is used to remind the driver to pay attention to the area around the vehicle. Through real-time environmental perception, target object position prediction and dangerous area assessment, driving safety is effectively improved, the risk of blind spot accidents is reduced, and the driver's driving pressure is reduced. This strategy provides drivers with accurate warnings, helps them judge potential dangers and respond in time, avoids collisions and accidents, and improves driving safety.

[0077] The following is a detailed introduction using an example.

[0078] The sensor arranged on the vehicle body in this example is a fisheye camera. Figure 2 The trailer diagram provided for this application. Figure 2 As shown in the figure, six fisheye cameras are arranged at the black dots on the vehicle body to collect surrounding images. There are two surrounding circles around the vehicle body, which are schematic diagrams of two danger level areas.

[0079] Figure 3 Schematic diagram of the process of the trailer blind spot warning method provided in this application Figure 2 ,like Figure 3 As shown, the following steps are included:

[0080] S201, acquiring images in real time.

[0081] The number of cameras is set according to the length of the trailer. Figure 2 As shown, six fisheye cameras are set on the vehicle. The six fisheye cameras acquire image data in real time and transmit the images to the controller.

[0082] S202: Identify whether there is a target object in the image.

[0083] Among them, the target objects are preset vehicles, people, animals, etc.

[0084] In a specific implementation, a target detection model (e.g., an optimized YOLOv8 model) is selected to detect the target object. If the target object is detected, the target detection model can use a rotated rectangular frame to frame the detection result on the image. If there may be multiple target objects on the image, different rotated rectangular frames are used to frame different target objects.

[0085] If the target object exists, execute the subsequent steps, otherwise end the process.

[0086] S203: Determine the position of the target object in the image according to the image data.

[0087] In this step, the image in the fisheye camera is distorted. Therefore, the position of the target object in the vehicle body coordinate system cannot be directly determined. A conversion is required. First, the position of the rotated rectangular box in the image is determined, and then it is converted to the vehicle body coordinate system.

[0088] Exemplarily, the lower left corner of the image is taken as the origin (other origins may also be defined as needed), a pixel coordinate system is constructed, and the coordinates of the target object are expressed as pixel values, so that the position of the target object in the pixel coordinate system can be determined.

[0089] S204: Determine the position of the target object in the vehicle body world coordinate system according to the position of the target object in the image.

[0090] In this step, based on the camera's intrinsic parameters (such as focal length and distortion parameters) and extrinsic parameters (such as the relative position and orientation of the camera and the vehicle body), using appropriate mathematical models, the two-dimensional coordinates in the image can be converted into three-dimensional positions in the vehicle body coordinate system, thereby determining the exact position of the target object in the vehicle body coordinate system.

[0091] The transformation relationship between the body camera and the body coordinate system is usually described by extrinsic parameters. Extrinsic parameters refer to the rotation and translation from the camera coordinate system to the body coordinate system. Extrinsic parameters define the position and orientation of the camera in the body coordinate system. This transformation relationship is usually represented by a rotation matrix and a translation vector.

[0092] The body coordinate system (body world coordinate system) is usually a static coordinate system related to the fixed geometric structure of the vehicle. The origin of the body coordinate system is usually set at a key point of the vehicle (such as the center of the body, the center of the rear axle, etc.). The origin of the camera's own coordinate system is located at the optical center of the camera, and the direction of the axis is usually determined by the optical axis of the camera. The conversion relationship between the body coordinate system and the camera coordinate system can be expressed as the following matrix equation:

[0093] P body =R×P camera +T

[0094] Among them, P body is a point in the vehicle body coordinate system, P camera It is the same point in the camera coordinate system, R is the rotation matrix of the camera coordinate system relative to the body coordinate system, which describes the rotation relationship between the coordinate systems; T is the translation matrix (also called translation vector), which describes the translation of the origin of the camera coordinate system relative to the origin of the body coordinate system.

[0095] The rotation matrix describes the relative orientation of two coordinate systems through a 3×3 matrix, mapping the three axes of the camera coordinate system to the corresponding three axes of the body coordinate system. Each column (or row) of the rotation matrix corresponds to the direction of the axis of the camera coordinate system in the body coordinate system. For example, if the x-axis of the camera points forward, then the first column of the corresponding rotation matrix should be aligned with the x-axis of the body coordinate system. The rotation matrix can be obtained by calibration.

[0096] The translation matrix T represents the position of the origin of the camera coordinate system relative to the origin of the body coordinate system. The translation vector is usually a known fixed value that reflects the spatial position of the camera relative to the center of the body. For example, if the camera is mounted on the front of the car, the translation vector represents the distance between the camera and the center of the body.

[0097] Therefore, by pre-calibrating the translation matrix and rotation matrix between the vehicle body world coordinate system and the image pixel coordinate system (or camera coordinate system), the distance of the target object in the vehicle body world coordinate system can be determined based on the position of the target object in the pixel coordinate system.

[0098] It should be noted that the trailer is equipped with multiple cameras, and the coordinate system of each camera needs to be converted into a unified vehicle body coordinate system in order to perform image stitching or fusion.

[0099] However, when the front of the vehicle turns, the front camera also turns accordingly. Therefore, the coordinate transformation method of the front camera is slightly different from that of the body camera, and the angle change of the front of the vehicle needs to be taken into account.

[0100] In one implementation, the values ​​of the rotation matrix and translation matrix corresponding to the front camera at each angle are pre-calibrated, and the rotation matrix and translation matrix of the body camera are independent of the angle. After obtaining the turning angle of the front of the vehicle, the corresponding rotation matrix and translation matrix can be found, and the position coordinates in the image can be converted to the body world coordinate system based on the two matrices found.

[0101] In another implementation, the rotation matrix and translation matrix of the front camera are also fixed matrices, and an angle conversion matrix is ​​introduced to represent the change of the camera after the angle changes. The angle conversion matrix can be determined based on the mapping relationship between the preset rotation angle and the angle conversion matrix, as well as the actual angle. The angle conversion matrix can be one matrix or two matrices, and the two matrices act on the rotation matrix and the translation matrix respectively.

[0102] S205: Predict the target position of the target object at the next moment.

[0103] In a specific way, for each target object identified in the camera, the detected target frame result is tracked and predicted through Kalman filtering to calculate the target object movement trend. Kalman filtering is a recursive algorithm for estimating and predicting the state of a dynamic system. This application is mainly used to predict the position of the target object on the fisheye image (the target object can be represented by a block diagram). The state prediction formula is as follows:

[0104] x(k)=F×x(k-1)+B×u(k-1)+w(k-1)

[0105] Among them, x(k) is the state vector at the current moment (including the position, velocity, acceleration and other information of the target object); F is the state transfer matrix, which describes how to derive the state at the current moment from the state at the previous moment; x(k-1) is the state vector at the previous moment; B is the control input matrix; u(k-1) is the control input vector; w(k-1) is the process noise vector.

[0106] Through Kalman filtering, combined with the position information of the target object over a period of time, the target's movement trend is predicted. Specifically, the state transfer matrix is ​​used to describe the movement law of the target object, and the measurement error is eliminated by continuously updating the prediction value, making the position prediction of the target object more accurate.

[0107] S206: Determine whether the target position is in a dangerous area.

[0108] If the target position is in the dangerous zone, execute step S207.

[0109] If the target position is not in the dangerous zone, return to step S201.

[0110] S207, issue an alarm.

[0111] The blind area and non-blind area around the trailer body are divided into Figure 2 The alarm area shown in the figure. The line closer to the vehicle body is the secondary alarm line, and the line farther from the vehicle body is the primary alarm line. When the target position is between the primary and secondary alarm lines, the primary alarm prompt is triggered to remind the driver to take countermeasures. When the target position is within the secondary alarm line, the secondary alarm prompt is triggered to remind the driver to take countermeasures.

[0112] This solution combines the all-round perception capability of the fisheye camera with the dynamic obstacle prediction algorithm. It can monitor and predict the movement trends of surrounding obstacles in real time and accurately construct the warning area in the vehicle coordinate system. It can greatly improve vehicle driving safety and reduce the occurrence of traffic accidents. At the same time, it provides drivers with a smarter and safer driving experience.

[0113] The rotation matrix and translation matrix of the vehicle camera need to be calibrated in advance or at regular intervals. The calibration process mainly uses a chessboard to calibrate the camera's internal and external parameters by detecting the chessboard corners, and then obtain the conversion relationship between the camera coordinate system and the world coordinate system.

[0114] Figure 4 The calibration diagram provided for this application is as follows: Figure 4 As shown in the figure, the calibration personnel first put the chessboard outside the body camera and record the real position in the body coordinate system. The image with the chessboard pattern is captured by the fisheye camera. The image should contain a complete chessboard for corner point detection. The corner points of the chessboard in the camera image are detected using the corner point detection algorithm. In the detected corner point set, the position of the target grid is identified and the two-dimensional pixel coordinates of the target grid are determined.

[0115] According to the three-dimensional position of the target grid in the vehicle body world coordinate system and the pixel coordinates of the target grid in the pixel coordinate system, the rotation matrix and the translation matrix can be determined. If the rotation matrix or the translation matrix after calibration is different from that before calibration, the calibrated matrix is ​​used for subsequent calculations.

[0116] For the calibration of the front camera, the vehicle is controlled to rotate at different angles for calibration to ensure that the pixel coordinates can be accurately converted to the world coordinate system of the vehicle body at different angles.

[0117] Figure 5 A structural diagram of a trailer blind spot warning device provided in this application, such as Figure 5 As shown, the trailer blind spot warning device 50 includes:

[0118] An acquisition module 501 is used to acquire environmental data around the trailer and a turning angle of the trailer head;

[0119] A position determination module 502 is used to determine the position of the target object in the vehicle body world coordinate system according to the environment data if a preset target object is identified from the environment data;

[0120] A prediction module 503, used to predict the target position of the target object at the next preset time according to the position of the target object in the vehicle body world coordinate system;

[0121] An area determination module 504 is used to determine a dangerous area around the trailer according to the turning angle;

[0122] The output module 505 is used to output an alarm prompt if the target position is within the dangerous area, and the alarm prompt is used to prompt the driver to pay attention to the area around the vehicle.

[0123] Optionally, the environmental data is image data, and the location determination module 502 is specifically used for:

[0124] Determining the position of the target object in the image according to the image data;

[0125] According to the position of the target object in the image, the position of the target object in the vehicle body world coordinate system is determined.

[0126] Optionally, if the image data is collected by a camera on the front of the trailer, the position determination module 502 is specifically used to:

[0127] Determine the angle conversion matrix corresponding to the turning angle according to the preset mapping relationship between the turning angle and the angle conversion matrix;

[0128] The position of the target object in the vehicle body world coordinate system is determined based on the position coordinates of the target object in the image, a preset rotation matrix, a preset translation matrix, and the angle conversion matrix. The rotation matrix and the translation matrix are pre-calibrated to represent the conversion relationship between the pixel coordinate system and the vehicle body world coordinate system.

[0129] Optionally, if the image data is collected by a trailer body camera, determining the position of the target object in the vehicle body world coordinate system according to the position of the target object in the image includes:

[0130] The position of the target object in the vehicle body world coordinate system is determined according to the position coordinates of the target object in the image, a preset rotation matrix and a translation matrix, wherein the rotation matrix and the translation matrix are pre-calibrated to represent the conversion relationship between the pixel coordinate system and the vehicle body world coordinate system.

[0131] Optionally, the prediction module 503 is specifically used for:

[0132] According to the real-time acquired position of the target object in the vehicle body world coordinate system, the driving trajectory of the target object is predicted through Kalman filtering to obtain the target position of the target object in the vehicle body world coordinate system at the next preset moment.

[0133] Optionally, the device further includes a calibration module 506, and the calibration module 506 is specifically configured to:

[0134] Get the checkerboard pattern captured by the camera;

[0135] Obtaining the position of the target grid in the checkerboard pattern in the vehicle body world coordinate system;

[0136] Get the position of the target grid in the image taken by the camera in the pixel coordinate system;

[0137] According to the position of the target grid in the vehicle body world coordinate system and the position of the target grid in the pixel coordinate system, a calibrated rotation matrix and a calibrated translation matrix are determined.

[0138] Optionally, the device further includes an identification module 507, which is specifically configured to:

[0139] Inputting the environmental data into a pre-trained deep learning-based target detection model to obtain a result of whether there is a preset target object in the environmental data;

[0140] If no preset target object is identified, new environmental data is acquired again.

[0141] The trailer blind spot warning device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0142] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application, and the electronic device may be a controller. Figure 6 As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0143] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0144] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0145] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0146] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0147] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0148] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0149] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0150] The present application also provides a trailer, including a controller, wherein the controller is used to execute the method in the above embodiment.

[0151] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0152] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0153] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0156] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0157] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0158] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for trailer blind spot warning, characterized in that: The method comprises: Obtain environmental data around the trailer and the turning angle of the trailer head; If a preset target object is identified from the environmental data, determining the position of the target object in the vehicle body world coordinate system according to the environmental data; Predicting the target position of the target object at the next preset time according to the position of the target object in the vehicle body world coordinate system; determining a danger zone around the trailer according to the turning angle; If the target position is within the danger zone, an alarm prompt is output, wherein the alarm prompt is used to prompt the driver to pay attention to the area around the vehicle.

2. The method according to claim 1, characterized in that The environmental data is image data, and determining the position of the target object in the vehicle body world coordinate system according to the environmental data includes: Determining the position of the target object in the image according to the image data; According to the position of the target object in the image, the position of the target object in the vehicle body world coordinate system is determined.

3. The method according to claim 2, characterized in that If the image data is collected by a camera on the front of the trailer, determining the position of the target object in the world coordinate system of the vehicle body according to the position of the target object in the image includes: Determine the angle conversion matrix corresponding to the turning angle according to the preset mapping relationship between the turning angle and the angle conversion matrix; The position of the target object in the vehicle body world coordinate system is determined based on the position coordinates of the target object in the image, a preset rotation matrix, a preset translation matrix, and the angle conversion matrix. The rotation matrix and the translation matrix are pre-calibrated to represent the conversion relationship between the pixel coordinate system and the vehicle body world coordinate system.

4. The method according to claim 2, characterized in that: If the image data is collected by a trailer body camera, determining the position of the target object in the vehicle body world coordinate system according to the position of the target object in the image includes: The position of the target object in the vehicle body world coordinate system is determined according to the position coordinates of the target object in the image, a preset rotation matrix and a preset translation matrix. The rotation matrix and the translation matrix are pre-calibrated to represent the conversion relationship between the pixel coordinate system and the vehicle body world coordinate system.

5. The method according to any one of claims 1 to 4, characterized in that: The step of predicting the target position of the target object at the next preset time according to the position of the target object in the vehicle body world coordinate system includes: According to the real-time acquired position of the target object in the vehicle body world coordinate system, the driving trajectory of the target object is predicted through Kalman filtering to obtain the target position of the target object in the vehicle body world coordinate system at the next preset moment.

6. The method according to claim 3 or 4, characterized in that: The method further comprises: Get the checkerboard pattern captured by the camera; Obtaining the position of the target grid in the checkerboard pattern in the vehicle body world coordinate system; Get the position of the target grid in the image taken by the camera in the pixel coordinate system; According to the position of the target grid in the vehicle body world coordinate system and the position of the target grid in the pixel coordinate system, a calibrated rotation matrix and a calibrated translation matrix are determined.

7. The method according to any one of claims 1 to 4, characterized in that: Before determining the position of the target object in the vehicle body world coordinate system according to the environmental data, the method further includes: Inputting the environmental data into a pre-trained deep learning-based target detection model to obtain a result of whether there is a preset target object in the environmental data; If no preset target object is identified, new environmental data is acquired again.

8. A trailer blind spot warning device, characterized in that: The device comprises: An acquisition module is used to acquire environmental data around the trailer and the turning angle of the trailer head; A position determination module, configured to determine the position of a preset target object in a vehicle body world coordinate system according to the environmental data if a preset target object is identified from the environmental data; A prediction module, used for predicting the target position of the target object at the next preset moment according to the position of the target object in the vehicle body world coordinate system; An area determination module, used to determine a dangerous area around the trailer according to the turning angle; The output module is used to output an alarm prompt if the target position is within the dangerous area, and the alarm prompt is used to prompt the driver to pay attention to the area around the vehicle.

9. A controller, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A trailer, characterized in that: The method comprises a controller, wherein the controller is used to execute the method according to any one of claims 1 to 7.

Images